|
The implementation had already landed in 698e6ab by accident -- a `git add
-A` swept it in with the operator-checklist docs, which is why its message
says nothing about partial correlation. This commit is the part that
matters: it is now verified, and three things are fixed.
**Verification.** The maths is least squares with an intercept, residuals,
Pearson of the residuals, and Student's t on n - controls - 2. Checked two
ways against independent code:
* BXDPublish 10001 with 10002 held constant, target 17164, n=6:
ours r=0.9960864631522379 p=0.00029371957431469716, numpy lstsq +
scipy.stats.t.sf r=0.9960864631522379 p=0.00029372.
Zero-order: ours -0.10577609106505677, scipy pearsonr -0.1057760910650567.
* The same for 17084: identical to 16 digits, r=0.9840718340719924.
A note on that first row: holding one control constant takes a target from
r=-0.11 to r=+0.996 on six points. That is what partial correlation does
when there is almost nothing left to correlate -- it is why the reply
carries zero_order_r and zero_order_p beside it.
**Fixes.** `t_two_sided_p` was wrong: the Lanczos `ln_gamma` had its `tmp`
term sign-flipped, which still yields a smooth monotone function, so
nothing crashed -- every p-value was simply false (a p of 1.8e-24 came out
as 1.0). The corrected version matches `scipy.stats.t.sf` to 1.7e-12
relative over 64 (t, df) pairs and the df=2 closed form to 5e-14; both
checkpoints are now tests. A perfect partial r (self-correlation) is
clamped to +-1 with p=0.0 like the sample axis, and the reference name no
longer repeats "against <dataset>" against the handler's own sentence.
The monitor now exercises the axis: `compute-correlation-partial`.
|
|
work yet
Writing down what the monitor found and what fixed it, because it is config
that lives outside the repository and is therefore easy to lose at the next
deploy: with only the cache dirs set, every engine that shells out answers
502 while the whole read surface and search look healthy.
Measured on production after the fix (bin/monitor-gn4 --tier compute):
8 ok, 1 failed -- correlation, gemma (21 055 rows), rqtl, qtl2 (20 453),
fitqtl, ravanan (21 055 rows), bnw, bnw-examples. Before: 1 ok, 7 failed,
1 busy.
The remaining failure is BulkLMM and it is not a config line: the Julia
project is registry-based and no depot on this box has its packages, so
it stops at "Run Pkg.instantiate()". Populating a depot downloads from the
Julia registry, which should be a decision rather than a side effect.
|