about summary refs log tree commit diff
path: root/test/python
diff options
context:
space:
mode:
authorPjotr Prins <pjotr.public01@thebird.nl>2026-11-15 01:37:51 +0000
committerPjotr Prins <pjotr.public01@thebird.nl>2026-11-15 01:37:51 +0000
commitc8ce261f2bf499ab3260ff5ce186c801491b9fea (patch)
treea5d26ae5871df85fc13d7ec75e9aaccbdaebc171 /test/python
parent41c70f042731e36fb91b8fb3f4f393a4aa654e9d (diff)
downloadgenenetwork4-c8ce261f2bf499ab3260ff5ce186c801491b9fea.tar.gz
correlation: the sample-axis endpoint, synchronous
POST /compute/correlation now computes instead of returning canned data:
one reference trait against every trait of a target dataset, observations
being individuals, pearson or spearman, top-N by |r|.

The reply is positional -- trait, r, p, n (column names in meta.columns) --
with the method, the minimum overlap and the top-N stated in the notes.
`bicor` is refused with the methods that exist; a bad top_n is refused
rather than clamped; a dataset or trait with nothing to correlate says so
instead of returning an empty list.

src/db.rs gains the bulk fetch the sample axis needs: every value of every
trait of a dataset restricted to the reference trait's individuals, one
record per (trait, individual, value), ordered by trait so the caller
groups as it streams.  It never loads a dataset whole -- ProbeSetData is
5.3e9 rows, so the query narrows by dataset and by individuals, which is
also why the sweep, not the arithmetic, is the whole cost.

Measured on the live db_webqtl: HC_M2_0606_P / 1427571_at against its own
dataset, top 5, HTTP 200 in 16 s; the trait correlates with itself at
1.0 over 99 observations, then 0.802 (p 1.9e-23), 0.790, -0.789, -0.788.
Synchronous is affordable at this size; GN2's job indirection exists for
top-N 20000 across datasets, which is the next thing to time.

Known and deliberate for now: the publish/genotype axes are not wired (the
fetch is probeset-only), and a perfect correlation reports p as null --
GSL's t statistic is undefined at |r| = 1, whose r comes out as
1.0000000000000002, so 1 - r^2 is slightly negative and the engine returns
NaN.  Reproducing the engine exactly, rather than "fixing" the p-value
here, is the decision recorded in docs/correlations.md.
Diffstat (limited to 'test/python')
0 files changed, 0 insertions, 0 deletions