diff options
| author | Pjotr Prins <pjotr.public01@thebird.nl> | 2026-11-15 14:41:16 +0000 |
|---|---|---|
| committer | Pjotr Prins <pjotr.public01@thebird.nl> | 2026-11-15 14:41:16 +0000 |
| commit | edaef7fd56e4ce7acd833e929462293478025a34 (patch) | |
| tree | 3c8cc2166fd3c3e057842fa0d08a8b9d03dcc59f /test/python/server_test_case.py | |
| parent | 532825fed53fddb4f8f8f750986f7f9ab2a3d5dc (diff) | |
| download | genenetwork4-edaef7fd56e4ce7acd833e929462293478025a34.tar.gz | |
correlation: the matrix endpoint, and the rule that a deploy ends with a monitor run
`POST /compute/correlation/matrix` takes a list of `{dataset, trait_name}`
references -- a collection, as GN2's matrix form posts it, which spans
datasets -- and answers an N x N matrix of r, p and n.
The five decisions that are not obvious, each recorded where a reader will
look for them:
* **One method for the whole matrix.** GN2's `show_corr_matrix.py` flips to
Spearman mid-row once a Pearson r passes 0.999, which the self-pair always
does, so its matrix is asymmetric by construction. Here the method is
chosen once and r[i][j] == r[j][i] exactly.
* **Each unordered pair is computed once and mirrored** -- N(N-1)/2 engine
calls, not N^2.
* **The diagonal is by construction**: r = 1, p = null, n = the trait's own
count, so half the traits cannot come back as 1.0000000000000002 with a
NaN p.
* **Per-cell n is its own matrix**, and a cell under `min_overlap` (default
6) is null in r and p while its n still says how many points there were.
GN2 accepts 2, where |r| = 1 by definition.
* **The sample axis is the first trait's group** (#106), resolved once and
stated in the reply with the number of individuals it kept.
Fetches are one query per dataset (`get_dataset_values_for_traits`,
`get_publish_values_for_traits`), not one per trait: a matrix chooses a
handful out of twenty thousand. Capped at 1 000 traits, because the cost is
quadratic in the correlations and in the reply. No PCA -- the r matrix has
to be trusted on its own first.
Verified twice: cells agree with scipy to the last bit (BXDPublish
10001/10002/10005: 0.6756270426459695, 0.6950450737755965,
0.8819879006700077 against scipy 0.6756270426459695, 0.6950450737755964,
0.8819879006700073), and a cell equals what the row endpoint says for the
same pair (0.6950450737755965 in both). Edge cases exercised: a 1x1 matrix,
`min_overlap` above every cell's n, an unknown trait, `bicor`, and no
traits at all.
The operator checklist now says a deploy ends with `bin/monitor-gn4 --tier
all`, and says why: `/version` can report a commit and a reachable database
while every engine that shells out is dead. The monitor gained a matrix
check whose judge enforces square, symmetric and a unit diagonal.
Diffstat (limited to 'test/python/server_test_case.py')
0 files changed, 0 insertions, 0 deletions
