about summary refs log tree commit diff
path: root/test/python/test_groups.py
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-11-08 16:23:03 +0000
committerPjotr <wrk@bosa.lan>2026-11-08 16:23:14 +0000
commite934e755f14648fff6ebac07e9cf9e5b37cb3fe4 (patch)
treeb7b921fe7e83bed348d65f3ecda3cb284479cd19 /test/python/test_groups.py
parent0769a0ea0a8654e5e50e03365545c8ccf58d6b91 (diff)
downloadgenenetwork4-e934e755f14648fff6ebac07e9cf9e5b37cb3fe4.tar.gz
api: describe case attributes, and gate large computations only
Two things that came out of reading a real response rather than a test.

**Say what a case attribute means.** `GET /groups/{group}/case-attributes`
returned names and values, and the per-trait attribute endpoint returned
values, so a caller had to guess that `SeqCvge` is a sequencing depth.
MariaDB has had the answer all along in `CaseAttribute.Description`:

  SeqCvge   Depth of DNA sequencing for 1 male of each strain in 2016
  Status    Live= Available at JAX, Cryo=Cryopreserved only, Extinct
  Epoch     BXD family subgroups. Each number with common parents. ...
  Sex       F= female, M= male

Both endpoints now carry it -- a `descriptions` map on the group listing, a
`description` field on the per-trait one -- and the covariate notes quote it,
so `/compute/gemma` answers "Covariate SeqCvge (Depth of DNA sequencing for
1 male of each strain in 2016): categorical ..." instead of leaving the name
to be interpreted. It matters most for covariates: `SeqCvge` is a property
of the *genotype* data, its values include compound ones (`40+31L`, two
sequencing runs), and the server codes those as 20-odd levels rather than as
a depth -- a covariate fitted for the wrong reason, which the description is
the first hint against.

**Gate large computations, not every computation.** The gate was applied to
every job, so a 93-strain test request could be refused because somebody
else's 6 000-individual run held the slot -- which is how the test suite
started serialising itself behind a mutex, a workaround for the wrong
problem. What the gate is for is *large* work: a BXD panel of ~200 finishes
in seconds and two at once cost nothing, while UM-HET3's 6 382 write 207 MB
of kinship per chromosome. It now starts at `LARGE_PANEL` (1 000
individuals): below that no slot is taken, nothing waits and nothing is
refused; above it the old policy holds unchanged (1 slot for gemma, 1 for
ravanan, `RQTL_MAX_CONCURRENT` for rqtl, one large job per caller, 409
naming the run in the way).

The size is not known when the request arrives -- the panel is resolved
server-side -- so the decorator now makes a *light* claim and the handler
promotes it (`__gn4_claim.claim_slot_or_busy(individuals, rid)`) at the
point where it knows, then hands it to `Gate::run` around the call that
computes, so the slot still outlives a client disconnect. A light claim
registers nothing and its drop removes nothing: only a held claim may touch
the claimant map.

* src/compute_gate.rs: `LARGE_PANEL`/`is_large`, `Gate::claim_light`,
  `Claim::claim_slot_or_busy`/`is_held`, `busy_error` (the 409 policy, moved
  out of `claim_or_busy`), and `Gate::run` unchanged in spirit.
* gnauth-macros: binds the light claim; the body keeps one gate call.
* src/routes/{gemma,rqtl}.rs: promotion after the panel/sample count is
  known, and the compute call wrapped in `run`; error mapping moved inside
  the job so the future returns `ApiError`.
* src/rqtl.rs: `Sample` is `Clone` (the job owns its copy).
* tests: the gemma suite loses its lock entirely (the fixtures and BXD.4 are
  under the threshold); rqtl keeps its lock only for the machine's sake --
  concurrent `Rscript` scans would compete for cores and scratch, not for a
  slot. Four new gate tests pin the threshold, that a small job never takes
  the slot, that one caller cannot hold two large jobs, and that another
  caller waits rather than fails.
* README.md, docs/www/ai-instruction.md, docs/www/operator-checklist.md:
  what the threshold is and why it is 1 000.

Measured on :3001 after the change: two concurrent small-panel GEMMA
requests from one token both answer 200 (one was 409 before), while two
6 382-individual requests answer 502/409 -- the second gated with the usual
`compute_busy` naming the first.

277 tests pass; the same four as before do not.
Diffstat (limited to 'test/python/test_groups.py')
0 files changed, 0 insertions, 0 deletions