about summary refs log tree commit diff
path: root/tests/case_attributes.rs
AgeCommit message (Collapse)Author
10 daysapi: describe case attributes, and gate large computations onlyPjotr
Two things that came out of reading a real response rather than a test. **Say what a case attribute means.** `GET /groups/{group}/case-attributes` returned names and values, and the per-trait attribute endpoint returned values, so a caller had to guess that `SeqCvge` is a sequencing depth. MariaDB has had the answer all along in `CaseAttribute.Description`: SeqCvge Depth of DNA sequencing for 1 male of each strain in 2016 Status Live= Available at JAX, Cryo=Cryopreserved only, Extinct Epoch BXD family subgroups. Each number with common parents. ... Sex F= female, M= male Both endpoints now carry it -- a `descriptions` map on the group listing, a `description` field on the per-trait one -- and the covariate notes quote it, so `/compute/gemma` answers "Covariate SeqCvge (Depth of DNA sequencing for 1 male of each strain in 2016): categorical ..." instead of leaving the name to be interpreted. It matters most for covariates: `SeqCvge` is a property of the *genotype* data, its values include compound ones (`40+31L`, two sequencing runs), and the server codes those as 20-odd levels rather than as a depth -- a covariate fitted for the wrong reason, which the description is the first hint against. **Gate large computations, not every computation.** The gate was applied to every job, so a 93-strain test request could be refused because somebody else's 6 000-individual run held the slot -- which is how the test suite started serialising itself behind a mutex, a workaround for the wrong problem. What the gate is for is *large* work: a BXD panel of ~200 finishes in seconds and two at once cost nothing, while UM-HET3's 6 382 write 207 MB of kinship per chromosome. It now starts at `LARGE_PANEL` (1 000 individuals): below that no slot is taken, nothing waits and nothing is refused; above it the old policy holds unchanged (1 slot for gemma, 1 for ravanan, `RQTL_MAX_CONCURRENT` for rqtl, one large job per caller, 409 naming the run in the way). The size is not known when the request arrives -- the panel is resolved server-side -- so the decorator now makes a *light* claim and the handler promotes it (`__gn4_claim.claim_slot_or_busy(individuals, rid)`) at the point where it knows, then hands it to `Gate::run` around the call that computes, so the slot still outlives a client disconnect. A light claim registers nothing and its drop removes nothing: only a held claim may touch the claimant map. * src/compute_gate.rs: `LARGE_PANEL`/`is_large`, `Gate::claim_light`, `Claim::claim_slot_or_busy`/`is_held`, `busy_error` (the 409 policy, moved out of `claim_or_busy`), and `Gate::run` unchanged in spirit. * gnauth-macros: binds the light claim; the body keeps one gate call. * src/routes/{gemma,rqtl}.rs: promotion after the panel/sample count is known, and the compute call wrapped in `run`; error mapping moved inside the job so the future returns `ApiError`. * src/rqtl.rs: `Sample` is `Clone` (the job owns its copy). * tests: the gemma suite loses its lock entirely (the fixtures and BXD.4 are under the threshold); rqtl keeps its lock only for the machine's sake -- concurrent `Rscript` scans would compete for cores and scratch, not for a slot. Four new gate tests pin the threshold, that a small job never takes the slot, that one caller cannot hold two large jobs, and that another caller waits rather than fails. * README.md, docs/www/ai-instruction.md, docs/www/operator-checklist.md: what the threshold is and why it is 1 000. Measured on :3001 after the change: two concurrent small-panel GEMMA requests from one token both answer 200 (one was 409 before), while two 6 382-individual requests answer 502/409 -- the second gated with the usual `compute_busy` naming the first. 277 tests pass; the same four as before do not.
11 dayscase-attributes: one attribute for one trait, in that trait's sample orderPjotr
The trait-values endpoint stays about trait values; a covariate needs the study's metadata for the same samples, aligned the same way. So: GET /groups/{group}/traits/{trait_id}/case-attributes/{attribute} returns one attribute's values for exactly that trait's samples, **in the trait's own order**, which is what makes it usable without the caller re-sorting anything: /groups/BXD-Longevity/traits/10001/case-attributes/Sex -> 3281 values in AgeAtDeath's order, F x 3101, M x 180 -> order identical to /traits/10001/values (asserted in the tests) Alongside the values it reports `distinct` (each value with its count, commonest first) and `num_values`. That is the check that decides whether an attribute can serve as a covariate at all -- a constant one cannot -- and it saves a second request just to find out. Attributes are per group, so an unknown one is a 404 whose `next` is the group's attribute list: BXD has no Sex, BXD-Longevity does. * src/db.rs (get_trait_case_attribute, get_group_trait_samples): the group's attribute values aligned onto the trait's sample list, joining CaseAttribute/CaseAttributeXRefNew on (InbredSetId, CaseAttributeId) as the group endpoint does; empty text is reported as no value. * src/routes/groups.rs: the handler, token-gated, with the response shapes and the two `next` links (the trait's values, the group's attributes). * tests/case_attributes.rs: alignment against an independent /values response, `distinct` accounting, ordering, and the 404s for BXD (no Sex) and for a non-existent attribute. * README.md, docs/www/ai-instruction.md (3. Case attributes): the endpoint, and "ask per trait rather than sorting the group's attributes yourself".