| Age | Commit message (Collapse) | Author |
|
Two things that came out of reading a real response rather than a test.
**Say what a case attribute means.** `GET /groups/{group}/case-attributes`
returned names and values, and the per-trait attribute endpoint returned
values, so a caller had to guess that `SeqCvge` is a sequencing depth.
MariaDB has had the answer all along in `CaseAttribute.Description`:
SeqCvge Depth of DNA sequencing for 1 male of each strain in 2016
Status Live= Available at JAX, Cryo=Cryopreserved only, Extinct
Epoch BXD family subgroups. Each number with common parents. ...
Sex F= female, M= male
Both endpoints now carry it -- a `descriptions` map on the group listing, a
`description` field on the per-trait one -- and the covariate notes quote it,
so `/compute/gemma` answers "Covariate SeqCvge (Depth of DNA sequencing for
1 male of each strain in 2016): categorical ..." instead of leaving the name
to be interpreted. It matters most for covariates: `SeqCvge` is a property
of the *genotype* data, its values include compound ones (`40+31L`, two
sequencing runs), and the server codes those as 20-odd levels rather than as
a depth -- a covariate fitted for the wrong reason, which the description is
the first hint against.
**Gate large computations, not every computation.** The gate was applied to
every job, so a 93-strain test request could be refused because somebody
else's 6 000-individual run held the slot -- which is how the test suite
started serialising itself behind a mutex, a workaround for the wrong
problem. What the gate is for is *large* work: a BXD panel of ~200 finishes
in seconds and two at once cost nothing, while UM-HET3's 6 382 write 207 MB
of kinship per chromosome. It now starts at `LARGE_PANEL` (1 000
individuals): below that no slot is taken, nothing waits and nothing is
refused; above it the old policy holds unchanged (1 slot for gemma, 1 for
ravanan, `RQTL_MAX_CONCURRENT` for rqtl, one large job per caller, 409
naming the run in the way).
The size is not known when the request arrives -- the panel is resolved
server-side -- so the decorator now makes a *light* claim and the handler
promotes it (`__gn4_claim.claim_slot_or_busy(individuals, rid)`) at the
point where it knows, then hands it to `Gate::run` around the call that
computes, so the slot still outlives a client disconnect. A light claim
registers nothing and its drop removes nothing: only a held claim may touch
the claimant map.
* src/compute_gate.rs: `LARGE_PANEL`/`is_large`, `Gate::claim_light`,
`Claim::claim_slot_or_busy`/`is_held`, `busy_error` (the 409 policy, moved
out of `claim_or_busy`), and `Gate::run` unchanged in spirit.
* gnauth-macros: binds the light claim; the body keeps one gate call.
* src/routes/{gemma,rqtl}.rs: promotion after the panel/sample count is
known, and the compute call wrapped in `run`; error mapping moved inside
the job so the future returns `ApiError`.
* src/rqtl.rs: `Sample` is `Clone` (the job owns its copy).
* tests: the gemma suite loses its lock entirely (the fixtures and BXD.4 are
under the threshold); rqtl keeps its lock only for the machine's sake --
concurrent `Rscript` scans would compete for cores and scratch, not for a
slot. Four new gate tests pin the threshold, that a small job never takes
the slot, that one caller cannot hold two large jobs, and that another
caller waits rather than fails.
* README.md, docs/www/ai-instruction.md, docs/www/operator-checklist.md:
what the threshold is and why it is 1 000.
Measured on :3001 after the change: two concurrent small-panel GEMMA
requests from one token both answer 200 (one was 409 before), while two
6 382-individual requests answer 502/409 -- the second gated with the usual
`compute_busy` naming the first.
277 tests pass; the same four as before do not.
|
|
Pure formatting sweep (import ordering, line wrapping); the tree
had accumulated rustfmt drift. Separated from any semantic change
so it stays reviewable and revertable on its own.
|
|
Breaking: every success body is now {"data", "meta": {"state",
"took_ms", "request_id"}, "notes", "next"}; clients read `data`. Only
`/` (plain text) and `/openapi.json` are unwrapped.
* src/response.rs (Envelope, Meta, State, Ctx, RequestInfo): handlers
take Ctx first and build the response with ctx.envelope(data), so
timing and the request id are never filled in by hand. `state` is
`complete` or `partial`; notes are server-authored only; `next` are
concrete relative hrefs (path_segment percent-encodes names).
* src/routes/*.rs, src/lib.rs (version): every handler returns
Envelope<..> with next links; /search is `partial` when the locus
lookup fails or a sorted 1000-hit window may be truncated, and
/publications/{pubmed_id}/traits when datasets were left out for
authorization.
* src/error.rs (ApiError): add `took_ms`, an RFC 7807 extension member.
* src/auth.rs (AuthToken): the 401 now carries request_id and took_ms
(it never had a request_id); Ctx stores RequestInfo in the request
extensions for it.
* tests: common::data / envelope_data parse every success body through
the envelope; tests/envelope.rs and test/python/test_envelope.py check
the shape, errors, and that every `next` link resolves;
tests/envelope_partial.rs checks the `partial` search without a
database; tests/openapi.rs fails if a JSON success response is not an
Envelope_* schema.
* README.md, docs/SKILL.md, docs/gn-auth.md, RELEASE_NOTES.md: document
the envelope and took_ms on errors.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Per-strain metadata beyond genotype/phenotype data (age, sex,
treatment, etc.), backed by MariaDB CaseAttribute/CaseAttributeXRefNew.
Combines GeneNetwork3's separate /case-attribute/{id}/names and
/case-attribute/{id}/values (gn3.api.case_attributes) into one
response -- neither list is large enough on its own to need splitting.
This is the only one of GeneNetwork2/3/gn-guile with the concept at
all: GeneNetwork2 delegates to GeneNetwork3 rather than storing case
attributes itself, and gn-guile doesn't have it.
db::get_group_case_attributes returns {names: [...], values: [{strain,
case_attributes: {...}}]}; values is built by folding adjacent
(strain, attribute, value) rows (ordered by strain name) rather than
grouping afterwards, since SQL already guarantees the adjacency.
Found and fixed a real bug while verifying this live: GeneNetwork3's
own query joins CaseAttribute to CaseAttributeXRefNew on
CaseAttributeId alone, but CaseAttributeId is not unique across groups
-- it's a shared vocabulary (id 37 means "Epoch" for dozens of
unrelated groups simultaneously); the actual key is
(InbredSetId, CaseAttributeId). Confirmed against this database:
without also joining on InbredSetId, BXD's result included a spurious
"Strain": "Live" attribute belonging to a different, unrelated group.
Fixed on genenetwork4's side by adding the missing join condition.
A group with no case attributes defined returns 200 with both lists
empty, not a 404 (the group itself exists) -- verified against LXS,
a real group with zero CaseAttribute rows.
tests/group_case_attributes.rs and its Python mirror cover BXD's 4
real attributes (asserting each strain has exactly those 4, not more,
to catch the join bug above), the empty-group case, missing token, and
unknown group.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
|