about summary refs log tree commit diff
path: root/gnauth-macros/src
AgeCommit message (Collapse)Author
11 daysapi: describe case attributes, and gate large computations onlyPjotr
Two things that came out of reading a real response rather than a test. **Say what a case attribute means.** `GET /groups/{group}/case-attributes` returned names and values, and the per-trait attribute endpoint returned values, so a caller had to guess that `SeqCvge` is a sequencing depth. MariaDB has had the answer all along in `CaseAttribute.Description`: SeqCvge Depth of DNA sequencing for 1 male of each strain in 2016 Status Live= Available at JAX, Cryo=Cryopreserved only, Extinct Epoch BXD family subgroups. Each number with common parents. ... Sex F= female, M= male Both endpoints now carry it -- a `descriptions` map on the group listing, a `description` field on the per-trait one -- and the covariate notes quote it, so `/compute/gemma` answers "Covariate SeqCvge (Depth of DNA sequencing for 1 male of each strain in 2016): categorical ..." instead of leaving the name to be interpreted. It matters most for covariates: `SeqCvge` is a property of the *genotype* data, its values include compound ones (`40+31L`, two sequencing runs), and the server codes those as 20-odd levels rather than as a depth -- a covariate fitted for the wrong reason, which the description is the first hint against. **Gate large computations, not every computation.** The gate was applied to every job, so a 93-strain test request could be refused because somebody else's 6 000-individual run held the slot -- which is how the test suite started serialising itself behind a mutex, a workaround for the wrong problem. What the gate is for is *large* work: a BXD panel of ~200 finishes in seconds and two at once cost nothing, while UM-HET3's 6 382 write 207 MB of kinship per chromosome. It now starts at `LARGE_PANEL` (1 000 individuals): below that no slot is taken, nothing waits and nothing is refused; above it the old policy holds unchanged (1 slot for gemma, 1 for ravanan, `RQTL_MAX_CONCURRENT` for rqtl, one large job per caller, 409 naming the run in the way). The size is not known when the request arrives -- the panel is resolved server-side -- so the decorator now makes a *light* claim and the handler promotes it (`__gn4_claim.claim_slot_or_busy(individuals, rid)`) at the point where it knows, then hands it to `Gate::run` around the call that computes, so the slot still outlives a client disconnect. A light claim registers nothing and its drop removes nothing: only a held claim may touch the claimant map. * src/compute_gate.rs: `LARGE_PANEL`/`is_large`, `Gate::claim_light`, `Claim::claim_slot_or_busy`/`is_held`, `busy_error` (the 409 policy, moved out of `claim_or_busy`), and `Gate::run` unchanged in spirit. * gnauth-macros: binds the light claim; the body keeps one gate call. * src/routes/{gemma,rqtl}.rs: promotion after the panel/sample count is known, and the compute call wrapped in `run`; error mapping moved inside the job so the future returns `ApiError`. * src/rqtl.rs: `Sample` is `Clone` (the job owns its copy). * tests: the gemma suite loses its lock entirely (the fixtures and BXD.4 are under the threshold); rqtl keeps its lock only for the machine's sake -- concurrent `Rscript` scans would compete for cores and scratch, not for a slot. Four new gate tests pin the threshold, that a small job never takes the slot, that one caller cannot hold two large jobs, and that another caller waits rather than fails. * README.md, docs/www/ai-instruction.md, docs/www/operator-checklist.md: what the threshold is and why it is 1 000. Measured on :3001 after the change: two concurrent small-panel GEMMA requests from one token both answer 200 (one was 409 before), while two 6 382-individual requests answer 502/409 -- the second gated with the usual `compute_busy` naming the first. 277 tests pass; the same four as before do not.
11 dayscompute: gate R/qtl at eight concurrent scans, one per callerPjotr
`POST /compute/rqtl` now takes up to `RQTL_MAX_CONCURRENT` slots (default 8, read when the gate is built) with the same one-job-per-caller rule as GEMMA: a second submission from the same token while its scan is running or queued is refused `409 compute_busy`; a submission that finds every slot taken waits for one to come free. Eight because an R/qtl scan is a single-threaded Rscript on a 96-core box; not more, because a scan on a 3 000-individual cross is minutes of CPU and the same machine serves the database. Unlike GEMMA's single slot, concurrent scans share no cache (R/qtl results are uncached), so the only cost is the CPU they take. Two flaws only appeared once there was more than one slot: - the "what is running" record was a single `Option<Active>`, so a refusal quoted whichever job had most recently taken a slot -- robw's 409 named JoM's run id. It is now a map keyed by subject, and `a_refusal_quotes_the_callers_own_run_id` pins it; - `Busy::describe` took the endpoint identifier, so the message read "your rqtl job"; the decorator takes a separate `label` ("R/qtl") from the status path. The test suite hit the rule itself: six compute tests in tests/rqtl.rs share one token, and cargo runs them on several threads, so three were refused instead of run. They now serialise on a module mutex, with a comment saying why -- the same lesson clients need, that parallelism means more tokens, not more requests from one. * src/compute_gate.rs (rqtl()): eight slots from `RQTL_MAX_CONCURRENT`; `Gate::new(permits)` was already parameterised. `running` becomes a map keyed by subject, since one record cannot name N jobs. Module doc talks about slots rather than a single slot. Tests: eight subjects run and the ninth waits, a caller still may not take two even with slots free, and a refusal quotes the caller's own run id. * src/config.rs: RQTL_MAX_CONCURRENT, default 8, fixed at first use. * gnauth-macros/src/lib.rs: `label = "…"`, so prose and status path are independent. * src/routes/rqtl.rs: the decorator, and `_token` becomes `token`, which it needs. * tests/rqtl.rs: compute tests serialised. * README.md (/compute/rqtl), docs/www/ai-instruction.md (4a), docs/www/operator-checklist.md: the numbers per endpoint, and that more parallelism means more tokens.
11 dayscompute: one gemma job at a time, per caller, via #[compute_gate]Pjotr
`POST /compute/gemma` now takes a single slot: one job runs at a time across all callers, a second submission from the same token is refused immediately with `409 compute_busy` (whatever it would have waited for, its own job is already in the way), and a submission from another token waits for the running job and then runs. Nothing else is gated -- ravanan and /compute/rqtl are untouched. The gate is one `tokio::sync::Semaphore(1)` (fair: waiters take the permit in arrival order) plus a `Mutex<HashSet<String>>` of subjects holding the slot or queued for it, in a per-process static. The claim is moved into the spawned job task, so a client disconnect -- which does not kill the child process -- cannot hand the slot to somebody else while the CPU is still busy. The endpoint carries none of that: `#[compute_gate(gemma, run_id = params.id.as_deref())]` expands to a claim and a run, and requires the handler to take `Ctx` and `AuthToken` (every compute endpoint is token-only, and the token's sub is the key), so an endpoint that is not token-gated cannot be decorated. The policy -- the 409's status, code, wording and next link -- stays in plain Rust (`Gate::claim_or_busy`), so the gate's own tests cover it; the attribute is decoration over two calls. * src/compute_gate.rs: new. Gate, Claim (released on drop, including a panic), Busy, claim/claim_or_busy/run, and four tests, two of them on a four-thread runtime (the earlier ones ran single-threaded and so never exercised the cross-thread hand-off; one now asserts at most one task inside the slot across eight tasks). * gnauth-macros/src/lib.rs: the `compute_gate` attribute, alongside the `gnauth` markers it shares `syn`/`quote` with; arguments documented (gate, endpoint, status, run_id) for the endpoints still to be gated. * src/routes/gemma.rs: the decorator replaces 57 lines of claim and spawn plumbing in compute_gemma. * docs/www/ai-instruction.md (4a, 5): one job at a time per caller, and that a `compute_busy` 409 means poll, not resubmit.
2026-10-13auth: add gnauth compile-time tags (secured/token_only/disabled)Pjotr
Adds a gnauth-macros proc-macro crate and a private-field Authorized capability type in src/auth.rs, so #[gnauth(secured)] is a real, compile-time-checked guarantee that a handler performs the gn-auth resource-authorisation round trip before sharing trait/probeset values, rather than just a comment. #[gnauth(token_only)] marks handlers that require a token but skip the round trip, and #[gnauth(disabled)] marks the two handlers that need no token at all. Also fixes groups.rs: group_trait_values, group_trait_info and group_traits previously performed no gn-auth round trip at all for published-trait data; they now resolve the group's PublishFreeze row and call the same auth::check used elsewhere, denying by default when no such row exists. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>