about summary refs log tree commit diff
path: root/docs
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-11-08 14:15:13 +0000
committerPjotr <wrk@bosa.lan>2026-11-08 14:15:24 +0000
commit51739ea82f3543ac6e28c8392d6d62957c021b3a (patch)
tree7663b917022f709b343933ca036e6f420a3a460a /docs
parent1f76ef2ef19ae36882236be7c016f0ab98d31f11 (diff)
downloadgenenetwork4-51739ea82f3543ac6e28c8392d6d62957c021b3a.tar.gz
compute: gate R/qtl at eight concurrent scans, one per caller
`POST /compute/rqtl` now takes up to `RQTL_MAX_CONCURRENT` slots (default
8, read when the gate is built) with the same one-job-per-caller rule as
GEMMA: a second submission from the same token while its scan is running
or queued is refused `409 compute_busy`; a submission that finds every
slot taken waits for one to come free. Eight because an R/qtl scan is a
single-threaded Rscript on a 96-core box; not more, because a scan on a
3 000-individual cross is minutes of CPU and the same machine serves the
database. Unlike GEMMA's single slot, concurrent scans share no cache
(R/qtl results are uncached), so the only cost is the CPU they take.

Two flaws only appeared once there was more than one slot:

- the "what is running" record was a single `Option<Active>`, so a
  refusal quoted whichever job had most recently taken a slot -- robw's
  409 named JoM's run id. It is now a map keyed by subject, and
  `a_refusal_quotes_the_callers_own_run_id` pins it;
- `Busy::describe` took the endpoint identifier, so the message read
  "your rqtl job"; the decorator takes a separate `label` ("R/qtl") from
  the status path.

The test suite hit the rule itself: six compute tests in tests/rqtl.rs
share one token, and cargo runs them on several threads, so three were
refused instead of run. They now serialise on a module mutex, with a
comment saying why -- the same lesson clients need, that parallelism
means more tokens, not more requests from one.

* src/compute_gate.rs (rqtl()): eight slots from `RQTL_MAX_CONCURRENT`;
`Gate::new(permits)` was already parameterised. `running` becomes a map
keyed by subject, since one record cannot name N jobs. Module doc talks
about slots rather than a single slot. Tests: eight subjects run and the
ninth waits, a caller still may not take two even with slots free, and a
refusal quotes the caller's own run id.
* src/config.rs: RQTL_MAX_CONCURRENT, default 8, fixed at first use.
* gnauth-macros/src/lib.rs: `label = "…"`, so prose and status path are
independent.
* src/routes/rqtl.rs: the decorator, and `_token` becomes `token`, which
it needs.
* tests/rqtl.rs: compute tests serialised.
* README.md (/compute/rqtl), docs/www/ai-instruction.md (4a),
docs/www/operator-checklist.md: the numbers per endpoint, and that more
parallelism means more tokens.
Diffstat (limited to 'docs')
-rw-r--r--docs/www/ai-instruction.md17
-rw-r--r--docs/www/operator-checklist.md9
2 files changed, 18 insertions, 8 deletions
diff --git a/docs/www/ai-instruction.md b/docs/www/ai-instruction.md
index 4e8498b..bc185e2 100644
--- a/docs/www/ai-instruction.md
+++ b/docs/www/ai-instruction.md
@@ -447,14 +447,15 @@ against a genome-wide threshold. (gemma-wrapper itself can shuffle
 phenotypes with `--permutate n`; this API does not pass it — see
 docs/genotypes.md.)
 
-**One job at a time, per caller.** `/compute/gemma` runs one job at a
-time across all callers: a second submission from *your* token while your
-job is still running — or still queued — is refused at once with
-`409 compute_busy`, naming the run and linking its status endpoint; a
-submission from *another* token waits for the running job to finish and
-then runs. So a 409 here is not a retry-able failure: poll
-`/compute/gemma/status/{id}`, and only then submit the next job. Map one
-trait, read it, submit the next.
+**A few jobs at a time, one per caller.** Each compute endpoint has a
+fixed number of slots — **1** for `/compute/gemma`, **8** for
+`/compute/rqtl` — and one job per caller within them. A second submission
+from *your* token while your job is still running or queued is refused at
+once with `409 compute_busy`, naming the run and linking its status
+endpoint; a submission that finds every slot taken waits for one to come
+free and then runs. So a 409 here is not a retry-able failure: poll the
+status endpoint it names, and only then submit the next job. (If you want
+more parallelism, that is more *tokens*, not more requests from one.)
 
 Results are **cached** by input, so running the same request twice costs
 little the second time (a cached GEMMA run on the BXD fixtures comes back
diff --git a/docs/www/operator-checklist.md b/docs/www/operator-checklist.md
index 14c5c82..6f49501 100644
--- a/docs/www/operator-checklist.md
+++ b/docs/www/operator-checklist.md
@@ -205,6 +205,15 @@ restarts — that is what happened on 2026-11-07. `GET /version` now carries
 `"database": "reachable" | "unreachable"` (cached ten seconds), so an
 up-but-useless service is visible without calling a data endpoint.
 
+## Compute concurrency
+
+`POST /compute/gemma` takes one job at a time and `POST /compute/rqtl`
+`RQTL_MAX_CONCURRENT` (default 8), one job per caller each; both are
+in-process gates (`src/compute_gate.rs`), so they are per server process
+and reset on restart -- a job's child process, however, outlives the
+server. `/compute/gemma/ravanan` is not gated yet. Idle waits are bounded
+only by nginx's `proxy_read_timeout`.
+
 ## Plotting
 
 `POST /plot/rqtl` and `/plot/gemma` take a **posted mapping response** --