diff options
| author | Pjotr <wrk@bosa.lan> | 2026-11-08 16:32:59 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-11-08 16:32:59 +0000 |
| commit | ab5fef05237309667dd2ca7f835a5fbf9c71a12a (patch) | |
| tree | 94267f7fa6c5f27fe6d5093d1280c48170ffe164 /test/python/test_openapi.py | |
| parent | e934e755f14648fff6ebac07e9cf9e5b37cb3fe4 (diff) | |
| download | genenetwork4-ab5fef05237309667dd2ca7f835a5fbf9c71a12a.tar.gz | |
compute: gate GEMMA from 50 individuals, R/qtl from 1 000
One threshold for both engines was the wrong shape. For R/qtl the interesting line is where a `scanone` stops being seconds (a few hundred individuals is seconds of one core), so 1 000 stays. For GEMMA it is much earlier, and not because of the runtime: - every GEMMA run goes through **one shared cache directory** (`GEMMA_CACHE_DIR`), where `gemma-wrapper` extracts per-chromosome kinship archives of hundreds of megabytes by content hash. Two runs at once are two writers over the same archives, and a half-extracted kinship another run reads surfaces as a GEMMA failure; - a real BXD WGS run is ~19 s of every core it can get; UM-HET3's 6 382 individuals write 207 MB of kinship per chromosome. So `GEMMA_LARGE_PANEL` is 50 -- the smallest real panel, which means every mapping request is gated in practice, and only a handful of individuals (a four-way cross's parents, a probe of the pipeline) runs free. `ravanan` uses the same number, since it writes into the same cache. The threshold moved onto the `Gate` (`large_panel()`, `is_large()`), so each endpoint carries its own; the policy, the slots and the 409 are unchanged. Tests, asked and answered: at 50 the three gemma tests that do real work (fixtures 198 individuals, BXD.4 with `Status` 93, ravanan 198) are gated and share one token, so on a cold cache two of them at once would collide into `409 compute_busy` -- a gate doing its job, failing a test that only wanted to map something. They now take a lock, and only they do; the validation tests stay parallel, and the rqtl suite is untouched (its gate still starts at 1 000). Verified: with `GEMMA_CACHE_DIR` pointed at an empty directory, `cargo test --test gemma` is green twice in a row, 18 s per run, i.e. really recomputing. * src/compute_gate.rs: `RQTL_LARGE_PANEL` (1 000), `GEMMA_LARGE_PANEL` (50), `Gate::large_panel`/`is_large`, `Gate::new(permits, large_panel)`; the test that pinned one number now pins both and that each endpoint is wired to its own. * tests/gemma.rs: the three real-job tests take the lock, and say why. * README.md, docs/www/ai-instruction.md, docs/www/operator-checklist.md: the two thresholds and the cache-directory reason for the lower one. 277 tests pass; the same four as before do not.
Diffstat (limited to 'test/python/test_openapi.py')
0 files changed, 0 insertions, 0 deletions
