From c822ba22ce00c1615bf47fc8a13f28590aba424a Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sat, 14 Nov 2026 14:03:14 +0000 Subject: skills: ravanan is the default GEMMA runner, gemma-wrapper is the reference docs/www/SKILL.md: the workflow example now calls /compute/gemma/ravanan, and a new "Which GEMMA endpoint" section says when not to: Slurm down or submission failing, the user asks, a field ravanan refuses (covariates, normalization), or verifying results -- where the wrapper endpoint is the reference and scripts/gemma-compare.py is the check. docs/dev/SKILL.md: the same rule from the developer side, in Testing -- verify mapping changes by sending one body to both endpoints and never 'fix' a difference by switching endpoints, plus the reminder that the wrapper's cache only hits from 5eb53fa on, so a stale gemma-wrapper on PATH looks like a broken cache. --- docs/dev/SKILL.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'docs/dev') diff --git a/docs/dev/SKILL.md b/docs/dev/SKILL.md index ce936d7..0ba66dd 100644 --- a/docs/dev/SKILL.md +++ b/docs/dev/SKILL.md @@ -243,6 +243,20 @@ Use `utoipa` + `utoipa-axum`, not a hand-maintained spec file: ## Testing +- **GEMMA mapping: ravanan is the default runner, `/compute/gemma` is the + reference.** Any change that touches mapping is verified by sending the same + request body to both endpoints -- `scripts/gemma-compare.py --base URL --body + FILE` prints both wall-clock times and exits non-zero if a row disagrees on + `rs`, `beta`, `se`, `l_mle` or `p_lrt`. The wrapper endpoint is only the + fallback when Slurm is unavailable, so never "fix" a mapping difference by + switching endpoints: the difference is the finding. +- Both are content-cached (ravanan's store; gemma-wrapper's `--cache-dir`), so a + repeat of an identical request is ~1-2 s against ~12-15 s cold -- cheap enough + to re-run for verification. Note the wrapper's cache only hits from + gemma-wrapper `5eb53fa` on (archives without the per-run `.log.txt` can never + satisfy its cache-hit check); check `gemma-wrapper --help | head -1` for the + version before concluding a cache is broken. + Two parallel suites exercising the same endpoints: - Rust integration tests (`tests/*.rs`) call the router in-process via -- cgit 1.4.1