diff options
Diffstat (limited to 'docs/dev')
| -rw-r--r-- | docs/dev/SKILL.md | 14 |
1 files changed, 14 insertions, 0 deletions
diff --git a/docs/dev/SKILL.md b/docs/dev/SKILL.md index ce936d7..0ba66dd 100644 --- a/docs/dev/SKILL.md +++ b/docs/dev/SKILL.md @@ -243,6 +243,20 @@ Use `utoipa` + `utoipa-axum`, not a hand-maintained spec file: ## Testing +- **GEMMA mapping: ravanan is the default runner, `/compute/gemma` is the + reference.** Any change that touches mapping is verified by sending the same + request body to both endpoints -- `scripts/gemma-compare.py --base URL --body + FILE` prints both wall-clock times and exits non-zero if a row disagrees on + `rs`, `beta`, `se`, `l_mle` or `p_lrt`. The wrapper endpoint is only the + fallback when Slurm is unavailable, so never "fix" a mapping difference by + switching endpoints: the difference is the finding. +- Both are content-cached (ravanan's store; gemma-wrapper's `--cache-dir`), so a + repeat of an identical request is ~1-2 s against ~12-15 s cold -- cheap enough + to re-run for verification. Note the wrapper's cache only hits from + gemma-wrapper `5eb53fa` on (archives without the per-run `.log.txt` can never + satisfy its cache-hit check); check `gemma-wrapper --help | head -1` for the + version before concluding a cache is broken. + Two parallel suites exercising the same endpoints: - Rust integration tests (`tests/*.rs`) call the router in-process via |
