about summary refs log tree commit diff
path: root/test/python/test_dev_token.py
diff options
context:
space:
mode:
authorPjotr Prins <pjotr.public01@thebird.nl>2026-11-15 15:22:01 +0000
committerPjotr Prins <pjotr.public01@thebird.nl>2026-11-15 15:22:01 +0000
commit9c3cd6f2300bb290ddd05c5a46c5b813b22c0ce1 (patch)
treef218f79b3c7798313ab814a40ce3244f18d26b2b /test/python/test_dev_token.py
parentedaef7fd56e4ce7acd833e929462293478025a34 (diff)
downloadgenenetwork4-9c3cd6f2300bb290ddd05c5a46c5b813b22c0ce1.tar.gz
monitor: run the cached engines cold, not just warm
A warm run answers from caches the previous binary filled, so it can pass
while the cold path -- what a new caller's first request actually does -- is
broken. `--cold` sends each cached engine a payload with one sample value
nudged by a thousandth, then repeats the original: every engine here keys its
cache on content (gemma-wrapper's hash, ravanan's interned inputs,
rqtl/qtl2/bulklmm's own), so the first is a miss and the second a hit. Both
runs are judged; the line reports "cold X / warm Y".

Measured on production with the 30-sample fixture:

  compute-ravanan   102.0 s cold /   3.7 s warm
  compute-gemma       3.1 s cold /   2.0 s warm
  compute-rqtl        7.5 s cold /   7.3 s warm
  compute-qtl2        3.3 s cold /   3.3 s warm
  compute-fitqtl      6.7 s cold /   6.9 s warm
  compute-bulklmm    19.6 s cold /  19.0 s warm

ravanan is the one where the cache is the difference between minutes and
seconds; the rest are compute-bound at this size, and the honest thing is to
print the pair rather than assert a speedup. A check whose payload cannot be
perturbed skips with that reason instead of reporting two warm runs as cold
and warm.

The operator checklist gains the cold run as part of deploying, with both
costs named: ravanan's cold run takes minutes, and every cold run adds cache
entries.
Diffstat (limited to 'test/python/test_dev_token.py')
0 files changed, 0 insertions, 0 deletions