about summary refs log tree commit diff
path: root/docs
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-11-07 16:28:14 +0000
committerPjotr <wrk@bosa.lan>2026-11-07 16:28:14 +0000
commit14dc58f070c2950c13ae822c4a371cf3d303b16a (patch)
tree60799fc8b97344f496205dd3fea59edff0d307d1 /docs
parentb5f5209642f422e87a4d0041471c5654491a8d0c (diff)
downloadgenenetwork4-14dc58f070c2950c13ae822c4a371cf3d303b16a.tar.gz
docs: name the real genotype files, panels and timings everywhere
docs/genotypes.md explained the convention; the endpoint docs still left
a caller guessing which file to pass and how big the panel is. So:

* README.md (/compute/gemma): GEMMA wants the BIMBAM pair
`genotype/bimbam/<file>_geno.txt` + `<file>_snps.txt`, not the `.geno`
file; `trait_values` is one entry per individual in that file's column
order; BXD's current panel stated (235 x 21056), with the 17.1 s
measurement against the 19 s the 7 320-marker fixtures take.
* README.md (/compute/rqtl): the file is `genotype/<file>.geno`
(BXD.8: 235 x 21057; HET3-ITP.12: 6447 x 894), `MappingMethodId`
decides the engine, and the two measured runs with the UM-HET3 peak.
* README.md (/compute/gemma/ravanan): same input convention as gemma.
* docs/genotypes.md: how each measurement was made, step by step, so
they can be repeated; and that step 3 (expanding values onto the panel)
is the API's job in the next iteration.
* docs/www/ai-instruction.md (4a): a table of the BXD files per engine
with panel sizes, the UM-HET3 file, "ask the operator for any other
panel" with where GN2 records which one is current, and the measured
times. Values must be one per individual, in file order -- position, not
name -- and the API does not re-index yet, so a mapping is prepared for
the caller rather than guessed.
* docs/www/operator-checklist.md: MappingMethodId per group, and the
three measurements.
Diffstat (limited to 'docs')
-rw-r--r--docs/genotypes.md19
-rw-r--r--docs/www/ai-instruction.md39
-rw-r--r--docs/www/operator-checklist.md12
3 files changed, 61 insertions, 9 deletions
diff --git a/docs/genotypes.md b/docs/genotypes.md
index fed2932..e2d91da 100644
--- a/docs/genotypes.md
+++ b/docs/genotypes.md
@@ -99,6 +99,25 @@ and were posted to the API.
 | BXD, `BXD.8.geno` | R/qtl scanone | 10001 | 235 | 34 | 7.5 s | 21 056 markers |
 | HET3-ITP, `HET3-ITP.12.geno` (Apr 2023) | R/qtl scanone | 10000 lifespan | 6 447 | 6 438 | 83.1 s | 893 markers, peak LOD 7.95 at `1_24042124` (chr 1, 24.04 Mb) |
 
+## Reproducing the measurements
+
+The pattern for all three, and for any mapping this deployment runs:
+
+1. take the sample order from the genotype file's header (first non-`#`/
+   `@` line, column 5 on if column 4 is `Mb`, else column 4 on);
+2. fetch `/groups/{group}/traits/{id}/values` and index those values by
+   sample name;
+3. build one entry per panel member, in header order, `null` where the
+   trait has none;
+4. POST that to `/compute/rqtl` (`samples: [{name, value}]` with
+   `genotype_file: .../<file>.geno`) or `/compute/gemma`
+   (`trait_values: [...]` with the `bimbam/<file>_geno.txt` and
+   `<file>_snps.txt` pair).
+
+Step 3 is the API's job in the next iteration; until then, a request
+built without it is caught rather than trusted -- R/qtl refuses a count
+mismatch, and GEMMA fails on one.
+
 ## What is still messy
 
 `HET3-ITP` has eleven `.geno` versions on disk (`.2` … `.12`) and **no
diff --git a/docs/www/ai-instruction.md b/docs/www/ai-instruction.md
index 0c8bfb4..e6bdb8c 100644
--- a/docs/www/ai-instruction.md
+++ b/docs/www/ai-instruction.md
@@ -262,18 +262,39 @@ itself, and the choice of genotypes, is the server's.
 the server's own filesystem. A valid token is the only requirement: they
 are not dataset-checked, so nothing about §4 applies — but the paths you
 name have to be paths the server can read, which means ones the
-deployment already holds. You cannot upload a genotype file, and you
-should not go looking for one: ask the operator which paths the
-deployment uses for the panel you are working on.
+deployment already holds. You cannot upload a genotype file.
+
+Which file, and which panel version, is a deployment convention you
+cannot guess; the current BXD panel is
+
+| engine | file | panel |
+|---|---|---|
+| R/qtl | `genotype/BXD.8.geno` (WGS-based, Mar 2022) | 235 individuals × 21 057 markers |
+| GEMMA | `genotype/bimbam/BXD.8_geno.txt` + `BXD.8_snps.txt` | 235 individuals × 21 056 markers |
+
+under `/home/wrk/tmp/ai-root/factory/gn4-data/genotype-files/`, alongside
+`genotype/HET3-ITP.12.geno` (6 447 individuals × 894 markers; UM-HET3 is
+a 4-way cross, so R/qtl is the engine there, not GEMMA). Ask the operator
+for the path and version for any *other* panel: which file is current is
+recorded in GeneNetwork2's per-group `genotype/<Group>.json`, whose
+`genofile` list is newest-first, and `InbredSet.MappingMethodId` says
+which engines a group offers at all.
 
 The values and the genotype file have to describe the **same
-individuals, in the same order, one for one**. Both engines match them by
+individuals, in the same order, one for one**: one entry per panel
+member, `null` where the trait has no value. Both engines match by
 position, not by name. Get that wrong and the computation is meaningless;
-GEMMA refuses outright, and R/qtl now refuses too rather than returning
-plausible-looking numbers for the wrong strains. So: use the values from
-the values endpoint and the genotype file the deployment pairs with that
-panel — never a genotype file from a different panel, and never a
-hand-built trait vector.
+GEMMA refuses outright, and R/qtl refuses a count mismatch rather than
+returning plausible-looking numbers for the wrong strains.
+
+The API does not re-index a trait's values onto the panel yet, so a
+mapping request is something the operator prepares: the panel's sample
+order lives in the genotype file's header, not in this API. Ask for the
+mapping (or for the panel order) rather than guessing — and never use a
+genotype file from a different panel, or a hand-built trait vector. For
+reference, the deployment maps BXD (235 individuals, 34 with values) in
+about 17 s with GEMMA and 8 s with R/qtl, and UM-HET3 lifespan (6 447
+individuals, 893 markers) in about 83 s with R/qtl.
 
 Results are **cached** by input, so running the same request twice costs
 little the second time (a cached GEMMA run on the BXD fixtures comes back
diff --git a/docs/www/operator-checklist.md b/docs/www/operator-checklist.md
index 333ed68..f21337f 100644
--- a/docs/www/operator-checklist.md
+++ b/docs/www/operator-checklist.md
@@ -147,6 +147,18 @@ re-indexing yet, so whoever prepares the request has to. It is the first
 thing to fix: pass the values a caller has, let the server align them
 from the `.geno` header.
 
+Which engines a group offers is `InbredSet.MappingMethodId` (1 → GEMMA,
+QTLReaper, R/qtl; 2 → GEMMA; 3 → R/qtl; 4 → GEMMA, PLINK): BXD = 1,
+HET3-ITP = 3. So UM-HET3 is R/qtl only, correct for a 4-way cross even
+though `bimbam/HET3-ITP.*` exists.
+
+Measured on this box with the newest genotypes: BXD (`BXD.8`, 235
+individuals, trait 10001 with 34 values) 17.1 s / 21 055 marker rows on
+GEMMA and 7.5 s / 21 056 markers on R/qtl; UM-HET3 lifespan over
+`HET3-ITP.12` (6 447 individuals, 6 438 with values) 83.1 s / 893
+markers, peak LOD 7.95 at chr 1, 24.04 Mb. Details and the misgivings
+about HET3-ITP's versioning are in `docs/genotypes.md`.
+
 `/compute/gemma/ravanan` additionally needs `ravanan` and `ccwl` on
 `PATH`, and with `RAVANAN_BATCH_SYSTEM=slurm-api` a reachable slurmrestd
 (see `docs/ravanan-slurm.md`).