about summary refs log tree commit diff
path: root/tests/test_data
AgeCommit message (Collapse)Author
8 daysrqtl: map the cM-only genotype filesPjotr
44 of this deployment's `.geno` files carry `cM` and no `Mb` column, and none of them could be scanned. GeneNetwork3's wrapper (copied verbatim here) assumes the position columns are `Chr, Locus, cM, Mb` and then starts the samples at a hardcoded column 5. On a cM-only file that is one column too far, so the genotype block loses its first individual and the whole scan dies in `do.call(rbind, ...)` with "number of columns of result is not a multiple of vector length" -- a message that says nothing about the cause. The wrapper now picks the map column from whichever of the requested scale and the file's own columns exists (falling back, and saying so), and starts the samples right after the position columns; the sex default and the sample-count guard use the same count. `scripts/qtl_fitqtl.R` had the same assumption and got the same fix. This is the second deliberate deviation from GeneNetwork3's script, after the sample-count guard, and both are commented as such at the code. * scripts/rqtl_wrapper.R, scripts/qtl_fitqtl.R: the position column and the first sample column are computed, not assumed. * tests/test_data/rqtl_sample1_cm.geno: the existing fixture with `Mb` dropped -- same panel, cM only. * tests/rqtl.rs: `a_cm_only_genotype_file_maps` scans it and asserts all six markers come back (the failure mode was five-vs-six columns, not an error message that pointed at the genotype file). * src/rqtl.rs, src/routes/rqtl.rs: the docs said the script never forwards `--scale`; it does, and now falls back, so the request field says what `pos` is in. Verified through the API: DGRP (cM-only) now maps (3 markers), and AXB (499 markers) and BXD.4 (3 811) are unchanged. 307 tests pass; the same four as before do not.
2026-10-12rqtl: add POST /compute/rqtl, running GeneNetwork3's own rqtl_wrapper.RPjotr
Same request/response shape as /compute/gemma: synchronous (no job queue), ?id= + GET /compute/rqtl/status/{run_id} progress-streaming. Extracted that streaming infrastructure (run_streaming, create_run_log/ RunLogError, status/StatusError, RunStatus/RunStatusParams, is_valid_run_id, COMPLETION_MARKER) out of src/gemma.rs into a new src/streaming.rs so both compute endpoints share it instead of duplicating it; src/routes/gemma.rs updated to the shared names, confirmed its own 8 tests still pass unchanged. scripts/rqtl_wrapper.R is GeneNetwork3's script, copied here verbatim and run via Rscript -- same reasoning as reusing gemma-wrapper rather than reimplementing GEMMA. Scope: a single scan (scanone) plus, if nperm > 0, its permutation thresholds (67th/95th percentile, via a Rust port of numpy.percentile's default linear interpolation) -- the common case. Not implemented: covariates, interval mapping, permutation strata, pair-scan (scantwo) -- scantwo in particular needs cross-referencing the genotype file for nearest-marker lookups and a second (MAP_) output file, a materially bigger port than this covers. One fix relative to GeneNetwork3, found by reading the code (no test covers the real caller's keys): generate_rqtl_cmd's content hash only ever looks at kwargs keyed "g"/"p", but gn3/api/rqtl.py's real caller uses keys "geno"/"pheno" -- they never match, so the genotype/ phenotype file contents never actually enter the cache-key hash in production, only the other options. Two different phenotypes mapped with the same other options would get the same cached output filename and silently reuse each other's result. Fixed here: both files' contents are always hashed (std DefaultHasher -- a cache key, not a security boundary, so no need for GeneNetwork3's md5). guix.scm: r, r-qtl, r-optparse, r-stringi and r-stringr added as propagated-inputs (same category as gemma-wrapper, and same reasoning: invoked as a subprocess, not linked). All stock Guix -- confirmed the existing -L (needed only for gemma-wrapper, from guix-bioinformatics) is not needed for these; the dev shell has Rscript/R-qtl on PATH from guix.scm's own propagated-inputs alone. Test fixture: a small, hand-written, valid BXD riset genotype/ phenotype pair (tests/test_data/rqtl_sample1.*), not GeneNetwork3's own tests/unit/db/.../genotype_sample1.geno -- that fixture has a genotype code outside its own header's declared set and duplicate marker names across chromosomes, harmless for the unrelated parsing unit it's written for there, but both would break a real R/qtl run. Verified end to end, not just unit tested: tests/rqtl.rs runs the real Rscript/R-qtl against that fixture (a real scanone scan, real permutation thresholds with suggestive <= significant, the run-id- collision 409 pre-seeded with no Rscript needed, and the completion marker landing in a real log after the run finishes). cargo test passes in full. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-11gemma: add POST /compute/gemma, backed by the real gemma-wrapper/GEMMAPjotr
Maps a QTL from a genotype file, a SNP annotation file and trait values, by running gemma-wrapper's K-matrix step then its LMM association step, both synchronously (axum/tokio awaits the subprocess directly, no job queue needed) and returning the parsed .assoc.txt rows in the standard envelope. Both steps use a bare --loco (gemma-wrapper derives the chromosome list from the SNP annotation file). This was arrived at empirically, not from GeneNetwork3's or GeneNetwork2's own gemma code: run for real against the BXD fixtures, - GeneNetwork3's gn3/computations/gemma.py composes a GWA command that always keeps "-gk" alongside "-lmm"; real GEMMA rejects that combination outright ("only one of -gk ... -lmm ... options is allowed"), so none of its gwa-compute/k-gwa-compute endpoints can produce output. Unsurprising: tests/integration/test_gemma.py is entirely @unittest.skip, so this was never exercised. - GeneNetwork2's wqflask/marker_regression/gemma_mapping.py correctly omits "-gk" from the GWA step, but its non-LOCO branch fails too, the same way my own first (non-LOCO) attempt did: gemma-wrapper only ever archives a non-LOCO kinship matrix into a .tar.xz, with no loose file at the path its own non-LOCO GWA code assumes ("Enforce failed for file_kin ... open file"). Only --loco, which gemma-wrapper's own USAGE banner documents and which does unpack that archive, actually works. A --loco run's .assoc.txt files (one per chromosome) are themselves only ever inside gemma-wrapper's own output archive, never loose, so gemma.rs shells out to `tar` (one of gemma-wrapper's own propagated dependencies) to extract them before concatenating all chromosomes' rows into the response. guix.scm now depends on (gn packages gemma) from guix-bioinformatics for gemma-wrapper (propagated-input: it's invoked as a subprocess, not linked, and this makes it land in a consuming profile too, same as GeneNetwork3's and GeneNetwork2's own packages do it). That dependency needed guix-bioinformatics' own gn/packages/gemma.scm fixed first (see that repo's fix/gemma-which-shadowing branch) -- gemma-gn2 would not build at all before that fix. Verified end to end: `cargo test` passes in full, including a real (non-mocked, non-skipped) run of gemma-wrapper/GEMMA against the BXD fixtures copied from genenetwork3/tests/test_data, producing exactly 7320 association rows (one per marker in BXD_snps.txt) in ~19s. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>