about summary refs log tree commit diff
path: root/tests/bulklmm.rs
AgeCommit message (Collapse)Author
10 daysbulklmm: report Julia's error line, not its last linePjotr
A failing scan came back as BulkLMM failed: in expression starting at scripts/bulklmm_scan.jl:26 which names neither the package nor the reason. Julia had said it plainly, two lines up: ERROR: LoadError: ArgumentError: Package CSV not found in current path. - Run `import Pkg; Pkg.add("CSV")` to install the CSV package. The complaint finder looked for a line starting "Error" -- Julia writes "ERROR:" -- and fell back to the last non-empty line, which is the line number. That is the same defect as gemma-wrapper's Ruby backtrace, in our own error handling: the wrapped tool said why, and we threw it away. julia_complaint takes the first ERROR:/LoadError line plus Julia's hint line when there is one, with the last line kept only as a fallback for a driver that fails silently. Three unit tests cover the three shapes. The integration tests also stop failing on a half-instantiated Julia project: bulklmm_available used to accept any directory with a Project.toml, and /tmp/gj-env has one naming BulkLMM and nothing else, so the scan died on `using CSV` instead of skipping. It now requires the project to name the three packages the driver loads, which is what a usable env is.
10 daysplot/bulklmm: the 0.05 and 0.01 lines, never 0.10 unless askedPjotr
Checked what the plot actually draws: it has never drawn the alpha 0.10 line -- the plotter matches 0.05 and 0.01 only. What lists all three is the reply's notes ("alpha 0.01: LOD 4.58, alpha 0.05: 3.65, alpha 0.1: 3.50"), which is where the 0.10 came from. Still, the point stands on its own: 0.10 is the loosest bar in the set and drawing it invites reading it as the line to clear. So the defaults are stated and pinned -- 0.05 and 0.01 on the plot, 0.10 only on request (?all_thresholds=true, or an explicit ?thresholds=) -- alongside GEMMA's Bonferroni, which is opt-in for the same reason: a reference line is an annotation, not a result. tests/bulklmm.rs counts the lines: exactly two by default, and no alpha 0.10 label.
10 daysbulklmm: BULKLMM_BLOCKS as a config option, default 4Pjotr
The marker blocks a single-trait scan is split into were the driver's default (Threads.nthreads()) and changed with --threads. They are a separate knob now: BULKLMM_THREADS Julia threads (--threads) default 4 BULKLMM_BLOCKS independent marker blocks, scanned default 4 in parallel (1 = serial scan) Blocks are exact either way -- LOD 9.589938 and the same permutation thresholds at 1 and 4 -- so the knob only buys wall clock. The reply says what ran ("split into 4 independent blocks and scanned on 4 Julia thread(s) (BULKLMM_BLOCKS / BULKLMM_THREADS ...)"), because the answer to "is it multicore?" should be in the response and not in the log.
10 daysgemma: the permutation refusal names both routes, including running GEMMA itselfPjotr
We do not run GEMMA permutations -- each repeats the whole LOCO scan and the gate gives GEMMA one slot -- but "not supported here" is not the same as "not available". The refusal now says both: 1. POST /compute/bulklmm with nperm: 1000 -- an LMM whose thresholds come from rotations that keep the kinship (R/qtl's nperm does not, and is anti-conservative on a structured panel); 2. run GEMMA's own permutation locally. The reply's notes already name the bimbam pair and the phenotype file, so `gemma-wrapper --permutate` or `gemma -lmm 9 --permutate` on those inputs reproduces exactly the null GEMMA would have used -- hours instead of seconds, which is the honest price of that particular null. docs/bulklmm.md gains the table this belongs in: three ways to get a threshold, ordered by how honest the null is for a structured panel (rotation with kinship; GEMMA's own LOCO permutation run locally; R/qtl's nperm as a sanity check only). SKILL.md says the same in two lines. tests/gemma.rs now asserts both halves of the refusal, and the earlier assertion on /compute/rqtl -- which stopped being true when the refusal was rewritten -- is replaced rather than left passing by accident.
10 daysdocs: what BulkLMM models, what its cores do, and how to package itPjotr
docs/bulklmm.md records the three things that are easy to get wrong when a BulkLMM result is quoted: * where it stands against the other engines -- Haley-Knott with no kinship (R/qtl), LOCO LMM whose permutations are refused (GEMMA), LOCO LMM on HMM probabilities (R/qtl2), and this: a global-kinship LMM whose permutation thresholds are rotation-based and therefore cheap -- the only engine here that can say "genome-wide significant at alpha 0.05" against a null that respects relatedness; * what it is not: global kinship rather than LOCO, so its p-values are not GEMMA's; h2_null says how much relatedness it found (near zero here, which is why it reproduces R/qtl's 9.59 where GEMMA differs); one degree of freedom, so multiparent scans stay with /compute/qtl2; and no standard errors in 1.2.2's scan(), so the rows are marker, chr, pos, lod, beta, h2; * the cores, measured: Threads.@threads is in bulkscan.jl and bulkscan_helpers.jl, never in scan.jl, so a single-trait scan is serial by design and the visible parallelism was BLAS (96 threads 10.5 s against 4 threads 7.7 s). What the driver does about it -- exact marker blocks, cached matrix and kinship, capped BLAS, and the breakdown printed into the reply -- with the numbers (20.9 s at one block against 18.3 s at four, same LOD 9.589938). Also the packaging picture, since it is not obvious from the outside: guix-bioinformatics has a working julia-bin and a julia-build-system, but only three of BulkLMM's closure are packaged, so a pure-Guix build is 35-45 recipes; the pragmatic route is julia-bin plus a depot built once from the pinned Manifest. scripts/prepare-julia-depot.sh is that step, written down, and the depot stays small and GPU-free (79 packages, 139 MB) with JULIA_PKG_OFFLINE=true at request time. The plot test learns that both thresholds are drawn, not one.
10 daysbulklmm: exact marker-block parallelism, and what the cores are actually doingPjotr
Reported: a BulkLMM run does not look multicore. It is not a misconfiguration. BulkLMM's single-trait `scan()` -- what this endpoint calls -- has no `Threads.@threads` at all; the package threads `bulkscan()` (many traits) and bulkscan_helpers, not scan. The parallelism that was visible before was BLAS (OpenBLAS defaulting to all 96 cores here), which is why `user` time ran 4x `real` while Julia threads made no difference: --threads 1 13.7 s, --threads 4 12.3 s. * The driver splits the markers into independent blocks and scans them on Julia threads (--blocks, default = BULKLMM_THREADS). It is exact: the LOD for each marker is independent given the null variance components, which depend on y and K only. Verified identical -- peak LOD 9.589938 and the same permutation thresholds at 1 and 4 blocks -- and permutation thresholds combine by taking each permutation's maximum across blocks before the quantiles, which is what a genome-wide threshold is. * The parsed matrix and the kinship are cached per panel (read 1.3 s -> 0.4 s, kinship 0.9 s -> 0), because both depend only on the cross. * The summary in `notes` now says what ran: julia threads, blocks, BLAS threads, and the read/scan split. * The hand parser is back: CSV.jl into a DataFrame is an order of magnitude slower for a 21056 x 235 matrix (14-17 s against 2 s). * BLAS is capped at 4 threads -- 96 was slower (10.5 s against 7.7 s for one scan). Measured on BXD.8 trait 16627 (21056 markers): 1 block 20.9 s, 4 blocks 18.3 s, same answer. The gain is modest because the LMM loop is ~8 s of the run and each block re-estimates its variance components. Where the cores pay is bulkscan() -- many traits in one process, threaded over trait blocks -- which is the shape the next endpoint wants; documented in the README rather than left as folklore.
11 daysbulklmm: an LMM scan that can be asked for thresholdsPjotr
GEMMA's LOCO scan is the reference for a biallelic panel, but its p-values have no threshold we can compute: a GEMMA permutation repeats the whole scan, which is why /compute/gemma refuses nperm. R/qtl's nperm is the usual stand-in and it is the wrong null -- it shuffles phenotypes as if the individuals were unrelated. BulkLMM.jl fits an LMM per marker with a global kinship matrix and permutes by rotating the data through the kinship's eigen decomposition, so a threshold costs seconds. POST /compute/bulklmm {group, genotype_version, samples|basket, nperm, seed, reml, assumption} Rows are marker, chr, pos (Mb), lod, beta, h2. BXD trait 16627 (BXD.8, 35 phenotyped animals, 21056 markers) in 22 s: peak rsm10000000888 chr2 4.92 Mb, LOD 9.59, and thresholds alpha 0.10 -> 3.51, 0.05 -> 3.81, 0.01 -> 4.47 from 200 rotations with seed 42. That is the trait's published locus and LOD, and a significant peak against a threshold this API could not compute before. Two differences from GEMMA are in every reply, not implied: a global kinship matrix rather than leave-one-chromosome-out, and h2_null saying how much relatedness was found (here ~0, so the fit is effectively fixed-effects -- which is why it agrees with R/qtl where GEMMA differs). * src/bulklmm.rs prepares what Julia reads -- the BIMBAM dosages as a strains x markers matrix (our BIMBAM pair is already the allele dosages BulkLMM wants), a marker map, the phenotype and covariates -- runs the driver through streaming::run_streaming_with_env, and reads the CSV back. A row whose LOD is NaN is dropped: NaN is a valid f64 in Rust and would otherwise be reported as a value. * scripts/bulklmm_scan.jl is the driver (scan/calcKinship/get_thresholds), with the seed recorded so a threshold set is reproducible. * gate::bulklmm (2 slots, GEMMA-sized threshold): a permutation here is cheap, unlike R/qtl's. * Julia comes from the deployment's own store binary with a prepared depot -- 79 packages, 139 MB, no GPU/CUDA stack -- and every request runs JULIA_PKG_OFFLINE=true, so a scan can never download anything. `--project=<path>`, not `--project <path>`: Julia 1.8 takes the space-separated form as a program argument. tests/bulklmm.rs: the peak stays on chr2 above LOD 3, the thresholds and seed are in the notes, and a scan without permutations says there is no threshold rather than leaving a bare LOD to be read as significant.