|
A failing scan came back as
BulkLMM failed: in expression starting at scripts/bulklmm_scan.jl:26
which names neither the package nor the reason. Julia had said it plainly,
two lines up:
ERROR: LoadError: ArgumentError: Package CSV not found in current path.
- Run `import Pkg; Pkg.add("CSV")` to install the CSV package.
The complaint finder looked for a line starting "Error" -- Julia writes
"ERROR:" -- and fell back to the last non-empty line, which is the line
number. That is the same defect as gemma-wrapper's Ruby backtrace, in our
own error handling: the wrapped tool said why, and we threw it away.
julia_complaint takes the first ERROR:/LoadError line plus Julia's hint
line when there is one, with the last line kept only as a fallback for a
driver that fails silently. Three unit tests cover the three shapes.
The integration tests also stop failing on a half-instantiated Julia
project: bulklmm_available used to accept any directory with a
Project.toml, and /tmp/gj-env has one naming BulkLMM and nothing else, so
the scan died on `using CSV` instead of skipping. It now requires the
project to name the three packages the driver loads, which is what a
usable env is.
|
|
"is the kinship term doing anything?" should be answerable from the reply.
The driver now prints h2 and the residual variance under the null model, and
the reply carries both:
Variance components: h2 = 0.000000, sigma2_e = 0.402171, estimated once
under the null (intercept) model from y and K, not per marker. With a few
dozen phenotyped individuals these estimates pile up at the boundaries 0
and 1 -- the likelihood is flat there -- so read h2 as "kinship matters a
lot / not at all" rather than as a precise proportion. At a boundary
(h2 ~ 0 the model is ordinary regression, which is why its LODs can match
R/qtl's to the digit).
Checked with a strain-ID-driven trait (values = rank of strain), which has
h2 ~ 1 by construction: BulkLMM reports h2 = 1.000000, sigma2_e = 3e-06 and
a peak of LOD 3.1 -- the kinship term absorbs the strain structure -- while
R/qtl, which has no kinship term at all, reports a spurious peak of 5.06 on
the same values. That is the discrimination the h2 column was missing.
|
|
GEMMA's LOCO scan is the reference for a biallelic panel, but its p-values
have no threshold we can compute: a GEMMA permutation repeats the whole
scan, which is why /compute/gemma refuses nperm. R/qtl's nperm is the usual
stand-in and it is the wrong null -- it shuffles phenotypes as if the
individuals were unrelated. BulkLMM.jl fits an LMM per marker with a global
kinship matrix and permutes by rotating the data through the kinship's
eigen decomposition, so a threshold costs seconds.
POST /compute/bulklmm {group, genotype_version, samples|basket,
nperm, seed, reml, assumption}
Rows are marker, chr, pos (Mb), lod, beta, h2. BXD trait 16627 (BXD.8,
35 phenotyped animals, 21056 markers) in 22 s: peak rsm10000000888 chr2
4.92 Mb, LOD 9.59, and thresholds alpha 0.10 -> 3.51, 0.05 -> 3.81,
0.01 -> 4.47 from 200 rotations with seed 42. That is the trait's
published locus and LOD, and a significant peak against a threshold this
API could not compute before.
Two differences from GEMMA are in every reply, not implied: a global
kinship matrix rather than leave-one-chromosome-out, and h2_null saying
how much relatedness was found (here ~0, so the fit is effectively
fixed-effects -- which is why it agrees with R/qtl where GEMMA differs).
* src/bulklmm.rs prepares what Julia reads -- the BIMBAM dosages as a
strains x markers matrix (our BIMBAM pair is already the allele
dosages BulkLMM wants), a marker map, the phenotype and covariates
-- runs the driver through streaming::run_streaming_with_env, and
reads the CSV back. A row whose LOD is NaN is dropped: NaN is a valid
f64 in Rust and would otherwise be reported as a value.
* scripts/bulklmm_scan.jl is the driver (scan/calcKinship/get_thresholds),
with the seed recorded so a threshold set is reproducible.
* gate::bulklmm (2 slots, GEMMA-sized threshold): a permutation here is
cheap, unlike R/qtl's.
* Julia comes from the deployment's own store binary with a prepared
depot -- 79 packages, 139 MB, no GPU/CUDA stack -- and every request
runs JULIA_PKG_OFFLINE=true, so a scan can never download anything.
`--project=<path>`, not `--project <path>`: Julia 1.8 takes the
space-separated form as a program argument.
tests/bulklmm.rs: the peak stays on chr2 above LOD 3, the thresholds and
seed are in the notes, and a scan without permutations says there is no
threshold rather than leaving a bare LOD to be read as significant.
|