about summary refs log tree commit diff

BulkLMM: an LMM scan whose thresholds we can actually compute

POST /compute/bulklmm runs BulkLMM.jl (Apache-2.0) through scripts/bulklmm_scan.jl. This note records why the engine is here, what it does and does not model, and what its cores are doing — three things that are easy to get wrong when a result is quoted.

Why it sits beside GEMMA and R/qtl

model kinship permutations
/compute/rqtl Haley-Knott (single marker) none yes, but shuffles phenotypes — the wrong null for related individuals
/compute/gemma LMM, LOCO leave-one-chromosome-out refused: each one re-runs the whole LOCO scan
/compute/qtl2 LMM, LOCO, HMM probabilities LOCO no
/compute/bulklmm LMM, global kinship one K for the whole genome yes, cheaply: rotates by K's eigen decomposition

Three ways to get a threshold, in order of honesty for a structured panel:

where null cost
/compute/bulklmm with nperm rotations that keep the kinship seconds
GEMMA's own --permutate, run locally GEMMA's exact LOCO null one full LOCO scan per permutation: hours
R/qtl nperm on the same values phenotypes shuffled across individuals — relatedness destroyed, anti-conservative on RI lines and DO collapses minutes

The GEMMA endpoint does not run permutations: each repeats the whole LOCO scan and the gate gives GEMMA one slot. Its refusal says both alternatives — and names the files it used (the bimbam/ pair and the phenotype file) so a user who wants GEMMA's own null can reproduce it locally, which is the only way to get that null.

So BulkLMM is the only engine here that can say "this peak is genome-wide significant at α = 0.05" against a null that respects relatedness. That is what /compute/gemma's permutations_not_supported refusal now points at.

Worked example, BXD trait 16627 on BXD.8 (35 phenotyped animals, 21 056 markers):

peak rsm10000000888  chr2 4.92 Mb  LOD 9.59  beta 0.0027  h2_null ≈ 0
thresholds (200 rotations, seed 42): alpha 0.10 → 3.51, 0.05 → 3.81, 0.01 → 4.47

rsm10000000888 is the trait's published locus and 9.59 its published LOD, so the engine agrees with the record — and the peak clears the α = 0.01 threshold.

What it is not

  • A global kinship matrix, not LOCO. One K for every marker; GEMMA excludes each chromosome in turn. Different p-values on the same data, and the reply says so. Do not present a BulkLMM p-value as GEMMA's.
  • h2_null is the honest tell. It is the heritability estimated under the null (intercept) model, from y and K alone. Near zero — as here — the fit is effectively a fixed-effects model, which is exactly why it reproduces R/qtl's answer on this trait where GEMMA's LOCO scan differs.
  • One degree of freedom. BulkLMM 1.2.2 tests one coefficient per marker; a multiparent founder-allele model is /compute/qtl2's job (crosstype: do).
  • No standard errors yet. scan() in 1.2.2 returns (b, sigma2_e, h2_null, lod, L_perms); SE is not among them, so the reply's se column does not exist and the rows are marker, chr, pos, lod, beta, h2.

When the two engines' h2 disagree, and what settles it

Reported from a real analysis (BXD.8, 35 phenotyped animals):

trait BulkLMM h2 GEMMA l_mle reading
16769 1.000, σ²_e = 0 at the upper clamp on every marker except the chr2 peak both say kinship dominates; the peak's drop is the marker absorbing the effect
16627 0.000, σ²_e = 0.402 at the upper clamp everywhere, peak included a genuine disagreement

Both estimates sit on a boundary, which is expected at n = 35 — the likelihood is flat there — so neither number is a precise proportion. But opposite boundaries on the same phenotype and the same genotypes means the two engines are estimating different things, and it is worth knowing which:

  1. Different kinship matrices. GEMMA computes its own relatedness from the BIMBAM dosages (and, in LOCO mode, one per chromosome); BulkLMM's calcKinship is 2·(X−0.5)(X−0.5)ᵀ/p + 0.5 with a unit diagonal. Two matrices, two variance components — not comparable until their correlation is known.
  2. Different encodings. GEMMA gets 0 / 0.5 / 1 dosages with NA for missing; BulkLMM gets the same dosages with missing values mean-imputed per marker, and R/qtl's convert2riself (two of three codes) is a third. Trait 16627 is nearly determined by the chr2 genotype (B above 1.0, D below 0.6), so the residual variance differs a lot between those encodings.
  3. Different variance-component step. BulkLMM estimates once under the null with assumption = "null" (and REML or ML); GEMMA optimises per chromosome under LOCO. On a boundary both optimisers clamp, and they clamp to different conventions.

Measured on BXD::10001, 34 animals, once both engines' variance components were readable: BulkLMM's global null h2 is 0.819 (σ²_e = 5.381), and GEMMA's LOCO pve over the 20 chromosomes runs 0.3125-0.9558, median 0.8015, mean 0.7769. So the two agree on the variance split to within a couple of percent -- 0.80 against 0.82 -- which is the answer this comparison was for: the engines were never in disagreement about heritability, only about which chromosome's kinship to hold the rest of the genome out from. Quote GEMMA's pve with its range (it is per chromosome), BulkLMM's h2 as one number (it is global), and do not compare the p-values: --loco weights each chromosome differently, so neither set is the other's null.

What would settle it, in order of cost:

  • GEMMA's own variance components — available, one flag away. Tested on a synthetic BIMBAM trio (40 individuals, 60 markers) with GEMMA 0.98.6: -lmm 9 -pve is accepted and writes the block

    ## pve estimate in the null model = 1.62846e-06 ## se(pve) in the null model = -nan ## vg estimate in the null model = 0 ## ve estimate in the null model = 0

So the flag works with -lmm 9; what was missing was the plumbing, and that is now in. gemma-wrapper 5eb53fa copies each run's .log.txt beside the archive (they were only inside the .tar.xz, so reading pve meant unpacking a multi-megabyte archive), and /compute/gemma passes -pve on the GWA step and reports the components per chromosome:

GEMMA -pve, null model, per chromosome (LOCO): chr1 0.6674±0.3451, chr2 0.8426±0.3668, ..., chrX 0.8563±0.3525. That is pve 0.3118 to 0.9549 over 20 chromosomes; lowest chr8 pve 0.3118 ± 0.5009 (vg 23.3659, ve 12.3389), highest chr7 pve 0.9549 ± 0.3570 (vg 76.0933, ve 0.8661).

every one of the twenty (only the ends shown here), because under --loco each belongs to a different kinship matrix and a caller told only the range cannot ask which chromosome is which. The unrounded values also go to the run log as variance: lines, so a ?id= run's /compute/gemma/status stream carries them while the run is in flight. (BXD::10001, 34 animals, LOCO, -lmm 9.) That is the number the 16627/16769 comparison lacked: GEMMA's own h2, per chromosome, objective function named. Where the two engines still disagree, the disagreement is now between two printed variance components rather than between a number and an inference.

Two caveats, both measured. GEMMA prints -nan for se(pve) when the estimate sits on a boundary, and this build reports that as "not available" rather than as a number; a boundary pve is a direction, not a proportion — the same caveat as BulkLMM's h2, from the other engine. And the components are on the phenotype's own scale, so a trait measured in thousandths gives variances in thousandths: BXD::16769 (values 0.0012-0.0122, 36 animals) yields pve 1.0000 ± 0.0016 (vg 3.411e-5, ve 3.411e-10) — pve is scale-free and still reads 1.0, but the absolute numbers are meaningless to compare across traits. Compare pve with h2, not vg with σ²_e. * BulkLMM's likelihood over an h2 grid. The package has BulkLMM.profile_LL(y::Matrix, G, covar, K, h2_grid, markerID; reml) (internal, not exported), returning (ll_list_null, ll_list_alt) over the grid. It wants the phenotype as a one-column matrix and a covariate matrix (an all-zero column is what scan uses with no covariates). Printing that profile at the peak marker says whether an h2 of 0.000 is the maximum or the flat edge of one — the difference between "no heritability" and "the estimator cannot tell". * The kinship matrices, or a summary. BulkLMM's calcKinship(g) is one line and can be written out per panel; GEMMA's is already cached as a -gk *.k.json. Correlating their off-diagonals (or reporting min/median/max) is enough to say whether the two engines disagree about relatedness before arguing about what the variance components mean.

Until then, the honest reading is the one the analysis reached: BulkLMM's h2 and the thresholds that depend on it are approximate where h2 is on a boundary, and GEMMA's p-values for 16627 come from a kinship-dominated model — its peak should not be compared with a fixed-effects LOD.

Cores: scan() is serial by design

This was measured, not assumed. Threads.@threads appears in bulkscan.jl and bulkscan_helpers.jl — never in scan.jl. The single-trait scan() this endpoint calls runs on one Julia thread; the parallel speed-up that is visible comes from BLAS:

--threads 1  scan 13.7 s        BLAS 96 threads  scan 10.5 s
--threads 4  scan 12.3 s        BLAS  4 threads  scan  7.7 s   (better!)

OpenBLAS defaults to every core on the machine (96 here) and oversubscribes on matrices of this shape. scripts/bulklmm_scan.jl therefore:

  • splits the markers into independent blocks and scans them on Julia threads (--blocks, BULKLMM_BLOCKS, default 4; 1 is a plain serial scan). It is exact: each marker's LOD is independent once the null variance components are fixed, and those depend on y and K only. Verified identical output — LOD 9.589938 and the same thresholds at 1 and 4 blocks — and the permutation thresholds are combined by taking each permutation's maximum across blocks before the quantiles, which is precisely what a genome-wide threshold is;
  • caches the parsed matrix and the kinship per panel (read 1.3 → 0.4 s, kinship 0.9 → 0);
  • caps BLAS at 4 threads when serial, 1 per block when parallel;
  • draws two reference lines: the genome-wide alpha 0.05 and the stricter alpha 0.01. The reply reports alpha 0.10 as well, but that is the loosest bar in the set, so it is drawn only on request (?all_thresholds=true, or an explicit ?thresholds=), and GEMMA's Bonferroni line on /plot/gemma is opt-in in the same spirit;
  • reports what ran in the reply: julia threads 4, blocks 4, BLAS threads 96 | read 1.5 s, scan 10.5 s.

What that buys at this size: 20.9 s (1 block) → 18.3 s (4 blocks). Modest, because the LMM loop is ~8 s of the run, each block re-estimates its variance components, and Julia's startup is the rest. Where cores pay is bulkscan() — many traits in one process, threaded over trait blocks — which is the shape an eQTL endpoint wants and the next thing to add here.

Configuration and the Julia side

key meaning
BULKLMM_CMD the julia binary (default: the store's julia-1.8.5)
BULKLMM_ENV the project its packages are instantiated into
BULKLMM_DEPOT the depot those packages live in (default $HOME/.julia)
BULKLMM_SCRIPT scripts/bulklmm_scan.jl
BULKLMM_THREADS Julia threads (--threads), default 4
BULKLMM_BLOCKS independent marker blocks scanned in parallel, default 4 (1 = serial scan, same answer)
BULKLMM_CACHE_DIR scratch: prepared genotypes, cached matrix and kinship
BULKLMM_PLOTTER scripts/plot-bulklmm.py for POST /plot/bulklmm

The depot is small and deliberately so — 79 packages, 139 MB, no GPU/CUDA stack: BulkLMM's closure is CSV, DataFrames, Distributions, Optim and their dependencies. Every request runs with JULIA_PKG_OFFLINE=true, so a scan can never fetch a package; if something is missing, the run fails saying which package, and the fix is to prepare the depot — a deliberate operator action, not a side effect of a request.

Packaging, current state. The deployment uses the store's own julia-1.8.5 with a depot under ~/.julia, and the project (BULKLMM_ENV) pinned by Manifest.toml (BulkLMM 1.2.2 + CSV + DataFrames). guix-bioinformatics already carries a working julia-bin (1.10.9) and a julia-build-system, but of BulkLMM's closure only julia-distributions, julia-optim and julia-nlsolversbase are packaged there — a pure-Guix build would need ~35–45 new package recipes. The pragmatic route, if the depot should stop being ~/.julia, is julia-bin plus a depot built once from our Manifest.toml (scripts/prepare-julia-depot.sh) and pointed at by BULKLMM_DEPOT; the alternative, PackageCompiler, would bake the driver into a single executable and remove Julia startup from the request path entirely.

Traps worth remembering

  • --project=<path>, not --project <path>: Julia 1.8 takes the space-separated form as a program argument and the run produces nothing, with exit status 0.
  • A NaN LOD parses as a valid f64 in Rust; the parser filters non-finite values, or a marker the model could not fit would be reported as a LOD of 0.
  • The gate gives a caller one BulkLMM scan at a time (compute_busy), which is why two tests that scan concurrently must tolerate a 409 rather than fail.

Deployed state (2026-11-11)

Both processes were restarted with GEMMA_WRAPPER_CMD pointing at gemma-wrapper-1.00-pre1-4.5eb53fa (the package is pinned to 5eb53fa, revision 4, in guix-bioinformatics' gn/packages/gemma.scm), because the profile's own wrapper is older than the pin. Verified against the running production instance rather than a fixture: POST /compute/gemma for BXD::10001 returns pve 0.3125 to 0.9558 over 20 chromosomes in its notes.