| Age | Commit message (Collapse) | Author |
|
The note reported the range and named the two chromosomes that bounded it,
which is not enough to answer the question the numbers exist for. Under
--loco each pve belongs to a *different* kinship matrix, and GEMMA's
per-chromosome .log.txt files live on the server -- so a caller told "0.31
to 0.95" could not ask which chromosome was which, or compare a single
chromosome against BulkLMM.
The note now lists all twenty, compactly, with the range and the two
extremes keeping their vg/ve:
GEMMA -pve, null model, per chromosome (LOCO): chr1 0.6674±0.3451,
chr2 0.8426±0.3668, ..., chrX 0.8563±0.3525. That is pve 0.3118 to
0.9549 over 20 chromosomes; lowest chr8 pve 0.3118 ± 0.5009 (vg
23.3659, ve 12.3389), highest chr7 pve 0.9549 ± 0.3570 (vg 76.0933,
ve 0.8661).
and streaming::append_lines puts the unrounded values in the run log as
variance: chr7 pve=0.954938 se_pve=0.357032 vg=76.0933 ve=0.86612
so a ?id= run's /compute/gemma/status stream carries them while the run is
still in flight, rather than only in a reply someone has to keep. Both
measured on BXD::10001 through the dev server before committing.
The values are printed with four significant figures (scientific when the
magnitude needs it), unchanged: rounding variance components to four
decimals was the earlier bug, and these are the same numbers a reader
compares with BulkLMM's h2.
Docs: README's /compute/gemma section, SKILL's gemma bullet ("quote
per-chromosome values, not an average: they are twenty different kinship
matrices") and docs/bulklmm.md's example.
|
|
Once both engines' variance components were readable, BXD::10001 (34
animals) gives:
BulkLMM, global null h2 = 0.819 (sigma2_e 5.381)
GEMMA, LOCO per chr pve 0.3125-0.9558, median 0.8015, mean 0.7769
0.80 against 0.82. The engines never disagreed about the variance split;
they disagreed about which chromosome's kinship to hold the rest of the
genome out from. That closes the question this comparison started with.
Also says how to quote them: GEMMA's pve with its range because it is per
chromosome, BulkLMM's h2 as one number because it is global, and neither
set of p-values against the other.
|
|
GEMMA_WRAPPER_CMD is the lever: the profile's gemma-wrapper may be older
than the pin, and pointing the variable at the store path is enough --
-pve reaches GEMMA either way, but the per-run .log.txt files where pve
lives are only kept beside the archive from 5eb53fa on.
Recorded the verified production state too: both processes restarted with
gemma-wrapper-1.00-pre1-4.5eb53fa, and POST /compute/gemma for BXD::10001
on the running instance returns pve 0.3125 to 0.9558 over 20 chromosomes
-- measured there, not on a fixture.
|
|
The GWA step now passes -pve, and the reply gains a note with the
variance components GEMMA computed for the null model. Under --loco
there is one per chromosome, so the note gives their range and names
the two that bound it:
GEMMA -pve, null model, per chromosome (LOCO): pve 0.3118 to 0.9549
over 20 chromosomes -- lowest 8 pve 0.3118 +/- 0.5009 (vg 23.3659, ve
12.3389), highest 7 pve 0.9549 +/- 0.3570 (vg 76.0933, ve 0.8661).
That is BXD::10001 measured, not assumed. pve is the proportion of
variance the kinship explains -- the same quantity as /compute/bulklmm's
h2, per chromosome rather than genome-wide -- which is what the
16627/16769 comparison against BulkLMM needed.
Reading the numbers took three decisions worth stating:
- Four decimals is wrong for variance components. The first live run
reported 'vg 0.0000, ve 0.0000' for a BXD::16769 null fit whose real
components are 3.411e-5 and 3.411e-10 -- both rounded to zero, which
reads as 'no variance' when the split is 1.0. Components are printed
with four significant figures, scientific when the magnitude needs it,
the way GEMMA's own logs do with %g.
- se(pve) is -nan on a boundary, and Rust parses -nan as a NaN f64, so
the numeric parse alone let it through as NaN. Non-finite values are
now 'not available': a boundary estimate has no standard error.
- Twenty chromosomes of values bury the other notes, and every value is
in the .log.txt beside the archive anyway, so the note names the ends.
Requires gemma-wrapper 5eb53fa, which keeps each run's .log.txt beside
its archive instead of only inside the .tar.xz.
Docs: README's /compute/gemma section, SKILL's gemma bullet (compare pve
with h2, never vg with sigma2_e -- pve is scale-free), and
docs/bulklmm.md's 'what would settle it', which now says the plumbing is
in rather than pending.
|
|
Tested -lmm 9 with -pve on a synthetic BIMBAM trio (40 individuals, 60
markers, GEMMA 0.98.6): the flag is accepted and the log carries
## pve estimate in the null model = ...
## se(pve) in the null model = -nan
## vg estimate in the null model = 0
## ve estimate in the null model = 0
So what was missing is plumbing, not support. Two gemma-wrapper changes are
in its working tree (5eb53fa): the per-run logs are copied beside the
archive -- they were only inside the .tar.xz -- and a failing child's own
ERROR lines are printed before the Ruby backtrace. Once pushed and pinned,
/compute/gemma can pass -pve and report pve/vg/ve per chromosome beside
l_mle, which is what the 16627 and 16769 comparison needs.
Also recorded: on random data GEMMA returns se(pve) = -nan with vg = ve = 0,
so a boundary pve is a direction rather than a proportion -- the same
caveat as BulkLMM's h2, from the other engine.
|
|
The earlier note said passing -pve crashes gemma-wrapper. It does not --
my hand-made invocation fails identically with and without the flag, inside
the parallel batch of per-chromosome jobs, so the flag was never isolated
and its compatibility with -lmm 9 under --loco is unknown.
What that control run did find is the real defect: the wrapper does not
surface the failing job's error (the batch's stderr goes to a temporary
directory that is removed), which hides every GEMMA failure, not just this
one.
|
|
Tried it: passing -pve through gemma-wrapper after -- crashes the wrapper
(Ruby backtrace, no output), and the logs the wrapper keeps are the kinship
step's -- 223 .log.txt in the cache dir, none containing a pve block,
because the per-chromosome LMM logs live in the wrapper's temp dirs and are
not retained.
Two routes remain, both written down: run GEMMA ourselves (-gk for a text K
matrix, then -lmm 1 -pve on one marker, seconds each) and parse 'pve
estimate in the null model'; or get -pve into the wrapper's allowlist
upstream. The first needs the kinship as plain text, which the wrapper only
caches as JSON.
|
|
settle it
Written down from a real analysis (BXD.8, 35 phenotyped animals): trait
16769 has both engines saying kinship dominates (BulkLMM h2 = 1.000,
sigma2_e = 0; GEMMA clamps l_mle on every marker but the chr2 peak), while
16627 is a genuine disagreement (h2 = 0.000 against a clamped l_mle
everywhere). Both sit on a boundary, which is expected at n = 35 -- but
opposite boundaries mean the engines are estimating different things:
* different kinship matrices (GEMMA's own relatedness from the BIMBAM
dosages, one per chromosome under LOCO, against calcKinship's
2*(X-0.5)(X-0.5)'/p + 0.5 with a unit diagonal);
* different encodings (0/0.5/1 with NA, against the same dosages with
missing values mean-imputed, against R/qtl's two-of-three codes) --
and 16627 is nearly determined by the chr2 genotype;
* different variance-component step (once under the null against
per-chromosome LOCO, clamped differently at a boundary).
The three measurements that would settle it, with what is known about each:
1. GEMMA's pve: it prints "pve estimate" when run with -pve, and
gemma-wrapper keeps 223 per-chromosome .log.txt files under its cache
dir -- none contain pve, because the flag is not passed. One argument
on the GWA step plus a parser.
2. BulkLMM's profile over an h2 grid: BulkLMM.profile_LL(y::Matrix, G,
covar, K, h2_grid, markerID; reml) is internal (not exported) and
returns (ll_list_null, ll_list_alt); it wants a one-column phenotype
matrix and a covariate matrix. An attempt to call it from the driver
is not landed -- the signature mismatch is the whole of the remaining
work.
3. The kinship matrices, or a summary: calcKinship is one line and can be
written per panel, GEMMA's is already cached as *.k.json; correlating
off-diagonals says whether the engines disagree about relatedness
before arguing about variance components.
Until those exist, the reading in the analysis holds: BulkLMM's h2 and the
thresholds that rest on it are approximate on a boundary, and GEMMA's
16627 p-values come from a kinship-dominated model.
|
|
Checked what the plot actually draws: it has never drawn the alpha 0.10
line -- the plotter matches 0.05 and 0.01 only. What lists all three is the
reply's notes ("alpha 0.01: LOD 4.58, alpha 0.05: 3.65, alpha 0.1: 3.50"),
which is where the 0.10 came from.
Still, the point stands on its own: 0.10 is the loosest bar in the set and
drawing it invites reading it as the line to clear. So the defaults are
stated and pinned -- 0.05 and 0.01 on the plot, 0.10 only on request
(?all_thresholds=true, or an explicit ?thresholds=) -- alongside GEMMA's
Bonferroni, which is opt-in for the same reason: a reference line is an
annotation, not a result.
tests/bulklmm.rs counts the lines: exactly two by default, and no
alpha 0.10 label.
|
|
The marker blocks a single-trait scan is split into were the driver's
default (Threads.nthreads()) and changed with --threads. They are a
separate knob now:
BULKLMM_THREADS Julia threads (--threads) default 4
BULKLMM_BLOCKS independent marker blocks, scanned default 4
in parallel (1 = serial scan)
Blocks are exact either way -- LOD 9.589938 and the same permutation
thresholds at 1 and 4 -- so the knob only buys wall clock. The reply says
what ran ("split into 4 independent blocks and scanned on 4 Julia
thread(s) (BULKLMM_BLOCKS / BULKLMM_THREADS ...)"), because the answer to
"is it multicore?" should be in the response and not in the log.
|
|
We do not run GEMMA permutations -- each repeats the whole LOCO scan and the
gate gives GEMMA one slot -- but "not supported here" is not the same as
"not available". The refusal now says both:
1. POST /compute/bulklmm with nperm: 1000 -- an LMM whose thresholds come
from rotations that keep the kinship (R/qtl's nperm does not, and is
anti-conservative on a structured panel);
2. run GEMMA's own permutation locally. The reply's notes already name the
bimbam pair and the phenotype file, so `gemma-wrapper --permutate` or
`gemma -lmm 9 --permutate` on those inputs reproduces exactly the null
GEMMA would have used -- hours instead of seconds, which is the honest
price of that particular null.
docs/bulklmm.md gains the table this belongs in: three ways to get a
threshold, ordered by how honest the null is for a structured panel
(rotation with kinship; GEMMA's own LOCO permutation run locally; R/qtl's
nperm as a sanity check only). SKILL.md says the same in two lines.
tests/gemma.rs now asserts both halves of the refusal, and the earlier
assertion on /compute/rqtl -- which stopped being true when the refusal was
rewritten -- is replaced rather than left passing by accident.
|
|
docs/bulklmm.md records the three things that are easy to get wrong when a
BulkLMM result is quoted:
* where it stands against the other engines -- Haley-Knott with no
kinship (R/qtl), LOCO LMM whose permutations are refused (GEMMA),
LOCO LMM on HMM probabilities (R/qtl2), and this: a global-kinship
LMM whose permutation thresholds are rotation-based and therefore
cheap -- the only engine here that can say "genome-wide significant
at alpha 0.05" against a null that respects relatedness;
* what it is not: global kinship rather than LOCO, so its p-values are
not GEMMA's; h2_null says how much relatedness it found (near zero
here, which is why it reproduces R/qtl's 9.59 where GEMMA differs);
one degree of freedom, so multiparent scans stay with /compute/qtl2;
and no standard errors in 1.2.2's scan(), so the rows are
marker, chr, pos, lod, beta, h2;
* the cores, measured: Threads.@threads is in bulkscan.jl and
bulkscan_helpers.jl, never in scan.jl, so a single-trait scan is
serial by design and the visible parallelism was BLAS (96 threads
10.5 s against 4 threads 7.7 s). What the driver does about it --
exact marker blocks, cached matrix and kinship, capped BLAS, and the
breakdown printed into the reply -- with the numbers (20.9 s at one
block against 18.3 s at four, same LOD 9.589938).
Also the packaging picture, since it is not obvious from the outside:
guix-bioinformatics has a working julia-bin and a julia-build-system, but
only three of BulkLMM's closure are packaged, so a pure-Guix build is
35-45 recipes; the pragmatic route is julia-bin plus a depot built once
from the pinned Manifest. scripts/prepare-julia-depot.sh is that step,
written down, and the depot stays small and GPU-free (79 packages,
139 MB) with JULIA_PKG_OFFLINE=true at request time.
The plot test learns that both thresholds are drawn, not one.
|