diff options
| author | Pjotr <wrk@bosa.lan> | 2026-11-11 15:22:07 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-11-11 15:22:07 +0000 |
| commit | 8a0b4059114d2628fe3362c816efcfc4262fc897 (patch) | |
| tree | e095c0e348ad124efa0c05461f2c71be0932db0f /docs | |
| parent | ee007a3731d186dc12c1b8b81589a03e8eb95acb (diff) | |
| download | genenetwork4-8a0b4059114d2628fe3362c816efcfc4262fc897.tar.gz | |
docs: why the two engines' h2 disagree, and the three measurements that settle it
Written down from a real analysis (BXD.8, 35 phenotyped animals): trait
16769 has both engines saying kinship dominates (BulkLMM h2 = 1.000,
sigma2_e = 0; GEMMA clamps l_mle on every marker but the chr2 peak), while
16627 is a genuine disagreement (h2 = 0.000 against a clamped l_mle
everywhere). Both sit on a boundary, which is expected at n = 35 -- but
opposite boundaries mean the engines are estimating different things:
* different kinship matrices (GEMMA's own relatedness from the BIMBAM
dosages, one per chromosome under LOCO, against calcKinship's
2*(X-0.5)(X-0.5)'/p + 0.5 with a unit diagonal);
* different encodings (0/0.5/1 with NA, against the same dosages with
missing values mean-imputed, against R/qtl's two-of-three codes) --
and 16627 is nearly determined by the chr2 genotype;
* different variance-component step (once under the null against
per-chromosome LOCO, clamped differently at a boundary).
The three measurements that would settle it, with what is known about each:
1. GEMMA's pve: it prints "pve estimate" when run with -pve, and
gemma-wrapper keeps 223 per-chromosome .log.txt files under its cache
dir -- none contain pve, because the flag is not passed. One argument
on the GWA step plus a parser.
2. BulkLMM's profile over an h2 grid: BulkLMM.profile_LL(y::Matrix, G,
covar, K, h2_grid, markerID; reml) is internal (not exported) and
returns (ll_list_null, ll_list_alt); it wants a one-column phenotype
matrix and a covariate matrix. An attempt to call it from the driver
is not landed -- the signature mismatch is the whole of the remaining
work.
3. The kinship matrices, or a summary: calcKinship is one line and can be
written per panel, GEMMA's is already cached as *.k.json; correlating
off-diagonals says whether the engines disagree about relatedness
before arguing about variance components.
Until those exist, the reading in the analysis holds: BulkLMM's h2 and the
thresholds that rest on it are approximate on a boundary, and GEMMA's
16627 p-values come from a kinship-dominated model.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/bulklmm.md | 55 |
1 files changed, 55 insertions, 0 deletions
diff --git a/docs/bulklmm.md b/docs/bulklmm.md index fbb3b37..21e7f48 100644 --- a/docs/bulklmm.md +++ b/docs/bulklmm.md @@ -58,6 +58,61 @@ the engine agrees with the record — and the peak clears the α = 0.01 threshol lod, L_perms)`; `SE` is not among them, so the reply's `se` column does not exist and the rows are `marker, chr, pos, lod, beta, h2`. +## When the two engines' h2 disagree, and what settles it + +Reported from a real analysis (BXD.8, 35 phenotyped animals): + +| trait | BulkLMM h2 | GEMMA `l_mle` | reading | +|---|---|---|---| +| 16769 | 1.000, σ²_e = 0 | at the upper clamp on every marker except the chr2 peak | both say kinship dominates; the peak's drop is the marker absorbing the effect | +| 16627 | 0.000, σ²_e = 0.402 | at the upper clamp everywhere, peak included | a genuine disagreement | + +Both estimates sit on a boundary, which is expected at n = 35 — the likelihood +is flat there — so neither number is a precise proportion. But *opposite* +boundaries on the same phenotype and the same genotypes means the two engines +are estimating different things, and it is worth knowing which: + +1. **Different kinship matrices.** GEMMA computes its own relatedness from the + BIMBAM dosages (and, in LOCO mode, one per chromosome); BulkLMM's + `calcKinship` is `2·(X−0.5)(X−0.5)ᵀ/p + 0.5` with a unit diagonal. Two + matrices, two variance components — not comparable until their correlation is + known. +2. **Different encodings.** GEMMA gets `0 / 0.5 / 1` dosages with `NA` for + missing; BulkLMM gets the same dosages with missing values mean-imputed per + marker, and R/qtl's `convert2riself` (two of three codes) is a third. + Trait 16627 is nearly determined by the chr2 genotype (B above 1.0, D below + 0.6), so the residual variance differs a lot between those encodings. +3. **Different variance-component step.** BulkLMM estimates once under the null + with `assumption = "null"` (and REML or ML); GEMMA optimises per chromosome + under LOCO. On a boundary both optimisers clamp, and they clamp to different + conventions. + +**What would settle it, in order of cost:** + +* **GEMMA's own variance components.** GEMMA prints `pve estimate` and + `se(pve)` in its `.log.txt` when it is run with `-pve`, and gemma-wrapper keeps + the per-chromosome logs under its cache directory (223 `.log.txt` files here, + **none** containing `pve` — the flag is not passed). The change is one + argument on the GWA step and a parser over those logs; then `/compute/gemma` + can report pve per chromosome, as `/compute/bulklmm` reports h2. +* **BulkLMM's likelihood over an h2 grid.** The package has + `BulkLMM.profile_LL(y::Matrix, G, covar, K, h2_grid, markerID; reml)` (internal, + not exported), returning `(ll_list_null, ll_list_alt)` over the grid. It wants + the phenotype as a one-column matrix and a covariate matrix (an all-zero column + is what `scan` uses with no covariates). Printing that profile at the peak + marker says whether an h2 of 0.000 is the maximum or the flat edge of one — the + difference between "no heritability" and "the estimator cannot tell". +* **The kinship matrices, or a summary.** BulkLMM's `calcKinship(g)` is one line + and can be written out per panel; GEMMA's is already cached as a `-gk` + `*.k.json`. Correlating their off-diagonals (or reporting min/median/max) is + enough to say whether the two engines disagree about *relatedness* before + arguing about what the variance components mean. + +Until then, the honest reading is the one the analysis reached: BulkLMM's `h2` +and the thresholds that depend on it are approximate where `h2` is on a +boundary, and GEMMA's p-values for 16627 come from a kinship-dominated model — +its peak should not be compared with a fixed-effects LOD. + ## Cores: `scan()` is serial by design This was measured, not assumed. `Threads.@threads` appears in `bulkscan.jl` and |
