diff options
| author | Pjotr <wrk@bosa.lan> | 2026-11-11 15:42:40 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-11-11 15:42:40 +0000 |
| commit | 54d704ab7a6ec6e39ccb66e08c2334652a437cfa (patch) | |
| tree | 12fdfd2b3e56d5a6cdfdf9d65739b55c26b97cb9 /docs | |
| parent | 48e33540c4a00f0077963c3410628a6258448558 (diff) | |
| download | genenetwork4-54d704ab7a6ec6e39ccb66e08c2334652a437cfa.tar.gz | |
docs: GEMMA's pve is one flag away, not a dead end
Tested -lmm 9 with -pve on a synthetic BIMBAM trio (40 individuals, 60 markers, GEMMA 0.98.6): the flag is accepted and the log carries ## pve estimate in the null model = ... ## se(pve) in the null model = -nan ## vg estimate in the null model = 0 ## ve estimate in the null model = 0 So what was missing is plumbing, not support. Two gemma-wrapper changes are in its working tree (5eb53fa): the per-run logs are copied beside the archive -- they were only inside the .tar.xz -- and a failing child's own ERROR lines are printed before the Ruby backtrace. Once pushed and pinned, /compute/gemma can pass -pve and report pve/vg/ve per chromosome beside l_mle, which is what the 16627 and 16769 comparison needs. Also recorded: on random data GEMMA returns se(pve) = -nan with vg = ve = 0, so a boundary pve is a direction rather than a proportion -- the same caveat as BulkLMM's h2, from the other engine.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/bulklmm.md | 41 |
1 files changed, 23 insertions, 18 deletions
diff --git a/docs/bulklmm.md b/docs/bulklmm.md index 83d4686..84ebe4d 100644 --- a/docs/bulklmm.md +++ b/docs/bulklmm.md @@ -89,24 +89,29 @@ are estimating different things, and it is worth knowing which: **What would settle it, in order of cost:** -* **GEMMA's own variance components.** GEMMA prints `pve estimate` and - `se(pve)` in its `.log.txt` only when run with `-pve`, which our GWA step does - not pass. Whether `-pve` is *compatible* with `-lmm 9` under `--loco` is - **untested**, not refuted: a hand-made invocation with and without the flag - fails identically inside gemma-wrapper's `parallel` batch of per-chromosome - jobs, so the flag was never isolated. What the wrapper does not surface is the - failing job's own error — the batch's stderr goes to a temporary directory - that is removed — and that is the defect worth fixing first, because it hides - every GEMMA failure, not just this one. The logs the wrapper keeps are the - *kinship* step's — 223 `.log.txt` files under `/tmp/gn4-gemma`, none with a - `pve` block, because the per-chromosome LMM runs write theirs inside - gemma-wrapper's temporary directories and they are not retained. Two routes - remain, neither landed: (a) run GEMMA ourselves for the null fit — `-gk` to - write a text K matrix, then `-lmm 1 -pve` on a single marker, both seconds — - and read `pve estimate in the null model` from its log; (b) ask gemma-wrapper - to forward `-pve` (it is not in its allowlist). (a) needs the kinship as plain - text: the wrapper caches it as JSON under a per-instance cache directory, and - the two attempts at writing it out by hand were not completed. +* **GEMMA's own variance components — available, one flag away.** Tested on a + synthetic BIMBAM trio (40 individuals, 60 markers) with GEMMA 0.98.6: `-lmm 9 + -pve` is accepted and writes the block + + ## pve estimate in the null model = 1.62846e-06 + ## se(pve) in the null model = -nan + ## vg estimate in the null model = 0 + ## ve estimate in the null model = 0 + + So the flag works with `-lmm 9`; what was missing is the plumbing. Two + wrapper changes are in `gemma-wrapper` (5eb53fa, awaiting push and a pin + bump): the per-run `.log.txt` files are copied **beside** the archive (they + were only inside the `.tar.xz`, so reading `pve` meant unpacking a + multi-megabyte archive), and a failing child's own `ERROR`/`****` lines are + printed before the Ruby backtrace. With those, `/compute/gemma` can pass + `-pve` on the GWA step and read `pve`/`vg`/`ve` per chromosome out of + `<hash>.<chr>.cXX.txt.log.txt` — and report them beside `l_mle`, which is what + the 16627/16769 comparison needs. + + Note what GEMMA itself returned on random data: `se(pve) = -nan` with + `vg = ve = 0`. On a boundary GEMMA's own standard error is not a number, so a + boundary pve is a direction, not a proportion — the same caveat as BulkLMM's + `h2`, from the other engine. * **BulkLMM's likelihood over an h2 grid.** The package has `BulkLMM.profile_LL(y::Matrix, G, covar, K, h2_grid, markerID; reml)` (internal, not exported), returning `(ll_list_null, ll_list_alt)` over the grid. It wants |
