about summary refs log tree commit diff
path: root/src/param.cpp
diff options
context:
space:
mode:
authorPjotr Prins <pjotr.public01@thebird.nl>2026-11-17 11:25:45 +0000
committerPjotr Prins <pjotr.public01@thebird.nl>2026-11-17 11:25:45 +0000
commitcebca3b5cc041731df65bb5008fd3e7d5202c299 (patch)
tree845fe569565f033b2a57b763b1817cb2e0c3355e /src/param.cpp
parent314dbfd4b834cd7ad30e808587cd1ec3dc30feec (diff)
downloadpangemma-cebca3b5cc041731df65bb5008fd3e7d5202c299.tar.gz
param: count analyzed individuals properly, which revives the -r2 filter
ProcessInclusionIndicators summed ni_test with a range-for over the values
but used each value as an index:

    for (auto &i : indicator_idv) {
      ni_test += indicator_idv[i];
    }

so ni_test came out as the number of elements times indicator_idv[0] or
indicator_idv[1] -- 198 on the BXD fixture, where 67 individuals are
analyzed. Everything sized from it there was sized wrong: the covariate
matrix W was allocated with 198 rows while genotypes were written only at
the 67 analyzed positions, leaving 131 entries of uninitialised memory.

That memory is what killed the -r2 covariate filter. The filter computes
v_x = genotype . genotype over the whole vector, so a single garbage entry
(a value of 1.26e248 in the run that was traced) makes v_x infinite and the
ratio v_w/v_x zero, and no SNP is ever dropped. Measured on the BXD
fixture with two covariates: GEMMA 0.98.6 drops three SNPs and reports
7317; pangemma reported 7320 at every -r2 setting, including -r2 0.0001,
which should have dropped nearly everything. Those three SNPs, at lines
615-617 of the genotype file, were the entire remaining difference between
the two builds' association output, and they are not duplicates: one of
them is unique among all 7320 markers.

With the count fixed:

  -r2 0.9999 (default)   pangemma 7317   gemma 7317
  -r2 0.5                pangemma 3029   gemma 3029

and the BXD association output is identical to GEMMA's for all 7317 rows,
cell for cell, allele columns included. The GRM is unchanged (max |diff| 0
against GEMMA) and the lmdb LOCO path is unchanged (chr 1: 636 identical
rows).
Diffstat (limited to 'src/param.cpp')
-rw-r--r--src/param.cpp15
1 files changed, 13 insertions, 2 deletions
diff --git a/src/param.cpp b/src/param.cpp
index e2a61ea..7e34830 100644
--- a/src/param.cpp
+++ b/src/param.cpp
@@ -2039,9 +2039,20 @@ void PARAM::ProcessInclusionIndicators() {
   }
 
   // Obtain ni_test.
+  //
+  // This used to read `for (auto &i : indicator_idv) ni_test += indicator_idv[i];'
+  // -- a range-for binding each *element* (0 or 1) and then using it as an
+  // index, so the sum was ni_test == ni_total * indicator_idv[0 or 1], i.e. 0 or
+  // the total number of individuals.  Everything downstream then sized its
+  // matrices to the wrong ni_test: the covariate matrix W got ni_total rows
+  // while genotypes were written only at the ni_test analyzed positions, leaving
+  // the tail uninitialised -- which made the -r2 covariate filter compute
+  // v_x = infinity and therefore never drop a SNP, and made the
+  // "number of analyzed individuals equals 0" check fire whenever the first
+  // individual happened to be missing.
   ni_test = 0;
-  for(auto &i : indicator_idv) {
-    ni_test += indicator_idv[i];
+  for (auto i : indicator_idv) {
+    ni_test += i;
   }
 
   // If subsample number is set, perform a random sub-sampling