From cebca3b5cc041731df65bb5008fd3e7d5202c299 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Tue, 17 Nov 2026 11:25:45 +0000 Subject: param: count analyzed individuals properly, which revives the -r2 filter ProcessInclusionIndicators summed ni_test with a range-for over the values but used each value as an index: for (auto &i : indicator_idv) { ni_test += indicator_idv[i]; } so ni_test came out as the number of elements times indicator_idv[0] or indicator_idv[1] -- 198 on the BXD fixture, where 67 individuals are analyzed. Everything sized from it there was sized wrong: the covariate matrix W was allocated with 198 rows while genotypes were written only at the 67 analyzed positions, leaving 131 entries of uninitialised memory. That memory is what killed the -r2 covariate filter. The filter computes v_x = genotype . genotype over the whole vector, so a single garbage entry (a value of 1.26e248 in the run that was traced) makes v_x infinite and the ratio v_w/v_x zero, and no SNP is ever dropped. Measured on the BXD fixture with two covariates: GEMMA 0.98.6 drops three SNPs and reports 7317; pangemma reported 7320 at every -r2 setting, including -r2 0.0001, which should have dropped nearly everything. Those three SNPs, at lines 615-617 of the genotype file, were the entire remaining difference between the two builds' association output, and they are not duplicates: one of them is unique among all 7320 markers. With the count fixed: -r2 0.9999 (default) pangemma 7317 gemma 7317 -r2 0.5 pangemma 3029 gemma 3029 and the BXD association output is identical to GEMMA's for all 7317 rows, cell for cell, allele columns included. The GRM is unchanged (max |diff| 0 against GEMMA) and the lmdb LOCO path is unchanged (chr 1: 636 identical rows). --- src/param.cpp | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) (limited to 'src/param.cpp') diff --git a/src/param.cpp b/src/param.cpp index e2a61ea..7e34830 100644 --- a/src/param.cpp +++ b/src/param.cpp @@ -2039,9 +2039,20 @@ void PARAM::ProcessInclusionIndicators() { } // Obtain ni_test. + // + // This used to read `for (auto &i : indicator_idv) ni_test += indicator_idv[i];' + // -- a range-for binding each *element* (0 or 1) and then using it as an + // index, so the sum was ni_test == ni_total * indicator_idv[0 or 1], i.e. 0 or + // the total number of individuals. Everything downstream then sized its + // matrices to the wrong ni_test: the covariate matrix W got ni_total rows + // while genotypes were written only at the ni_test analyzed positions, leaving + // the tail uninitialised -- which made the -r2 covariate filter compute + // v_x = infinity and therefore never drop a SNP, and made the + // "number of analyzed individuals equals 0" check fire whenever the first + // individual happened to be missing. ni_test = 0; - for(auto &i : indicator_idv) { - ni_test += indicator_idv[i]; + for (auto i : indicator_idv) { + ni_test += i; } // If subsample number is set, perform a random sub-sampling -- cgit 1.4.1