about summary refs log tree commit diff
path: root/scripts
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-11-16 00:30:42 +0000
committerPjotr <wrk@bosa.lan>2026-11-16 00:30:42 +0000
commit1ae2f5d712e18c016fdb7bd04da99eab4da6915b (patch)
tree31bb20e52585506351d81198b65c13ec3a229af0 /scripts
parent9c3cd6f2300bb290ddd05c5a46c5b813b22c0ce1 (diff)
downloadgenenetwork4-1ae2f5d712e18c016fdb7bd04da99eab4da6915b.tar.gz
qtl2: permutations for the LOD thresholds
POST /compute/qtl2 gains nperm, perm_seed and perm_strata, and the reply
gains meta.permutation {n, seed, significant, suggestive, values} -- the
genome-wide maximum of every permutation, so a histogram needs no second
call. `data` stays the positional rows array on purpose: /plot/rqtl posts
it straight back and the monitor counts its rows, so the null went beside
the rows rather than around them (a generic Envelope::with_meta/Meta::extra,
skip_serializing_if, so no other endpoint's response changes).

Thresholds come from R/qtl2's scan1perm over the same genotype
probabilities, kinship and covariates as the scan, with chr_lengths from
the map -- without those it derives chromosome lengths from marker
positions and the numbers stop being comparable with R/qtl's. The
permutation CSV is written in R/qtl's two-column shape so one parser and
one percentile routine serve both engines: PermutationResult::from_values
is now shared with /compute/rqtl instead of duplicated.

Semantics, each with the note that says it:

* scan1perm shuffles phenotypes across individuals -- right for an
  outbred cross, only approximate for a selfing RIL panel, where qtl2perm
  is the correct package (not installed here, and not packaged);
* perm_strata takes a case attribute name (sex, Epoch, ...) and permutes
  within it, because shuffling across sexes inflates the null;
* fewer than 100 permutations gets a note that the 0.05 quantile is a
  quantile of too few values; more than 10 000 is refused, since each
  permutation is a whole genome scan;
* a seed is reported always -- without one, two identical requests would
  give different thresholds.

Cost, measured on BXD trait 10001 (34 phenotyped, 20 chromosomes, LOCO
kinship, 4 cores): 20 permutations 0.6 s, 100 in 2.2 s, 500 in 14 s,
1000 in 24 s through the API; 0.022-0.03 s per permutation. Thresholds
converge as expected (0.05: 4.331 at 100 and 200 permutations, 4.374 at
1000 on the raw probe) and are reproducible: the same seed gives the same
numbers on 1 and 4 cores.

Four things that only surfaced by running it, all commented where they
bit: optparse keeps the dash (--perm-out is opt[["perm-out"]], so
opt$perm_out was always NULL and the file landed somewhere the caller did
not look); perm_strata must be a named character vector, not a factor;
the LOD in the CSV is the LAST numeric field (R/qtl's shape puts the
permutation index first, and a first-field parser reads that as the LOD);
and percentiles interpolate at (n-1)p/100, so 100 maxima give 0.6733, not
0.67.

bin/monitor-gn4 gains compute-qtl2-perm (nperm 10, judging
meta.permutation) so a deploy that loses the R path shows up as a red
check rather than a surprise.
Diffstat (limited to 'scripts')
-rwxr-xr-xscripts/qtl2_scan.R74
1 files changed, 74 insertions, 0 deletions
diff --git a/scripts/qtl2_scan.R b/scripts/qtl2_scan.R
index 6044db3..724eb69 100755
--- a/scripts/qtl2_scan.R
+++ b/scripts/qtl2_scan.R
@@ -44,6 +44,14 @@ option_list = list(
   make_option(c("--cores"), type="integer", default=1),
   make_option(c("--cache-dir"), type="character", default=NULL,
               help="where to keep genotype probabilities and kinship"),
+  make_option(c("--nperm"), type="integer", default=0,
+              help="permutations for the LOD thresholds; 0 (default) skips them"),
+  make_option(c("--seed"), type="integer", default=NULL,
+              help="seed for the permutations, so a run is reproducible"),
+  make_option(c("--strata"), type="character", default=NULL,
+              help="permute within strata (e.g. sex): a phenotype-file-shaped column, Sample and stratum"),
+  make_option(c("--perm-out"), type="character", default=NULL,
+              help="where to write one LOD per permutation (default <out>.perm)"),
   make_option(c("--out"), type="character", required=TRUE),
   make_option(c("--quiet"), action="store_true", default=FALSE)
 )
@@ -256,3 +264,69 @@ write.csv(result, opt$out, row.names = FALSE, quote = FALSE)
 best <- result[which.max(result$lod), ]
 say(sprintf("wrote %s: %d markers; strongest %s chr%s %.1f Mb LOD %.2f",
             opt$out, nrow(result), best$marker, best$chr, best$pos, best$lod))
+
+# ---- permutations: the null distribution of the genome-wide maximum ---
+# One permutation is a whole genome scan of a shuffled phenotype, so the
+# cost is nperm x the scan above -- which is why the caller decides how
+# many, and why qtl2's own `cores` argument is used here (the permutations
+# fork cleanly, unlike scan1 on allele probabilities).
+#
+# The output is deliberately the same two-column shape R/qtl's scanone
+# permutations produce ("", "lod"), so one parser reads both. Two things
+# that are NOT the same as R/qtl, and are the caller's business:
+#   * scan1perm permutes phenotypes across individuals, which is the right
+#     null for an outbred cross and only approximate for a selfing RIL
+#     panel (qtl2perm is the statistically correct package for those);
+#   * thresholds at alpha depend on how many permutations there are: with
+#     fewer than ~100 the 0.05 quantile is a coin toss.
+if (opt$nperm > 0) {
+  perm_started <- Sys.time()
+  if (!is.null(opt$seed)) set.seed(opt$seed)
+  strata <- NULL
+  if (!is.null(opt$strata)) {
+    strata_df <- read.table(opt$strata, sep = "\t", header = TRUE,
+                            check.names = FALSE, stringsAsFactors = FALSE)
+    strata_values <- strata_df[[2]][match(ind, strata_df$Sample)]
+    if (all(is.na(strata_values))) {
+      stop("none of the stratum file's samples appear in the cross", call. = FALSE)
+    }
+    # qtl2's perm_strata is a vector *of strata per individual*, named by
+    # individual: a factor reaches get_common_ids and it answers "Not sure
+    # what to do with object of class factor". Characters, names, and the
+    # unassigned left as NA so qtl2 drops those individuals from the
+    # permutation rather than refusing.
+    strata <- stats::setNames(as.character(strata_values), ind)
+    say("permutation strata:", paste(sort(unique(strata[!is.na(strata)])), collapse = ","),
+        "|", sum(!is.na(strata)), "individuals assigned")
+  }
+  # chr_lengths from the map: without it scan1perm derives chromosome
+  # lengths from the marker positions, and the thresholds stop being
+  # comparable with R/qtl's on the same cross.
+  lengths <- qtl2::chr_lengths(map)
+  perm <- scan1perm(scan_probs, pheno, kinship = kinship, addcovar = addcovar,
+                    n_perm = opt$nperm, perm_strata = strata,
+                    chr_lengths = lengths, cores = scan_cores,
+                    model = opt$model)
+  perm_values <- as.numeric(perm)
+  if (all(is.na(perm_values))) {
+    # A placeholder or constant phenotype makes every permutation NaN
+    # without an error, and summary() then reports "0 permutations" with NA
+    # thresholds -- a plausible answer that means nothing. Refuse instead.
+    stop("every permutation returned NA (a constant or placeholder phenotype?)",
+         call. = FALSE)
+  }
+  # optparse keeps the dash in the name (`--perm-out` is opt[["perm-out"]]),
+  # so opt$perm_out is always NULL: the fallback silently took over and the
+  # file landed at <out>.perm while the caller read <out>-perm.csv.
+  perm_path <- if (is.null(opt[["perm-out"]])) paste0(opt$out, ".perm") else opt[["perm-out"]]
+  write.csv(data.frame(lod = perm_values), perm_path, row.names = FALSE,
+            quote = FALSE)
+  thresholds <- stats::quantile(perm_values, probs = c(0.99, 0.95, 0.63),
+                                na.rm = TRUE, names = FALSE)
+  say(sprintf("permutations: %d in %.1f s (%.3f s each); thresholds 0.01=%.3f 0.05=%.3f 0.37=%.3f",
+              opt$nperm,
+              as.numeric(difftime(Sys.time(), perm_started, units = "secs")),
+              as.numeric(difftime(Sys.time(), perm_started, units = "secs")) / opt$nperm,
+              thresholds[1], thresholds[2], thresholds[3]))
+  say("wrote ", perm_path, ": ", length(perm_values), " permutation maxima")
+}