about summary refs log tree commit diff
path: root/src/qtl2.rs
AgeCommit message (Collapse)Author
3 daysqtl2: permutations for the LOD thresholdsPjotr
POST /compute/qtl2 gains nperm, perm_seed and perm_strata, and the reply gains meta.permutation {n, seed, significant, suggestive, values} -- the genome-wide maximum of every permutation, so a histogram needs no second call. `data` stays the positional rows array on purpose: /plot/rqtl posts it straight back and the monitor counts its rows, so the null went beside the rows rather than around them (a generic Envelope::with_meta/Meta::extra, skip_serializing_if, so no other endpoint's response changes). Thresholds come from R/qtl2's scan1perm over the same genotype probabilities, kinship and covariates as the scan, with chr_lengths from the map -- without those it derives chromosome lengths from marker positions and the numbers stop being comparable with R/qtl's. The permutation CSV is written in R/qtl's two-column shape so one parser and one percentile routine serve both engines: PermutationResult::from_values is now shared with /compute/rqtl instead of duplicated. Semantics, each with the note that says it: * scan1perm shuffles phenotypes across individuals -- right for an outbred cross, only approximate for a selfing RIL panel, where qtl2perm is the correct package (not installed here, and not packaged); * perm_strata takes a case attribute name (sex, Epoch, ...) and permutes within it, because shuffling across sexes inflates the null; * fewer than 100 permutations gets a note that the 0.05 quantile is a quantile of too few values; more than 10 000 is refused, since each permutation is a whole genome scan; * a seed is reported always -- without one, two identical requests would give different thresholds. Cost, measured on BXD trait 10001 (34 phenotyped, 20 chromosomes, LOCO kinship, 4 cores): 20 permutations 0.6 s, 100 in 2.2 s, 500 in 14 s, 1000 in 24 s through the API; 0.022-0.03 s per permutation. Thresholds converge as expected (0.05: 4.331 at 100 and 200 permutations, 4.374 at 1000 on the raw probe) and are reproducible: the same seed gives the same numbers on 1 and 4 cores. Four things that only surfaced by running it, all commented where they bit: optparse keeps the dash (--perm-out is opt[["perm-out"]], so opt$perm_out was always NULL and the file landed somewhere the caller did not look); perm_strata must be a named character vector, not a factor; the LOD in the CSV is the LAST numeric field (R/qtl's shape puts the permutation index first, and a first-field parser reads that as the LOD); and percentiles interpolate at (n-1)p/100, so 100 maxima give 0.6733, not 0.67. bin/monitor-gn4 gains compute-qtl2-perm (nperm 10, judging meta.permutation) so a deploy that loses the R path shows up as a red check rather than a surprise.
8 daysqtl2: name the genotype model, and say what the conversion droppedPjotr
Two engines reading one riset panel can fit different predictors, and the reply now says which one it used and what it cost -- with this panel's own numbers. * `genotype_model: "riself" | "f2"` on /compute/qtl2 (`het` kept as the older spelling: drop/keep). riself is the default: R/qtl's convert2riself coding, the H calls become missing and the HMM re-imputes them. f2 keeps them as a third genotype -- a diagnostic. * The conversion now counts the calls it drops and keeps the count beside the cross (conversion.json), so a reply can say "610684 heterozygous call(s) (12.7 %) were dropped". A cross converted before that file existed measures the count once, on first use, rather than never. * Crossed with the note that already explained the H drop, so the two read together: what riself does, how many calls it cost, and what f2 would give. Documented in the README as "Why R/qtl and R/qtl2 disagree on a riset panel", with the measurements (BXD trait 16627, BXD.8: R/qtl 9.59 at rs33299937; qtl2 riself 3.73; qtl2 f2 9.68), R/qtl's adaptive convert2riself rule quoted from its own source, and qtl2's side: riself has two states, f2 three, and `genril2` -- from qtl2 0.18, "the progeny are inbred lines", cross_info first column the number of generations -- is the honest model for residual heterozygosity and is available in the 0.42 we run. What is missing for it is recorded precisely: a founder-based cross expects individual genotypes coded as founder alleles rather than 1/2/3 (a prototype with numeric codes read every genotype as missing), so founder_geno from the two parental alleles and an allele-coded het remain to be done, with DOL as the counter-example (its H is a collapsed DO, not residual heterozygosity). tests/qtl2.rs: the model is reported (Cross: riself / Cross: f2), the dropped count is in the reply, keeping the calls finds more, and an unknown model is refused rather than ignored.
8 daysqtl2: an option to keep heterozygous calls, to test what dropping them costsPjotr
R/qtl's convert2riself drops the third genotype code (see the note below), and so does this converter -- but qtl2 then never sees the call, the HMM re-imputes it from flanking markers, and on a marker that is 14% H that is a different predictor from the one R/qtl fits. {"group":"BXD","genotype_version":"BXD.8","het":"keep"} -> crosstype "f2" On BXD trait 16627 (BXD.8, 35 phenotyped animals): marker Mb R/qtl qtl2 drop qtl2 keep rsm10000000889 5.652 9.59 3.79 9.98 rs33299937 5.653 9.59 3.73 9.68 rs13476332 6.311 9.59 3.70 ... rsm10000000890 6.312 9.59 3.67 ... So dropping the H calls is what costs qtl2 the peak: kept as a third genotype it recovers R/qtl's number (9.68 against 9.59). The option is a DIAGNOSTIC: an inbred line has no heterozygotes, so an f2-shaped cross is wrong for inference even when it reproduces the other engine -- it is how one finds out that the conversion, not the model, is the difference. The policy is part of the conversion cache key (v3), and the reply says when it is not the default. `het` applies only to a panel this server converts: a shipped control-file cross carries its own coding and is refused rather than silently ignored.
8 dayscompute/qtl2: multiparent crosses, by namePjotr
A multiparent population cannot come from a .geno. That file carries two codes per marker (B/D, A/H/B) -- a collapse, not a founder allele -- and the collapse is not invertible, so DOL's 194 DO animals cannot be fitted as a DO however much reference data appears. What can be is a cross that ships its own qtl2 control file with founder genotypes, so the endpoint now takes one by name: {"rqtl2_cross": "do", "trait_name": "DO::10001 (WBC)", "samples": ...} * resolve_control_file looks in genotype/rqtl2/{name}.json, then {name}/{name}.json (do/do.json), then {name}.json under GENOTYPE_FILES; prepare_control_cross reads the cross's pheno, maps and founder_geno headers without converting anything. It reports crosstype, founders and whether the cross is multiparent, and refuses a multiparent crosstype with no founder_geno rather than silently scanning it as biallelic. * Samples are matched to the cross's own population by name and the strays are counted in notes; rqtl2_cross cannot be combined with group or genotype_file. * The gate is given individuals x founders, because a multiparent scan fits one coefficient per founder allele at every marker. * Autosomes only now applies to shipped crosses as well: the DO's X is left out (369 markers) instead of being fitted without a sex covariate, and the script subsets with cross[, chr = autosomal]. Measured on the DO (WBC, 742 individuals, 8 founders, 6044 autosomal markers): probabilities 174 s / 1.39 GB (cached), LOCO kinship 15 s (cached), scan 21 s. scan1 runs on one core for a multiparent cross because its forked path stalls on allele probabilities here -- 25+ minutes at 8 cores with no CPU consumed, against 21 s serial. Two bugs found by running an actual published trait through it: * do_pmap.csv is in Mb, not the bp qtl2 documents, so every DO peak was reported at 0.0 Mb. The map's unit is now detected from its largest position and printed. * The phenotype file was read with the default whitespace separator, so a label with a space ("DO::10001 (WBC)") split into three fields and read.table stopped with "line 1 did not have 3 elements". Both the pheno and covariate reads are tab-separated now. The R error extraction also joins R's wrapped continuation line, so a complaint reads as a sentence rather than "Error in scan(...) :".
9 dayscompute: R/qtl2 as a third scan, POST /compute/qtl2Pjotr
R/qtl sits between the existing two scans: Haley-Knott on marker genotypes, no kinship. GEMMA is an LMM but biallelic BIMBAM with mean-imputed genotypes. Neither can analyse a multiparent population, and neither models relatedness from the actual genotype data. R/qtl2 does both -- HMM genotype probabilities, then an LMM with leave-one-chromosome-out kinship -- and it is what names the third peak on DOL trait 10003 that R/qtl's scan cannot see (chr16, LOD 4.24, against GEMMA's -log10 p 7.15 there). * src/qtl2.rs converts this deployment's own .geno into a qtl2 cross under QTL2_CACHE_DIR: control.json plus geno/gmap/pmap/pheno, with the genotype-code map, the orientation and the paths written out explicitly. The shipped genotype/rqtl2/ control files cannot be used -- they point outside the data directory, declare no code map (every genotype reads as missing) and no orientation (individuals read as markers) -- and each of those produces a plausible flat scan instead of an error, which is why the conversion is ours. * Heterozygous calls are dropped on a riself panel, as R/qtl's convert2riself does. Left in, qtl2 reads the cross as multiparent and its scan1 takes a path the panel cannot satisfy ("Error in scan1_pg ... : No individuals in common"). * Autosomes only: X needs a sex covariate to be fitted properly and Y/MT degenerate the per-chromosome kinship. GEMMA's LOCO scan is restricted the same way. The note says how many markers were left out. * scripts/qtl2_scan.R caches genotype probabilities and kinship by a hash of the control file *and its directory*; two conversions of the same panel can have identical control files and identical file sizes, and reusing the first one's probabilities silently scans the wrong cross. It also takes the individual names from whichever shape the cross's pheno slot has -- a single-column pheno file leaves it a plain matrix, and the resulting empty phenotype reads as scan1's "No individuals in common". * Rows are marker, chr, pos, lod, so /plot/rqtl draws the reply unchanged -- post the same samples as /compute/rqtl. * tests/qtl2.rs: the fixture scans and its notes name the cross; BXD trait 10001 peaks on chr8 under both scans; refusals carry missing_genotype / invalid_kinship. cost: 2 concurrent scans, gated from RQTL_LARGE_PANEL individuals like R/qtl; a DO cross costs minutes of genotype probabilities, paid once per panel. QTL2_SCRIPT_PATH and QTL2_CACHE_DIR configure it.