about summary refs log tree commit diff
path: root/tests/genotypes.rs
AgeCommit message (Collapse)Author
8 daysgenotypes: call frequencies, marker names, and a location per markerPjotr
Two questions the genotype surface could not answer: what a panel is actually called as, and which markers it holds. The listing gives counts and the compute endpoints give scans; a panel whose calls are not the codes its header declares, or a marker that is 99% missing, stayed invisible until a scan came back flat -- and then the search for the cause started at the wrong end. * GET /groups/{group}/genotypes/{version}/frequencies counts the calls in the version's .geno and returns them as call -> frequency ({"B": 0.4889, "D": 0.492109, "H": 0.018991}). Frequencies are over the calls that are present, so they sum to 1; missing codes (U, N, NA, an empty cell) are counted separately and reported in notes, because the missing rate is the diagnosis. * A probability-valued file is binned 0.05 wide (twenty bins over [0, 1], keys are the bin edges), since 0.4187 and 0.4191 are the same statement about a marker; ?bin= sets another width. A file with a few distinct numbers (0, 0.5, 1 for a biallelic dosage) is reported exactly -- binning three values hides what is being looked at. * .../markers/{marker}/frequencies is one locus, and ?marker= on the panel route is the same call. Counting goes through an interner, so cost is one hash lookup per cell and one allocation per distinct call; a .geno over 256 MB is refused (413 genotype_too_large) rather than read, which is what HSNIH-Palmer_true (3.7 GB, 134927 markers x 6147 individuals) demanded. * The version's /markers now lists marker NAMES only, one column, and .../markers/{marker}/location returns {"name","chr","cM","Mb"} as named fields: a page of names stays small, and a position is a lookup rather than a table the client must hold. The same size guard applies, so an oversized panel's listing is a 413 in 3 ms. * Files no group lists -- a panel in preparation, a fixture, the DO's do/ cross -- live at /groups/unlisted/genotypes/{name}/... with the same four calls. `unlisted` is a namespace, not a group: only its genotype subtree answers. One prefix for the whole genotype surface beats a second spelling for the same tally, so the earlier /genotypes/{name} spelling is gone. * genotypes::resolve_named (moved out of the frequencies module, since it is genotype metadata) resolves a version, a .geno name, or a path relative to the genotype files directory; MAX_SCAN_BYTES is shared by every whole-file read. Tests limited to the feature: genotypes 3 (names-only columns, paging, location fields, marker_not_found, unlisted parity, 413), frequencies 5, frequencies unit 7.
11 daysgenotypes: expose the metadata, never the genotype callsPjotr
The listing named versions but not files; a caller could pick one and still not see what was in it. Three token-gated routes now describe a version completely, short of the matrix: GET /groups/{group}/genotypes versions, files, sizes, current GET /groups/{group}/genotypes/{version} file name, header labels, individuals, strains, chromosomes GET /groups/{group}/genotypes/{version}/markers markers (name, chr, cM, Mb), paged The detail reads the file's own `@`-labels (type, mat, pat, het, unk, scale) and its panel in column order, and reports both the whole panel and the strains a mapping uses after the parents/F1s rule -- so a caller can check a request before sending it and interpret a result after. Markers come back as positional rows with `meta.columns`, paged with `?page`/`?limit` and filterable with `?chr`, following /compute/gemma's shape rather than inventing one. The genotype calls stay on the server: file_meta() reads the matrix to skip it, and nothing returns a cell. * src/genotypes.rs (Marker, FileMeta, file_meta; GenotypeFile::file): header labels, individuals and marker positions; unit test. * src/routes/genotypes.rs: the two new handlers, token-gated like the listing, plus `file` in the listing and counts in the detail. * tests/genotypes.rs: the listing carries file names but no paths. * README.md, docs/genotypes.md, docs/www/ai-instruction.md: what the three routes give, and that the matrix is not among them.
11 daysgenotypes: list a group's genotype files, and map by group + versionPjotr
Until now /compute/{rqtl,gemma} took server paths, so the caller had to know the deployment's layout, pick a version, and expand a trait's values onto the panel in the file's order -- none of which the API told it. The listing endpoint and the group-scoped request remove all three: GET /groups/{group}/genotypes versions, newest first, no paths POST /compute/rqtl {group, [genotype_version], samples} POST /compute/gemma {group, [genotype_version], samples} The resolver (src/genotypes.rs) applies GeneNetwork2's rule -- the group's genotype/<Group>.json `genofile` list is newest-first and its first entry is current; groups with no JSON (HET3-ITP) fall back to the highest version number, and the listing's notes say which happened -- reads the panel from the .geno header the way GN3 does, and excludes parents and F1s by default (GN2's `incparentsf1` off, GN3's `get_sample_list(inc_par=False, inc_f1=False)`). Excluding them has to subset the genotype matrix, not null-fill the values: a null keeps an individual out of the fit but not out of GEMMA's kinship or R/qtl's cross. So `prepare` writes a panel-subset file under the engine's cache directory, keyed by panel, and reuses it. Nothing in the listing or the mapping response carries a path: the endpoint reports versions and sizes, and `notes` names the resolved version, the panel size and how many values aligned. Explicit `genotype_file`/`snps_file` remain as an override for files outside the layout, and the two request shapes are mutually exclusive (422). * src/genotypes.rs: new. Discovery, resolution, panel parse, parent/F1 exclusion, subset preparation, name alignment; 10 unit tests. * src/routes/genotypes.rs: new endpoint, with a test that its list has no path or file name in it. * src/routes/{rqtl,gemma}.rs: group + genotype_version + include_parents_f1; named `samples` aligned server-side; notes record what resolved. gemma also accepts named samples where it used to insist on positional trait_values. * src/suggest.rs: suggestions from /groups/{group}/traits/{id}/values now carry `group` and no placeholders; the dataset-scoped ones keep the placeholder and say to add the group. * src/routes/groups.rs: the group record points at /genotypes. * src/config.rs: GENOTYPE_FILES (default the gn4-data directory). * tests/genotypes.rs: new integration test (skips without the directory). * README.md, docs/genotypes.md, docs/www/*: the flow is now "ask for the versions, map by group" -- measured runs, the parent/F1 rule, the subsetting requirement, and the restart list unchanged.