| Age | Commit message (Collapse) | Author |
|
Two questions the genotype surface could not answer: what a panel is
actually called as, and which markers it holds. The listing gives counts
and the compute endpoints give scans; a panel whose calls are not the
codes its header declares, or a marker that is 99% missing, stayed
invisible until a scan came back flat -- and then the search for the
cause started at the wrong end.
* GET /groups/{group}/genotypes/{version}/frequencies counts the calls
in the version's .geno and returns them as call -> frequency
({"B": 0.4889, "D": 0.492109, "H": 0.018991}). Frequencies are over
the calls that are present, so they sum to 1; missing codes (U, N,
NA, an empty cell) are counted separately and reported in notes,
because the missing rate is the diagnosis.
* A probability-valued file is binned 0.05 wide (twenty bins over
[0, 1], keys are the bin edges), since 0.4187 and 0.4191 are the same
statement about a marker; ?bin= sets another width. A file with a few
distinct numbers (0, 0.5, 1 for a biallelic dosage) is reported
exactly -- binning three values hides what is being looked at.
* .../markers/{marker}/frequencies is one locus, and ?marker= on the
panel route is the same call. Counting goes through an interner, so
cost is one hash lookup per cell and one allocation per distinct
call; a .geno over 256 MB is refused (413 genotype_too_large) rather
than read, which is what HSNIH-Palmer_true (3.7 GB, 134927 markers x
6147 individuals) demanded.
* The version's /markers now lists marker NAMES only, one column, and
.../markers/{marker}/location returns {"name","chr","cM","Mb"} as
named fields: a page of names stays small, and a position is a
lookup rather than a table the client must hold. The same size guard
applies, so an oversized panel's listing is a 413 in 3 ms.
* Files no group lists -- a panel in preparation, a fixture, the DO's
do/ cross -- live at /groups/unlisted/genotypes/{name}/... with the
same four calls. `unlisted` is a namespace, not a group: only its
genotype subtree answers. One prefix for the whole genotype surface
beats a second spelling for the same tally, so the earlier
/genotypes/{name} spelling is gone.
* genotypes::resolve_named (moved out of the frequencies module, since
it is genotype metadata) resolves a version, a .geno name, or a path
relative to the genotype files directory; MAX_SCAN_BYTES is shared by
every whole-file read.
Tests limited to the feature: genotypes 3 (names-only columns, paging,
location fields, marker_not_found, unlisted parity, 413), frequencies 5,
frequencies unit 7.
|
|
The listing named versions but not files; a caller could pick one and
still not see what was in it. Three token-gated routes now describe a
version completely, short of the matrix:
GET /groups/{group}/genotypes versions, files, sizes, current
GET /groups/{group}/genotypes/{version} file name, header labels,
individuals, strains, chromosomes
GET /groups/{group}/genotypes/{version}/markers markers (name, chr, cM, Mb), paged
The detail reads the file's own `@`-labels (type, mat, pat, het, unk,
scale) and its panel in column order, and reports both the whole panel
and the strains a mapping uses after the parents/F1s rule -- so a caller
can check a request before sending it and interpret a result after.
Markers come back as positional rows with `meta.columns`, paged with
`?page`/`?limit` and filterable with `?chr`, following /compute/gemma's
shape rather than inventing one.
The genotype calls stay on the server: file_meta() reads the matrix to
skip it, and nothing returns a cell.
* src/genotypes.rs (Marker, FileMeta, file_meta; GenotypeFile::file):
header labels, individuals and marker positions; unit test.
* src/routes/genotypes.rs: the two new handlers, token-gated like the
listing, plus `file` in the listing and counts in the detail.
* tests/genotypes.rs: the listing carries file names but no paths.
* README.md, docs/genotypes.md, docs/www/ai-instruction.md: what the
three routes give, and that the matrix is not among them.
|
|
Until now /compute/{rqtl,gemma} took server paths, so the caller had to
know the deployment's layout, pick a version, and expand a trait's values
onto the panel in the file's order -- none of which the API told it. The
listing endpoint and the group-scoped request remove all three:
GET /groups/{group}/genotypes versions, newest first, no paths
POST /compute/rqtl {group, [genotype_version], samples}
POST /compute/gemma {group, [genotype_version], samples}
The resolver (src/genotypes.rs) applies GeneNetwork2's rule -- the
group's genotype/<Group>.json `genofile` list is newest-first and its
first entry is current; groups with no JSON (HET3-ITP) fall back to the
highest version number, and the listing's notes say which happened --
reads the panel from the .geno header the way GN3 does, and excludes
parents and F1s by default (GN2's `incparentsf1` off, GN3's
`get_sample_list(inc_par=False, inc_f1=False)`).
Excluding them has to subset the genotype matrix, not null-fill the
values: a null keeps an individual out of the fit but not out of GEMMA's
kinship or R/qtl's cross. So `prepare` writes a panel-subset file under
the engine's cache directory, keyed by panel, and reuses it.
Nothing in the listing or the mapping response carries a path: the
endpoint reports versions and sizes, and `notes` names the resolved
version, the panel size and how many values aligned. Explicit
`genotype_file`/`snps_file` remain as an override for files outside the
layout, and the two request shapes are mutually exclusive (422).
* src/genotypes.rs: new. Discovery, resolution, panel parse, parent/F1
exclusion, subset preparation, name alignment; 10 unit tests.
* src/routes/genotypes.rs: new endpoint, with a test that its list has no
path or file name in it.
* src/routes/{rqtl,gemma}.rs: group + genotype_version +
include_parents_f1; named `samples` aligned server-side; notes record
what resolved. gemma also accepts named samples where it used to insist
on positional trait_values.
* src/suggest.rs: suggestions from /groups/{group}/traits/{id}/values now
carry `group` and no placeholders; the dataset-scoped ones keep the
placeholder and say to add the group.
* src/routes/groups.rs: the group record points at /genotypes.
* src/config.rs: GENOTYPE_FILES (default the gn4-data directory).
* tests/genotypes.rs: new integration test (skips without the directory).
* README.md, docs/genotypes.md, docs/www/*: the flow is now "ask for the
versions, map by group" -- measured runs, the parent/F1 rule, the
subsetting requirement, and the restart list unchanged.
|