| Age | Commit message (Collapse) | Author |
|
POST /compute/qtl2 gains nperm, perm_seed and perm_strata, and the reply
gains meta.permutation {n, seed, significant, suggestive, values} -- the
genome-wide maximum of every permutation, so a histogram needs no second
call. `data` stays the positional rows array on purpose: /plot/rqtl posts
it straight back and the monitor counts its rows, so the null went beside
the rows rather than around them (a generic Envelope::with_meta/Meta::extra,
skip_serializing_if, so no other endpoint's response changes).
Thresholds come from R/qtl2's scan1perm over the same genotype
probabilities, kinship and covariates as the scan, with chr_lengths from
the map -- without those it derives chromosome lengths from marker
positions and the numbers stop being comparable with R/qtl's. The
permutation CSV is written in R/qtl's two-column shape so one parser and
one percentile routine serve both engines: PermutationResult::from_values
is now shared with /compute/rqtl instead of duplicated.
Semantics, each with the note that says it:
* scan1perm shuffles phenotypes across individuals -- right for an
outbred cross, only approximate for a selfing RIL panel, where qtl2perm
is the correct package (not installed here, and not packaged);
* perm_strata takes a case attribute name (sex, Epoch, ...) and permutes
within it, because shuffling across sexes inflates the null;
* fewer than 100 permutations gets a note that the 0.05 quantile is a
quantile of too few values; more than 10 000 is refused, since each
permutation is a whole genome scan;
* a seed is reported always -- without one, two identical requests would
give different thresholds.
Cost, measured on BXD trait 10001 (34 phenotyped, 20 chromosomes, LOCO
kinship, 4 cores): 20 permutations 0.6 s, 100 in 2.2 s, 500 in 14 s,
1000 in 24 s through the API; 0.022-0.03 s per permutation. Thresholds
converge as expected (0.05: 4.331 at 100 and 200 permutations, 4.374 at
1000 on the raw probe) and are reproducible: the same seed gives the same
numbers on 1 and 4 cores.
Four things that only surfaced by running it, all commented where they
bit: optparse keeps the dash (--perm-out is opt[["perm-out"]], so
opt$perm_out was always NULL and the file landed somewhere the caller did
not look); perm_strata must be a named character vector, not a factor;
the LOD in the CSV is the LAST numeric field (R/qtl's shape puts the
permutation index first, and a first-field parser reads that as the LOD);
and percentiles interpolate at (n-1)p/100, so 100 maxima give 0.6733, not
0.67.
bin/monitor-gn4 gains compute-qtl2-perm (nperm 10, judging
meta.permutation) so a deploy that loses the R path shows up as a red
check rather than a surprise.
|
|
A caller that knows only /version should be able to reach anything by
following links. Until now `next` was thin and uneven: the genotype
subtree offered none at all, the group record offered one, /version
offered nothing, and two links it did offer could not be followed.
* suggest:: now builds the graph: root_links (groups, species,
mapping-methods, openapi), genotype_version_links (frequencies,
marker names, a marker's location and tally, the group, the scanning
actions), unlisted_genotype_links, marker_links (location, tally,
marker list, panel tally, PheWAS at that marker), group_links and
trait_links. Envelope::with_links / with_links_except add them,
deduplicating and never suggesting the response the caller is
already reading.
* Wired through the genotype surface (listing, version detail, markers,
location), both frequencies routes, the compute endpoints, groups,
traits, both value routes, species, datasets, publications,
accessions, search, mapping methods, phewas, correlation and
/version.
* Fixed what was not followable, because `next` promises a GET that
returns 200 and a client is the one that finds out otherwise:
- /groups/{g}/phewas without a locus is a 422. Bare PheWAS links are
gone; comput endpoints now link ?locus=<their strongest marker>
(suggest::peak_marker_of, computed before the rows move).
- /datasets and /publications are not routes; root_links offers the
listings that exist (/groups, /species, /mapping-methods) and
/openapi.json.
- /search?q= with an empty query is a 400, so it is not a link.
- the correlation endpoint's "/compute/correlation" was a POST in
`next`; it is in `actions` now.
- marker_links pointed at /markers/{marker}, which is not a route;
it points at /markers/{marker}/location.
tests/links.rs walks the graph two hops from a dozen seeds and fails on
any link that does not answer 200, plus a check that a tally leads to the
marker list and to a marker's location, and a location to that marker's
tally and its PheWAS.
|
|
Fetching one trait's value per strain is usually the step before mapping
that trait, and the API can do the mapping. Those calls are POSTs with a
body, so they cannot go in `next` -- a `next` link is a GET that returns
200, which is what makes `next` safe to follow blindly and what
tests/envelope.rs relies on. Hence a fifth envelope member, `actions`,
holding method + href + description + body, and omitted entirely when
there is nothing to suggest (most responses keep the four-member shape).
The three per-trait values endpoints now carry a `POST /compute/rqtl`
and a `POST /compute/gemma` suggestion with the values already in the
body, in the response's own sample order. Genotype/SNP paths stay
bracketed placeholders: they must exist on the server, which the server
cannot know on the caller's behalf.
The instruction page says, in the same breath, never to run R/qtl or
GEMMA locally -- a local scan would use genotypes that are not the
panel's and would disagree with GeneNetwork's own results -- and that
results are cached, so repeat calls are cheap (measured: ~2 s cached
against ~20 s cold on the BXD fixtures).
Also fixes a flake the new test exposed rather than caused:
config::tests read and write process-global env vars and Config::load()
reads all of them, so the RAVANAN_* tests could fail
search_engine_keys_follow_env_over_default_layering on an unrelated
assertion. One module-level lock serialises them.
* src/response.rs (SuggestedCall, Envelope::actions/with_action/
with_actions): the new member, omitted when empty; unit tests for both.
* src/suggest.rs: new. Builds the two suggestions for either value type
(db::TraitValue, db::ProbeSetValue), with its own unit tests.
* src/routes/{traits,groups,datasets}.rs: the three values handlers.
* tests/compute_suggestions.rs: new. Suggestions present with values in
order, on all three routes; absent elsewhere; `next` still GET-only and
still returns 200.
* src/config.rs (tests): env_guard, taken by the 11 config tests.
* README.md (Success responses): the `actions` member, with an example.
* docs/www/ai-instruction.md (4a): never compute locally, why, caching,
and how to use the suggested bodies.
* docs/www/operator-checklist.md: the caching numbers.
|
|
So a caller of /compute/gemma(/ravanan) or /compute/rqtl doesn't need
to count data's rows or columns.len() itself. with_columns now takes
the row count too and derives num_cols from the column list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
|
Cuts repeated key names out of large /compute/gemma(/ravanan) responses:
data is now a list of positional row arrays instead of one keyed object
per row, with the column names moved to a new meta.columns field shared
with any other endpoint that wants the same shape.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
|
Breaking: every success body is now {"data", "meta": {"state",
"took_ms", "request_id"}, "notes", "next"}; clients read `data`. Only
`/` (plain text) and `/openapi.json` are unwrapped.
* src/response.rs (Envelope, Meta, State, Ctx, RequestInfo): handlers
take Ctx first and build the response with ctx.envelope(data), so
timing and the request id are never filled in by hand. `state` is
`complete` or `partial`; notes are server-authored only; `next` are
concrete relative hrefs (path_segment percent-encodes names).
* src/routes/*.rs, src/lib.rs (version): every handler returns
Envelope<..> with next links; /search is `partial` when the locus
lookup fails or a sorted 1000-hit window may be truncated, and
/publications/{pubmed_id}/traits when datasets were left out for
authorization.
* src/error.rs (ApiError): add `took_ms`, an RFC 7807 extension member.
* src/auth.rs (AuthToken): the 401 now carries request_id and took_ms
(it never had a request_id); Ctx stores RequestInfo in the request
extensions for it.
* tests: common::data / envelope_data parse every success body through
the envelope; tests/envelope.rs and test/python/test_envelope.py check
the shape, errors, and that every `next` link resolves;
tests/envelope_partial.rs checks the `partial` search without a
database; tests/openapi.rs fails if a JSON success response is not an
Envelope_* schema.
* README.md, docs/SKILL.md, docs/gn-auth.md, RELEASE_NOTES.md: document
the envelope and took_ms on errors.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Lets OpenAPI document the success envelope as a first-class schema
(Envelope<SearchHit> as a response body) instead of an opaque blob,
so generated specs stay useful to LLM callers.
|
|
Line-wrapping and canonical import ordering only (verified with
git diff -w). Establishes rustfmt's output as the canonical style
before the search/correlation work lands.
|
|
* src/config.rs: New module. Layered configuration,
defaults -> GN4_CONF -> GN4_SECRETS -> environment, mirroring
GeneNetwork3's SQL_URI convention so the two services can share
configuration. Missing config files are ignored.
* src/response.rs: New module. The descriptive success envelope
(data + next actions) from the foundation design note
(topics/gn4/gn4-api-foundation.gmi). Not yet adopted by existing
endpoints; that is a deliberate wire-format change to agree on
before it ships.
* src/routes/mod.rs, species.rs, groups.rs: Move the species and
groups handlers out of lib.rs into one module per domain, per the
foundation structure. Handlers are pub; lib.rs just registers them.
* src/lib.rs (app): Register the moved routes. hello_world and
version predate the domain split and stay as top-level utility
routes.
* src/db.rs (sql_uri): Resolve through config::Config instead of
reading the environment directly.
* src/main.rs: Bind the port from config::Config.
|