about summary refs log tree commit diff
path: root/src/response.rs
AgeCommit message (Collapse)Author
2 daysqtl2: permutations for the LOD thresholdsPjotr
POST /compute/qtl2 gains nperm, perm_seed and perm_strata, and the reply gains meta.permutation {n, seed, significant, suggestive, values} -- the genome-wide maximum of every permutation, so a histogram needs no second call. `data` stays the positional rows array on purpose: /plot/rqtl posts it straight back and the monitor counts its rows, so the null went beside the rows rather than around them (a generic Envelope::with_meta/Meta::extra, skip_serializing_if, so no other endpoint's response changes). Thresholds come from R/qtl2's scan1perm over the same genotype probabilities, kinship and covariates as the scan, with chr_lengths from the map -- without those it derives chromosome lengths from marker positions and the numbers stop being comparable with R/qtl's. The permutation CSV is written in R/qtl's two-column shape so one parser and one percentile routine serve both engines: PermutationResult::from_values is now shared with /compute/rqtl instead of duplicated. Semantics, each with the note that says it: * scan1perm shuffles phenotypes across individuals -- right for an outbred cross, only approximate for a selfing RIL panel, where qtl2perm is the correct package (not installed here, and not packaged); * perm_strata takes a case attribute name (sex, Epoch, ...) and permutes within it, because shuffling across sexes inflates the null; * fewer than 100 permutations gets a note that the 0.05 quantile is a quantile of too few values; more than 10 000 is refused, since each permutation is a whole genome scan; * a seed is reported always -- without one, two identical requests would give different thresholds. Cost, measured on BXD trait 10001 (34 phenotyped, 20 chromosomes, LOCO kinship, 4 cores): 20 permutations 0.6 s, 100 in 2.2 s, 500 in 14 s, 1000 in 24 s through the API; 0.022-0.03 s per permutation. Thresholds converge as expected (0.05: 4.331 at 100 and 200 permutations, 4.374 at 1000 on the raw probe) and are reproducible: the same seed gives the same numbers on 1 and 4 cores. Four things that only surfaced by running it, all commented where they bit: optparse keeps the dash (--perm-out is opt[["perm-out"]], so opt$perm_out was always NULL and the file landed somewhere the caller did not look); perm_strata must be a named character vector, not a factor; the LOD in the CSV is the LAST numeric field (R/qtl's shape puts the permutation index first, and a first-field parser reads that as the LOD); and percentiles interpolate at (n-1)p/100, so 100 maxima give 0.6733, not 0.67. bin/monitor-gn4 gains compute-qtl2-perm (nperm 10, judging meta.permutation) so a deploy that loses the R path shows up as a red check rather than a surprise.
8 daysapi: every response suggests where to go nextPjotr
A caller that knows only /version should be able to reach anything by following links. Until now `next` was thin and uneven: the genotype subtree offered none at all, the group record offered one, /version offered nothing, and two links it did offer could not be followed. * suggest:: now builds the graph: root_links (groups, species, mapping-methods, openapi), genotype_version_links (frequencies, marker names, a marker's location and tally, the group, the scanning actions), unlisted_genotype_links, marker_links (location, tally, marker list, panel tally, PheWAS at that marker), group_links and trait_links. Envelope::with_links / with_links_except add them, deduplicating and never suggesting the response the caller is already reading. * Wired through the genotype surface (listing, version detail, markers, location), both frequencies routes, the compute endpoints, groups, traits, both value routes, species, datasets, publications, accessions, search, mapping methods, phewas, correlation and /version. * Fixed what was not followable, because `next` promises a GET that returns 200 and a client is the one that finds out otherwise: - /groups/{g}/phewas without a locus is a 422. Bare PheWAS links are gone; comput endpoints now link ?locus=<their strongest marker> (suggest::peak_marker_of, computed before the rows move). - /datasets and /publications are not routes; root_links offers the listings that exist (/groups, /species, /mapping-methods) and /openapi.json. - /search?q= with an empty query is a 400, so it is not a link. - the correlation endpoint's "/compute/correlation" was a POST in `next`; it is in `actions` now. - marker_links pointed at /markers/{marker}, which is not a route; it points at /markers/{marker}/location. tests/links.rs walks the graph two hops from a dozen seeds and fails on any link that does not answer 200, plus a check that a tally leads to the marker list and to a marker's location, and a location to that marker's tally and its PheWAS.
11 daysvalues: suggest R/qtl and GEMMA in the envelope's `actions`Pjotr
Fetching one trait's value per strain is usually the step before mapping that trait, and the API can do the mapping. Those calls are POSTs with a body, so they cannot go in `next` -- a `next` link is a GET that returns 200, which is what makes `next` safe to follow blindly and what tests/envelope.rs relies on. Hence a fifth envelope member, `actions`, holding method + href + description + body, and omitted entirely when there is nothing to suggest (most responses keep the four-member shape). The three per-trait values endpoints now carry a `POST /compute/rqtl` and a `POST /compute/gemma` suggestion with the values already in the body, in the response's own sample order. Genotype/SNP paths stay bracketed placeholders: they must exist on the server, which the server cannot know on the caller's behalf. The instruction page says, in the same breath, never to run R/qtl or GEMMA locally -- a local scan would use genotypes that are not the panel's and would disagree with GeneNetwork's own results -- and that results are cached, so repeat calls are cheap (measured: ~2 s cached against ~20 s cold on the BXD fixtures). Also fixes a flake the new test exposed rather than caused: config::tests read and write process-global env vars and Config::load() reads all of them, so the RAVANAN_* tests could fail search_engine_keys_follow_env_over_default_layering on an unrelated assertion. One module-level lock serialises them. * src/response.rs (SuggestedCall, Envelope::actions/with_action/ with_actions): the new member, omitted when empty; unit tests for both. * src/suggest.rs: new. Builds the two suggestions for either value type (db::TraitValue, db::ProbeSetValue), with its own unit tests. * src/routes/{traits,groups,datasets}.rs: the three values handlers. * tests/compute_suggestions.rs: new. Suggestions present with values in order, on all three routes; absent elsewhere; `next` still GET-only and still returns 200. * src/config.rs (tests): env_guard, taken by the 11 config tests. * README.md (Success responses): the `actions` member, with an example. * docs/www/ai-instruction.md (4a): never compute locally, why, caching, and how to use the suggested bodies. * docs/www/operator-checklist.md: the caching numbers.
2026-10-15response: add meta.num_rows/num_cols alongside meta.columnsPjotr
So a caller of /compute/gemma(/ravanan) or /compute/rqtl doesn't need to count data's rows or columns.len() itself. with_columns now takes the row count too and derives num_cols from the column list. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-15gemma: return .assoc.txt rows positionally, columns in meta.columnsPjotr
Cuts repeated key names out of large /compute/gemma(/ravanan) responses: data is now a list of positional row arrays instead of one keyed object per row, with the column names moved to a new meta.columns field shared with any other endpoint that wants the same shape. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08* src/response.rs: wrap every success response in one envelope.Pjotr
Breaking: every success body is now {"data", "meta": {"state", "took_ms", "request_id"}, "notes", "next"}; clients read `data`. Only `/` (plain text) and `/openapi.json` are unwrapped. * src/response.rs (Envelope, Meta, State, Ctx, RequestInfo): handlers take Ctx first and build the response with ctx.envelope(data), so timing and the request id are never filled in by hand. `state` is `complete` or `partial`; notes are server-authored only; `next` are concrete relative hrefs (path_segment percent-encodes names). * src/routes/*.rs, src/lib.rs (version): every handler returns Envelope<..> with next links; /search is `partial` when the locus lookup fails or a sorted 1000-hit window may be truncated, and /publications/{pubmed_id}/traits when datasets were left out for authorization. * src/error.rs (ApiError): add `took_ms`, an RFC 7807 extension member. * src/auth.rs (AuthToken): the 401 now carries request_id and took_ms (it never had a request_id); Ctx stores RequestInfo in the request extensions for it. * tests: common::data / envelope_data parse every success body through the envelope; tests/envelope.rs and test/python/test_envelope.py check the shape, errors, and that every `next` link resolves; tests/envelope_partial.rs checks the `partial` search without a database; tests/openapi.rs fails if a JSON success response is not an Envelope_* schema. * README.md, docs/SKILL.md, docs/gn-auth.md, RELEASE_NOTES.md: document the envelope and took_ms on errors. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-05* src/response.rs (Envelope, NextAction): derive utoipa::ToSchemaAlexander_Kabui
Lets OpenAPI document the success envelope as a first-class schema (Envelope<SearchHit> as a response body) instead of an opaque blob, so generated specs stay useful to LLM callers.
2026-10-04Format all Rust sources with cargo fmt (no behavior change)Alexander_Kabui
Line-wrapping and canonical import ordering only (verified with git diff -w). Establishes rustfmt's output as the canonical style before the search/correlation work lands.
2026-10-03Add foundation structure: config layering, response envelope, routes modules.Alexander_Kabui
* src/config.rs: New module. Layered configuration, defaults -> GN4_CONF -> GN4_SECRETS -> environment, mirroring GeneNetwork3's SQL_URI convention so the two services can share configuration. Missing config files are ignored. * src/response.rs: New module. The descriptive success envelope (data + next actions) from the foundation design note (topics/gn4/gn4-api-foundation.gmi). Not yet adopted by existing endpoints; that is a deliberate wire-format change to agree on before it ships. * src/routes/mod.rs, species.rs, groups.rs: Move the species and groups handlers out of lib.rs into one module per domain, per the foundation structure. Handlers are pub; lib.rs just registers them. * src/lib.rs (app): Register the moved routes. hello_world and version predate the domain split and stay as top-level utility routes. * src/db.rs (sql_uri): Resolve through config::Config instead of reading the environment directly. * src/main.rs: Bind the port from config::Config.