| Age | Commit message (Collapse) | Author |
|
tests/envelope.rs follows every href in every reply, and running it end to
end found six that were not addresses:
- /species, /species/{name}/datasets and /groups/{group}/datasets offered
/species/{species} and /datasets/{dataset} -- templates, which 404 when
followed. Each now points at a real row from the listing in hand, and
the description says any other name can go in its place.
- /datasets/{name} offered /datasets/{dataset}/probesets/{probeset}: the
listing it already links is the callable one, so the shape moved into
that link's description.
- /traits/{dataset}/{trait}/values offered a GET
/traits/{ds}/{trait}/correlation, which does not exist: correlation is
POST /compute/correlation and that reply already carries it in
"actions". Removed, with the reason written where it was.
- the same reply built its /groups/... links from the *dataset* name.
BXDPublish's traits live in group BXD, and /groups/BXDPublish/phewas is
a 404 (no genotype files for it). db::get_group_name resolves the group
from InbredSet.Id and the links use it; when a dataset has no group the
group links are simply absent.
- /groups/{g}/phewas with no parameters is a 422 ("give either locus, or
chr and mb"), so that link is now a shape with placeholders.
Placeholders stay where the caller must supply a value
(.../markers/{marker}/classes?trait={trait_id}), but the crawl now skips
them only when the description says what to fill in -- otherwise a client
reading the next list cannot tell a call from a template.
Two stale expectations alongside: /version carries the root links now (it
is the natural first call), and /search's results put its own follow-ups
first and the root links after them.
Tests are honest about their environment too: links skips when no search
index is configured (an empty XAPIAN_DB_PATH answers 502), and the
envelope crawl exempts /openapi.json, the API document, which is
deliberately not enveloped.
|
|
DOL (200 individuals, 112 913 markers, one genotype version) failed on
R/qtl with "There are missing chromosome IDs" and on GEMMA with a bare
wrapper bail-out. Two unrelated causes, both in our own code.
**R/qtl: a genotype code collided with a chromosome name.** DOL's header
says `@unk:X` -- X is the code for a missing genotype -- and the wrapper
passed that to `read.csv(na.strings=...)` for the *whole* file, so every
marker on the X chromosome lost its chromosome ID and `read.cross.csv`
refused the map. The unknown code now applies to the genotype columns
only. DOL scans: 112 913 markers in 42 s, where before it never got past
reading the file. Same fix in scripts/qtl_fitqtl.R, which shares the read.
**GEMMA: its leave-one-chromosome-out run cannot fit the Y.** With LOCO the
DOL panel is scanned per chromosome, and the Y run dies inside BLAS --
`Enforce failed for N>0 in src/fastblas.cpp ... fast_cblas_dgemm` -- taking
the whole request with it (a 502 whose detail is the wrapper's banner).
The mitochondrial chromosome has the same shape of problem. `prepare` for
the GEMMA engine now writes the bimbam pair without the Y and M markers
(cached beside the individual subset, line-aligned so the geno and snps
stay in step), and the note says what was left out: "46 Y/M markers left
out (GEMMA's LOCO cannot fit them)". DOL maps: 106 515 rows in 49 s.
**And the four tests that had been failing since before this work**, all
stale expectations rather than product bugs:
* `benchmark_questions` pinned two traits to "page 2" of a 100-per-page
search when they rank ~190th and ~219th of 246 -- one re-index pushes
them over. It now walks up to three pages and asserts presence and LOD,
not the page; and the helper `search()` pages, because a benchmark row
can name a trait that ranks outside the first hundred of its own query.
* `no_db_mode_degrades_gracefully` asserted exactly one hit for "alcohol"
from the index alone. The index grew; the behaviour under test is that
the search still answers, within the limit, with the database-backed
fields null. That is what it asserts now.
* `search_proxy_scenarios` and `search_parity_replay` never pinned the
engine they test, so an ambient `SEARCH_ENGINE=native` made the proxy
scenarios answer from the local index: a 200 where a 422 was expected,
and "expected 2 upstream calls, got 0" for every replayed query. Both now
set `SEARCH_ENGINE=forward`.
* src/genotypes.rs: `DEGENERATE_CHROMOSOMES`, `drop_degenerate_chromosomes`,
`Prepared.markers_dropped`.
* src/routes/gemma.rs: also the 500 that `eb5320d` introduced -- counting
parents/F1s from `panel - keep` instead of subtracting the exclusion
*set*, whose length can exceed the panel (subtract overflow).
* scripts/rqtl_wrapper.R, scripts/qtl_fitqtl.R: na.strings scoped to the
genotype columns.
* tests/{benchmark,no_db,search,search_parity}.rs: as above.
Verified through the API on DOL trait 10003: R/qtl 112 913 rows, GEMMA
106 515 rows. gemma/rqtl/fitqtl suites green (14, 10, 4).
|
|
Pure formatting sweep (import ordering, line wrapping); the tree
had accumulated rustfmt drift. Separated from any semantic change
so it stays reviewable and revertable on its own.
|
|
Breaking: every success body is now {"data", "meta": {"state",
"took_ms", "request_id"}, "notes", "next"}; clients read `data`. Only
`/` (plain text) and `/openapi.json` are unwrapped.
* src/response.rs (Envelope, Meta, State, Ctx, RequestInfo): handlers
take Ctx first and build the response with ctx.envelope(data), so
timing and the request id are never filled in by hand. `state` is
`complete` or `partial`; notes are server-authored only; `next` are
concrete relative hrefs (path_segment percent-encodes names).
* src/routes/*.rs, src/lib.rs (version): every handler returns
Envelope<..> with next links; /search is `partial` when the locus
lookup fails or a sorted 1000-hit window may be truncated, and
/publications/{pubmed_id}/traits when datasets were left out for
authorization.
* src/error.rs (ApiError): add `took_ms`, an RFC 7807 extension member.
* src/auth.rs (AuthToken): the 401 now carries request_id and took_ms
(it never had a request_id); Ctx stores RequestInfo in the request
extensions for it.
* tests: common::data / envelope_data parse every success body through
the envelope; tests/envelope.rs and test/python/test_envelope.py check
the shape, errors, and that every `next` link resolves;
tests/envelope_partial.rs checks the `partial` search without a
database; tests/openapi.rs fails if a JSON success response is not an
Envelope_* schema.
* README.md, docs/SKILL.md, docs/gn-auth.md, RELEASE_NOTES.md: document
the envelope and took_ms on errors.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Lists the published traits (across groups) a publication reports.
Datasets the caller may not view are left out, using the same check as
GET /traits/{dataset} (traits.rs is_authorised is now pub(crate)).
* src/db.rs (TraitInfo, trait_info_sql): TraitInfo gains `dataset`,
`abbreviation` (Phenotype.Post_publication_abbreviation, else Pre) and
`url` (the GeneNetwork show_trait page). The three TraitInfo queries
now share one SELECT. New get_group_traits_by_abbreviation and
get_publication_traits.
* src/routes/groups.rs (group_traits): add ?abbreviation= filter.
* src/db.rs (Group), src/routes/groups.rs (groups): groups carry
`family` and `species`; add family, species and q filters.
* src/routes/search.rs (search): add `page` (gn3's page unsorted, a
page of the ranked list with `sort`); declare `pubmed_id`; hits gain
`abbreviation` next to `locus` (get_publish_annotations).
* src/db.rs (LRS_PER_LOD): LOD = LRS / 4.604, the divisor the
benchmark's RDF data uses (was 4.61).
* tests, test/python, test/data/benchmark_search.json, README.md: cover
all of the above; tests/benchmark.rs now also answers rows 11, 16, 25,
30, 34 and 37 of small_benchmark.csv, and compares LOD exactly.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Search hits gain `lod` (lrs / 4.61) and `locus` (the peak marker name,
looked up in PublishXRef in one batched query; null when the database
is unavailable). `sort=lrs|lod` with `order=asc|desc` ranks hits: gn3
is asked for its first 1000 matches, those are ranked, then `limit`
applies.
* src/db.rs (LRS_PER_LOD, lod, get_publish_loci): add the shared
LRS-to-LOD constant and the locus lookup; TraitInfo and ProbeSetInfo
gain `lod`, computed in SQL from the same constant.
* tests/search.rs, test/python/test_search.py: add the "highest LOD,
locus marker and additive effect for serotonin in BXD male midbrain"
scenario over a recorded production fixture, plus sort/order tests.
* tests/group_trait_info.rs, tests/dataset_probeset_info.rs and their
Python mirrors: assert lod (and locus for trait 17985).
* README.md (Endpoints): document sort, order, lod and locus.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
- src/routes/search.rs: AuthToken per correlation.rs pattern (+ Unauthorized).
- tests/search.rs: stub /auth/public-jwks + locally-signed token; scenarios
run offline again (multi_thread runtime: the blocking ureq JWKS fetch
starved the single-thread stub); add invalid-token 401 case.
- test/python/test_search.py: fetch token once, reuse.
- RELEASE_NOTES.md: /search now requires a token.
- test/data: test-only JWKS + RSA key fixtures.
|
|
total timeout for gn3 calls
reqwest::get built a fresh client per query (no keepalive) and set no
timeout, while gn3 sets none of its own -- a stalled upstream would
hang the handler past any front-proxy timeout. One shared client in a
OnceLock now, mirroring the shared-DB-pool pattern.
* src/config.rs (Config): add GN3_TIMEOUT_SECS (default 30);
normalize GN3_URL to end with a slash.
* tests/search.rs test/python/test_search.py (search): one scenario
mapping a gn3 5xx to the 502 backend_error shape.
|
|
Production gn3 puts the diagnostic under "error" and leaks Python
reprs under "error_type", so the old "parsererror" match never
fired and malformed queries fell to 502 with no actionable feedback.
Match on "error" alone; gn3 5xx stays 502.
* tests/search.rs test/python/test_search.py (search): stub replays
production's error body shape.
|
|
search proxy.
Validates q/type/limit at the boundary (400 with suggestions; limit
capped at 100), forwards to {GN3_URL}/search/ with proper
percent-encoding, and re-wraps hits in the descriptive envelope.
Hits are typed as SearchHit (all 14 gn3 fields optional, unknown
fields preserved via serde flatten) so /openapi.json documents a
real schema while staying transparent to gn3 shape changes. gn3's
400+parsererror becomes 422 invalid_query carrying gn3's own
diagnostic; every other backend failure is one 502 shape
distinguished by code.
* src/lib.rs, src/routes/mod.rs (routes): register the search route.
* tests/search.rs, test/data/search_alcohol_phenotype.json (tests):
in-process gn3 stub replays the recorded production fixture; four
scenarios run offline -- passthrough+envelope, validation, 422
forwarding, limit cap.
|