| Age | Commit message (Collapse) | Author |
|
/accessions.
Answers four more questions of aisearch's full_benchmark.csv from
tables the API did not expose.
* src/db.rs (Group, GroupDetail), src/routes/groups.rs: groups gain
`mapping_method`, the name of the MappingMethod row their
mapping_method_id points to (qtlreaper, Rqtl, ...), or null.
* src/db.rs (Dataset): datasets gain `tissue`, the Tissue their
ProbeFreeze points to (e.g. "Brain mRNA").
* src/routes/mapping_methods.rs: GET /mapping-methods lists the
MappingMethod table, including tools no group uses (PLINK, FastMap).
* src/routes/accessions.rs: GET /accessions/{accession} resolves a
GeneNetwork accession number (GN602, gn602 or 602) via InfoFiles to the
dataset's name, title, category (publish/probeset) and group.
* docs/genenetwork_datatypes.md: MappingMethodId is a key into the
MappingMethod table (the authoritative reading); GN2's code-based reading
is noted as superseded.
* tests/benchmark.rs, test/data/benchmark_search.json: add the 60 new
full-benchmark rows that pass (full rows 0..114; search rows run against
recorded production gn3, the rest need the database), and full rows 33
(mapping tools), 34 (Rqtl for outbred sets), 35 (GN602) and 44 (only
GTEx v8 has TPM datasets). The untested rows are listed in the file's
header comment with the reason for each.
* tests, test/python: tests for the new fields and endpoints; both
endpoints added to the OpenAPI and next-link crawl tests.
* README.md, RELEASE_NOTES.md: document the new fields and endpoints.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Composes the new main above the search engine seam:
* src/routes/search.rs (page): parsed and threaded through the seam;
the gn3 forwarder passes page to gn3 unsorted (sorted fetches the
wide first page), the native engine takes the mset window. Ranked
pages slice the sorted list, as before.
* src/routes/search.rs (add_publish_annotations): enrichment now
fills locus and abbreviation in one PublishXRef/Phenotype lookup.
* src/search/mod.rs (Engine::search): page parameter; SearchHit
gains abbreviation and pubmed_id (gn3's future fields flow via
extra, as before).
* src/db.rs (LRS_PER_LOD): 4.604 — lod shifts accordingly everywhere.
All 34 test binaries green; parity replay and shape gates pass
against the dev index with the new lod constant (grading compares
identifier sets, not lod); page disjointness, abbreviation and locus
verified live (native engine, dev index, SQL_URI set).
|
|
Lists the published traits (across groups) a publication reports.
Datasets the caller may not view are left out, using the same check as
GET /traits/{dataset} (traits.rs is_authorised is now pub(crate)).
* src/db.rs (TraitInfo, trait_info_sql): TraitInfo gains `dataset`,
`abbreviation` (Phenotype.Post_publication_abbreviation, else Pre) and
`url` (the GeneNetwork show_trait page). The three TraitInfo queries
now share one SELECT. New get_group_traits_by_abbreviation and
get_publication_traits.
* src/routes/groups.rs (group_traits): add ?abbreviation= filter.
* src/db.rs (Group), src/routes/groups.rs (groups): groups carry
`family` and `species`; add family, species and q filters.
* src/routes/search.rs (search): add `page` (gn3's page unsorted, a
page of the ranked list with `sort`); declare `pubmed_id`; hits gain
`abbreviation` next to `locus` (get_publish_annotations).
* src/db.rs (LRS_PER_LOD): LOD = LRS / 4.604, the divisor the
benchmark's RDF data uses (was 4.61).
* tests, test/python, test/data/benchmark_search.json, README.md: cover
all of the above; tests/benchmark.rs now also answers rows 11, 16, 25,
30, 34 and 37 of small_benchmark.csv, and compares LOD exactly.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
* test/data/parity-fixtures-dev/README.md (dev-oracle): a trait Id can
appear twice in one response with different dataset labels (BXDPublish
and GITrMetPublish both cover InbredSetId 1) and identical LOD; all
(dataset,name) keys are unique, so these are not duplicates. Verified
against the source database and the index 2026-10-08.
|
|
corpus. The 54 harness shapes recorded from the self-built 47k index via the native engine (tests/search_native_dev_corpus.rs), reduced with the committed tool. Dev-truth counts; the production corpus stays the production grading truth. Pins the dev-oracle for reproducibility before any sign-off decision makes it load-bearing.
|
|
native-search-engine.
Resolution: his route-level additions compose above the engine seam —
the handler now validates sort/order, asks the engine for
SORT_FETCH_LIMIT (1000) when ranking, maps SearchError via the seam's
engine_error, then computes lod, sorts, truncates to limit and fills
loci; lod/locus join SearchHit in src/search/mod.rs. His tests pass
through the seam.
KNOWN at this commit: the parity floor is BLOCKED (36 shapes) — the
route's new truncate(limit) exposes that route-level full-set grading
was a calibration-stub artifact (real gn3 caps at per_page; the stub
ignored it). Harness fix queued: set-equality moves to engine level,
status + top-10 stay route level. Landed together with that fix.
|
|
Checks the 30 of the 39 questions in aisearch/data/small_benchmark.csv
that the API can answer today against the benchmark's own answers:
species and group metadata, group datasets, trait metadata, and
search-based trait questions (marker, additive effect, PubMed id, mean,
LOD within 0.5%). The benchmark's LOD is LRS / 4.604, ours LRS / 4.61.
The uncovered questions are listed in the file's header comment.
* test/data/benchmark_search.json: recorded production gn3 search
responses for those questions (2026-10-07, trimmed to the relevant
hits), so the test needs no network. Database-backed checks skip when
no usable database is reachable.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Search hits gain `lod` (lrs / 4.61) and `locus` (the peak marker name,
looked up in PublishXRef in one batched query; null when the database
is unavailable). `sort=lrs|lod` with `order=asc|desc` ranks hits: gn3
is asked for its first 1000 matches, those are ranked, then `limit`
applies.
* src/db.rs (LRS_PER_LOD, lod, get_publish_loci): add the shared
LRS-to-LOD constant and the locus lookup; TraitInfo and ProbeSetInfo
gain `lod`, computed in SQL from the same constant.
* tests/search.rs, test/python/test_search.py: add the "highest LOD,
locus marker and additive effect for serotonin in BXD male midbrain"
scenario over a recorded production fixture, plus sort/order tests.
* tests/group_trait_info.rs, tests/dataset_probeset_info.rs and their
Python mirrors: assert lod (and locus for trait 17985).
* README.md (Endpoints): document sort, order, lod and locus.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
discriminating values; re-record Q02 (insulin) through a verified
pagination walk (50000 + 7632 = 57632 hits, truncation closed). 12
formerly zero-hit shapes now carry real matches so clause (b) bites
on 36/36 recorded shapes; Q26 re-shaped from chr:2 (>1.1M matches,
unrecordable) to the bounded chr:m (9374 hits), same chr:->XC boolean
filter grammar; Q04/Q11/Q12/Q21 retained as documented empty (the
deployed index has no positional data and XD is unpopulated).
* test/data/parity-fixtures (Q01-Q54, reduced-v1): re-baseline the
corpus with the 14 re-recorded fixtures (production snapshot
2026-10-07); parity floor PASS 54/54, clause-(b) 36/36, min Jaccard
1.000.
* test/data/parity-fixtures/reduce_fixtures.py (SHAPE_RE): tolerate
rustfmt-expanded shape! tuples; the strict regex parsed only 4/54 of
the current harness file and would fatal the generator.
* test/data/parity-fixtures/README.md: document the re-recording, the
truncation closure and the retained-empty shapes.
|
|
* test/data/parity-fixtures (Q01-Q54, reduced-v1): commit the corpus.
* test/data/parity-fixtures/reduce_fixtures.py: regenerate it from a full
recording corpus.
Grades any implementation behind /search against 54 recorded
production shapes per parity-metric.md: status, full match-set
equality, a Jaccard >= 0.8 top-10 floor for sets of 10 or more,
error-class mapping (recorded gn3 4xx -> 422 invalid_query, recorded
500 -> 502 backend_error, surface -> 400 invalid_parameter with no
upstream call), and exactly-once upstream-call assertions. Hermetic:
axum stubs stand in for gn3 and gn-auth, tokens are minted with the
committed test key, no network.
The committed corpus is reduced-v1: 109 MB of recorded payloads
reduced to 6.6 MB of graded essentials (sorted identifier set, the
recorded first-ten order, status); the raw corpus stays out of git.
The loader sniffs full and reduced files in one directory, and the
stub serves reduced bodies reconstructed top-ten-first so grading is
identical either way; reduced files also self-verify against the
shape table at load time. The generator parses the shape table from
this test so the two cannot drift, and fails loudly on any hit
lacking (dataset, name).
Calibrated 54/54 against the forwarding route on both the full
and the reduced corpus; negative controls prove it fails loudly.
Runs by default from the committed corpus; GN4_PARITY_FIXTURES
overrides for external corpora, GN4_PARITY_REPORTS relocates the
report (default target/parity-reports).
|
|
- src/routes/search.rs: AuthToken per correlation.rs pattern (+ Unauthorized).
- tests/search.rs: stub /auth/public-jwks + locally-signed token; scenarios
run offline again (multi_thread runtime: the blocking ureq JWKS fetch
starved the single-thread stub); add invalid-token 401 case.
- test/python/test_search.py: fetch token once, reuse.
- RELEASE_NOTES.md: /search now requires a token.
- test/data: test-only JWKS + RSA key fixtures.
|
|
search proxy.
Validates q/type/limit at the boundary (400 with suggestions; limit
capped at 100), forwards to {GN3_URL}/search/ with proper
percent-encoding, and re-wraps hits in the descriptive envelope.
Hits are typed as SearchHit (all 14 gn3 fields optional, unknown
fields preserved via serde flatten) so /openapi.json documents a
real schema while staying transparent to gn3 shape changes. gn3's
400+parsererror becomes 422 invalid_query carrying gn3's own
diagnostic; every other backend failure is one 502 shape
distinguished by code.
* src/lib.rs, src/routes/mod.rs (routes): register the search route.
* tests/search.rs, test/data/search_alcohol_phenotype.json (tests):
in-process gn3 stub replays the recorded production fixture; four
scenarios run offline -- passthrough+envelope, validation, 422
forwarding, limit cap.
|