about summary refs log tree commit diff
path: root/test/data
AgeCommit message (Collapse)Author
2026-10-08* src/db.rs, src/routes: add mapping_method, tissue, /mapping-methods, ↵Pjotr
/accessions. Answers four more questions of aisearch's full_benchmark.csv from tables the API did not expose. * src/db.rs (Group, GroupDetail), src/routes/groups.rs: groups gain `mapping_method`, the name of the MappingMethod row their mapping_method_id points to (qtlreaper, Rqtl, ...), or null. * src/db.rs (Dataset): datasets gain `tissue`, the Tissue their ProbeFreeze points to (e.g. "Brain mRNA"). * src/routes/mapping_methods.rs: GET /mapping-methods lists the MappingMethod table, including tools no group uses (PLINK, FastMap). * src/routes/accessions.rs: GET /accessions/{accession} resolves a GeneNetwork accession number (GN602, gn602 or 602) via InfoFiles to the dataset's name, title, category (publish/probeset) and group. * docs/genenetwork_datatypes.md: MappingMethodId is a key into the MappingMethod table (the authoritative reading); GN2's code-based reading is noted as superseded. * tests/benchmark.rs, test/data/benchmark_search.json: add the 60 new full-benchmark rows that pass (full rows 0..114; search rows run against recorded production gn3, the rest need the database), and full rows 33 (mapping tools), 34 (Rqtl for outbred sets), 35 (GN602) and 44 (only GTEx v8 has TPM datasets). The untested rows are listed in the file's header comment with the reason for each. * tests, test/python: tests for the new fields and endpoints; both endpoints added to the OpenAPI and next-link crawl tests. * README.md, RELEASE_NOTES.md: document the new fields and endpoints. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08Merge origin/main (54f1388): page, publish annotations, publications route.Alexander_Kabui
Composes the new main above the search engine seam: * src/routes/search.rs (page): parsed and threaded through the seam; the gn3 forwarder passes page to gn3 unsorted (sorted fetches the wide first page), the native engine takes the mset window. Ranked pages slice the sorted list, as before. * src/routes/search.rs (add_publish_annotations): enrichment now fills locus and abbreviation in one PublishXRef/Phenotype lookup. * src/search/mod.rs (Engine::search): page parameter; SearchHit gains abbreviation and pubmed_id (gn3's future fields flow via extra, as before). * src/db.rs (LRS_PER_LOD): 4.604 — lod shifts accordingly everywhere. All 34 test binaries green; parity replay and shape gates pass against the dev index with the new lod constant (grading compares identifier sets, not lod); page disjointness, abbreviation and locus verified live (native engine, dev index, SQL_URI set).
2026-10-08* src/routes/publications.rs: add GET /publications/{pubmed_id}/traits.Pjotr
Lists the published traits (across groups) a publication reports. Datasets the caller may not view are left out, using the same check as GET /traits/{dataset} (traits.rs is_authorised is now pub(crate)). * src/db.rs (TraitInfo, trait_info_sql): TraitInfo gains `dataset`, `abbreviation` (Phenotype.Post_publication_abbreviation, else Pre) and `url` (the GeneNetwork show_trait page). The three TraitInfo queries now share one SELECT. New get_group_traits_by_abbreviation and get_publication_traits. * src/routes/groups.rs (group_traits): add ?abbreviation= filter. * src/db.rs (Group), src/routes/groups.rs (groups): groups carry `family` and `species`; add family, species and q filters. * src/routes/search.rs (search): add `page` (gn3's page unsorted, a page of the ranked list with `sort`); declare `pubmed_id`; hits gain `abbreviation` next to `locus` (get_publish_annotations). * src/db.rs (LRS_PER_LOD): LOD = LRS / 4.604, the divisor the benchmark's RDF data uses (was 4.61). * tests, test/python, test/data/benchmark_search.json, README.md: cover all of the above; tests/benchmark.rs now also answers rows 11, 16, 25, 30, 34 and 37 of small_benchmark.csv, and compares LOD exactly. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08docs: parity-fixtures-dev README notes the BXD two-dataset mirroring.Alexander_Kabui
* test/data/parity-fixtures-dev/README.md (dev-oracle): a trait Id can appear twice in one response with different dataset labels (BXDPublish and GITrMetPublish both cover InbredSetId 1) and identical LOD; all (dataset,name) keys are unique, so these are not duplicates. Verified against the source database and the index 2026-10-08.
2026-10-08test/data/parity-fixtures-dev (Q01-Q54, reduced-v1): commit the dev-oracle ↵Alexander_Kabui
corpus. The 54 harness shapes recorded from the self-built 47k index via the native engine (tests/search_native_dev_corpus.rs), reduced with the committed tool. Dev-truth counts; the production corpus stays the production grading truth. Pins the dev-oracle for reproducibility before any sign-off decision makes it load-bearing.
2026-10-07Merge origin/main (e912c87: /search gains lod, locus, sort/order) intoAlexander_Kabui
native-search-engine. Resolution: his route-level additions compose above the engine seam — the handler now validates sort/order, asks the engine for SORT_FETCH_LIMIT (1000) when ranking, maps SearchError via the seam's engine_error, then computes lod, sorts, truncates to limit and fills loci; lod/locus join SearchHit in src/search/mod.rs. His tests pass through the seam. KNOWN at this commit: the parity floor is BLOCKED (36 shapes) — the route's new truncate(limit) exposes that route-level full-set grading was a calibration-stub artifact (real gn3 caps at per_page; the stub ignored it). Harness fix queued: set-equality moves to engine level, status + top-10 stay route level. Landed together with that fix.
2026-10-07* tests/benchmark.rs: add the answerable small_benchmark.csv questions.Pjotr
Checks the 30 of the 39 questions in aisearch/data/small_benchmark.csv that the API can answer today against the benchmark's own answers: species and group metadata, group datasets, trait metadata, and search-based trait questions (marker, additive effect, PubMed id, mean, LOD within 0.5%). The benchmark's LOD is LRS / 4.604, ours LRS / 4.61. The uncovered questions are listed in the file's header comment. * test/data/benchmark_search.json: recorded production gn3 search responses for those questions (2026-10-07, trimmed to the relevant hits), so the test needs no network. Database-backed checks skip when no usable database is reachable. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07* src/routes/search.rs (search): add lod, locus, and sort/order.Pjotr
Search hits gain `lod` (lrs / 4.61) and `locus` (the peak marker name, looked up in PublishXRef in one batched query; null when the database is unavailable). `sort=lrs|lod` with `order=asc|desc` ranks hits: gn3 is asked for its first 1000 matches, those are ranked, then `limit` applies. * src/db.rs (LRS_PER_LOD, lod, get_publish_loci): add the shared LRS-to-LOD constant and the locus lookup; TraitInfo and ProbeSetInfo gain `lod`, computed in SQL from the same constant. * tests/search.rs, test/python/test_search.py: add the "highest LOD, locus marker and additive effect for serotonin in BXD male midbrain" scenario over a recorded production fixture, plus sort/order tests. * tests/group_trait_info.rs, tests/dataset_probeset_info.rs and their Python mirrors: assert lod (and locus for trait 17985). * README.md (Endpoints): document sort, order, lod and locus. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07tests/search_parity.rs (search parity): re-shape 13 weak shapes toAlexander_Kabui
discriminating values; re-record Q02 (insulin) through a verified pagination walk (50000 + 7632 = 57632 hits, truncation closed). 12 formerly zero-hit shapes now carry real matches so clause (b) bites on 36/36 recorded shapes; Q26 re-shaped from chr:2 (>1.1M matches, unrecordable) to the bounded chr:m (9374 hits), same chr:->XC boolean filter grammar; Q04/Q11/Q12/Q21 retained as documented empty (the deployed index has no positional data and XD is unpopulated). * test/data/parity-fixtures (Q01-Q54, reduced-v1): re-baseline the corpus with the 14 re-recorded fixtures (production snapshot 2026-10-07); parity floor PASS 54/54, clause-(b) 36/36, min Jaccard 1.000. * test/data/parity-fixtures/reduce_fixtures.py (SHAPE_RE): tolerate rustfmt-expanded shape! tuples; the strict regex parsed only 4/54 of the current harness file and would fatal the generator. * test/data/parity-fixtures/README.md: document the re-recording, the truncation closure and the retained-empty shapes.
2026-10-07* tests/search_parity.rs (search parity): add the 54-shape harness.Alexander_Kabui
* test/data/parity-fixtures (Q01-Q54, reduced-v1): commit the corpus. * test/data/parity-fixtures/reduce_fixtures.py: regenerate it from a full recording corpus. Grades any implementation behind /search against 54 recorded production shapes per parity-metric.md: status, full match-set equality, a Jaccard >= 0.8 top-10 floor for sets of 10 or more, error-class mapping (recorded gn3 4xx -> 422 invalid_query, recorded 500 -> 502 backend_error, surface -> 400 invalid_parameter with no upstream call), and exactly-once upstream-call assertions. Hermetic: axum stubs stand in for gn3 and gn-auth, tokens are minted with the committed test key, no network. The committed corpus is reduced-v1: 109 MB of recorded payloads reduced to 6.6 MB of graded essentials (sorted identifier set, the recorded first-ten order, status); the raw corpus stays out of git. The loader sniffs full and reduced files in one directory, and the stub serves reduced bodies reconstructed top-ten-first so grading is identical either way; reduced files also self-verify against the shape table at load time. The generator parses the shape table from this test so the two cannot drift, and fails loudly on any hit lacking (dataset, name). Calibrated 54/54 against the forwarding route on both the full and the reduced corpus; negative controls prove it fails loudly. Runs by default from the committed corpus; GN4_PARITY_FIXTURES overrides for external corpora, GN4_PARITY_REPORTS relocates the report (default target/parity-reports).
2026-10-05search: require a token; restore hermetic /search coverageAlexander_Kabui
- src/routes/search.rs: AuthToken per correlation.rs pattern (+ Unauthorized). - tests/search.rs: stub /auth/public-jwks + locally-signed token; scenarios run offline again (multi_thread runtime: the blocking ureq JWKS fetch starved the single-thread stub); add invalid-token 401 case. - test/python/test_search.py: fetch token once, reuse. - RELEASE_NOTES.md: /search now requires a token. - test/data: test-only JWKS + RSA key fixtures.
2026-10-05* src/routes/search.rs (search): add GET /search, the phase-A gn3Alexander_Kabui
search proxy. Validates q/type/limit at the boundary (400 with suggestions; limit capped at 100), forwards to {GN3_URL}/search/ with proper percent-encoding, and re-wraps hits in the descriptive envelope. Hits are typed as SearchHit (all 14 gn3 fields optional, unknown fields preserved via serde flatten) so /openapi.json documents a real schema while staying transparent to gn3 shape changes. gn3's 400+parsererror becomes 422 invalid_query carrying gn3's own diagnostic; every other backend failure is one 502 shape distinguished by code. * src/lib.rs, src/routes/mod.rs (routes): register the search route. * tests/search.rs, test/data/search_alcohol_phenotype.json (tests): in-process gn3 stub replays the recorded production fixture; four scenarios run offline -- passthrough+envelope, validation, 422 forwarding, limit cap.