about summary refs log tree commit diff
path: root/src/search/mod.rs
AgeCommit message (Collapse)Author
12 dayssearch: lowercase boolean prefix valuesPjotr
The index stores every boolean term in lowercase and holds no mixed-case variants at all: XDSbxdpublish (15 060 docs), Awilliams (10 958), XSmouse, XCx, XYbdnf, XImrna all exist, while XDSBXDPublish, AWilliams, XSMouse, XCX, XYBDNF and XIMRNA report "not in database" on both the dev index and a full production one. Matching the value exactly therefore rejected the way a caller naturally writes it -- dataset:BXDPublish returned 0 hits against an index holding 15 060 BXDPublish traits, author:Williams 0 against 10 958 williams authors. Lowercase the value for every boolean prefix, generalising the group-only rule gn3 already has (search.py:204-209). No graded parity shape changes: they all use lowercase values already. Pinned per prefix in boolean_prefixes_parse_to_exact_index_terms, so a registration that regresses to an exact add_boolean_prefix fails there instead of silently returning zero hits.
2026-10-08Merge origin/main (54f1388): page, publish annotations, publications route.Alexander_Kabui
Composes the new main above the search engine seam: * src/routes/search.rs (page): parsed and threaded through the seam; the gn3 forwarder passes page to gn3 unsorted (sorted fetches the wide first page), the native engine takes the mset window. Ranked pages slice the sorted list, as before. * src/routes/search.rs (add_publish_annotations): enrichment now fills locus and abbreviation in one PublishXRef/Phenotype lookup. * src/search/mod.rs (Engine::search): page parameter; SearchHit gains abbreviation and pubmed_id (gn3's future fields flow via extra, as before). * src/db.rs (LRS_PER_LOD): 4.604 — lod shifts accordingly everywhere. All 34 test binaries green; parity replay and shape gates pass against the dev index with the new lod constant (grading compares identifier sets, not lod); page disjointness, abbreviation and locus verified live (native engine, dev index, SQL_URI set).
2026-10-07Typed-422 body + Q42 re-baseline (C3.7, ACTIVE — the owner-blessed contract)Alexander_Kabui
One commit, both halves of AMENDMENT E / the 2026-10-06 contract: (i) The 422 invalid_query body gains the blessed typed shape. The SearchError::InvalidQuery payload becomes {message, xapian_error_type}, threaded from every producer: - native parser rejects (parse_query_checked): the ParseError's own class, e.g. QueryParserError for Q51/Q52; - the C3.4 numeric-range guard: QueryParserError ("Unknown range operation", Q52's recorded production diagnostic); - the C3.6 invalid-location guard: InvalidOperationError — the class production actually threw for Q42; - the gn3 forwarder: gn3's error_type field, with the bytes-repr leak normalized (b'QueryParserError' -> QueryParserError — the binding- report R1 / OPEN-16 wrapper-layer normalization), or the exception class name itself for gn3's escaping-exception 500 bodies. The route's engine_error builds the typed body inside the existing ApiError envelope, under "error": {type: "invalid_query", message, param: "query", detail: {xapian_error_type}} — the envelope's existing fields (code, detail, ...) keep their meaning, so every current 422 (Q51/Q52 parser rejects, Q42's invalid-location class, mean:abc..def) emits it and the harness's body["code"] grading is untouched. The forwarder additionally maps gn3's malformed-processor-value 500s (the escaping xapian exceptions — Q42's recorded {"error": "InvalidOperationError", ...} body, the parity-metric amendment's curl-verified ValueError) onto InvalidQuery, scoped to the exception-class shape (a bare CamelCase ...Error name, or "Class: message"): the blessed contract folds all malformed input into 422, while prose 500 diagnostics ("gn3 exploded") keep the 502 backend_error class — pinned by both the rust and python route tests. (ii) The harness comparator's Q42 mapping rule changes: recorded production 500 -> expected 422 invalid_query. The recorded fixture itself stays untouched as production truth (recorded_status 500, served verbatim by the stub); only grade_error_class's Server500 arm re-baselines, with its doc comments and the comparator unit tests. Gate evidence (~/spikes/port-execution-r2/logs/c37-*): production floor 54/54 PASS with the NEW rule — Q42 rec=500 gn4=422 code=invalid_query through the real route + forwarding engine + stub (match200 49/49, clause-b 36/36, min J 1.000, parity-report.json floor PASS); FULL pinned cargo test --release EXIT=0 (30 suites, incl. the re-based comparator unit tests and the route unit tests pinning the typed body's presence for invalid_query and absence for backend errors); shape-class gate 52/52 unchanged; hermetic lib 23 green incl. the typed-payload assertions (Q51 message byte-identical, QueryParserError classification). rustfmt --edition 2024 clean on all nine touched files.
2026-10-07Merge origin/main (e912c87: /search gains lod, locus, sort/order) intoAlexander_Kabui
native-search-engine. Resolution: his route-level additions compose above the engine seam — the handler now validates sort/order, asks the engine for SORT_FETCH_LIMIT (1000) when ranking, maps SearchError via the seam's engine_error, then computes lod, sorts, truncates to limit and fills loci; lod/locus join SearchHit in src/search/mod.rs. His tests pass through the seam. KNOWN at this commit: the parity floor is BLOCKED (36 shapes) — the route's new truncate(limit) exposes that route-level full-set grading was a calibration-stub artifact (real gn3 caps at per_page; the stub ignored it). Harness fix queued: set-equality moves to engine level, status + top-10 stay route level. Landed together with that fix.
2026-10-07src/search/native.rs (native engine): port the xapian query path behind the ↵Alexander_Kabui
Engine seam (C3.1). New native engine, a Rust port of gn3 search path (gn3/api/search.py), opt-in via SEARCH_ENGINE=native (the gn3 forwarder stays the default; src/search/mod.rs now dispatches on config, routes/search.rs delegates): - startup-cached QueryParser behind a mutex: custom-prefix registration is Box::leak-based inside the binding, so a per-request parser would leak; built once per process (binding-report §5 item 3, R2 pattern). Minimal configuration this commit per PORT-DESIGN C3.1: Stem("en") + STEM_ALL_Z only. - checked parse via the fork's parse_query_checked: malformed queries are typed errors, not panics; Err maps to SearchError::InvalidQuery (422 invalid_query, the owner-blessed contract), carrying the parser diagnostic. Parse happens before the index is opened, as in gn3 (search.py:291 vs :296), so a bad query reports InvalidQuery even when the index is misconfigured. - XT{type} document filter: OP_FILTER(parsed, "XT"+type) (search.py:298-300). - FULL-match-set retrieval ignoring the forwarded limit (per-page invariant, PORT-DESIGN §5): mset bound is a deliberately huge fixed constant; the route keeps its validation and limit logic unchanged. - per-hit mapping: doc data JSON -> SearchHit via serde (search.py:302 equivalent); unknown fields flow through extra. Decode failure maps to SearchError::Misbehaving. - index open behind an app-level guard (the binding aborts on open failure; see the module doc): unset XAPIAN_DB_PATH -> Internal, unopenable path -> Unreachable. tests/search_native.rs: error-contract scenarios (hermetic) + the per-page invariant + dispatch pinning, one sequential test fn per the process-global env rule; the dev-index part skips loudly when XAPIAN_DB_PATH is unset. tests/search_native_shapes.rs: the shape-CLASS gate (this lane s AMENDMENT-B translation of the C3.1 stage gate) -- 16 shapes (Q01,Q03-Q13,Q47,Q50,Q51,Q54), each expected class cited to its tests/search_parity.rs shape! line; dev-index counts are printed as reference data only, never compared to production. Evidence: logs/c3-1.log (build EXIT=0, native tests green, lib 14/14), logs/c3-1-full.log (FULL suite EXIT=0, parity floor green), logs/shape-class-c3-1.log (SHAPE-CLASS GATE 16/16 PASS vs the dev index; Q51 byte-identical diagnostic; per-page invariant: limit=10 forwarded, 2330-hit full set returned).
2026-10-06* src/search/mod.rs (Engine, SearchError, SearchHit): add the seam.Alexander_Kabui
* src/search/gn3.rs (Gn3): forwarding engine over the shared reqwest client. * src/lib.rs (search): declare the search module. The seam isolates backend choice behind one trait: routes validate, wrap the envelope, and map errors; the engine resolves queries. A native engine can replace the gn3 proxy without touching the route contract, and the proxy stays as the transitional oracle bridge until the native engine reaches parity. The module docs record the xapian query grammar any native engine must accept.