| Age | Commit message (Collapse) | Author |
|
The index stores every boolean term in lowercase and holds no
mixed-case variants at all: XDSbxdpublish (15 060 docs), Awilliams
(10 958), XSmouse, XCx, XYbdnf, XImrna all exist, while XDSBXDPublish,
AWilliams, XSMouse, XCX, XYBDNF and XIMRNA report "not in database" on
both the dev index and a full production one. Matching the value
exactly therefore rejected the way a caller naturally writes it --
dataset:BXDPublish returned 0 hits against an index holding 15 060
BXDPublish traits, author:Williams 0 against 10 958 williams authors.
Lowercase the value for every boolean prefix, generalising the
group-only rule gn3 already has (search.py:204-209). No graded parity
shape changes: they all use lowercase values already.
Pinned per prefix in boolean_prefixes_parse_to_exact_index_terms, so a
registration that regresses to an exact add_boolean_prefix fails there
instead of silently returning zero hits.
|
|
Composes the new main above the search engine seam:
* src/routes/search.rs (page): parsed and threaded through the seam;
the gn3 forwarder passes page to gn3 unsorted (sorted fetches the
wide first page), the native engine takes the mset window. Ranked
pages slice the sorted list, as before.
* src/routes/search.rs (add_publish_annotations): enrichment now
fills locus and abbreviation in one PublishXRef/Phenotype lookup.
* src/search/mod.rs (Engine::search): page parameter; SearchHit
gains abbreviation and pubmed_id (gn3's future fields flow via
extra, as before).
* src/db.rs (LRS_PER_LOD): 4.604 — lod shifts accordingly everywhere.
All 34 test binaries green; parity replay and shape gates pass
against the dev index with the new lod constant (grading compares
identifier sets, not lod); page disjointness, abbreviation and locus
verified live (native engine, dev index, SQL_URI set).
|
|
One commit, both halves of AMENDMENT E / the 2026-10-06 contract:
(i) The 422 invalid_query body gains the blessed typed shape. The
SearchError::InvalidQuery payload becomes {message, xapian_error_type},
threaded from every producer:
- native parser rejects (parse_query_checked): the ParseError's own
class, e.g. QueryParserError for Q51/Q52;
- the C3.4 numeric-range guard: QueryParserError ("Unknown range
operation", Q52's recorded production diagnostic);
- the C3.6 invalid-location guard: InvalidOperationError — the class
production actually threw for Q42;
- the gn3 forwarder: gn3's error_type field, with the bytes-repr leak
normalized (b'QueryParserError' -> QueryParserError — the binding-
report R1 / OPEN-16 wrapper-layer normalization), or the exception
class name itself for gn3's escaping-exception 500 bodies.
The route's engine_error builds the typed body inside the existing
ApiError envelope, under "error": {type: "invalid_query", message,
param: "query", detail: {xapian_error_type}} — the envelope's existing
fields (code, detail, ...) keep their meaning, so every current 422
(Q51/Q52 parser rejects, Q42's invalid-location class, mean:abc..def)
emits it and the harness's body["code"] grading is untouched.
The forwarder additionally maps gn3's malformed-processor-value 500s
(the escaping xapian exceptions — Q42's recorded
{"error": "InvalidOperationError", ...} body, the parity-metric
amendment's curl-verified ValueError) onto InvalidQuery, scoped to the
exception-class shape (a bare CamelCase ...Error name, or "Class:
message"): the blessed contract folds all malformed input into 422,
while prose 500 diagnostics ("gn3 exploded") keep the 502
backend_error class — pinned by both the rust and python route tests.
(ii) The harness comparator's Q42 mapping rule changes: recorded
production 500 -> expected 422 invalid_query. The recorded fixture
itself stays untouched as production truth (recorded_status 500,
served verbatim by the stub); only grade_error_class's Server500 arm
re-baselines, with its doc comments and the comparator unit tests.
Gate evidence (~/spikes/port-execution-r2/logs/c37-*): production
floor 54/54 PASS with the NEW rule — Q42 rec=500 gn4=422
code=invalid_query through the real route + forwarding engine + stub
(match200 49/49, clause-b 36/36, min J 1.000, parity-report.json
floor PASS); FULL pinned cargo test --release EXIT=0 (30 suites,
incl. the re-based comparator unit tests and the route unit tests
pinning the typed body's presence for invalid_query and absence for
backend errors); shape-class gate 52/52 unchanged; hermetic lib 23
green incl. the typed-payload assertions (Q51 message byte-identical,
QueryParserError classification). rustfmt --edition 2024 clean on all
nine touched files.
|
|
native-search-engine.
Resolution: his route-level additions compose above the engine seam —
the handler now validates sort/order, asks the engine for
SORT_FETCH_LIMIT (1000) when ranking, maps SearchError via the seam's
engine_error, then computes lod, sorts, truncates to limit and fills
loci; lod/locus join SearchHit in src/search/mod.rs. His tests pass
through the seam.
KNOWN at this commit: the parity floor is BLOCKED (36 shapes) — the
route's new truncate(limit) exposes that route-level full-set grading
was a calibration-stub artifact (real gn3 caps at per_page; the stub
ignored it). Harness fix queued: set-equality moves to engine level,
status + top-10 stay route level. Landed together with that fix.
|
|
Engine seam (C3.1).
New native engine, a Rust port of gn3 search path (gn3/api/search.py),
opt-in via SEARCH_ENGINE=native (the gn3 forwarder stays the default;
src/search/mod.rs now dispatches on config, routes/search.rs delegates):
- startup-cached QueryParser behind a mutex: custom-prefix registration
is Box::leak-based inside the binding, so a per-request parser would
leak; built once per process (binding-report §5 item 3, R2 pattern).
Minimal configuration this commit per PORT-DESIGN C3.1: Stem("en") +
STEM_ALL_Z only.
- checked parse via the fork's parse_query_checked: malformed queries
are typed errors, not panics; Err maps to SearchError::InvalidQuery
(422 invalid_query, the owner-blessed contract), carrying the
parser diagnostic. Parse happens before the index is opened, as in
gn3 (search.py:291 vs :296), so a bad query reports InvalidQuery
even when the index is misconfigured.
- XT{type} document filter: OP_FILTER(parsed, "XT"+type)
(search.py:298-300).
- FULL-match-set retrieval ignoring the forwarded limit (per-page
invariant, PORT-DESIGN §5): mset bound is a deliberately huge fixed
constant; the route keeps its validation and limit logic unchanged.
- per-hit mapping: doc data JSON -> SearchHit via serde
(search.py:302 equivalent); unknown fields flow through extra.
Decode failure maps to SearchError::Misbehaving.
- index open behind an app-level guard (the binding aborts on open
failure; see the module doc): unset XAPIAN_DB_PATH -> Internal,
unopenable path -> Unreachable.
tests/search_native.rs: error-contract scenarios (hermetic) + the
per-page invariant + dispatch pinning, one sequential test fn per the
process-global env rule; the dev-index part skips loudly when
XAPIAN_DB_PATH is unset. tests/search_native_shapes.rs: the shape-CLASS
gate (this lane s AMENDMENT-B translation of the C3.1 stage gate) --
16 shapes (Q01,Q03-Q13,Q47,Q50,Q51,Q54), each expected class cited to
its tests/search_parity.rs shape! line; dev-index counts are printed
as reference data only, never compared to production.
Evidence: logs/c3-1.log (build EXIT=0, native tests green, lib 14/14),
logs/c3-1-full.log (FULL suite EXIT=0, parity floor green),
logs/shape-class-c3-1.log (SHAPE-CLASS GATE 16/16 PASS vs the dev
index; Q51 byte-identical diagnostic; per-page invariant: limit=10
forwarded, 2330-hit full set returned).
|
|
* src/search/gn3.rs (Gn3): forwarding engine over the shared reqwest client.
* src/lib.rs (search): declare the search module.
The seam isolates backend choice behind one trait: routes validate,
wrap the envelope, and map errors; the engine resolves queries. A
native engine can replace the gn3 proxy without touching the route
contract, and the proxy stays as the transitional oracle bridge
until the native engine reaches parity. The module docs record the
xapian query grammar any native engine must accept.
|