diff options
| author | Alexander_Kabui <alexanderkabua@gmail.com> | 2026-10-07 23:12:27 +0300 |
|---|---|---|
| committer | Alexander_Kabui <alexanderkabua@gmail.com> | 2026-10-07 23:12:27 +0300 |
| commit | 3dfe9ff07c229d2813edcdd9606884c38f93dd19 (patch) | |
| tree | 15ea452c2e141704ec5e8739b933ea67569ccc2f /src | |
| parent | 9a76f1c7b855ace11de51d1e625fe4205f0096af (diff) | |
| download | genenetwork4-3dfe9ff07c229d2813edcdd9606884c38f93dd19.tar.gz | |
src/search/native.rs (native engine): register the Hs/Mm location processors (C3.6)
gn3's species-location shorthands (search.py:246-272), WITHOUT the Hs liftover OR-branch (OPEN-4, owner-decided; a C3.6b commit would add it): `Hs:chr<N>:<pos>` / `Mm:...` expand to the gn3 triplet (search.py: 185-193) — AND of the species boolean term (XShuman / XSmouse), the chromosome term (XC + chromosome number, value lowercased before the split exactly like gn3's split_query, search.py:178-183) and the slot-2 position range (the C3.5 machinery: SI suffixes, point +/-50 kb clamp, one-sided VALUE_GE/VALUE_LE, bases->megabases). THE INVALID LEAF (Q42, AMENDMENT E): every condition where gn3 raises ValueError (search.py:179-180, 197-201) — a value not splitting into exactly chr<chromosome>:<position-spec>, a chromosome not starting with "chr", or an unparseable position spec — emits the sentinel term GN4InvalidLocationLeaf (verified absent from the dev index; untypeable as free text under STEM_ALL_Z). The engine then detects the sentinel in the parsed query's unique terms and reports SearchError::InvalidQuery BEFORE the index is opened and any mset call — the blessed 422 for the recorded production mset crash (Q42's 500 InvalidOperationError), via the app-level guard (evidence-backed option (b), PORT-DESIGN C3.6, OPEN-31 closed by AMENDMENT E). The C++ try/catch shim is deliberately NOT extended; no crash-mimicry. An invalid leaf anywhere in the tree poisons the whole query, exactly as gn3's mset-time OP_INVALID rejection did (search.py:293-301 has no try/except). Gate evidence (~/spikes/port-execution-r2/logs/c36-*): hermetic pins green (23 lib tests: Q40/Q41 triplet forms XShuman+XC2 / XSmouse+XC1 + slot-2 VALUE_RANGE, value lowercasing, the 7-case invalid matrix, OR-combined poisoning, and the guard firing InvalidQuery with NO index configured — gn3's parse-before-open order preserved); shape-CLASS gate 52/52 cumulative vs the dev index (Q40/Q41 parse Ok with dev count 0 — REFERENCE ONLY per AMENDMENT B; probe evidence the triplet machinery is live: species:mouse AND chr:1 AND mb:1..200 matches, the 4.95..5.05 Mb chr1 window is genuinely empty in the 2025-02 dev data; Q42 -> InvalidQuery); route-level scenarios: Q52 AND Q42 both 422 invalid_query through the real app with SEARCH_ENGINE=native. rustfmt --edition 2024 clean on the touched files.
Diffstat (limited to 'src')
| -rw-r--r-- | src/search/native.rs | 206 |
1 files changed, 205 insertions, 1 deletions
diff --git a/src/search/native.rs b/src/search/native.rs index a7b02cf..114d68a 100644 --- a/src/search/native.rs +++ b/src/search/native.rs @@ -139,6 +139,25 @@ fn build_query_parser() -> QueryParser { // inside the binding (the one-per-process parser absorbs that). qp.add_custom_boolean_prefix::<_, &str>("position", PositionProcessor, None); qp.add_custom_boolean_prefix::<_, &str>("pos", PositionProcessor, None); + // Hs / Mm species-location shorthands (search.py:246-272; C3.6): + // XShuman / XSmouse. WITHOUT the Hs liftover OR-branch — OPEN-4, + // owner-decided; a C3.6b commit would add it. The invalid-location + // leaf these processors may emit is detected by the engine guard + // below (see INVALID_LOCATION_SENTINEL). + qp.add_custom_boolean_prefix::<_, &str>( + "Hs", + LocationProcessor { + species_term: "XShuman", + }, + None, + ); + qp.add_custom_boolean_prefix::<_, &str>( + "Mm", + LocationProcessor { + species_term: "XSmouse", + }, + None, + ); qp } @@ -240,6 +259,73 @@ impl FieldProcessor for PositionProcessor { } } +/// The invalid Hs/Mm leaf marker (C3.6): the Hs/Mm processors emit this +/// TERM where gn3's `parse_location_field` raises `ValueError` +/// (search.py:178-183, 199 — a value that does not split into exactly +/// `chr<chromosome>:<position-spec>`, a chromosome not starting with +/// `chr`, or an unparseable position spec). gn3 maps that leaf to +/// `OP_INVALID`, which `Enquire` then rejects with an uncatchable +/// `InvalidOperationError` — production's recorded Q42 500 (the mset +/// crash), a process-abort hazard under the binding. The blessed 422 +/// contract (AMENDMENT E) instead surfaces the condition as a typed +/// invalid query BEFORE the mset call — the app-level guard, +/// evidence-backed option (b) from PORT-DESIGN C3.6; the C++ try/catch +/// shim is deliberately NOT extended. The engine detects the term in +/// the parsed query's unique terms and reports InvalidQuery. The term +/// matches nothing (verified: absent from the dev index) and free text +/// cannot produce it under STEM_ALL_Z (free text is stemmed and +/// lowercase-prefixed with Z). +const INVALID_LOCATION_SENTINEL: &str = "GN4InvalidLocationLeaf"; + +/// The invalid-location leaf (see [`INVALID_LOCATION_SENTINEL`]). +fn invalid_location_leaf() -> Query { + Query::term(INVALID_LOCATION_SENTINEL, None, None) +} + +/// gn3's `parse_location_field` (search.py:168-202) WITHOUT the Hs +/// liftover OR-branch (OPEN-4 — liftover stays out; a C3.6b commit +/// would add it): an AND of the species term (XShuman / XSmouse), the +/// chromosome term (XC + chromosome number) and the slot-2 position +/// range. The whole value is lowercased before splitting, exactly like +/// gn3's `split_query` (search.py:178-183). +#[derive(Clone)] +struct LocationProcessor { + /// The species boolean term: `XShuman` for Hs, `XSmouse` for Mm + /// (search.py:246-247). + species_term: &'static str, +} + +impl FieldProcessor for LocationProcessor { + fn process(&self, term: &str) -> Option<Query> { + let value = term.to_lowercase(); + let mut parts = value.split(':'); + let (chromosome, position_spec) = match (parts.next(), parts.next(), parts.next()) { + (Some(chromosome), Some(position_spec), None) => (chromosome, position_spec), + // gn3's `chromosome, position_spec = query.split(":")` + // unpack raises ValueError on any other part count. + _ => return Some(invalid_location_leaf()), + }; + let Some(chrom_number) = chromosome.strip_prefix("chr") else { + // search.py:179-180: the chromosome must start with "chr". + return Some(invalid_location_leaf()); + }; + let Some((begin, end)) = parse_position(position_spec) else { + // The ValueError escapes gn3's split_query into the same + // except-ValueError arm (search.py:197-201). + return Some(invalid_location_leaf()); + }; + Some(Query::combine( + Operator::And, + Query::term(self.species_term, None, None), + Query::combine( + Operator::And, + Query::term(format!("XC{chrom_number}"), None, None), + position_value_query(begin, end), + ), + )) + } +} + /// Guard the gn3 numeric-range contract at the app level (C3.4). /// /// Xapian's C++ `NumberRangeProcessor` rejects a range whose non-empty @@ -341,7 +427,26 @@ fn search_blocking(q: &str, type_: &str, limit: u64) -> Result<Vec<SearchHit>, S .map_err(|e| SearchError::InvalidQuery(e.message))? }; - // 2. Guard the numeric-range contract the fork's decline bridge no + // 2. Detect an invalid Hs/Mm leaf BEFORE the mset call (C3.6): the + // processors mark gn3's ValueError class (bad split, chromosome + // not starting with "chr", unparseable position) with a sentinel + // term, which surfaces here as InvalidQuery — the blessed 422 for + // the recorded production mset crash (Q42; AMENDMENT E). gn3's + // own leaf is OP_INVALID, which aborts under the binding; the + // app-level guard is the evidence-backed option (b) (PORT-DESIGN + // C3.6, OPEN-31 closed) and deliberately does NOT extend the C++ + // try/catch shim. An invalid leaf anywhere in the tree poisons + // the whole query, exactly as gn3's mset-time rejection did. + if parsed + .unique_terms() + .any(|term| term.to_string() == INVALID_LOCATION_SENTINEL) + { + return Err(SearchError::InvalidQuery( + "InvalidOperationError: Query is invalid (invalid Hs/Mm location field)".to_string(), + )); + } + + // 3. Guard the numeric-range contract the fork's decline bridge no // longer enforces at the parser level (C3.4; see // [`guard_numeric_ranges`]). Parser syntax errors keep // precedence, as in gn3; the guard runs before the index is @@ -714,6 +819,105 @@ mod tests { ); } + /// The Hs/Mm registrations (C3.6): valid locations expand to the + /// gn3 triplet — species term AND chromosome term AND slot-2 range + /// (search.py:185-193, no liftover); every gn3 ValueError condition + /// produces the sentinel leaf instead (search.py:178-183, 197-201), + /// which the engine guard maps to InvalidQuery before any mset call. + #[test] + fn hs_mm_locations_parse_to_species_chromosome_ranges() { + let mut qp = build_query_parser(); + let parse = |qp: &mut QueryParser, q: &str| { + qp.parse_query_checked::<&str>(q, None, None) + .unwrap_or_else(|e| panic!("{q} must parse: {e:?}")) + }; + let terms = |query: &Query| { + query + .unique_terms() + .map(|t| t.to_string()) + .collect::<std::collections::BTreeSet<_>>() + }; + + // Q40's shape: Hs:chr2:1M..1.2M -> XShuman AND XC2 AND a slot-2 + // range (r2-grammar's measured parse form, without liftover). + let hs = parse(&mut qp, "Hs:chr2:1M..1.2M"); + let hs_terms = terms(&hs); + assert!(hs_terms.contains("XShuman"), "got {hs_terms:?}"); + assert!(hs_terms.contains("XC2"), "got {hs_terms:?}"); + assert!(hs.to_string().contains("VALUE_RANGE 2 "), "got {hs}"); + + // Q41's shape: Mm:chr1:5M -> XSmouse AND XC1 AND a ±50 kb range. + let mm = parse(&mut qp, "Mm:chr1:5M"); + let mm_terms = terms(&mm); + assert!(mm_terms.contains("XSmouse"), "got {mm_terms:?}"); + assert!(mm_terms.contains("XC1"), "got {mm_terms:?}"); + assert!(mm.to_string().contains("VALUE_RANGE 2 "), "got {mm}"); + + // The whole value lowercases before splitting (search.py:178). + let upper = parse(&mut qp, "Mm:CHR1:5M"); + let upper_terms = terms(&upper); + assert!(upper_terms.contains("XC1"), "got {upper_terms:?}"); + + // Every gn3 ValueError condition marks the query with the + // sentinel leaf — the engine guard's detection signal. + let invalid = [ + "Hs:foo:1M", + "Mm:foo:5M", + "Hs:chr1", + "Mm:chr1", + "Hs:chr1:5M:extra", + "Hs:chr1:abc", + "Mm:chr1:1.5", + ]; + for q in invalid { + let query = parse(&mut qp, q); + assert!( + terms(&query).contains(INVALID_LOCATION_SENTINEL), + "{q} must carry the invalid-location sentinel" + ); + } + + // An invalid leaf poisons the whole query, as gn3's mset-time + // OP_INVALID rejection did (search.py:293-301 has no try/except). + let combined = parse(&mut qp, "Hs:foo:1M OR alcohol"); + assert!(terms(&combined).contains(INVALID_LOCATION_SENTINEL)); + + // And valid locations never carry it. + let clean = parse(&mut qp, "Mm:chr1:5M OR alcohol"); + assert!(!terms(&clean).contains(INVALID_LOCATION_SENTINEL)); + } + + /// The C3.6 engine guard: the sentinel anywhere in the parsed query + /// surfaces as InvalidQuery BEFORE the index is opened — so the + /// invalid-location class reports as a query fault (the blessed 422) + /// even with no index configured, mirroring gn3's parse-before-open + /// order (search.py:291 vs :296). + #[test] + fn hs_mm_engine_guard_fires_before_the_index() { + let prior = std::env::var("XAPIAN_DB_PATH").ok(); + // SAFETY: hermetic single-threaded unit test; restored below. + unsafe { std::env::set_var("XAPIAN_DB_PATH", "") }; + let outcome = futures_of_native_guard("Hs:foo:1M"); + match prior { + Some(value) => unsafe { std::env::set_var("XAPIAN_DB_PATH", value) }, + None => unsafe { std::env::remove_var("XAPIAN_DB_PATH") }, + } + assert!( + matches!(outcome, Err(SearchError::InvalidQuery(_))), + "the guard must report InvalidQuery before the index is touched" + ); + } + + /// Drive [`Native::search`] synchronously for the guard test above + /// (the engine is async only to keep xapian off the runtime + /// workers; the tokio single-threaded runtime here is inert). + fn futures_of_native_guard(query: &str) -> Result<Vec<SearchHit>, SearchError> { + let rt = tokio::runtime::Builder::new_current_thread() + .build() + .expect("guard test runtime"); + rt.block_on(Native.search(query, "phenotype", 5)) + } + /// The index-open guards, without touching a real index. #[test] fn open_database_maps_failures_onto_the_seam_vocabulary() { |
