about summary refs log tree commit diff
path: root/test/data/parity-fixtures/README.md
blob: 415e3596b36f9c327348ea01d377626f058046c7 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# Parity fixtures (reduced-v1)

The 54 recorded production search responses, reduced to what the parity
harness (`tests/search_parity.rs`) actually grades: the recorded status,
the full sorted `(dataset, name)` identifier set, and the first-ten-in-
recorded-order top 10. Error shapes carry their diagnostic body instead.

- Recorded from `https://genenetwork.org/api3/search/` on 2026-10-05
  (`per_page=50000`, `page=1`; see the recording script's parameters);
  14 fixtures re-recorded (Q02, Q14, Q19, Q20, Q22, Q23, Q26, Q32,
  Q33, Q34, Q40, Q41, Q44, Q45) on 2026-10-07, see below.
- The raw payload corpus (~121 MB) deliberately stays OUT of git; these
  reduced files (7.2 MB) are what grading consumes, and the harness's
  loader accepts both formats (top-level array = full fixture, object
  with `"format": "reduced-v1"` = reduced) in one directory.
- Regenerate from a full corpus with:
  `python3 reduce_fixtures.py <full_corpus_dir> . ../../tests/search_parity.rs`
  — the shape table is parsed from the harness itself so the two cannot
  drift; the generator fails loudly on any hit missing its
  `(dataset, name)` identity.

The 2026-10-07 re-recording closed the truncation gap and made the
zero-hit shapes discriminating where possible:

- Q02 (`insulin`, gene) was re-recorded through a verified pagination
  walk: 50000 + 7632 = 57632 hits, pages proven disjoint — no fixture
  in this corpus is truncated any more.
- Q26 (`chr:2`) was proven to match >1.1M documents and was re-shaped
  to the bounded `chr:m` (9374 hits), which exercises the same
  `chr:`→`XC` boolean-filter grammar.
- 12 shapes that previously recorded zero hits (Q14, Q19, Q20, Q22,
  Q23, Q32, Q33, Q34, Q40, Q41, Q44, Q45) were re-shaped to real
  values, so clause (b) of the parity metric bites on 36/36 shapes
  instead of 24/36.
- Q04, Q11, Q12, Q21 honestly remain empty-vs-empty: the deployed
  index carries no positional data (phrase/NEAR/ADJ can never match)
  and the `description:` (XD) field is unpopulated. They exercise
  parser acceptance only until the indexer is fixed.