|
index; INCREMENTAL crawls, hashes each document's complete source,
replaces only changed documents by identity term, deletes vanished
identity terms, and writes nothing when nothing changed.
* src/indexer/schema.rs: the identity/type/prefix/slot definitions
shared by the writer and the native reader.
* src/indexer/rows.rs: the gene and phenotype crawl. Both queries
carry confidentiality < 1 AND public > 0; the Python phenotype query
has no WHERE clause (the metadata leak this closes by construction).
* src/indexer/build.rs: document construction, mirroring index_genes /
index_phenotypes call for call (TermGenerator configuration, prefix
order, value slots, popped-before-storage fields, identity term).
* src/indexer/pyjson.rs: byte-exact reproduction of Python json.dumps
(separators, ensure_ascii escaping, shortest round-trip float repr
with CPython's half-to-even tie-breaking).
* src/indexer/cache.rs: the RDF RIF/wiki dictionaries, reproducing
build_rdf_cache(remove_common_words=True) from the authoritative
cache file; each dictionary is one monolithic hashed input.
* src/indexer/sha256.rs: dependency-free SHA-256 for the source and
cache hashes.
* src/indexer/sidecar.rs, src/indexer/run.rs: per-document source
hashes in a sidecar next to the index (atomic rename strictly after
the index commit), FULL and INCREMENTAL orchestration, identity-term
uniqueness assertion, and the startup key-set mismatch fallback.
* src/lib.rs: register the indexer module.
* tests/data/pyfloat.txt: reference-docdata float-token fixture for the
serializer test (generated read-only from the reference index).
|