|
SQLite stays the truth; this is a derived index that lets a reader enter by
concept rather than by symbol. Two commands, one prefix vocabulary:
ai-memory --index DIR update-index [--rebuild]
ai-memory --index DIR search 'topic:pve repo:genenetwork4'
Prefixes: topic:, repo:, type: (reasoning|commit), session:, commit:,
date:. Free text is stemmed; default operator AND, --or to widen. Topic
terms come from the session id split on hyphens, so a session named
"gemma-pve" answers topic:pve and topic:gemma -- naming a session after a
concept is how an entry becomes findable by it.
Why Guile and a subprocess: libxapian is C++-only, so a foreign call from
Odin means a hand-written C shim with its own exception boundary; guile-xapian
already ships one, tested and maintained. So scripts/index.scm owns every
Xapian call (426 lines: schema constants, record builders, updater, query
parser, JSON writer) and ai-memory shells out to it -- index.odin is 145 lines
of process plumbing. The boundary is a store path in and JSON out, which also
means the search reply needs no parser on this side.
Incremental in the way that matters: documents are deleted by their unique
term and re-added, and Xapian applies that as changed postings only. The whole
body is re-tokenised each run (~1 s at a few hundred documents) which buys not
keeping any "what did I index last time" state that could go stale. Two
discoveries shaped it, both from guile-xapian 0.4.0's SWIG surface: replace by
term silently appends (the string lands on the docid overload), so delete then
add; and Database-get-document by term returns nothing, so no digest compare
was attempted.
The package's check phase now exercises it end to end -- throwaway store,
one entry, update-index, then `search topic:index | grep -q '"count":1'` --
because a check that only links proves nothing about the four Guile
dependencies. guix.scm installs scripts/index.scm to share/ai-memory (where
index.odin looks) and wraps the binary with GUILE and GUILE_LOAD_PATH for
guile-xapian and guile-sqlite3.
doc/SKILL.md gains the usage, the prefix list and the reason for the
subprocess.
|