about summary refs log tree commit diff
path: root/scripts
AgeCommit message (Collapse)Author
33 hoursindex: title a reasoning doc with its gist, and make --rebuild stand alonePjotr Prins
A reasoning document was titled with the subject of the commit it is attached to (COALESCE(c.subject,'') through a LEFT JOIN), which is wrong twice over: an entry recorded with --commit none -- every entry before its commit exists -- was indexed with an empty title, and its gist, the assessed one-line claim that `list` and `why` show, was not indexed as text at all; an attached entry merely repeated the title its commit document already carries. The title is now the entry's own gist, from reasoning.gist, and the join is gone. `update-index --rebuild` also died with "mkdir failed (No such file or directory)" whenever the index directory was absent -- the state of every store that has never been indexed, and the state README.md tells a reader to produce by deleting the index. Only the non-rebuild path created the directory; `mkdir -p` now runs before the database handle either way. The check phase records a gist deliberately different from the body and asserts both that it answers a search and that it is the document's title, so a regression fails the build rather than the reader. .gitignore covers *.sqlite.d/, the default index directory, which neither *.sqlite nor *.sqlite-* matched -- the README already claimed the index was ignored. doc/SKILL.md says what `title` means in a search reply: the gist for a reasoning entry, the subject for a commit.
39 hoursinstall.sh: replace an existing root instead of failing on itPjotr
The first version worked exactly once. guix build --root cannot replace a root that already exists -- it answers guix build: error: failed to create GC root '.../root': File exists -- so the second run, which is the normal case, died before building. The script now removes the old symlink first and refuses to touch anything that is not a symlink, since a directory at that path is someone else's mistake. Verified by running it three times in a row against a tree that changed between runs (adding this file moves the store path, which is what makes the root worth having).
39 hoursinstall.sh: build, root and link, so no store path is ever written downPjotr
guix build -f guix.scm prints a path that changes with every rebuild of the tree, so the practical way to use it has been to copy a store hash into a script -- and a copied hash outlives the source. That is not hypothetical: a store on this box was created by a binary that predated the gist column, and the mismatch surfaced as an empty 'sql error (Error):' at write time. scripts/install.sh builds from this checkout, GC-roots the result (AI_MEMORY_ROOT, default ~/.local/state/ai-memory/root) so a guix gc cannot take it away, links AI_MEMORY_LINK (default ~/.local/bin/ai-memory) so nobody has to remember a hash, and then runs the binary once to prove it starts. Two builds in a row give the same store path, which is what makes the root stable between runs and not merely between rebuilds of the same day. README and doc/SKILL.md now lead with it, and keep the bare guix build as the unpinned alternative.
5 daysindex: a commit inherits the topics of the reasoning attached to itPjotr
topic:pve answered with nine reasoning entries and no commits, which is backwards for anyone following the story: the pve thread is carried by the commits (cd3e65b, 64e63bd, 3a42b5f, 8e11ec0) as much as by the notes. Commit documents now take the sessions of the reasoning attached to them (one GROUP_CONCAT query over reasoning, joined in memory) and gain the same S<session> and O<topic> terms, so the same query returns 13 documents instead of 9. Also documents the honest limit of session-derived topics in doc/SKILL.md: a session named gemma-wrapper-diagnostics about the same work does not answer topic:pve, while a plain-text pve search still finds it. topic: is the curated answer, free text is the recall answer -- name a session after the concept you want it found by.
5 daysAdd a topic index: Xapian over the store, driven through GuilePjotr
SQLite stays the truth; this is a derived index that lets a reader enter by concept rather than by symbol. Two commands, one prefix vocabulary: ai-memory --index DIR update-index [--rebuild] ai-memory --index DIR search 'topic:pve repo:genenetwork4' Prefixes: topic:, repo:, type: (reasoning|commit), session:, commit:, date:. Free text is stemmed; default operator AND, --or to widen. Topic terms come from the session id split on hyphens, so a session named "gemma-pve" answers topic:pve and topic:gemma -- naming a session after a concept is how an entry becomes findable by it. Why Guile and a subprocess: libxapian is C++-only, so a foreign call from Odin means a hand-written C shim with its own exception boundary; guile-xapian already ships one, tested and maintained. So scripts/index.scm owns every Xapian call (426 lines: schema constants, record builders, updater, query parser, JSON writer) and ai-memory shells out to it -- index.odin is 145 lines of process plumbing. The boundary is a store path in and JSON out, which also means the search reply needs no parser on this side. Incremental in the way that matters: documents are deleted by their unique term and re-added, and Xapian applies that as changed postings only. The whole body is re-tokenised each run (~1 s at a few hundred documents) which buys not keeping any "what did I index last time" state that could go stale. Two discoveries shaped it, both from guile-xapian 0.4.0's SWIG surface: replace by term silently appends (the string lands on the docid overload), so delete then add; and Database-get-document by term returns nothing, so no digest compare was attempted. The package's check phase now exercises it end to end -- throwaway store, one entry, update-index, then `search topic:index | grep -q '"count":1'` -- because a check that only links proves nothing about the four Guile dependencies. guix.scm installs scripts/index.scm to share/ai-memory (where index.odin looks) and wraps the binary with GUILE and GUILE_LOAD_PATH for guile-xapian and guile-sqlite3. doc/SKILL.md gains the usage, the prefix list and the reason for the subprocess.