name: gn-memory description: Keep a repository's ai-memory store current — import git history, record reasoning while working, and answer "why does this code look like this?" (or "how did we run that?") after a session's context has been cleared.
gn-memory
ai-memory is a per-repository store that links what changed (derived from
git) to why it changed (written by you). Three tables:
| table | source | contents |
|---|---|---|
commits |
derived | hash, author timestamp, subject, body |
symbols + touches |
derived | every declaration a commit added, removed, or merely enclosed (hunk context) |
reasoning |
authored | append-only thoughts, conclusions and questions; each records its basis and may supersede an earlier entry |
Query flow: symbol → commits that touched it → reasoning behind those commits.
Getting the binary
scripts/install.sh # build, GC-root, link ~/.local/bin/ai-memory
That is the one to use: a bare `guix build' prints a path that changes with every rebuild, so anything you write down goes stale, and a stale binary against a newer store is the schema skew that fails at write time rather than at start-up.
guix build -f guix.scm # build (odin + sqlite are in guix.scm)
$(guix build -f guix.scm)/bin/ai-memory --help
Or build locally into bin/ (gitignored) with the flags listed at the top of
guix.scm:
guix shell -D -f guix.scm -- odin build . -out:bin/ai-memory -vet -strict-style \
-collection:sqlite3=$GUIX_ENVIRONMENT/share/odin-sqlite3 \
-define:SQLITE3_SYSTEM_LIB=true -define:SQLITE3_DYNAMIC_LIB=true \
"-extra-linker-flags:-L$GUIX_ENVIRONMENT/lib -Wl,-rpath,$GUIX_ENVIRONMENT/lib"
guix build -f guix.scm also runs the package's check phase, which builds a
throwaway store, records an entry, indexes it, queries it, and then makes a
two-commit git repository holding one C++ file to prove the ctags half (an added
function is add, a changed body is context).
universal-ctags is what turns C and C++ files into symbols; the Guix package
bakes its absolute path into $AI_MEMORY_CTAGS, so nothing depends on PATH. If
the binary is missing the store still works -- C and C++ simply keep hunk
context and get no add/remove -- and sync says so once on stderr.
The store defaults to <repo>/ai-memory.sqlite; override with --db PATH or
$AI_MEMORY_DB. --repo PATH / -C PATH selects the repository, and may be
given before or after the command.
Session protocol
- Once per repo:
ai-memory -C REPO init(idempotent, creates/upgrades the schema). - After every commit:
ai-memory -C REPO sync. It is cheap and idempotent — re-running it only refreshes derived rows. Useimportinstead ofsynconly to pull in a whole history the first time (importhas no limit). - Before changing code: ask the store, do not guess.
ai-memory -C REPO why SYMBOLandai-memory -C REPO search TEXT. - While working: record as you go, not at the end.
--kind thoughtfor what you are weighing,--kind conclusionfor what you settled on, and--kind questionwhen you need an answer from the user. - Before the commit exists there is nothing to attach to: use
--commit none, then after committing runai-memory -C REPO attach HEAD --session S. - Write for a reader who has none of this conversation. The next session starts with an empty context window; see Session continuity for what that means for operational work in particular.
- Correcting yourself is not a normal record.
--supersedes IDchanges whatwhyandsearchanswer from then on, so it needs the user's consent first — see the Rules below.
ai-memory -C REPO record --session fix-parser --step 1 \
--kind thought --basis "read at git.odin:120-141" \
--gist "Hunk headers name the enclosing fn; a new fn still needs the scan" \
--body "Hunk headers name the enclosing fn; a new fn needs the declarative scan." --commit none
ai-memory -C REPO record --session fix-parser --step 2 \
--kind conclusion --basis "measured on genenetwork4, 284 commits in 2.5 s" \
--gist "Keep both signals; add ctags only if add/remove proves noisy" \
--body "Keep both signals; add ctags only if add/remove proves noisy."
ai-memory -C REPO attach HEAD --session fix-parser
record and attach import the commit on demand, so they work immediately after
committing even if sync has not run yet.
Session continuity: recovering after the context is cleared
A model's context is finite -- a million tokens if you are lucky, and the conversation that produced a piece of work is gone when it ends. The store is the part that outlives it: git-derived history for what happened, authored reasoning for why, and a topic index over both for finding it by concept. Treat recording as part of the work, not bookkeeping after it, because the next session starts with none of this conversation.
What recovery looks like in practice, measured on the GEMMA/ravanan work in
genenetwork4 (2026-11-12):
| step | cost |
|---|---|
search "slurm container" (SQLite FTS, no index needed) |
16 ms |
| the same through the topic index | ~100 ms (1.3 s to build the index once) |
| re-running what the entry says (container up → endpoint verified) | ~20 s of machine time |
The first two lines are the point: rediscovering what to type costs milliseconds, where reconstructing it from the diff, the docs and a guess would cost the session.
So, what to record so that a cold reader arrives at a working command:
- A RUNBOOK entry for anything operational, with the commands themselves —
not just what you concluded. An entry that says "the container has a stale
JWT and needs
HOMEwritable" is useful; an entry that also holds the four commands, the port, the fingerprint and the fastest test is recovery. Write it as if the reader has none of your context, because they will not. - Name the session after the concept you want to find it by (
topic:terms come from it), and attach reasoning to its commit sowhy SYMBOLandtopic:both reach it. - Record the gotchas, not just the happy path. In that session three of the
four things that had to be right were wrong at first (a ravanan build without
the endpoint fix, the job's
HOMEnot writable in the container,ccwlmissing fromPATH). Those are the sentences that save the next hour. - Point at the long-form docs (
docs/…, a session note in another repo) rather than copying them: the store is for the shortest path to the right document, not a second copy of it.
Two honest limits. The store is per repository, so a fact that spans repos
(the pve story in genenetwork4, gemma-wrapper and guix-bioinformatics)
needs recording in each — or one shared index with a repo: filter. And the
store is local, gitignored SQLite: it survives a cleared session, not a lost
machine. Anything that must outlive the box belongs in commits and docs as well
— which is also why every entry here is paired with a commit message that says
the same thing.
The topic index (optional, derived)
A second, derived store: a Xapian index over the SQLite one, so a reader can
enter by concept ("topic:pve") or by repository and kind ("repo:genenetwork4
type:commit") instead of by symbol name. It is disposable -- delete the
directory and update-index --rebuild -- and SQLite stays the truth.
ai-memory --index DIR update-index # bring it up to date
ai-memory --index DIR update-index --rebuild # after a schema change
ai-memory --index DIR search 'topic:pve'
ai-memory --index DIR search --limit 5 'type:commit ravanan repo:genenetwork4'
(The directory defaults to <store>.d/index, override with --index or
$AI_MEMORY_INDEX. search without --index keeps using SQLite FTS.)
Prefixes: topic:, repo:, type: (reasoning|commit), session:,
commit: (7-character hash), date: (yyyymmdd). Everything else is free
text, stemmed. The default operator is AND; --or widens it.
Topic terms come from the session id, split on hyphens: a session called
gemma-pve answers topic:gemma and topic:pve. So the cheapest way to make
an entry findable by concept is to name the session after the concept -- a
session called gemma-wrapper-diagnostics about the same work does not answer
topic:pve, while a plain-text pve search still finds it. That is the
difference between the two: topic: is the curated answer, free text is the
recall answer.
A commit inherits the topics of the reasoning attached to it, so
topic:pve returns both the reasoning and the commits that carry it.
Why a subprocess and a Guile script: libxapian is C++-only, so a foreign call
from Odin needs a hand-written C shim. guile-xapian already has one, tested
and maintained (scripts/index.scm owns every Xapian call; ai-memory shells
out to it). The boundary is a store path in, JSON out.
The JSON a search returns is the record as indexed -- key, repo, session,
kind, date, commit, title, body, terms -- plus docid and
percent. count is how many came back, matches how many the index found.
For a reasoning entry title is its gist (the assessed one-line claim), not
the subject of the commit it is attached to: that subject already titles the
commit document, and an entry recorded with --commit none has no commit to
borrow from. The gist is indexed as text too, so a claim written only in the
gist is findable; a commit document's title is its subject, as before.
Rules
- Never write to
commits,symbolsortouchesby hand. They are derived;syncowns them and hand edits will be silently overwritten or duplicated. reasoningis append-only. Do not edit or delete an entry to change your mind: record a correction with--supersedes ID. The old entry stays visible as#12 [superseded by #15],whyandsearchanswer with the correction, and--allshows the whole chain.attachis the only other update the tool makes (setting a NULLcommit_hash).- Ask the user before superseding.
--supersedesrewrites history as far as every later reader is concerned:whyandsearchstop answering with the old entry, and that entry collapses to a one-line stub. The row survives, but the record's answer does not. So: say which entry id you propose to replace, what it claimed, what the replacement says, and on what basis — then wait for a yes. Do not bundle it into a batch of records, and never supersede an entry written by someone else without their consent. If the user declines, either record the correction as a new standing entry without--supersedes(both claims stay visible and the contradiction is on the record) or do not record it at all. - Every entry says how it was established (
--basis):read at src/lib.rs:412-470,from commit message,measured on the gn4 instance, or plainlyinferred. Inference and observation look identical in prose, and that is exactly what makes a wrong entry unrecognisable later. - Every entry carries a gist (
--gist), and the gist is a claim, not a subject line: "GEMMA does accept -lmm 9 -pve; boundary se is -nan, so store None" beats "About pve reporting". Write it while recording, because that is the only moment the context is in hand —list,whyandsearchshow the gist and hide the body behind--full. Without--gistthe tool falls back to the body's first sentence and says so on stderr; that fallback says what the entry is about, which is why prose-heavy entries read as truncated fragments. - One session per task, monotonically increasing
--step.(session_id, step, kind)is unique, so re-running a step is a no-op rather than a duplicate. - Make bodies self-contained. Say what you considered, what you chose, what you rejected and why. A future reader has the diff but not the conversation.
- Name what you renamed. Symbols are
<file-base>.<identifier>(e.g.gemma.compute_gemma); an old name keeps its history, so mention the new name in the body to keepsearchuseful. - Never commit the store or the build output.
/bin/andai-memory.sqlite*are in.gitignore; commit only source. Importing history is read-only —-Cnever touches the working tree.
Rewriting history
git commit --amend, rebase and squash change commit hashes. Every
touches and reasoning row attached to the old hash is then stale, and sync
will not clean it up — it only adds. The repair, in this order:
- detach the reasoning for the old hash:
update reasoning set commit_hash = null where commit_hash = '<old>' ai-memory -C REPO sync— imports the new hashai-memory -C REPO attach <new|HEAD> --session S— re-links the entries- delete the old commit row:
pragma foreign_keys=on; delete from commits where hash = '<old>'
Step 4 needs the pragma: the sqlite3 CLI leaves foreign keys off, so the
delete runs without cascading and the old commit's touches stay behind as
orphans. Nothing complains — status just reports double the touches, and
why points at a hash that is no longer in git log. Check afterwards with
pragma foreign_key_check and by counting rows whose commit_hash has no
matching commits row.
This is the one place where hand-written SQL is correct: the tool repairs nothing. Amending before recording is cheaper than repairing afterwards.
Interpretation
[fn gemma.compute_gemma, context]means a hunk landed inside that function, not that the function was declared.add/remove/modare declaration changes.- Declarations come from three places: a line scanner for
.odinand.rs, and for C and C++ (.c .h .cc .cpp .cxx .hpp .hh .hxx .ipp .tpp .inl)universal-ctagsover the file itself, diffed between the commit's blob and its parent's -- which is why a C++addmeans the declaration is new, not that a line mentioning it moved. Everything else still gets hunk-context symbols ([fn name, context]) thanks to git's userdiff drivers, so "which function was I in" works for Python, Go, … too. - The ctags pass is cached per git blob (table
blobs), keyed by object id: a file version is parsed once per store, and re-syncing an imported history runs no ctags at all. Dropping the table only costs time. - C and C++ keep their extension in the symbol name
(
lmm.cpp.mdb_analyze), so a declaration in a header and the definition in the source are two symbols thatwhy compute_mafshows side by side. Directories are still stripped, so a file that moved between directories reads as the same symbol (that collision predates ctags and applies to every language). - Kinds are the same five as everywhere else:
fnfor functions, prototypes and methods;typefor class, struct, union, enum, typedef;constfor macros except include guards;varfor file-scope variables. Class members, enumerators, lambdas and namespaces are deliberately not tags here -- they churn without changing what a hunk was about. why SYMBOLtries three things in order, all but the last indexed: the qualified name (gemma.compute_gemma), then the bare identifier viasymbols.basename(compute_gemma), then aLIKE '%…%'scan for partial names.logprints how many reasoning entries each commit carries — commits with(0)are the ones worth annotating later.listis the inventory view: one line per entry,short-hash dd/mm #id kind gist. Unattached entries come first (labelleduncommitted, dated by when they were written), then the newest commit's, and a superseded entry readskind->#N. Dates are UTC and carry the year only when it is not the current one, so a listing stays narrow without becoming ambiguous after a year.--session,--kindand--commitnarrow it; skimlistbefore opening entries withwhy.why SYMBOLandsearch TEXTprint the gist per entry;--fulladds the body indented beneath it, and--alladmits superseded entries. Gist-first is the default because a listing of twenty bodies is not read.
Troubleshooting
no commits to import— the repo has no history yet, or--repopoints elsewhere.whyfinds nothing — runsyncfirst; or the symbol lives in a file type we do not parse for declarations.- Touch counts looking double, or a commit in the output of
logthat is not ingit log— a hash was rewritten (amend/rebase). See Rewriting history; the stale rows do not repair themselves. searchfinds nothing after manual DB edits — FTS is trigger-maintained; rebuild withinsert into reasoning_fts(reasoning_fts) values('rebuild').- Direct inspection is fine (read-only):
sqlite3 ai-memory.sqlite \
"select c.hash, c.subject, r.kind, r.body
from symbols s join touches t on t.symbol_id = s.id
join commits c on c.hash = t.commit_hash
left join reasoning r on r.commit_hash = c.hash
where s.name like '%compute_gemma'
order by c.author_ts desc;"
commit,symbolandtouchare SQL keywords; the tables are plural.
