about summary refs log tree commit diff
path: root/README.md
blob: 01be428364613fe291ea1dfda88dacdac584aa8f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
# ai-memory

Memory for an AI assistant working in a git repository: it links **what
changed** (derived from git) to **why it changed** (written by the assistant
while working), so the next session can recover it.

A session's context window is finite, and the conversation that produced a piece
of work is gone when it ends. The diff, the tests and the commit messages say
*what* happened; they rarely say what was tried and rejected, what a number was
measured against, or which four things had to be fixed before something ran at
all. `ai-memory` keeps that, per repository, in one SQLite file, and answers
three questions quickly:

```sh
ai-memory -C REPO why SYMBOL    # which commits touched this, and what was the reasoning?
ai-memory -C REPO search TEXT   # full-text over the reasoning: ~15 ms
ai-memory -C REPO log           # recent commits, and how much reasoning each carries
```

## What it stores

| table | source | contents |
|---|---|---|
| `commits` | derived | hash, author timestamp, subject, body |
| `symbols` + `touches` | derived | every declaration a commit added, removed, or merely *enclosed* (hunk context) |
| `reasoning` | **authored** | append-only thoughts, conclusions and questions, each with its basis and an optional `supersedes` link |

`sync` owns the derived rows; only the `reasoning` table is written by hand
(through the tool). That split is the point: the git half is always true of the
tree, and the authored half can never be rewritten — a change of mind is a new
entry that supersedes the old one, and both stay on the record.

## Install

```sh
scripts/install.sh                # build, GC-root it, link ~/.local/bin/ai-memory
```

`guix build -f guix.scm` prints a store path that changes with every rebuild, so
the one-liner exists to make that path irrelevant: it builds from this checkout,
roots the result (`AI_MEMORY_ROOT`, default `~/.local/state/ai-memory/root`) so a
`guix gc` cannot take it, links `AI_MEMORY_LINK` (default `~/.local/bin/ai-memory`)
and then runs the binary to prove it works. Hard-coding a store path is how a
store gets written by a binary that predates the source -- which is exactly how
one was created with a schema too old for the code that later used it.

```sh
guix build -f guix.scm            # → /gnu/store/…-ai-memory/bin/ai-memory, unpinned
```

`guix.scm` builds the Odin compiler and the SQLite bindings it needs, so the
first build is not quick (Guix does not substitute Odin). For hacking on it,
the header of `guix.scm` lists the `odin build` line the package uses, and
`guix shell -D -f guix.scm -- odin version` gets you a compiler shell.

A C or C++ declaration cannot be read off a diff line -- inside a body
`foo(bar);` calls something, inside a class body the same bytes declare a
constructor -- so those files are handed to `universal-ctags`, and the tags of a
commit's blob are diffed against its parent's to say what was added or removed.
That is an optional runtime dependency: without `ctags` on the box (or in
`$AI_MEMORY_CTAGS`) the store still works, C and C++ files simply keep hunk
context and get no add/remove.  Each blob version is parsed once and cached by
its git object id, so re-syncing an imported history runs no ctags at all.

The optional topic index shells out to a small Guile program
(`scripts/index.scm`) because `libxapian` is C++-only and
[guile-xapian](https://git.systemreboot.net/guile-xapian/) already ships the C++
shim a foreign call would need. That pulls in `guile-3.0`,
`guile-xapian` and `guile-sqlite3`; core commands need none of them.

## Quick start

```sh
cd path/to/repo
ai-memory init                                  # create/upgrade ai-memory.sqlite
ai-memory import                                # whole history (first time only)
ai-memory sync                                  # after each commit: recent commits only
```

Record as you work. Bodies are for a reader who has none of the conversation,
and every entry says how it was established:

```sh
ai-memory record --session fix-parser --step 1 --kind thought \
  --basis 'read at src/parse.rs:120-180' \
  --body "Hunk headers name the enclosing fn; a new fn needs the declarative scan."

ai-memory record --session fix-parser --step 2 --kind conclusion \
  --basis 'measured on the bxd panel' \
  --body "Keep both signals. Add ctags only if add/remove proves noisy."

# before the commit exists, there is nothing to attach to
ai-memory record --session fix-parser --step 3 --kind question --commit none \
  --body "Should a symbol that moved keep its old name in the index?"
```

Then commit, and attach the entries to it:

```sh
git commit -am "parse: keep both signals; defer ctags"
ai-memory sync
ai-memory attach HEAD --session fix-parser
```

Asking afterwards — the output is one line per commit that touched the symbol,
followed by its reasoning:

```
$ ai-memory why variance_components_note
3a42b5f  2026-11-11T17:04:58Z  gemma: the pve note carries every chromosome  [fn gemma.variance_components_note, context]
    #5 conclusion: Only the range or every chromosome? The first version chose the range plus the two
       extremes to keep the note short -- wrong, and caught in review: under --loco each pve belongs to
       a different kinship matrix ...
cd3e65b  2026-11-11T15:54:15Z  gemma: report GEMMA's own pve for the null model  [fn gemma.variance_components_note, add]
    #1 thought: Whether GEMMA's -pve works with -lmm 9 under --loco was unknown ...
```

`[fn file.function, add]` means the declaration was added by that commit;
`context` means a hunk landed inside it. `log` shows which commits still carry
nothing, i.e. which are worth annotating:

```
$ ai-memory log --limit 3
7a78dce  2026-11-12T14:58:13Z  docs: say that project memory is how a session recovers  (1)
d507d32  2026-11-12T14:45:16Z  gemma/ravanan: resolve the same inputs as the direct runner  (3)
60420b6  2026-11-11T21:26:07Z  docs: which reference genome the coordinates are on  (3)

$ ai-memory status
commits: 286
symbols: 1901
touches: 3362
reasoning: 31
```

## The topic index (optional)

Derived, disposable, and over the same store: one Xapian document per reasoning
entry and per commit, so a reader can enter by **concept** instead of by symbol.

```sh
ai-memory --index DIR update-index [--rebuild]
ai-memory --index DIR search 'topic:pve repo:genenetwork4'
ai-memory --index DIR search --limit 5 'type:commit ravanan'
```

Prefixes: `topic:`, `repo:`, `type:` (`reasoning`|`commit`), `session:`,
`commit:` (7-character hash), `date:` (`yyyymmdd`). Free text is stemmed; the
default operator is AND, `--or` widens it. Topic terms come from the session id
split on hyphens — a session named `gemma-pve` answers `topic:gemma` and
`topic:pve`, and a commit inherits the topics of the reasoning attached to it.
Naming a session after a concept is how that concept becomes an entry point.

The index directory defaults to `<store>.d/index`; `--index` or
`$AI_MEMORY_INDEX` overrides it, `$AI_MEMORY_INDEX_SCRIPT` the script.
Deleting it and running `update-index --rebuild` is always safe: SQLite is the
source of truth, the index is a projection of it.

## Where the store lives

`<repo>/ai-memory.sqlite`, or `--db PATH`, or `$AI_MEMORY_DB`. Neither the store
nor the index belongs in git (they are in `.gitignore`).

Two consequences worth stating plainly:

- the store is **per repository**, so a fact that spans repositories has to be
  recorded in each, or found through one index with a `repo:` filter;
- the store is **local**, so it survives a cleared session but not a lost
  machine. Anything that must outlive the box belongs in a commit message and
  in the project's docs as well — which is why every entry here is paired with
  a commit that says the same thing in its own words.

## Rules the tool relies on

- **Never write to `commits`, `symbols` or `touches`.** They are derived;
  `sync` overwrites them.
- **`reasoning` is append-only.** A change of mind is a new entry with
  `--supersedes ID`: the old row stays, and `why`/`search` answer with the
  correction. `--all` shows the whole chain.
- **One session per task**, monotonically increasing `--step`; `(session, step,
  kind)` is unique, so re-running a step is a no-op.
- **Say how it was established** (`--basis`): a measurement, a file and line
  range, a commit message, or plainly `inferred`. Inference and observation look
  identical in prose, which is exactly what makes a wrong entry unrecognisable
  later.

Two files document the practice in more depth: [`doc/SKILL.md`](doc/SKILL.md)
(the working protocol, the index, and *session continuity* — how a cleared
context recovers), and the code itself, which is four `.odin` files plus
`schema.sql`.

## Licence

BSD 3-Clause. See [LICENSE](LICENSE).