about summary refs log tree commit diff
path: root/tests/baskets.rs
AgeCommit message (Collapse)Author
8 daysbaskets: every revision kept, files named for the user, all four enginesPjotr
Three things the first cut was missing, and both were asked for. * An automatic version history. Revision 1 is the column as first stored and each PATCH adds one -- with its own digest and Unix timestamp -- and earlier revisions are kept. GET /baskets/{name}/revisions is the history, ?revision=N reads an old column, and a mapping can pin one: {"basket": "wbc-edit@2"}. That is the point of a digest: a result quoted yesterday can still be reproduced after the column has moved on. * Files named for the user, not a hash of the subject: BASKET_DIR/<user>.sqlite (0600, WAL). Tokens rotate, subjects do not, so a new token finds the same baskets -- and an operator can see whose baskets a file holds. A subject that is not filename-safe is sanitised and disambiguated with a short digest, and a pre-naming hash file is renamed on first use rather than orphaned. * /compute/rqtl and /compute/gemma and /compute/fitqtl take `basket` too, so all four engines share one meaning: a basket resolves to the same (name, value) pairs a client would have posted, and every reply records the revision and digest it ran. The schema upgrade is a real one, not a reset: an older file's single column becomes revision 1 of the history, cells copied across with the digest and timestamp they already had. Tested by building the old schema by hand and opening it (an_older_file_is_upgraded_with_its_column_intact), which is also how I found that the migration was recreating a table the base schema had already made. Verified on all four engines, head and pinned: rqtl 3811 rows, gemma 3809, qtl2 3730, fitqtl 3 markers -- each reply carrying "Basket '...' revision N (digest ...)".
8 daysbaskets: a user's own column of values, in their own SQLite filePjotr
The next phase is changing data, and the constraint is that MariaDB is never written to. A basket is the storage half of that: take a trait's values, edit some of them (or build a covariate column that does not exist), keep the column under a name, and map that instead. * POST /baskets stores a complete column; GET /baskets lists yours; GET /baskets/{name} its metadata; GET /baskets/{name}/values the cells; PATCH /baskets/{name} replaces the column with a new revision; DELETE removes it. * POST /compute/qtl2 takes "basket": "<name>" in place of `samples`, and both at once is a 422 rather than a silent choice. The mapping code below it is untouched -- a basket resolves to the same (name, value) pairs a client would have posted. * One SQLite file per user: BASKET_DIR/<sha256(token sub)>.sqlite, 0600, WAL, busy_timeout. `GET /baskets` only ever lists the caller's own; the file name is a hash, so a directory listing does not name users. * The full column is stored, not a diff -- the caller has the original beside it, and a mapping needs one complete column. * Text with a parsed copy beside it: raw (what was typed), value (the number or NULL), status (ok/missing/bad). "NA" and 0 stay different claims, and a column with something that is neither a number nor a missing marker (NA, x, -, ., ?) is refused with the samples named -- silently dropping it would change a trait without saying so. * A revision and a digest: an edit bumps the revision and changes the digest, and a mapping reply records both ("Basket 'wbc-edit' revision 2 (digest 9559a70e...)"), so two runs on one name cannot look alike. A result cache must key on the digest, not the name. The loop, end to end on BXD trait 10001: 34 values posted as a basket, one bumped by 5 and one marked NA, revision 2 mapped with /compute/qtl2 -- 3730 rows, peak chr8 112.9 Mb, and the reply's note naming the revision and digest. tests/baskets.rs walks that loop (create, list, values, patch, map by name, delete) and the refusals: a non-number named in the error, a name that cannot become a path, an unknown basket, and basket-plus-samples. Unit tests cover parsing (NA/x/-/. vs NaN/inf vs "twelve"), the digest changing on edit, per-user isolation, and name validation. rqtl/gemma take the same 10-line change next -- their `samples` shapes differ, so they are a separate commit rather than a rushed one.