|
Three things the first cut was missing, and both were asked for.
* An automatic version history. Revision 1 is the column as first
stored and each PATCH adds one -- with its own digest and Unix
timestamp -- and earlier revisions are kept. GET
/baskets/{name}/revisions is the history, ?revision=N reads an old
column, and a mapping can pin one: {"basket": "wbc-edit@2"}. That is
the point of a digest: a result quoted yesterday can still be
reproduced after the column has moved on.
* Files named for the user, not a hash of the subject:
BASKET_DIR/<user>.sqlite (0600, WAL). Tokens rotate, subjects do not,
so a new token finds the same baskets -- and an operator can see
whose baskets a file holds. A subject that is not filename-safe is
sanitised and disambiguated with a short digest, and a pre-naming
hash file is renamed on first use rather than orphaned.
* /compute/rqtl and /compute/gemma and /compute/fitqtl take `basket`
too, so all four engines share one meaning: a basket resolves to the
same (name, value) pairs a client would have posted, and every reply
records the revision and digest it ran.
The schema upgrade is a real one, not a reset: an older file's single
column becomes revision 1 of the history, cells copied across with the
digest and timestamp they already had. Tested by building the old schema
by hand and opening it (an_older_file_is_upgraded_with_its_column_intact),
which is also how I found that the migration was recreating a table the
base schema had already made.
Verified on all four engines, head and pinned: rqtl 3811 rows, gemma
3809, qtl2 3730, fitqtl 3 markers -- each reply carrying "Basket '...'
revision N (digest ...)".
|
|
The next phase is changing data, and the constraint is that MariaDB is
never written to. A basket is the storage half of that: take a trait's
values, edit some of them (or build a covariate column that does not
exist), keep the column under a name, and map that instead.
* POST /baskets stores a complete column; GET /baskets lists yours;
GET /baskets/{name} its metadata; GET /baskets/{name}/values the
cells; PATCH /baskets/{name} replaces the column with a new revision;
DELETE removes it.
* POST /compute/qtl2 takes "basket": "<name>" in place of `samples`,
and both at once is a 422 rather than a silent choice. The mapping
code below it is untouched -- a basket resolves to the same
(name, value) pairs a client would have posted.
* One SQLite file per user: BASKET_DIR/<sha256(token sub)>.sqlite, 0600,
WAL, busy_timeout. `GET /baskets` only ever lists the caller's own;
the file name is a hash, so a directory listing does not name users.
* The full column is stored, not a diff -- the caller has the original
beside it, and a mapping needs one complete column.
* Text with a parsed copy beside it: raw (what was typed), value (the
number or NULL), status (ok/missing/bad). "NA" and 0 stay different
claims, and a column with something that is neither a number nor a
missing marker (NA, x, -, ., ?) is refused with the samples named --
silently dropping it would change a trait without saying so.
* A revision and a digest: an edit bumps the revision and changes the
digest, and a mapping reply records both ("Basket 'wbc-edit'
revision 2 (digest 9559a70e...)"), so two runs on one name cannot
look alike. A result cache must key on the digest, not the name.
The loop, end to end on BXD trait 10001: 34 values posted as a basket,
one bumped by 5 and one marked NA, revision 2 mapped with /compute/qtl2
-- 3730 rows, peak chr8 112.9 Mb, and the reply's note naming the
revision and digest.
tests/baskets.rs walks that loop (create, list, values, patch, map by
name, delete) and the refusals: a non-number named in the error, a name
that cannot become a path, an unknown basket, and basket-plus-samples.
Unit tests cover parsing (NA/x/-/. vs NaN/inf vs "twelve"), the digest
changing on edit, per-user isolation, and name validation.
rqtl/gemma take the same 10-line change next -- their `samples` shapes
differ, so they are a separate commit rather than a rushed one.
|