about summary refs log tree commit diff
path: root/src/db.rs
AgeCommit message (Collapse)Author
3 dayscorrelation: the matrix endpoint, and the rule that a deploy ends with a ↵Pjotr Prins
monitor run `POST /compute/correlation/matrix` takes a list of `{dataset, trait_name}` references -- a collection, as GN2's matrix form posts it, which spans datasets -- and answers an N x N matrix of r, p and n. The five decisions that are not obvious, each recorded where a reader will look for them: * **One method for the whole matrix.** GN2's `show_corr_matrix.py` flips to Spearman mid-row once a Pearson r passes 0.999, which the self-pair always does, so its matrix is asymmetric by construction. Here the method is chosen once and r[i][j] == r[j][i] exactly. * **Each unordered pair is computed once and mirrored** -- N(N-1)/2 engine calls, not N^2. * **The diagonal is by construction**: r = 1, p = null, n = the trait's own count, so half the traits cannot come back as 1.0000000000000002 with a NaN p. * **Per-cell n is its own matrix**, and a cell under `min_overlap` (default 6) is null in r and p while its n still says how many points there were. GN2 accepts 2, where |r| = 1 by definition. * **The sample axis is the first trait's group** (#106), resolved once and stated in the reply with the number of individuals it kept. Fetches are one query per dataset (`get_dataset_values_for_traits`, `get_publish_values_for_traits`), not one per trait: a matrix chooses a handful out of twenty thousand. Capped at 1 000 traits, because the cost is quadratic in the correlations and in the reply. No PCA -- the r matrix has to be trusted on its own first. Verified twice: cells agree with scipy to the last bit (BXDPublish 10001/10002/10005: 0.6756270426459695, 0.6950450737755965, 0.8819879006700077 against scipy 0.6756270426459695, 0.6950450737755964, 0.8819879006700073), and a cell equals what the row endpoint says for the same pair (0.6950450737755965 in both). Edge cases exercised: a 1x1 matrix, `min_overlap` above every cell's n, an unknown trait, `bicor`, and no traits at all. The operator checklist now says a deploy ends with `bin/monitor-gn4 --tier all`, and says why: `/version` can report a commit and a reachable database while every engine that shells out is dead. The monitor gained a matrix check whose judge enforces square, symmetric and a unit diagonal.
3 dayscorrelation: tissue and literature axes, paired the way the data actually isPjotr Prins
`type` on /compute/correlation now selects GN2's `corr_type`: sample (default), tissue or literature. Partial is refused by name rather than silently treated as sample. **tissue** reduces each gene symbol to its highest-Mean probe in TissueProbeSetXRef and pairs the two value vectors **by TissueID**. That last word is the difference from GN2, which keeps each vector as a bare list and pairs by position on the assumption that every probe covers the same tissues. On this database it does not: every probe carries exactly 26 values (liver, kidney, spleen, ... peripheral) or exactly 32 (striatum, hippocampus, neocortex, ... brain regions), the two sets are disjoint, and 45 281 probes are in each. Pairing those positionally correlates liver expression with striatum expression because both happen to be entry one. Pairing by tissue means a cross-panel pair has no correlation at all, which is the honest answer and makes the panel boundary visible instead of inventing a number. Verified independently for Shh vs Hcrtr1: 0.7222718302270719 from the endpoint, 0.7222719072696666 from a separate Pearson over the same TissueID-aligned pairs (the residue is MySQL's single-precision `float` column as printed by the client). **literature** looks the pair up in LCorrRamin3 in both directions after translating rat/human ids through GeneIDXRef, in chunks of 4 000 genes (the first attempt died on MySQL's 65 535-placeholder limit). It is a precomputed value, not a statistic: p and n are null. A trait with no gene id or no row is left out and counted in the note, where GN2 reports 0.0 and fills the tail of its top-N with them -- that reads as "measured, no relationship", which is not what happened. Gene ids come from ProbeSet.GeneId, not ProbeSetXRef.GeneId: the latter has no such column here, which GN3's own mapping also records.
3 dayscorrelation: run over the group's sample axis, and report a perfect ↵Pjotr Prins
correlation as 0.0 Two divergences from the live genenetwork.org, both measured against it. **The sample axis.** GN2's `get_sample_r_and_p_values` walks `target_dataset.group.samplelist` and takes the reference value by name: the correlation runs over the group's animals, not over every individual the trait happens to have a value for. Trait 1427571_at of HC_M2_0606_P carries 99 values, the BXD panel holds 67 of those individuals, and the answer changes materially -- GN2's top hit is a -0.845 probe that does not appear in our top five at all. `corr_samples_group` now takes `samples_primary` (default, the group's genotype panel, which is where GN2's list comes from) or `samples_all` (the old behaviour); the panel is read with `genotypes::resolve`, falling back to the group's used-for-mapping strains where a group has no genotype file on this machine. `samples_other` is refused rather than guessed. **A perfect correlation.** The engine's r can land at 1.0000000000000002, which makes 1 - r^2 negative and its p-value NaN. The live API answers 0.0 for that row (scipy's pearsonr), so the reply now clamps: r to +-1 and p to 0.0. That is the reference's *output* being reproduced, not a computation being redone -- #96 stays resolved in the same direction. With the axis fixed, n agrees with the live site exactly (67) and r agrees with GN2's own published per-sample values to 1.4e-8. What does not yet agree is GN2's reported r itself: over the same 67 individuals, its `/correlation` endpoint says -0.8446202819617522 for 1447043_at while correlating the values its own `/sample_data` endpoint publishes for the same two traits gives -0.8393085803732605; on a second pair the gap is -0.8479321530363810 against -0.8451666639589470. Both pairs are off in the same direction and by a similar amount, so the two GN2 endpoints are reading different values for the same samples -- not a difference of sample sets (all four BXD genotype files give the same 67) and not a transform (2**v, 10**v, exp, v**2 and log were all tried). Until that is settled, our numbers are the reference's published values correlated correctly, and the ranking cannot be identical to a number we cannot reproduce from the reference's own data.
3 dayscorrelation: phenotype traits and datasets, the common casePjotr Prins
Either side of a correlation may now be a Publish (phenotype) dataset, not just a ProbeSet one: a pheno trait correlated against an expression dataset is what GN2's mapping page is usually used for, and returning 404 for BXDPublish was the wrong answer. A publish reference trait is addressed by its numeric PublishXRef.Id -- that is how GN2 addresses phenotype traits, and a name there is refused with a message saying so rather than a confusing "no such trait". The bulk fetch gains the publish equivalent, keyed on the freeze's InbredSetId, so the same individuals line up on both sides. Measured live: BXDPublish/10001 (34 observations) against HC_M2_0606_P, top 5, HTTP 200 in 5.0 s; 1440079_at at r -0.757 (p 3.2e-6) leads.
3 dayscorrelation: the sample-axis endpoint, synchronousPjotr Prins
POST /compute/correlation now computes instead of returning canned data: one reference trait against every trait of a target dataset, observations being individuals, pearson or spearman, top-N by |r|. The reply is positional -- trait, r, p, n (column names in meta.columns) -- with the method, the minimum overlap and the top-N stated in the notes. `bicor` is refused with the methods that exist; a bad top_n is refused rather than clamped; a dataset or trait with nothing to correlate says so instead of returning an empty list. src/db.rs gains the bulk fetch the sample axis needs: every value of every trait of a dataset restricted to the reference trait's individuals, one record per (trait, individual, value), ordered by trait so the caller groups as it streams. It never loads a dataset whole -- ProbeSetData is 5.3e9 rows, so the query narrows by dataset and by individuals, which is also why the sweep, not the arithmetic, is the whole cost. Measured on the live db_webqtl: HC_M2_0606_P / 1427571_at against its own dataset, top 5, HTTP 200 in 16 s; the trait correlates with itself at 1.0 over 99 observations, then 0.802 (p 1.9e-23), 0.790, -0.789, -0.788. Synchronous is affordable at this size; GN2's job indirection exists for top-N 20000 across datasets, which is the next thing to time. Known and deliberate for now: the publish/genotype axes are not wired (the fetch is probeset-only), and a perfect correlation reports p as null -- GSL's t statistic is undefined at |r| = 1, whose r comes out as 1.0000000000000002, so 1 - r^2 is slightly negative and the engine returns NaN. Reproducing the engine exactly, rather than "fixing" the p-value here, is the decision recorded in docs/correlations.md.
7 daysnext: every link a client is offered is a call it can makePjotr
tests/envelope.rs follows every href in every reply, and running it end to end found six that were not addresses: - /species, /species/{name}/datasets and /groups/{group}/datasets offered /species/{species} and /datasets/{dataset} -- templates, which 404 when followed. Each now points at a real row from the listing in hand, and the description says any other name can go in its place. - /datasets/{name} offered /datasets/{dataset}/probesets/{probeset}: the listing it already links is the callable one, so the shape moved into that link's description. - /traits/{dataset}/{trait}/values offered a GET /traits/{ds}/{trait}/correlation, which does not exist: correlation is POST /compute/correlation and that reply already carries it in "actions". Removed, with the reason written where it was. - the same reply built its /groups/... links from the *dataset* name. BXDPublish's traits live in group BXD, and /groups/BXDPublish/phewas is a 404 (no genotype files for it). db::get_group_name resolves the group from InbredSet.Id and the links use it; when a dataset has no group the group links are simply absent. - /groups/{g}/phewas with no parameters is a 422 ("give either locus, or chr and mb"), so that link is now a shape with placeholders. Placeholders stay where the caller must supply a value (.../markers/{marker}/classes?trait={trait_id}), but the crawl now skips them only when the description says what to fill in -- otherwise a client reading the next list cannot tell a call from a template. Two stale expectations alongside: /version carries the root links now (it is the natural first call), and /search's results put its own follow-ups first and the root links after them. Tests are honest about their environment too: links skips when no search index is configured (an empty XAPIAN_DB_PATH answers 502), and the envelope crawl exempts /openapi.json, the API document, which is deliberately not enveloped.
8 daysprobesets: cross-dataset lookups by probeset id and by gene symbolPjotr
A probeset id is not a Xapian field, so /search cannot find 1427571_at exactly and the only route to it was guessing a dataset and calling /datasets/{ds}/probesets/{name} until one answered. A gene symbol could be searched, but the reply was capped at 100 hits, interleaved probesets and platforms, and carried no total -- so "did I see all of Shh?" had no answer. Both questions are one indexed query each: ProbeSet.Name_IDX and symbol_IDX. * GET /probesets/{name}: one row per dataset that carries the probeset, with the panel (group, species, platform) and the scan's statistics (mean, LRS, LOD, locus, additive). Suffixed chip variants match too -- 1427571_at also finds 1427571_at_A -- and each row's `name` says which matched, so the answer never hides a substitution. LIKE's wildcards are escaped, or every Affymetrix `_at` would match any character. 155 rows over 155 datasets in 32 ms. * GET /genes/{symbol}/traits: the same rows filtered on Symbol, with a real total, ?sort=lod, ?min_lod=, ?group=, ?species=, ?platform= and paging -- "BXD datasets where Shh is a strong eQTL" is one call. 1225 rows over 17 datasets and 13 probesets in 38 ms. * Both serve metadata, not values. A token is enough for the index; trait values stay per-dataset and authorized per dataset, and `next` goes to /datasets/{ds}/probesets/{name}/values, which checks. There is no cross-dataset values endpoint, deliberately: that is bulk work and /groups/{g}/traits is the bulk call. The one-statement join took 5.2 s for Shh -- MySQL rescans InbredSet/Species per row -- so the 933 ProbeSetFreeze rows and the 102 panels are fetched separately and joined in memory: 70 ms each. nsamples is left out: it is a ProbeSetData count per row (+0.5 s) and the dataset record already reports strains.
9 dayserrors: log every 5xx, and remember the last database failurePjotr
A user reported that the token was fine and the service was up, but "everything backed by the relational database is failing" -- while `/search` (Xapian) and `/version` answered. By the time I looked, every data endpoint answered 200 and `/version` said `database: reachable`, and there was **no record anywhere** of what they had hit: the service logged nothing at all, so a caller's report could not be matched to a cause, and `/version`'s `database` is a *now* probe that says nothing about the minute before. Two things now make that diagnosable: * every 5xx is written to stderr as one JSON line -- status, code, operation_id, **request_id**, took_ms and a truncated detail -- so "500 at 12:31, request id ..." can be looked up. 4xx stay unlogged: they are the caller's business, and a 401/422 stream would drown this; * the last database failure is remembered (`db::record_failure`, called from `ApiError::from_db`, the one place every database error passes through) and `/version` reports it in `notes` for ten minutes, with its kind (`unavailable`, `connection_lost`, `query`) and the driver's message. That is the gap that misled the user: a healthy probe beside failing endpoints looks like an auth problem from outside. `is_connection_loss` gained a `_detail` form so the caller that already has the rendered message does not render it twice. * src/error.rs: the log line, and recording failures by kind; * src/db.rs: `FailureKind`, `Failure`, `record_failure`, `recent_failure`, and two unit tests (a failure is remembered and ages out; a driver message is classified correctly); * src/lib.rs: the `/version` note, absent unless something failed recently. Checked against a deliberately broken `SQL_URI` on a scratch instance: the data endpoint answers `503 db_unavailable`, the log carries the request id, and `/version` shows the note while still answering 200.
10 daysapi: describe case attributes, and gate large computations onlyPjotr
Two things that came out of reading a real response rather than a test. **Say what a case attribute means.** `GET /groups/{group}/case-attributes` returned names and values, and the per-trait attribute endpoint returned values, so a caller had to guess that `SeqCvge` is a sequencing depth. MariaDB has had the answer all along in `CaseAttribute.Description`: SeqCvge Depth of DNA sequencing for 1 male of each strain in 2016 Status Live= Available at JAX, Cryo=Cryopreserved only, Extinct Epoch BXD family subgroups. Each number with common parents. ... Sex F= female, M= male Both endpoints now carry it -- a `descriptions` map on the group listing, a `description` field on the per-trait one -- and the covariate notes quote it, so `/compute/gemma` answers "Covariate SeqCvge (Depth of DNA sequencing for 1 male of each strain in 2016): categorical ..." instead of leaving the name to be interpreted. It matters most for covariates: `SeqCvge` is a property of the *genotype* data, its values include compound ones (`40+31L`, two sequencing runs), and the server codes those as 20-odd levels rather than as a depth -- a covariate fitted for the wrong reason, which the description is the first hint against. **Gate large computations, not every computation.** The gate was applied to every job, so a 93-strain test request could be refused because somebody else's 6 000-individual run held the slot -- which is how the test suite started serialising itself behind a mutex, a workaround for the wrong problem. What the gate is for is *large* work: a BXD panel of ~200 finishes in seconds and two at once cost nothing, while UM-HET3's 6 382 write 207 MB of kinship per chromosome. It now starts at `LARGE_PANEL` (1 000 individuals): below that no slot is taken, nothing waits and nothing is refused; above it the old policy holds unchanged (1 slot for gemma, 1 for ravanan, `RQTL_MAX_CONCURRENT` for rqtl, one large job per caller, 409 naming the run in the way). The size is not known when the request arrives -- the panel is resolved server-side -- so the decorator now makes a *light* claim and the handler promotes it (`__gn4_claim.claim_slot_or_busy(individuals, rid)`) at the point where it knows, then hands it to `Gate::run` around the call that computes, so the slot still outlives a client disconnect. A light claim registers nothing and its drop removes nothing: only a held claim may touch the claimant map. * src/compute_gate.rs: `LARGE_PANEL`/`is_large`, `Gate::claim_light`, `Claim::claim_slot_or_busy`/`is_held`, `busy_error` (the 409 policy, moved out of `claim_or_busy`), and `Gate::run` unchanged in spirit. * gnauth-macros: binds the light claim; the body keeps one gate call. * src/routes/{gemma,rqtl}.rs: promotion after the panel/sample count is known, and the compute call wrapped in `run`; error mapping moved inside the job so the future returns `ApiError`. * src/rqtl.rs: `Sample` is `Clone` (the job owns its copy). * tests: the gemma suite loses its lock entirely (the fixtures and BXD.4 are under the threshold); rqtl keeps its lock only for the machine's sake -- concurrent `Rscript` scans would compete for cores and scratch, not for a slot. Four new gate tests pin the threshold, that a small job never takes the slot, that one caller cannot hold two large jobs, and that another caller waits rather than fails. * README.md, docs/www/ai-instruction.md, docs/www/operator-checklist.md: what the threshold is and why it is 1 000. Measured on :3001 after the change: two concurrent small-panel GEMMA requests from one token both answer 200 (one was 409 before), while two 6 382-individual requests answer 502/409 -- the second gated with the usual `compute_busy` naming the first. 277 tests pass; the same four as before do not.
10 daysdb: recover from a connection the server closed, and report the databasePjotr
Every database-backed endpoint 500'd with `CodecError { IO error: Broken pipe }` while the process answered /version quite happily. The cause was ours: `pool()` turns off the driver's `check_health` (a ping at checkout, dead connections replaced) to save a round trip -- sound against a database 130 ms away, worth nothing against loopback, and it costs the pool its self-healing. One server-side close of an idle session then becomes one failed request per dead socket: ten in a row, until a restart. * src/db.rs: `SQL_CHECK_HEALTH` (default true) decides the ping; `is_connection_loss` recognises the errors that mean the peer went away (broken pipe, reset, went away, lost connection, closed); `forget_pool` drops the cached pool so the next call rebuilds it; `health` probes reachability, cached ten seconds. One pool static shared by pool()/forget_pool (they had drifted into two). * src/error.rs (ApiError::from_db): on a connection loss, forget the pool -- the one place every database error passes through, so recovery does not depend on each caller. * src/lib.rs (/version): `"database": "reachable" | "unreachable"`; a service that answers while nothing behind it works is now visible. * src/config.rs: the switch, with the trade written down. * tests/db_recovery.rs: kills every pooled session from a second connection, then asserts the next twelve requests all succeed -- the scenario that produced the outage, now a regression test. * tests/...: unit test for the classifier (its own IO error text and the "this is a bad query, do not retry" cases). * README.md ("Database connections"), docs/www/operator-checklist.md: the knob, why it defaults on, and what /version reports.
10 dayscase-attributes: one attribute for one trait, in that trait's sample orderPjotr
The trait-values endpoint stays about trait values; a covariate needs the study's metadata for the same samples, aligned the same way. So: GET /groups/{group}/traits/{trait_id}/case-attributes/{attribute} returns one attribute's values for exactly that trait's samples, **in the trait's own order**, which is what makes it usable without the caller re-sorting anything: /groups/BXD-Longevity/traits/10001/case-attributes/Sex -> 3281 values in AgeAtDeath's order, F x 3101, M x 180 -> order identical to /traits/10001/values (asserted in the tests) Alongside the values it reports `distinct` (each value with its count, commonest first) and `num_values`. That is the check that decides whether an attribute can serve as a covariate at all -- a constant one cannot -- and it saves a second request just to find out. Attributes are per group, so an unknown one is a 404 whose `next` is the group's attribute list: BXD has no Sex, BXD-Longevity does. * src/db.rs (get_trait_case_attribute, get_group_trait_samples): the group's attribute values aligned onto the trait's sample list, joining CaseAttribute/CaseAttributeXRefNew on (InbredSetId, CaseAttributeId) as the group endpoint does; empty text is reported as no value. * src/routes/groups.rs: the handler, token-gated, with the response shapes and the two `next` links (the trait's values, the group's attributes). * tests/case_attributes.rs: alignment against an independent /values response, `distinct` accounting, ordering, and the 404s for BXD (no Sex) and for a non-existent attribute. * README.md, docs/www/ai-instruction.md (3. Case attributes): the endpoint, and "ask per trait rather than sorting the group's attributes yourself".
11 daysphewas: every trait peaking within a window of one positionPjotr
PublishXRef records the marker each published trait's QTL peaks at (Locus) with its LRS, which is a PheWAS waiting to happen: fix a position, find the markers within a window of it, and ask which traits peak at those markers. GET /groups/{group}/phewas?locus=rs30524230&window=10000 GET /groups/{group}/phewas?chr=9&mb=29.940961&window=10000 The window is in base pairs, centred, default 10 kb either side -- the "10K up and downstream" case. Positions come from the group's genotype marker table (the endpoint added last round), so `genotype_version` selects whose markers define the window and the answer reports them alongside the traits: an empty `traits` list is then readable as "nothing peaks that close" rather than a failure, and a ten-times wider window is offered as `next`. Measured on real data, ±10 kb around rs30524230 (chr 9, 29.94 Mb, a BXD QTL peak at LRS 460): 4 markers, 10 traits, from LOD 99.9 down to 2.68 -- ANPEP_band, then microbiome/haematology/infectious-disease traits. That cluster of 99.9s is the 460 LRS cap, not a spectrum, which the notes say. * src/db.rs (get_phewas_traits): PublishXRef rows for a set of peak markers, joining Phenotype/Publication through the existing trait_info_sql, strongest first; marker names are re-validated before they reach the IN list. * src/routes/phewas.rs: the endpoint, token-gated; window/centre validation (422 for none or negative), 404 for an unknown locus, notes for the empty cases, and a unit test for the version/group helper. * tests/phewas.rs: markers in the window, traits sorted by LRS with lod = lrs/4.604, every hit peaking at a listed marker, zero-width window, the two failure modes, and that the suggested wider window resolves. * README.md, docs/www/ai-instruction.md (new 4b: how to read a PheWAS), docs/www/operator-checklist.md: the endpoint and its reading.
2026-10-14rustfmt: apply rustfmt across the treeAlexander_Kabui
Pure formatting sweep (import ordering, line wrapping); the tree had accumulated rustfmt drift. Separated from any semantic change so it stays reviewable and revertable on its own.
2026-10-14config: GN4_DB_MODE (mariadb optional on dev) + no-DB hardeningAlexander_Kabui
* src/config.rs (DbMode, Config.db_mode, parse_db_mode): add GN4_DB_MODE=dev|production in the layered config; dev runs with no configured DB; fail-closed parse (a typo refuses). * src/db.rs (DbError, pool, probe), src/error.rs (ApiError::from_db): mint a typed 503 db_unavailable at the db::pool() choke point, distinguishing Unavailable from Query. * src/main.rs: loud no-DB startup line; production refuses to start (exit non-zero) before axum::serve. * src/routes/{accessions,datasets,groups,mapping_methods,publications,species,traits}.rs: route every DB error through ApiError::from_db. * src/bin/gn-indexer.rs: refuse to build an index when the DB is unreachable. * tests/no_db.rs: pin no-DB degradation (search 200 partial, DB routes 503). His ask (DM 2026-10-12): "Make it an option to switch it off for development machines? I mean mariadb." The no-DB hardening is ours (review-driven). Default GN4_DB_MODE left unflipped (=dev); the flip needs owner sign-off.
2026-10-08* src/db.rs, src/routes: add mapping_method, tissue, /mapping-methods, ↵Pjotr
/accessions. Answers four more questions of aisearch's full_benchmark.csv from tables the API did not expose. * src/db.rs (Group, GroupDetail), src/routes/groups.rs: groups gain `mapping_method`, the name of the MappingMethod row their mapping_method_id points to (qtlreaper, Rqtl, ...), or null. * src/db.rs (Dataset): datasets gain `tissue`, the Tissue their ProbeFreeze points to (e.g. "Brain mRNA"). * src/routes/mapping_methods.rs: GET /mapping-methods lists the MappingMethod table, including tools no group uses (PLINK, FastMap). * src/routes/accessions.rs: GET /accessions/{accession} resolves a GeneNetwork accession number (GN602, gn602 or 602) via InfoFiles to the dataset's name, title, category (publish/probeset) and group. * docs/genenetwork_datatypes.md: MappingMethodId is a key into the MappingMethod table (the authoritative reading); GN2's code-based reading is noted as superseded. * tests/benchmark.rs, test/data/benchmark_search.json: add the 60 new full-benchmark rows that pass (full rows 0..114; search rows run against recorded production gn3, the rest need the database), and full rows 33 (mapping tools), 34 (Rqtl for outbred sets), 35 (GN602) and 44 (only GTEx v8 has TPM datasets). The untested rows are listed in the file's header comment with the reason for each. * tests, test/python: tests for the new fields and endpoints; both endpoints added to the OpenAPI and next-link crawl tests. * README.md, RELEASE_NOTES.md: document the new fields and endpoints. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08* src/db.rs (pool): turn off check_health and reset_connection.Pjotr
Each costs a full round trip to the database per pooled checkout (a ping on take, COM_RESET_CONNECTION on return), on top of the query: three round trips per database call instead of one. Against a database about 130 ms away that is 390 ms per call instead of 130 ms. Measured across 25 SQL endpoints: 1.3x (a 15,060-row list, dominated by row cost) to 3.0x (single-call endpoints) faster; the full Rust and Python suites pass. Nothing here needs the reset (queries are read-only and keep no session state). The ping only guarded against a connection the server closed while idle; that now surfaces as a database error on the next query that uses it (the pool then drops it), without a retry. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08* src/response.rs: wrap every success response in one envelope.Pjotr
Breaking: every success body is now {"data", "meta": {"state", "took_ms", "request_id"}, "notes", "next"}; clients read `data`. Only `/` (plain text) and `/openapi.json` are unwrapped. * src/response.rs (Envelope, Meta, State, Ctx, RequestInfo): handlers take Ctx first and build the response with ctx.envelope(data), so timing and the request id are never filled in by hand. `state` is `complete` or `partial`; notes are server-authored only; `next` are concrete relative hrefs (path_segment percent-encodes names). * src/routes/*.rs, src/lib.rs (version): every handler returns Envelope<..> with next links; /search is `partial` when the locus lookup fails or a sorted 1000-hit window may be truncated, and /publications/{pubmed_id}/traits when datasets were left out for authorization. * src/error.rs (ApiError): add `took_ms`, an RFC 7807 extension member. * src/auth.rs (AuthToken): the 401 now carries request_id and took_ms (it never had a request_id); Ctx stores RequestInfo in the request extensions for it. * tests: common::data / envelope_data parse every success body through the envelope; tests/envelope.rs and test/python/test_envelope.py check the shape, errors, and that every `next` link resolves; tests/envelope_partial.rs checks the `partial` search without a database; tests/openapi.rs fails if a JSON success response is not an Envelope_* schema. * README.md, docs/SKILL.md, docs/gn-auth.md, RELEASE_NOTES.md: document the envelope and took_ms on errors. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08* src/routes/publications.rs: add GET /publications/{pubmed_id}/traits.Pjotr
Lists the published traits (across groups) a publication reports. Datasets the caller may not view are left out, using the same check as GET /traits/{dataset} (traits.rs is_authorised is now pub(crate)). * src/db.rs (TraitInfo, trait_info_sql): TraitInfo gains `dataset`, `abbreviation` (Phenotype.Post_publication_abbreviation, else Pre) and `url` (the GeneNetwork show_trait page). The three TraitInfo queries now share one SELECT. New get_group_traits_by_abbreviation and get_publication_traits. * src/routes/groups.rs (group_traits): add ?abbreviation= filter. * src/db.rs (Group), src/routes/groups.rs (groups): groups carry `family` and `species`; add family, species and q filters. * src/routes/search.rs (search): add `page` (gn3's page unsorted, a page of the ranked list with `sort`); declare `pubmed_id`; hits gain `abbreviation` next to `locus` (get_publish_annotations). * src/db.rs (LRS_PER_LOD): LOD = LRS / 4.604, the divisor the benchmark's RDF data uses (was 4.61). * tests, test/python, test/data/benchmark_search.json, README.md: cover all of the above; tests/benchmark.rs now also answers rows 11, 16, 25, 30, 34 and 37 of small_benchmark.csv, and compares LOD exactly. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07* src/routes/search.rs (search): add lod, locus, and sort/order.Pjotr
Search hits gain `lod` (lrs / 4.61) and `locus` (the peak marker name, looked up in PublishXRef in one batched query; null when the database is unavailable). `sort=lrs|lod` with `order=asc|desc` ranks hits: gn3 is asked for its first 1000 matches, those are ranked, then `limit` applies. * src/db.rs (LRS_PER_LOD, lod, get_publish_loci): add the shared LRS-to-LOD constant and the locus lookup; TraitInfo and ProbeSetInfo gain `lod`, computed in SQL from the same constant. * tests/search.rs, test/python/test_search.py: add the "highest LOD, locus marker and additive effect for serotonin in BXD male midbrain" scenario over a recorded production fixture, plus sort/order tests. * tests/group_trait_info.rs, tests/dataset_probeset_info.rs and their Python mirrors: assert lod (and locus for trait 17985). * README.md (Endpoints): document sort, order, lod and locus. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07Add GET /groups/{group}/case-attributesPjotr Prins
Per-strain metadata beyond genotype/phenotype data (age, sex, treatment, etc.), backed by MariaDB CaseAttribute/CaseAttributeXRefNew. Combines GeneNetwork3's separate /case-attribute/{id}/names and /case-attribute/{id}/values (gn3.api.case_attributes) into one response -- neither list is large enough on its own to need splitting. This is the only one of GeneNetwork2/3/gn-guile with the concept at all: GeneNetwork2 delegates to GeneNetwork3 rather than storing case attributes itself, and gn-guile doesn't have it. db::get_group_case_attributes returns {names: [...], values: [{strain, case_attributes: {...}}]}; values is built by folding adjacent (strain, attribute, value) rows (ordered by strain name) rather than grouping afterwards, since SQL already guarantees the adjacency. Found and fixed a real bug while verifying this live: GeneNetwork3's own query joins CaseAttribute to CaseAttributeXRefNew on CaseAttributeId alone, but CaseAttributeId is not unique across groups -- it's a shared vocabulary (id 37 means "Epoch" for dozens of unrelated groups simultaneously); the actual key is (InbredSetId, CaseAttributeId). Confirmed against this database: without also joining on InbredSetId, BXD's result included a spurious "Strain": "Live" attribute belonging to a different, unrelated group. Fixed on genenetwork4's side by adding the missing join condition. A group with no case attributes defined returns 200 with both lists empty, not a 404 (the group itself exists) -- verified against LXS, a real group with zero CaseAttribute rows. tests/group_case_attributes.rs and its Python mirror cover BXD's 4 real attributes (asserting each strain has exactly those 4, not more, to catch the join bug above), the empty-group case, missing token, and unknown group. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-06* src/db.rs (row mappers): alias the SQL row tuple types.Alexander_Kabui
Clippy type_complexity fired three times on the ten- and eleven-column row closures. Naming each SELECT row shape in one place documents the column order and keeps the mappers readable.
2026-10-04Add GET /traits/{dataset}/... spanning ProbeSet and Publish dataPjotr Prins
A new, additional endpoint family that addresses both trait categories the same way -- (dataset, trait) -- matching GeneNetwork2's own generic wqflask.api.router.trait_sample_data/get_trait_info, rather than the two category-specific shapes genenetwork4 already has (/datasets/{dataset}/probesets... and /groups/{group}/traits...), which are unchanged and keep working exactly as before. - db::resolve_trait_dataset(name) resolves a dataset name against ProbeSetFreeze first, then the new db::PublishDataset (a PublishFreeze row, via the new get_publish_freeze_by_name) -- whichever matches decides the category. This means only dataset names are ever "guessed" across categories, not trait names/ids, which is the much smaller and safer surface: GN2's own equivalent tries ProbeSet.Name and PublishXRef.Id against the same bare trait string and silently falls back, which can return the wrong trait on a same-named collision. get_publish_freeze_by_name accepts a PublishFreeze's own Name/ShortName/Id or its owning group's Name/InbredSetName/FullName/ Id, mirroring GN2's own flexible matching (e.g. "BXDPublish" or "BXD"). - GET /traits/{dataset}: list every trait in a dataset, tagged by category. - GET /traits/{dataset}/{trait}: one trait's metadata. A non-numeric trait against a Publish dataset is a 404 (can't match a PublishXRef.Id), not a 500. - GET /traits/{dataset}/{trait}/values: that trait's per-strain values. TraitValue and ProbeSetValue are already identical in shape, so this unifies with no information loss, unlike the metadata endpoint above. Metadata responses are a tagged union ({"category": "probeset"| "publish", ...}) wrapping ProbeSetInfo/TraitInfo unchanged, rather than a lossy common-fields schema -- the two share little beyond mean/locus/lrs/additive. utoipa generates this as a oneOf with a category discriminator, verified against /openapi.json. Gated the same way as /datasets/{dataset}/probesets..., extended to Publish data for the first time: /groups/{group}/traits itself is still not gated, but reaching the same data through /traits/{group} now requires a token and checks PublishFreeze.public/confidentiality. No gn-auth-side change was needed -- POST /auth/data/authorisation already dispatches *Publish-suffixed dataset names to its phenotype resource resolver (gn_auth/auth/authorisation/data/phenotypes.py), the exact mechanism this integration already relies on for mRNA data. auth::VIEW_RESOURCE_PRIVILEGE moves from routes::datasets (private) to routes::auth (pub) so routes::traits can reuse it instead of duplicating the string. tests/traits.rs and its Python mirror cover both categories (including both ways of addressing a Publish dataset), the 401/403/404 cases, and the non-numeric-trait-against-Publish-dataset edge case, run against a real gn-auth instance. The Publish branch's deny path itself is only exercised via gn-auth's "unlinked dataset" fallback (already proven for ProbeSet) -- no non-public PublishFreeze exists in the database this was tested against. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Add GET /species/{species}/datasetsPjotr Prins
Lists the expression (ProbeSet) datasets belonging to a species, across all of its groups -- the species-scoped equivalent of /groups/{group}/datasets (db::get_species_datasets joins ProbeSetFreeze through ProbeFreeze and InbredSet to InbredSet.SpeciesId, reusing the existing Dataset struct). GeneNetwork2's own species-scoped variant of wqflask.api.router.get_datasets_for_group still requires a group name (species only disambiguates it); this doesn't. Not gated on gn-auth, matching its /groups/{group}/datasets sibling: this returns dataset metadata (names, public/confidentiality flags), not trait data, unlike the /datasets/{dataset}/probesets family. tests/species_datasets.rs and its Python mirror cover the mouse-group and unknown-species cases. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Add GET /groups/{group}/traits and GET /datasets/{dataset}/probesetsPjotr Prins
List endpoints for the two trait domains, each the listing equivalent of an existing singular endpoint: - GET /groups/{group}/traits: every published (phenotype) trait in a group (db::get_group_traits), same fields/join as /groups/{group}/traits/{trait_id}, no token required -- matching its singular sibling, which isn't gated either. A large group (BXD has 15000+ phenotype traits) can take several seconds and tens of megabytes to return in full; no pagination yet, matching GeneNetwork2's own wqflask.api.router.fetch_traits, which has no default limit either. - GET /datasets/{dataset}/probesets: every expression (ProbeSet) trait in a dataset (db::get_dataset_probesets), same fields/join as /datasets/{dataset}/probesets/{probeset}. Gated the same way as that singular endpoint -- a valid token is required unconditionally, and a non-public dataset is checked against gn-auth once for the whole dataset via a new auth::authorise_dataset, rather than once per probeset in the list. Leaving this listing ungated would have let a caller read out a whole non-public dataset's trait data in one request, bypassing the gate just added to the singular endpoint. auth::authorise_dataset wraps authorise_trait with an empty trait name: gn-auth resolves a trait's resource purely from its dataset name, so the trait half of a "Dataset::Trait" string never affects which privileges come back. Verified directly against the live instance both for an unlinked non-public dataset (403, deny) and folded into dataset_probesets's own test for the public/embargo-aged bypass path. tests/group_traits.rs, tests/dataset_probesets.rs and their Python mirrors cover both, run against a real gn-auth instance for the gated one. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Format with cargo fmt, matching 924bcf3's project-wide pass.Pjotr Prins
* src/db.rs, src/routes/groups.rs, tests/dataset_by_name.rs, tests/group_by_name.rs, tests/group_trait_info.rs: Run cargo fmt. No behavior change -- this code was written before the project adopted cargo fmt formatting in 924bcf3 and landed unformatted during the rebase that merged the two histories. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Add ProbeSetXRef endpoints: datasets, probesets, probeset values.Pjotr Prins
The expression (ProbeSet) counterpart to the PublishXRef endpoints: ProbeSetXRef has no single Id column of its own (its primary key is DataId; the natural key is the pair (ProbeSetFreezeId, ProbeSetId)), so a trait here is addressed by dataset name plus ProbeSet.Name instead of a single numeric trait id. * src/db.rs (Dataset): New struct, one ProbeSetFreeze row. (get_group_datasets): New function. Joins ProbeSetFreeze and ProbeFreeze by group Id, matching GeneNetwork2's wqflask.api.router.get_datasets_for_group. (get_dataset_by_name): New function. Matches Name, Name2, FullName, ShortName or numeric Id, matching wqflask.api.router.get_dataset_info's ProbeSet-branch matching logic. A dataset name is globally unique on its own, unlike a group's traits, so this isn't scoped by group. (ProbeSetInfo): New struct, a probe's ProbeSetXRef mapping stats plus its ProbeSet gene annotation. (get_dataset_probeset_info): New function, matching wqflask.api.router.get_trait_info's ProbeSet branch. (ProbeSetValue): New struct. (get_dataset_probeset_values): New function, matching wqflask.api.router.trait_sample_data's ProbeSet branch. * src/lib.rs (DatasetProbesetLookup): New enum, the expression-dataset equivalent of GroupTraitLookup -- distinguishes a missing dataset from a missing probeset in an existing dataset. (group_datasets): New handler for GET /groups/{group}/datasets. (dataset_by_name): New handler for GET /datasets/{dataset}. (get_dataset_probeset_info_result, dataset_probeset_info): New function and handler for GET /datasets/{dataset}/probesets/{probeset}. (get_dataset_probeset_values_result, dataset_probeset_values): New function and handler for GET /datasets/{dataset}/probesets/{probeset}/values. (app): Register all four. * tests/group_datasets.rs, tests/dataset_by_name.rs, tests/dataset_probeset_info.rs, tests/dataset_probeset_values.rs, test/python/test_group_datasets.py, test/python/test_dataset_by_name.py, test/python/test_dataset_probeset_info.py, test/python/test_dataset_probeset_values.py: New files. Br_U_0803_M (dataset) / 100001_at (probeset, gene symbol Cd3g) in BXD are real rows used as fixtures. * tests/openapi.rs, test/python/test_openapi.py: Check the four new paths are in the generated spec. * README.md: Document the four new endpoints and list the new test files. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Add GET /groups/{group}/traits/{trait_id}; time DB tests; share pool.Pjotr Prins
* src/db.rs (TraitInfo): New struct. Joins PublishXRef, Phenotype and Publication; LEFT JOIN on the latter two since a handful of PublishXRef rows have no matching row. SQL column aliases are written to match mysql_common's UpperCamelCase conversion exactly, so no field needs a #[mysql(rename = "...")] override. (get_group_trait_info): New function. (pool): New function. mysql::Pool::new eagerly opens its default 10 connections before returning (about 7s against this database); every query function here used to call it per request, paying that cost on every single API call. Builds the Pool once behind a OnceLock<Mutex<Option<Pool>>> and shares the cheap, Arc-backed handle from then on (OnceLock::get_or_init can't be used alone since Pool::new is fallible and get_or_try_init is still unstable). (get_all_species, get_species_by_name, get_all_groups, get_group_by_name, get_group_strains, get_group_trait_values, get_group_trait_info): Use it instead of building their own Pool. * src/lib.rs (GroupTraitLookup): Generalize over the found value (was Vec<db::TraitValue>-specific), so group_trait_info can share it with group_trait_values. (get_group_trait_info_result): New function. (group_trait_info): New handler for GET /groups/{group}/traits/{trait_id}. (app): Register it. * tests/common/mod.rs: New file. get() issues a GET against genenetwork4::app() and warns on stderr if it takes over a second -- this is what surfaced the Pool::new cost above. * tests/species.rs, tests/species_by_name.rs, tests/groups.rs, tests/group_by_name.rs, tests/group_strains.rs, tests/group_trait_values.rs: Use tests/common::get instead of building each request by hand. * tests/group_trait_info.rs, test/python/test_group_trait_info.py: New files. * tests/openapi.rs, test/python/test_openapi.py: Check the new path is in the generated spec. * README.md (GET /groups/{group}/traits/{trait_id}): Document it. (Tests): List the new test files; document tests/common::get's timing warnings and how to see them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Add GET /groups/{group}/traits/{trait_id}/values, publish trait data.Pjotr Prins
* src/db.rs (TraitValue): New struct. (get_group_trait_values): New function. Joins PublishData, Strain and PublishXRef by group Id and PublishXRef.Id, matching gn-guile's gn.data.dataset:get-bxd-publish-name-value-dict generalized beyond BXD: PublishXRef.Id alone isn't unique across groups, only the pair (InbredSetId, Id) is, the same way GeneNetwork2's wqflask.api.router.trait_sample_data combines trait_name with a group or dataset to resolve a Publish trait. * src/lib.rs (GroupTraitLookup): New enum, distinguishes a missing group from a missing trait in an existing group. (get_group_trait_values_result): New function, looks up a group then its trait, returning a GroupTraitLookup. (group_trait_values): New handler for GET /groups/{group}/traits/{trait_id}/values. 404s as group_not_found or trait_not_found depending on what get_group_trait_values_result found. (app): Register it. * tests/group_trait_values.rs, test/python/test_group_trait_values.py: New files. * tests/openapi.rs, test/python/test_openapi.py: Check the new path is in the generated spec. * README.md (GET /groups/{group}/traits/{trait_id}/values): Document it, including that only Publish (phenotype) traits are supported so far, not ProbeSet or Geno traits. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-04Format all Rust sources with cargo fmt (no behavior change)Alexander_Kabui
Line-wrapping and canonical import ordering only (verified with git diff -w). Establishes rustfmt's output as the canonical style before the search/correlation work lands.
2026-10-03Add foundation structure: config layering, response envelope, routes modules.Alexander_Kabui
* src/config.rs: New module. Layered configuration, defaults -> GN4_CONF -> GN4_SECRETS -> environment, mirroring GeneNetwork3's SQL_URI convention so the two services can share configuration. Missing config files are ignored. * src/response.rs: New module. The descriptive success envelope (data + next actions) from the foundation design note (topics/gn4/gn4-api-foundation.gmi). Not yet adopted by existing endpoints; that is a deliberate wire-format change to agree on before it ships. * src/routes/mod.rs, species.rs, groups.rs: Move the species and groups handlers out of lib.rs into one module per domain, per the foundation structure. Handlers are pub; lib.rs just registers them. * src/lib.rs (app): Register the moved routes. hello_world and version predate the domain split and stay as top-level utility routes. * src/db.rs (sql_uri): Resolve through config::Config instead of reading the environment directly. * src/main.rs: Bind the port from config::Config.
2026-10-03Add GET /groups/{group}/strains, mirroring gn-guile's strain-id-names.Pjotr Prins
* src/db.rs (Strain): New struct. (get_group_strains): New function. Joins Strain and StrainXRef by group Id, optionally filtered to strains used for mapping, matching gn-guile's gn.data.strains:strain-id-names (same InbredSetId and used-for-mapping? arguments), returning strain names as well as ids. * src/lib.rs (GroupStrainsParams): New struct, the used_for_mapping query parameter. (group_strains): New handler for GET /groups/{group}/strains. Looks the group up the same way group_by_name does, for the same 404 shape on an unknown group. (app): Register it. * tests/group_strains.rs, test/python/test_group_strains.py: New files. * tests/openapi.rs, test/python/test_openapi.py: Check the new path is in the generated spec. * README.md (GET /groups/{group}/strains): Document it, including the used_for_mapping filter excluding BXD's parent strains. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-03Match group by numeric Id, use Group alias for InbredSet fields.Pjotr Prins
* src/db.rs (get_group_by_name): Also match CAST(Id AS CHAR), so a numeric Id works like GeneNetwork2's wqflask.api.router.get_group_info plus gn-guile's gn.data.population:population-info, which looks a group up by Id only. (GroupDetail): Rename inbred_set_id, inbred_set_name, inbred_set_code fields to group_id, display_name, group_code, adding #[mysql(rename = "...")] overrides so FromRow still matches the InbredSetId, InbredSetName, InbredSetCode MariaDB columns; InbredSet stays a MariaDB-only name, not a Rust identifier. * src/lib.rs (group_by_name): Update the "group" path parameter description to mention matching by numeric Id too. * tests/group_by_name.rs (group_by_name_bxd_returns_group_row): Update assertion for the display_name field rename. (group_by_name_numeric_id_returns_same_group_row): New test. * test/python/test_group_by_name.py: Likewise. * README.md (GET /groups/{group}): Document matching by numeric Id and the group_id/display_name/group_code field names. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-03Add /groups and /groups/{group} endpoints.Pjotr Prins
* src/db.rs (Group): New struct. Summary of one InbredSet row, matching the shape of GeneNetwork2's GET /api/v_pre1/groups response. (get_all_groups): New function. Mirrors GeneNetwork2's wqflask.api.router.get_groups_list (no species filter yet). (GroupDetail): New struct, deriving FromRow instead of being built from a tuple like SpeciesDetail, since InbredSet has 14 columns, more than mysql_common's FromRow is implemented for tuples (up to 12). FromRow matches by column name (PascalCase of the field name via #[mysql(rename_all = "PascalCase")]), except public, which is lowercase in the schema and needs its own #[mysql(rename = "public")]. (get_group_by_name): New function. Mirrors GeneNetwork2's wqflask.api.router.get_group_info, but returns every column instead of a fixed subset. * src/lib.rs (app): Add /groups and /groups/{group} routes. (groups, group_by_name): New handlers, following the same ApiError/request-id/spawn_blocking pattern as species/species_by_name. (group_by_name): On no match, set ApiError's next to /groups. * tests/groups.rs: New file. Checks BXD is present (used throughout the GeneNetwork2 and GeneNetwork3 test suites as a fixture group, always present in the database); skips rather than fails when no database is reachable, same as species.rs. * test/python/test_groups.py: New file. Likewise. * tests/group_by_name.rs: New file. Checks BXD's full row and an unknown-group 404 case, same pattern as species_by_name.rs. * test/python/test_group_by_name.py: New file. Likewise. * tests/openapi.rs (openapi_json_describes_all_endpoints): Check for /groups and /groups/{group} among the documented paths. * test/python/test_openapi.py (test_openapi_json_describes_all_endpoints): Likewise. * README.md: Document /groups and /groups/{group}.
2026-10-03Add /species/{species} for a single species' full row.Pjotr Prins
Checked for an existing implementation first: GeneNetwork2's wqflask/api/router.py has /api/v_pre1/species/<species_name>, matching Name, FullName or SpeciesName, but returning only a 4-column subset (Id, Name, FullName, TaxonomyId). GeneNetwork3's gn3/db/wiki.py has only internal name-to-id lookups for the wiki feature; its /species is SPARQL-based metadata, not a Species table row. gn-guile's species code (gn/data/species.scm) is also SPARQL/RDF metadata, not backed by db_webqtl. * src/db.rs (params): Import. (SpeciesDetail): New struct, one full row of the Species table. (get_species_by_name): New function. Same match logic as GeneNetwork2's get_species_info (Name, FullName or SpeciesName), but selects every column instead of a fixed subset. * src/lib.rs (app): Add /species/{species} route. (species_by_name): New handler. 404 if no species matches. * tests/species_by_name.rs: New file. Tests mouse specifically (Name=mouse is always present) and an unknown-species 404 case; skips rather than fails when no database is reachable, same as species.rs. * test/python/test_species_by_name.py: New file. Likewise. * tests/openapi.rs (openapi_json_describes_all_endpoints): Check for /species/{species} among the documented paths. * test/python/test_openapi.py (test_openapi_json_describes_all_endpoints): Likewise. * README.md: Document /species/{species}.
2026-10-03Add /openapi.json describing the REST API.Pjotr Prins
* Cargo.toml (dependencies): Add utoipa, feature axum_extras; add utoipa-axum. * src/lib.rs: Use utoipa_axum::router::OpenApiRouter and routes! instead of axum::Router and route!, so paths register into an OpenApi document as well as the router. (ApiDoc): New struct, deriving utoipa::OpenApi with title, description and license. (app): Add /openapi.json, serving the OpenApi document split out of OpenApiRouter via split_for_parts. (hello_world, version, species): Add #[utoipa::path(...)] attributes describing each endpoint's responses. (VersionInfo): Derive utoipa::ToSchema in addition to Serialize. * src/db.rs (Species): Likewise. * tests/openapi.rs: New file. Checks /openapi.json's info.title and that it lists all three endpoints. * test/python/test_openapi.py: New file. Likewise. * README.md: Document /openapi.json. * rust-package.scm: Regenerate with guix import crate -f Cargo.lock genenetwork4. Adds crate-source definitions for utoipa, utoipa-gen, utoipa-axum and pastey. * guix.scm (genenetwork4-git)[native-inputs]: Add rust-utoipa-6.0.0, rust-utoipa-gen-6.0.1, rust-utoipa-axum-0.3.0 and rust-pastey-0.2.3.
2026-10-03Add /species endpoint backed by MariaDB.Pjotr Prins
* Cargo.toml (dependencies): Add mysql, default-features = false, features minimal-rust and derive (minimal pulls in flate2/zlib, which fails to compile here; minimal-rust uses flate2's pure-Rust backend instead). * Cargo.lock: Regenerate. Picks up mysql and its roughly 100 transitive dependencies; also bumps libc to 0.2.190 and tokio to 1.53.2 from re-resolution. * src/db.rs: New file. sql_uri, Species, get_all_species. Mirrors GeneNetwork3's gn3.db.species.get_all_species: same query, same ordering. Connection string from the SQL_URI environment variable (same config key GeneNetwork3 uses), defaulting to mysql://webqtlout:webqtlout@127.0.0.1/db_webqtl. * src/lib.rs (app): Add /species route. (species): New handler. Runs db::get_all_species on a blocking thread, since mysql::Conn is synchronous. * tests/species.rs: New file. Checks for a database on 127.0.0.1:3306 first and skips with a message instead of failing when none is reachable, since the Guix build sandbox has no network. * test/python/test_species.py: New file. Likewise. * README.md: Document /species, SQL_URI, and the skip-without-database behavior of species.rs and test_species.py. * rust-package.scm: Regenerate with guix import crate -f Cargo.lock genenetwork4. Adds crate-source definitions for mysql's transitive dependencies; bumps rust-libc and rust-tokio to the versions above. * guix.scm (genenetwork4-git)[native-inputs]: Add every new rust-package.scm entry mysql pulled in (roughly 100 packages, including rust-mysql-28.0.3, rust-mysql-common-0.37.3, rust-uuid, rust-url, rust-sha1, rust-sha2, rust-flate2).