--- name: rust-axum-mariadb-api description: Build a Rust/Axum REST API with OpenAPI docs, RFC-7807 errors, and a read-only MariaDB backend, packaged with Guix --- # Rust/Axum REST API over MariaDB Practices used to build genenetwork4. Reuse these when starting a similar Rust/Axum service backed by a MariaDB/MySQL database. ## mysql crate - Pin `mysql = { version = "...", default-features = false, features = ["minimal-rust", "derive"] }`. The default features pull in `flate2`'s system-zlib backend, which fails to build in sandboxes without a system zlib; `minimal-rust` uses a pure-Rust zlib instead. `derive` is needed for `#[derive(FromRow)]`. - The client is synchronous. Run every query inside `tokio::task::spawn_blocking` so it doesn't stall the async runtime. - Tuple-based `FromRow` (`conn.exec_first::<(A, B, ...), _, _>`) is only implemented up to 12 elements. For a wider row, define a struct and `#[derive(FromRow)]` instead. - `FromRow` derive matches columns by PascalCase of the field name by default (`#[mysql(rename_all = "PascalCase")]` on the struct). Add `#[mysql(rename = "ActualColumn")]` on a field whenever its Rust name doesn't PascalCase back to the real column -- a lowercase column (e.g. `public`), or a field deliberately renamed away from a database-specific name (see "Naming" below). - Keep `mysql::params!` queries (`:name` placeholders) over string interpolation even for read-only queries. - A query that joins several tables and selects same-named columns from more than one (`Id`, `Units`, `Year`, ...) needs an explicit SQL `AS` alias per column to disambiguate for `FromRow`. Rather than reaching for `#[mysql(rename = "...")]` on every such field, write each alias to already equal that field's default PascalCase conversion (e.g. a `pubmed_id` field needs the column aliased `AS PubmedId`, not `AS PubMedId` -- the derive capitalizes each underscore-separated segment, it doesn't preserve mid-word capitalization) -- then no renames are needed at all. Verify empirically with a real query against the live database rather than trusting the conversion by inspection; it is not compile-checked. - **`mysql::Pool::new` is expensive -- do not call it per request.** Its default `PoolConstraints` (`min: 10, max: 100`) makes it eagerly open 10 real connections before returning, which took about 7 seconds against a real database in this project despite running on localhost (a single already-established connection then takes under 200ms per query) -- plausibly a per-connection handshake/auth cost multiplied by 10, not network latency (a raw `TcpStream::connect` to the same address was sub-millisecond). A handler that builds a fresh `Pool` and drops it at the end of the function -- an easy mistake, since `Pool::new` reads just like opening any other resource -- pays this cost on *every single request*. Build one `Pool` lazily on first use and share it (it's a cheap, thread-safe, `Arc`-backed handle, so cloning it out of a cache is free): `OnceLock>>` works, since `Pool::new` is fallible and `OnceLock::get_or_init`'s closure isn't allowed to fail (`get_or_try_init` is still unstable, rust-lang/rust#109737) -- lock, check, build-and-cache on a miss, return a clone either way. This class of bug is exactly what per-request query timing (see Testing below) is for: it's invisible in a quick manual check (the person testing doesn't notice ~7s consistently, they just think the service is slow) but immediately obvious once requests are timed individually. ## Naming: database vs. code When a table/column name is domain jargon, internal, or otherwise not the name the API should expose (e.g. `InbredSet` where GeneNetwork's own code and docs call the concept "Group"), keep the raw name ONLY in: - SQL query strings (`"... FROM InbredSet WHERE ..."`) - `#[mysql(rename = "...")]` values Everywhere else -- struct names, field names, route paths, doc prose about the Rust API -- use the human-facing alias. This keeps the MariaDB schema as the single place the historical name has to appear. ## Error handling One shared RFC 7807 (Problem Details)-style error type for every endpoint, so clients parse one shape everywhere: - Fields beyond the RFC: `code` (machine-readable, stable), `detail`, `operation_id` (matches the OpenAPI operation name), `spec_url` (always the OpenAPI document's path, so a client can always look up what it just failed to understand), `request_id`, and an optional `next` hint pointing at a related endpoint (e.g. a 404 on `/thing/{name}` sets `next` to `/thing`, so a caller with a typo can discover valid values). - `type`/`spec_url` are relative paths, not absolute URLs, when the service has no fixed public domain -- RFC 7807 allows `type` to be a relative URI reference. - Document the same error schema on every route, including ones with no failure path today (no DB access, no I/O) -- a handler gaining one later doesn't need a doc update, and the spec stays honest about the shape every endpoint *could* return. - Generate a request id per request (`tower_http::request_id`: `SetRequestIdLayer` + `PropagateRequestIdLayer`, `x-request-id`) and include it in every error body and the response headers, so a report can be correlated with logs. Include the server's time so far (`took_ms`) too; it is an RFC 7807 extension member, and it matches the `meta.took_ms` of a success (see "Success responses"). - Errors raised by an extractor (the 401 for a bad token) have no handler to stamp them. Have the first extractor (`Ctx`) store the request's id and start time in the request extensions, and have the token extractor read them for its rejection; otherwise those errors silently lack a `request_id`. ## Success responses The success counterpart of the error type: one envelope for every endpoint, so clients (and LLM callers) parse one shape and are told what to do next. Do this from the start; retrofitting it later breaks every client and test at once. - Shape: `{ "data": ..., "meta": {"state", "took_ms", "request_id"}, "notes": [...], "next": [{"href", "description"}] }`. `data` is the payload and nothing else. - `meta.state` is `complete` or `partial`, not a copy of the HTTP status. Use `partial` (HTTP still 200) when a response is usable but something it could have had is missing or unavailable -- an optional lookup failed, results were left out for authorization, a ranking covers only a window -- and say what in `notes`. The alternative, quietly returning nulls, is invisible to callers. - `took_ms` is server time (it cannot be the client's round trip). Measure it with an extractor that is the handler's first parameter (`Ctx`: it records the start time and the request id), and build the response with `ctx.envelope(data)` at the end, so no handler fills in timing by hand and none forgets it. - `notes` are server-authored only. Never copy database text or request input into them: LLM callers read them as instructions. - `next` hrefs are relative, percent-encoded (a name with a space is `a%20b`), and must resolve: write a test that fetches an endpoint of each kind and follows every `next` it suggests (skip the huge listings, and say so in the suggestion's description). Suggest concrete calls (the record's children, its related records, the next page), not templates. - Exempt a plain-text liveness route and the OpenAPI document itself; nothing else. Enforce it: a test over `/openapi.json` that fails if any JSON 200 response is not an `Envelope_*` schema, so a new endpoint cannot forget it. - In `#[utoipa::path]`, `body = Envelope>`; the generic `Envelope` derives `ToSchema`. - In tests, parse every success body through one helper that checks the envelope and returns `data`, rather than indexing `["data"]` by hand at each site. ## OpenAPI Use `utoipa` + `utoipa-axum`, not a hand-maintained spec file: - `#[utoipa::path(...)]` on each handler, `#[derive(utoipa::ToSchema)]` on each request/response struct. - Register routes with `utoipa_axum::routes!(handler)` on an `OpenApiRouter`, not plain `axum::routing::get`. - A `#[derive(utoipa::OpenApi)]` struct supplies `info` (title, description, license) -- without it the spec carries utoipa-axum's placeholder metadata. - `OpenApiRouter::split_for_parts()` yields the plain `axum::Router` plus the assembled `OpenApi` value; serve the latter with `Json` at `/openapi.json`. - This way a route or schema change that doesn't update its `#[utoipa::path]`/`ToSchema` is a compile error or an obviously incomplete diff, not spec drift noticed weeks later. - When most endpoints require the same thing (a bearer token, an API key), set it once as the document default -- `#[openapi(security( ("scheme_name" = [])), ...)]` on the `#[derive(utoipa::OpenApi)]` struct -- instead of repeating `security(...)` on every `#[utoipa::path]`. The handful of endpoints that must NOT require it (health check, version, the endpoint that hands out the credential in the first place) override with an empty `security()` of their own; an absent `security(...)` on an operation inherits the document default, a present-but-empty one overrides to "none" -- standard OpenAPI semantics, not utoipa-specific. Audit every operation's resolved `security` in the generated `/openapi.json` after making this change (`"security": [...]` present vs. absent per operation) -- it's easy to silently document a public endpoint as requiring auth, or vice versa, since nothing fails to compile either way. - A response shape repeated across many handlers (e.g. "401, missing or invalid token", always the same error body type) doesn't need hand-copying into every `responses(...)` list: give it its own `#[derive(utoipa::IntoResponses)] #[response(status = 401, ...)]` marker struct wrapping the real body type (`struct Unauthorized( #[to_schema] ApiError)`), then reference it bare inside any handler's `responses(...)` (`responses(Unauthorized, (status = 200, ...))`). The marker struct is never constructed at runtime (the actual response still comes from wherever the rejection/error is produced, e.g. an extractor's `FromRequestParts` impl) -- `#[allow(dead_code)]` it with a comment saying why, rather than leaving an unexplained warning or actually constructing a throwaway value just to silence one. ## Authentication/authorization - Make requiring a token **opt-in per handler**, not a blanket middleware layer: a custom extractor (`struct AuthToken; impl FromRequestParts for AuthToken`) that verifies/rejects in `from_request_parts`. A handler adds `token: AuthToken` (or `_token: AuthToken` if it only needs the gate, not the claims) to its parameter list to require one; omitting the parameter is what makes it optional, with no central route-list to keep in sync. Non-body extractors like this can go anywhere in the parameter list except after a body-consuming one (`Json` etc. must stay last). - If some data is "public" and some isn't, don't let that become "some endpoints need a token and some don't" -- keep the token check universal (cheap: local signature verification, no network call) and make "is this specific resource public" a second, independent decision the handler makes afterward, with its own cache, only calling out to an authorization service for the resources that actually need it. - A resource addressed by `(container, item)` where `container` can belong to more than one underlying category (e.g. a "dataset" that's sometimes one database table, sometimes another) should resolve `container`'s category **first**, as its own lookup, before ever looking up `item`. Resolving `item` by guessing across categories (try category A's lookup, fall back to category B's) risks a same-named collision silently returning the wrong thing -- resolving `container` first instead means only a much smaller set of names (containers) is ever disambiguated across categories, not every item name ever. - An endpoint that hands out a credential (e.g. proxies an auth service's own token grant for a well-known test user) should decide per request, from the connection, whether to answer -- not from a mode flag or a bind address, which are easy to leave set wrongly in a deployment. Answer only a caller whose TCP peer address is loopback (`axum::extract::ConnectInfo`; serve with `into_make_service_with_connect_info`), and fail closed when there is no peer address. Never establish "local" from a header, but do refuse a request that carries `X-Forwarded-For`/`Forwarded`/`X-Real-IP`: a reverse proxy on the same host connects from loopback for every client, so without that refusal the whole internet looks local. Keep the decision in small plain functions with their own unit tests, test the handler with injected peer addresses (a local one, several remote ones, none at all, and a local one with a proxy header), and list the 403 in the OpenAPI responses. Default any credentials the endpoint needs to the target service's own published "dev fixtures" (a well-known test client/user every checkout of that service already has) so it works with zero extra configuration. Also give deployments a way to leave the route out entirely (a config switch checked when the router is built, so the route is a 404 and absent from the OpenAPI spec). Make that switch fail closed: parse it strictly so only a clearly affirmative value keeps the endpoint on and a typo ("flase") turns it off, since the safe state of a credential-issuing endpoint is "not running". service already has) so it works with zero extra configuration. ## Testing - **GEMMA mapping: ravanan is the default runner, `/compute/gemma` is the reference.** Any change that touches mapping is verified by sending the same request body to both endpoints -- `scripts/gemma-compare.py --base URL --body FILE` prints both wall-clock times and exits non-zero if a row disagrees on `rs`, `beta`, `se`, `l_mle` or `p_lrt`. The wrapper endpoint is only the fallback when Slurm is unavailable, so never "fix" a mapping difference by switching endpoints: the difference is the finding. - Both are content-cached (ravanan's store; gemma-wrapper's `--cache-dir`), so a repeat of an identical request is ~1-2 s against ~12-15 s cold -- cheap enough to re-run for verification. Note the wrapper's cache only hits from gemma-wrapper `5eb53fa` on (archives without the per-run `.log.txt` can never satisfy its cache-hit check); check `gemma-wrapper --help | head -1` for the version before concluding a cache is broken. Two parallel suites exercising the same endpoints: - Rust integration tests (`tests/*.rs`) call the router in-process via `tower::ServiceExt::oneshot(app())` -- no socket needed. - Python tests (`test/python/test_*.py`) spawn the actual compiled binary and talk to it over real HTTP, since that's the only way Python can reach it; share a `ServerTestCase` base class that spawns the binary and polls for the port. - Any test that needs the real database checks reachability first (`TcpStream::connect_timeout` / a `socket.create_connection` helper) and skips with a message rather than failing when unreachable -- the Guix build sandbox has no network access, and skip (not failure) is the correct result there. - Use one well-known, always-present fixture row per table (e.g. mouse and rat for a species table, BXD for a groups table) rather than inventing synthetic data -- grep the existing Python test suites of sibling services for what they already rely on being present. - Route every DB-backed Rust test's request through one shared helper (e.g. `tests/common::get`, a `mod common;` under `tests/` so every integration-test file can use it) that times the call and warns on stderr past a threshold (e.g. 1s). This is what actually caught the `Pool::new`-per-request bug above -- it was invisible until requests were timed individually. Warnings only show with `cargo test -- --nocapture` (or on a failing test; cargo captures output for passing tests otherwise), so say so in the README. Each integration test file compiles to its own process, so the one-time cost of building a share-once resource (like the Pool above) still shows up once per test *file*, not just once overall -- that's expected, not a sign the fix didn't work; what matters is that repeated calls inside one file (or one real server process) stay fast after it. - If endpoints need a bearer token and the service has a local-only token-issuing endpoint (see "Authentication/authorization" above), have the test suite fetch its own token from it rather than requiring a human to mint and export one by hand first. Try that first, fall back to an externally-provided token (an env var) when there is none -- so the suite still works unmodified either way. For the in-process Rust suite this means calling the router's own endpoint via the same `oneshot` plumbing used for every other request, with a local peer address injected into the request (there is no real connection); for a suite that spawns the real binary as a subprocess (Python, here), the test client is already local, so the subprocess answers its own token request over the same HTTP connection the test already uses. - Mutating process-wide state (env vars, global config) from a test is only safe when nothing else in the same process can read or write that same state concurrently. A single compiled test binary (one `tests/*.rs` file) runs its `#[tokio::test]`/`#[test]` functions concurrently on different threads by default -- so a test that must flip an env var through several values keeps all of those mutations in one sequential test function, with a comment saying why, rather than splitting them across multiple test functions in the same file. A suite that instead spawns a fresh subprocess per test class (own process, own env, e.g. Python's `ServerTestCase`) doesn't have this problem -- each class can set its own env independently and safely. On edition 2024, `std::env::set_var`/`remove_var` are `unsafe fn` (exactly because of this hazard); wrap each call in its own tiny `unsafe { ... }` with a `// SAFETY:` comment pointing at whichever of the two guarantees above makes it sound here, rather than one broad `unsafe` block around unrelated code. ## Guix packaging - `guix import crate -f Cargo.lock PACKAGE` regenerates the crate dependency list; diff the crate-name list before/after with `comm -23` (both directions) to confirm no unrelated version silently drifted. - `guix shell -C -D -N -f guix.scm` (isolated + network) can silently drop build dependencies on some Guix versions when `-D` and `-N` are combined -- if a fully isolated shell reports "cargo: command not found", fall back to `guix shell -D -f guix.scm` (no isolation). - Known trap: a committed `Cargo.lock` can pin crate versions newer than the Guix-packaged rust supports (crates.io keeps publishing; cargo's own relocking step inside the build will refuse with "rustc X.Y is not supported by CRATE, requires rustc Z"). This is a packaging-environment mismatch, not a code bug -- rule that out before assuming a build failure is caused by whatever code change prompted the build. To fix it: - `cargo update -p CRATE --precise VERSION`, one crate at a time (per the error list), picks the newest release still compatible -- `cargo generate-lockfile` alone re-derives these correctly for most transitive deps (it already favors the rustc-compatible version), but re-running it also undoes any manual `--precise` pin you already made, so apply pins one at a time without regenerating the whole lockfile afterwards. - A crate's own non-macro code can need a newer rustc regardless of which version you pick -- e.g. mysql_common adopted `let`-chains and `is_multiple_of` (both stabilized in rustc 1.88) in every 0.36.x/0.37.x release, so no `--precise` pin within those lines helps; the fix was downgrading the *dependent* (`mysql` in Cargo.toml) to the last major version pinning the still-compatible mysql_common 0.35.x. Check a candidate version actually builds, don't assume a lower semver number is automatically safe. - Query `https://index.crates.io/<2-or-3-letter-prefix-dirs>/` (the sparse registry index, one JSON line per version, with a `rust_version` field when the crate declares an MSRV) to compare versions without a full `cargo update` round-trip per guess. ## Commits Force the author explicitly (`--author="Name "`) if the local git identity doesn't match, and write messages in the GNU ChangeLog style: `* path/file (entity): what changed.`, one bullet per file/entity, most specific context first.