diff options
| author | Pjotr <wrk@bosa.lan> | 2026-11-08 13:28:15 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-11-08 13:28:15 +0000 |
| commit | e5c2aa0ab1e568fe91c63cd5257aee7ff08ebed9 (patch) | |
| tree | 3404fdf0a0c1a0cd34218832905bac0361074b64 | |
| parent | b8fdbb3d60603c07cacf9f406ed1a5c228223829 (diff) | |
| download | genenetwork4-e5c2aa0ab1e568fe91c63cd5257aee7ff08ebed9.tar.gz | |
docs: what the last two commits changed, and the dev/production split
* README.md (Running with cargo): "This deployment runs two instances:
3000 (production) and 3001 (development)" -- the table of ports, binaries
and cache directories, both start-up invocations, and why the caches are
separate while the database, index and auth stub are shared. Also notes
that in-memory state is per instance, so the dev server is not a capacity
test of production.
* docs/www/ai-instruction.md:
- /version now described honestly: git (with -dirty), dirty,
source_digest, database reachable/unreachable -- and that
`unreachable` means stop and report, not retry in a loop.
- error table: 413 (plain text, not the JSON object: over the 64 MiB
body limit), 500 database_error with a connection-loss detail is
transient (retry once), and 409 run_id_in_use pointing at the status
endpoint.
- covariates: case_attribute is the smallest form (nothing posted),
which is also the answer to a 413 from sending a value per individual.
| -rw-r--r-- | README.md | 51 | ||||
| -rw-r--r-- | docs/www/ai-instruction.md | 27 |
2 files changed, 76 insertions, 2 deletions
diff --git a/README.md b/README.md index dd3716b..a0f5958 100644 --- a/README.md +++ b/README.md @@ -1565,6 +1565,57 @@ GENENETWORK4_PORT=8080 cargo run GENENETWORK4_BIND=127.0.0.1 cargo run ``` +### This deployment runs two instances: 3000 (production) and 3001 (development) + +The reference deployment keeps a development instance next to the live +one, on the same box, so work can be tried out without disturbing what +the site is serving: + +| instance | port | binary | run caches | proxied by nginx | +|---|---|---|---|---| +| **production** | `3000` | `target/release/genenetwork4` | defaults (`/tmp/gn4-gemma`, `/tmp/gn4-rqtl`, …) | yes, at `/gn4/` (see [docs/nginx.md](docs/nginx.md)) | +| **development** | `3001` | `target/debug/genenetwork4` | `/tmp/gn4-dev/{gemma,rqtl,ravanan}` | no | + +They differ only in the port, the build profile and those cache +directories; both use the same read-only `SQL_URI`, the same Xapian index +and the same auth stub, so a request behaves the same on either. The +separate caches matter: a GEMMA or R/qtl run on the development instance +must not fill, or be sped up by, production's warm cache. + +```sh +P=$(guix shell -D -f guix.scm -- bash -c 'echo $GUIX_ENVIRONMENT') # needs R + gemma-wrapper on PATH + +# production +PATH=$P/bin:$PATH R_LIBS_SITE=$P/site-library/ \ +GENENETWORK4_BIND=127.0.0.1 GENENETWORK4_PORT=3000 GENENETWORK4_TEST_TOKEN=false \ +AUTH_SERVER_URL=http://127.0.0.1:8081/ \ +SQL_URI=mysql://webqtlout:webqtlout@127.0.0.1/db_webqtl \ +XAPIAN_DB_PATH=/home/wrk/tmp/ai-root/factory/gn4-index SEARCH_ENGINE=native \ +nohup ./target/release/genenetwork4 > /tmp/gn4-3000.log 2>&1 & # pid in /tmp/gn4.pid + +# development: same, plus the port and its own caches +PATH=$P/bin:$PATH R_LIBS_SITE=$P/site-library/ \ +GENENETWORK4_BIND=127.0.0.1 GENENETWORK4_PORT=3001 GENENETWORK4_TEST_TOKEN=false \ +AUTH_SERVER_URL=http://127.0.0.1:8081/ \ +SQL_URI=mysql://webqtlout:webqtlout@127.0.0.1/db_webqtl \ +XAPIAN_DB_PATH=/home/wrk/tmp/ai-root/factory/gn4-index SEARCH_ENGINE=native \ +GEMMA_CACHE_DIR=/tmp/gn4-dev/gemma RQTL_CACHE_DIR=/tmp/gn4-dev/rqtl \ +RAVANAN_CACHE_DIR=/tmp/gn4-dev/ravanan \ +nohup ./target/debug/genenetwork4 > /tmp/gn4-dev.log 2>&1 & # pid in /tmp/gn4-dev.pid +``` + +Both need the `PATH`/`R_LIBS_SITE` pair (see "Building with cargo, outside +a bare toolchain" below) or R/qtl fails with `there is no package called +'optparse'`, and both need the auth stub on `127.0.0.1:8081` to serve +`/auth/public-jwks` (see "Tests"). + +Anything the server keeps in memory is per instance: the pool, the JWKS +and authorisation caches, and any in-process compute limit. So the two +instances do not serialise against each other — a mapping on `:3001` runs +alongside one on `:3000` — which is the point of the split, and a good +reason not to treat the development instance as a capacity test of what +production will do. + ### Building with cargo, outside a bare toolchain `cargo build` in the package's own dependency shell is the supported diff --git a/docs/www/ai-instruction.md b/docs/www/ai-instruction.md index d38e207..794167b 100644 --- a/docs/www/ai-instruction.md +++ b/docs/www/ai-instruction.md @@ -123,9 +123,19 @@ Two routes are safe to call first and need no token: ```sh curl -s "$API/openapi.json" # every endpoint, parameter, schema -curl -s "$API/version" # build + git revision +curl -s "$API/version" # build, git revision, and whether the database answers ``` +`/version` is worth reading when something fails everywhere at once. It +returns `git` (the commit the running binary was built from; a `-dirty` +suffix means that build had uncommitted changes), `dirty`, +`source_digest` (which identifies the build when a commit hash cannot), +`database` (`reachable` or `unreachable`) and `date` (the *commit's* date, +not the build time). If `database` is `unreachable`, every data endpoint +will fail with `503 db_unavailable` or `500 database_error` and there is +nothing you can do but report it — do not retry in a loop and do not +conclude the resource does not exist. + Every response is one envelope: | member | meaning | @@ -409,7 +419,11 @@ A mapping's response ends with what to do next: **Covariates.** Add `covariates` to fit the study's own metadata — sex, diet, batch — alongside the trait. The easiest form names a case -attribute (§3), and the server codes it: +attribute (§3), and the server codes it. It is also the smallest: nothing +is posted, where an inline covariate sends a value per individual — for a +panel of thousands that is the difference between a few hundred bytes and +a multi-megabyte body, and bodies are capped at 64 MiB (`413`, in plain +text, if you pass it): ```json "covariates": [{"case_attribute": "Sex"}] @@ -522,6 +536,7 @@ curl -s -H "$AUTH" "$API/groups/BXD/phewas?chr=9&mb=29.940961&window=10000" | `window` | **half-width in base pairs, centred**: `10000` is 10 kb up *and* downstream (the default) | | `genotype_version` | whose marker table defines the window; omit for the group's current one | + ### What comes back `data` has `group`, `genotype_version`, `centre` (`locus`, `chr`, `mb`), @@ -594,9 +609,17 @@ Errors are RFC 7807-style JSON (`type`, `title`, `status`, `code`, | 403 | `forbidden` | the data is not public: do not retry | | 404 | `not_found` | check the name; `next` often lists the valid values | | 422 | `invalid_query`, `invalid_lmm` | fix the query or parameter; do not retry | +| 409 | `run_id_in_use` | the `?id=` you chose is already in use: pick another, or poll `/compute/{rqtl,gemma}/status/{id}` | +| 413 | — (plain text, **not** JSON) | the request body is over the 64 MiB limit: send fewer values, or none at all (§4a's `case_attribute`) | | 429 | — | back off, honour `Retry-After` (not currently emitted, but written into the API's contract) | +| 500 | `database_error` | if the `detail` mentions a connection (`Broken pipe`, `Connection reset`, `server has gone away`) it is transient: retry **once**, then report; otherwise treat it as a bug and report it with the `request_id` | | 502 / 503 | `backend_unreachable`, `db_unavailable` | transient: retry with exponential backoff, max ~5 tries | +Two things about the error shape itself: a **413 comes back as plain +text**, not the JSON error object (`Failed to buffer the request body: +length limit exceeded`) — the body limit is enforced before any handler +runs — and errors carry `took_ms` as well, like a success's `meta`. + Quote `request_id` when reporting a problem — it is in the body, in the `x-request-id` header, and in the server's access log. |
