about summary refs log tree commit diff
path: root/docs/nginx.md
AgeCommit message (Collapse)Author
9 daysapi: name the caller on the response, so the log can say whoPjotr
Answering "is robw using this successfully, and the others?" from the access logs took an hour of inference -- user agents, search terms, which traits were mapped -- because nginx records the address and the subject is only in the token. Two callers can share an address (an agent and a person behind one NAT), one caller can move between addresses (an LLM's egress is a /24), and the answer for a given person was "probably 80.115.1.132". The API now sets `x-gn4-user: <token sub>` on every **authenticated** response, and the `gn4` log format records it as `user=`. The subject is only known inside the handler (the `AuthToken` extractor verifies the token there), so a middleware opens a slot in the request extensions, the extractor fills it, and the header is set on the way out; a request with no valid token gets no header, which is the honest answer and reads as `user=-`. The caller only ever sees an identity it already presented. * src/auth.rs: `USER_HEADER`, `CallerSlot`, and the extractor recording its subject. * src/lib.rs: the `tag_caller` middleware, outermost so the slot exists before any extractor runs. * tests/auth.rs: `an_authenticated_response_names_its_caller` -- the header equals the token's `sub`, and `/version` (no identity) carries none. * etc/nginx-vesuvio.pu.ac.ke.conf (and the deployed copy of it): `user=` in the log format. * README.md, docs/nginx.md (the format, an `awk` one-liner for per-user counts, and the caveat that old and new lines differ in field count), docs/www/operator-checklist.md (`user=-` means no token, not a token that failed late). 281 tests pass; the same four as before do not.
10 dayshttp: accept bodies up to 64 MiB, not axum's 2 MiB defaultPjotr
A caller posting a mapping result back to /plot/rqtl -- or a compute request carrying a value per individual plus covariates -- was refused with `413 Failed to buffer the request body: length limit exceeded`. That is axum's DefaultBodyLimit, 2 MiB, which nothing here had raised: a real GEMMA run on the BXD WGS panel is ~1.9 MB, so the plot endpoints sat just under the cap and anything bigger (a genome-sized run, or every animal's covariate value) fell over it. nginx's 10m was never the binding limit. * src/lib.rs (BODY_LIMIT_BYTES): 64 MiB, with the reasoning and the note that a proxy has to allow as much or it refuses first. * etc/nginx-vesuvio.pu.ac.ke.conf, docs/nginx.md: client_max_body_size 64m, so the two limits agree instead of the proxy cutting in at 10m. * README.md ("Request body size"): both caps, and that a request does not need to be large -- `{"case_attribute": "Sex"}` has the server read the attribute and align it, where posting a value per animal is pure payload. Verified: the 11.9 MB body that was refused now returns an SVG; 67 MB still gets axum's 413.
11 dayscompute: proxy /compute/* for any valid tokenPjotr
R/qtl and GEMMA are part of the service now, so the blanket 403 goes: the handlers already require an AuthToken, which is the whole gate ("token_only", not resource-checked). What that buys and costs is now written down rather than implied -- any token holder can name server-side paths for R and gemma to read, and the public-data-only restriction on traits/datasets does not apply here at all. The box also has to be able to run them: PATH must carry Rscript, gemma-wrapper and gemma, *and* R_LIBS_SITE must point at the profile's site-library or scripts/rqtl_wrapper.R dies on `there is no package called 'optparse'` -- which reads like a token or proxy fault and is neither. Verified through nginx with a token: POST /gn4/compute/rqtl 200 in 0.4s with the fixture's marker rows, POST /gn4/compute/gemma 200 in 2.1s with 7 320 marker rows (identical to the documented run's first row), GET /gn4/compute/gemma/status/<id> ending in "GN4: computation complete", /compute/correlation 200; all four 401 without a token. * etc/nginx-vesuvio.pu.ac.ke.conf: drop the /compute 403 locations, keep them commented as the way back; timeouts/body-size comments now apply. * docs/www/ai-instruction.md: new ยง4a, "Running QTL mapping (compute)" -- shapes, server-side paths, ?id= + status polling, what to do with a 504. * docs/www/operator-checklist.md: compute moves from "must be off" to "proxied, know what it grants", plus what the box needs (PATH, R_LIBS_SITE, ravanan) and the measured results. * docs/nginx.md: same, in the deployment and checking sections.
11 daysdocs/www: rewrite the AI instructions for the live deploymentPjotr
ai-instruction.md was written before anything was deployed: it took a placeholder base URL, did not say where the API is mounted, and buried the operator's deployment notes in an appendix labelled "not for the AI reader" -- awkward to hand to an assistant, and confusing when it is. So: the AI-facing page now names the deployment (https://vesuvio.pu.ac.ke/gn4) and is self-contained, and the operator material moves to operator-checklist.md. Everything in the page was exercised against the running deployment. Content changes, beyond the move: * `next` hrefs arrive already prefixed (`/gn4/groups/BXD/strains`), so they resolve against the origin -- joining them to the current URL's directory double-prefixes them. Said plainly, since it is the one thing the prefix breaks. * says what cannot work and should not be retried: get-test-token is a 404, /compute/* is a 403 from the front end, and neither is a permissions problem to work around. * `/publications/{pubmed_id}/traits` wants the PubMed id, not the internal `publication_id` in the same trait record, which 404s. * states the current rate-limit position (none configured) instead of implying a 429 policy that does not exist yet. * docs/www/ai-instruction.md: rewritten, paste-ready. * docs/www/operator-checklist.md: new file, the former appendix, updated with what is now enforced at the proxy and a record of the verified request matrix. * README.md, docs/nginx.md, etc/nginx-vesuvio.pu.ac.ke.conf: point at the operator checklist by its new name.
11 daysdocs: SEARCH_ENGINE=native in the nginx deployment's environmentPjotr
Without it /search forwards to GeneNetwork3 at GN3_URL (:5003) and answers 502 backend_unreachable when nothing is listening there -- a correct error, with the error body's links rewritten, but easy to misread as the proxy being wrong. The deployment has an index, so native is what it wants. * docs/nginx.md (The server behind nginx): add the variable and say what the default does. * etc/nginx-vesuvio.pu.ac.ke.conf: same, in the start-up comment.
11 daysdocs: a certbot-edited default site silently wins over this blockPjotr
Reproduced from a mirror of the deployed /etc/nginx: with sites-enabled/default still present, nginx logs conflicting server name "vesuvio.pu.ac.ke" on 0.0.0.0:443, ignored and answers 443 with the default site -- stock welcome page at /, nginx's own 404 for /gn4/..., HTTP/1.1 despite `http2 on` in this block. sites-enabled/* is globbed alphabetically and the first block whose server_name matches wins, so `default` (edited by certbot to serve this hostname) shadows the gn4 block. Removing it gives the expected 302 / 301 / 200 / 404 / 403 matrix. * docs/nginx.md (nginx): say to install the snippet instead of the distribution default, and what the shadowing looks like from outside.
11 daysetc: the host is vesuvio.pu.ac.ke, not .ac.ukPjotr
The certificate on :443 carries subjectAltName DNS:vesuvio.pu.ac.ke, and vesuvio.pu.ac.uk does not resolve at all, so the config as committed could not have matched a request for the real name: nginx falls through to the default server, which is why /gn4/... answered 404 from nginx rather than a 502 from genenetwork4. * etc/nginx-vesuvio.pu.ac.uk.conf -> etc/nginx-vesuvio.pu.ac.ke.conf: server_name and the certificate paths. * docs/nginx.md: the same rename, and a "Checking it" section that starts from what actually gets in the way on the box -- TLS validates the URL hostname rather than a Host: header, so on-box checks need --resolve against the name on the certificate, and a mismatch there and in server_name look identical (default server answers).
11 daysetc: drop-in nginx config for vesuvio.pu.ac.uk/gn4Pjotr
docs/nginx.md explained the pieces; this is the assembled file for the deployment actually being set up, so it can be reviewed and installed without stitching anything together. It lives in etc/ rather than docs/, being deployable configuration rather than prose. It refuses /compute/*, which the README-based template did not: those endpoints take server-side file paths and run gemma/ravanan/R on them, so any token holder able to reach them can read the box -- the operator checklist in docs/www/ai-instruction.md already said to keep them on the internal listener, and this is that rule where it belongs. * etc/nginx-vesuvio.pu.ac.uk.conf: new file. TLS, http2, log format carrying x-request-id, / and /gn4 redirects, proxy_pass with the prefix-stripping trailing slash, raised timeouts, sub_filter rewriting of href/next/spec_url/type, and 403/404 refusals for /compute/ and /get-test-token. * docs/nginx.md: point at it, and say why /compute/ is refused.
12 daysdocs: publish genenetwork4 under a path prefix with nginxPjotr
The deployment being set up is https://vesuvio.pu.ac.uk/gn4 proxying to a loopback port, with no gn-auth yet. Two things there are not obvious and neither is discoverable from the README's /get-test-token section: every link the API emits is root-relative, so under /gn4 they resolve outside the mount point; and /compute/* runs for minutes in a single request, so nginx's one-minute default read timeout cuts it off mid-computation. The link rewriting is the delicate part -- the server's serde output is compact ("href":"/"), not the pretty-printed shape the README examples show, so a filter written from those examples silently does nothing. Both spellings of all four affected keys (href, next, spec_url, type) are in the config, and the envelope's "next":[ array is deliberately left alone. * docs/nginx.md: new file. Server environment, the nginx block, what the prefix does to links and the three ways to handle it, stub-minted tokens instead of gn-auth, and how to check the result. * README.md (Behind nginx or in a container): point at it.