# gn-auth gn-auth (`factory/code/gn-auth`, package `gn_auth`) is GeneNetwork's shared OAuth2 authentication/authorisation server (Flask + [authlib](https://authlib.org/)). It issues tokens to end users and client applications, and answers "does the bearer of this token have privilege X on resource Y" questions for the rest of the GeneNetwork suite. GeneNetwork4 doesn't run its own user/token database -- it calls gn-auth over HTTP for both of those things, the same way GeneNetwork3 (gn3) already does. Everything below is about **how to call gn-auth as a client**, not how gn-auth itself is built. In dev, gn-auth listens on `http://127.0.0.1:8081/`, every route below is under an `/auth` prefix (`gn_auth/__init__.py`, `gn_auth/auth/views.py`). ## Tokens are JWTs, not opaque strings Every grant type gn-auth actually wires up -- `password`, `urn:ietf:params:oauth:grant-type:jwt-bearer` and `refresh_token` -- shares one token generator (`gn_auth/auth/authentication/oauth2/server.py`) that produces an RS256-signed JWT as the access token, plus a 42-character opaque refresh token. The JWT's claims include `scope`, `client_id`, `sub` (the user id), `iat`, `exp` and `jti` (`gn_auth/auth/authentication/oauth2/grants/jwt_bearer_grant.py`). This matters because it means a consuming service can **verify a token itself**, locally, by checking its signature against gn-auth's public key -- it doesn't have to ask gn-auth "is this token valid" on every request. ## Verifying a token: fetch the JWKS, check the signature locally `GET /auth/public-jwks` (no auth required) returns gn-auth's current RSA public keys, oldest first: ```json { "documentation": "The keys are listed in order of creation, from the oldest (first) to the newest (last).", "jwks": [{"kty": "RSA", "kid": "...", "n": "...", "e": "...", ...}, ...] } ``` gn-auth rotates in a new key every `JWKS_ROTATION_AGE_DAYS` (default 7) and keeps old ones around for `JWKS_DELETION_AGE_DAYS` (default 14, `gn_auth/settings.py`) -- so more than one key can be valid at once; try each one rather than assuming the first or last. **This is exactly what gn3 already does** (so genenetwork4 should mirror gn3, not invent a new pattern): ```python # genenetwork3/gn3/oauth2/jwks.py def fetch_jwks(authserveruri, path="auth/public-jwks") -> KeySet: response = requests.get(urljoin(authserveruri, path), timeout=300) return KeySet([JsonWebKey.import_key(key) for key in response.json()["jwks"]]) def validate_token(token: str, keys: KeySet) -> dict: for key in keys.keys: try: return JsonWebToken(["RS256"]).decode(token, key=key) except BadSignatureError: pass raise TokenValidationError("No key was found for validation.") ``` `genenetwork3/gn3/oauth2/authorisation.py`'s `require_token` decorator then just reads the `Authorization: Bearer ` header and calls the above. Two things worth doing better than gn3's version, in a genenetwork4 implementation: - **gn3 refetches the JWKS on every single incoming request** (no caching). gn-auth's own internal validator caches the key set and only refreshes it every `jwt_refresh_frequency_hours` (default 6, `gn_auth/auth/authentication/oauth2/resource_server.py`) -- that's the better model: fetch once, cache, refresh periodically (e.g. hourly), not per-request. - **gn3's `validate_token` only checks the signature.** It never calls `.validate()` to check `exp`/`nbf`/`iss`, so an expired token with a valid signature still passes. genenetwork4 should check the signature *and* the standard time-based claims. ## Checking authorization for a specific resource (e.g. a group) Knowing a token is validly signed only proves *who* the caller is, not *what they're allowed to do*. For that, gn-auth models permissions as: - a **resource**: a UUID, a name, a category (`mrna`, `phenotype`, `genotype`, `system`, `group`, `inbredset-group`, ...), and whether it's public. - a **privilege**: an id string shaped `::`, e.g. `group:resource:view-resource`, `system:data:link-to-group`, `system:user:masquerade` (real examples from gn-auth's migrations). - **roles**, which bundle privileges and get assigned to users on specific resources. A GeneNetwork group/`InbredSet` is one such resource. To check "can this token's user do X on InbredSet Y", **mirror `genenetwork3/gn3/api/case_attributes.py`'s `required_access` function** exactly -- two calls: ```python # 1. Resolve (SpeciesId, InbredSetId) to a gn-auth resource id. No auth needed. GET {AUTH_SERVER_URL}/auth/resource/populations/resource-id/{species_id}/{inbredset_id} -> {"resource-id": ""} # 2. Ask what the bearer of this token can do on that resource. POST {AUTH_SERVER_URL}/auth/resource/authorisation Authorization: Bearer {"resource-ids": [""]} -> { "": { "public-read": false, "roles": [ {"role_id": "...", "role_name": "...", "privileges": [{"privilege_id": "group:resource:view-resource", ...}, ...]} ] } } ``` Collect every `privilege_id` across every role in the response and check the ones you need are in there (`case_attributes.py` does `all(level in privileges for level in required_levels)`). There's a dataset/trait-oriented equivalent, `POST /auth/data/authorisation`, keyed by `"Dataset::Trait"` strings instead of resource ids (e.g. `"BXDPublish::10001"`) -- useful if genenetwork4 ever needs to authorize a specific trait rather than a whole group; see `gn_auth/auth/authorisation/data/views.py` for its exact request/response shape. Both of these routes are deliberately **not** hard-gated by gn-auth: call them with no `Authorization` header and you get back only the public-resource view; add a valid bearer token and you get the full, user-specific answer. That's by design, not a bug to route around. ## Endpoints reference This table runs wider than 100 chars: its longest path, `/auth/resource/populations/resource-id//`, is 66 chars by itself, leaving no room for the Auth required/Purpose columns if kept under that width. | Method | Path | Auth required | Purpose | |--------|--------------------------------------------------------------------|------------------------------------------------------|--------------------------------------------------------------------------------------------------| | GET | `/auth/public-jwks` | none | RSA public keys for verifying JWTs locally. | | POST | `/auth/token` | client credentials | Obtain a token (any grant type). | | POST | `/auth/revoke` | client credentials | Revoke a token. | | POST | `/auth/introspect` | bearer token, `introspect` scope | RFC 7662 introspection -- see Known issues, do not rely on this yet. | | GET | `/auth/resource/populations/resource-id//` | none | Map a `(SpeciesId, InbredSetId)` pair to a gn-auth `resource-id`. | | POST | `/auth/resource/authorisation` | bearer token (falls back to public-only without one) | Given `{"resource-ids": [...]}`, return the caller's roles/privileges on each resource. | | GET | `/auth/resource/authorisation/` | bearer token | Roles/privileges for one resource looked up by name. | | POST | `/auth/data/authorisation` | bearer token (falls back to public-only without one) | Same as resource/authorisation, but keyed by `"Dataset::Trait"` strings instead of resource ids. | | GET | `/auth/data/species` | none | List species known to gn-auth. | These are the routes relevant to a service consuming gn-auth. gn-auth has many more (`/auth/role/...`, `/auth/user/...`, `/auth/system/administration/...`) for managing the auth system itself -- not needed here. ## Known issues -- don't build on these yet - **`/auth/introspect` doesn't currently work.** Three independent problems: the config key that allow-lists which clients may call it, `OAUTH2_CLIENTS_WITH_INTROSPECTION_PRIVILEGE`, is never set by default; no client-creation tooling in the repo ever creates a client with the `client_type: "internal"` that `check_permission` requires (`gn_auth/auth/authentication/oauth2/endpoints/introspection.py`); and tokens issued via the password/jwt-bearer/refresh_token grants are never written to the `oauth2_tokens` table in the first place (`server.py`'s `save_token` is wired to a no-op for JWT tokens), so there's nothing in the database for introspection to find even if the first two problems were fixed. Use local JWT verification (above) instead. - The introspection response's `sub` claim is dead code -- a helper function to compute it exists but its result is never actually included in the returned JSON. - The `password` grant is marked broken by gn-auth's own test suite (skipped, with a comment that JWTs broke the original opaque-token-shaped test). Prefer the `jwt-bearer` grant, or treat token issuance as already-solved by whatever front end mints the token for a user and focus genenetwork4's side on verification. ## What this means for genenetwork4 genenetwork4 doesn't need to talk to gn-auth for every request -- only for the two things above: verifying a presented token's signature (cache the JWKS, refresh periodically, don't refetch per-request), and -- for data that isn't public -- asking gn-auth what that token's holder is allowed to do on a specific resource. Every endpoint needs the first (see "Every endpoint requires a token" below); only some need the second. Whether data "is public" (and so skips the second, resource-level check) is answered by GeneNetwork's own schema, not by asking gn-auth: GeneNetwork2 treats a dataset (`ProbeSetFreeze`, `PublishFreeze`, `GenoFreeze`) as public exactly when `public > 0 AND confidentiality < 1` (`wqflask/api/router.py`, used consistently throughout GN2/GN3). `InbredSet`/group rows carry a `public` column too, but GN2 never actually filters on it anywhere -- it's informational only there -- so groups/species endpoints need no *resource-level* gating, matching GN2's own behavior; they still need a valid token like every endpoint does. For dataset/trait data that *is* public by the threshold above, likewise still require a valid token (don't skip verification entirely), but skip the resource-authorization call to gn-auth -- one less network round trip for what is, in practice, most of the data. A dataset older than 5 years is treated as public too, regardless of its `public`/`confidentiality` flags -- a confidentiality embargo isn't meant to last forever. This isn't a GN2 convention, just a rule applied here, based on `CreateTime`. ### Resolving a probeset/trait to a gn-auth resource A group/`InbredSet` resource's id is resolved via `GET /auth/resource/populations/resource-id//` (above). A probeset/trait doesn't have an equivalent "resolve this id to a resource id" endpoint -- instead, call `POST /auth/data/authorisation` directly with a `"Dataset::Trait"` string (e.g. `{"traits": ["HC_M2_0606_P::1442370_at"]}`); gn-auth resolves the resource itself, internally, via a join on `linked_mrna_data.dataset_name` (`gn_auth/auth/authorisation/data/ mrna.py`'s `resources_by_datasets_and_traits`). If the trait's dataset was never linked to a gn-auth resource at all (the common case for data nobody has ever restricted), gn-auth answers with a 404, which genenetwork4 treats the same as "no privileges" -- not a request failure. ### Implemented, in `src/auth.rs` - `AuthToken`, an Axum extractor (`impl FromRequestParts`): verifies a bearer token's RS256 signature and standard time claims (`exp`/`nbf`, checked when present -- gn-auth itself treats them as optional, confirmed against its own validator) against a cached, periodically refreshed JWKS. Add it as a handler parameter to require a valid token; omit it for endpoints that don't need one. - `is_publicly_readable(public, confidentiality, create_time)`: the GN2 threshold above, plus the 5-year embargo-age rule, as a plain function a handler calls on its own fetched row. - `authorise(auth_server_url, token, resource_id, privilege)`: the two gn-auth calls (resource-id resolution is the caller's job -- `GET /auth/resource/populations/resource-id/...` -- `authorise` itself only wraps `POST /auth/resource/authorisation`), with the per-(user, resource) answer cached briefly so repeated requests for the same resource by the same user don't each cost a round trip. - `authorise_trait(auth_server_url, token, dataset_name, trait_name, privilege)`: the trait-keyed equivalent of `authorise`, wrapping `POST /auth/data/authorisation` -- for data addressed by a `(dataset, probeset)` pair rather than a resource id directly. Same caching as `authorise`. - `authorise_dataset(auth_server_url, token, dataset_name, privilege)`: `authorise_trait` with an empty trait name, for checking a whole dataset at once (e.g. before listing every probeset in it) -- gn-auth resolves a trait's resource purely from its dataset name, so the trait half of the `"Dataset::Trait"` string never affects the answer. - `check(auth_server_url, token, dataset_name, trait_name, public, confidentiality, create_time)`: combines `is_publicly_readable` and `authorise_trait`/`authorise_dataset` (always with `"group:resource:view-resource"`, the only privilege any route here checks) into the one branch every gated handler below actually wants, returning `Option` -- the capability a handler can require to construct a "found and visible" result, rather than writing the branch out by hand. `is_authorised` is its boolean form, for a caller that filters a list instead of rejecting one request (`publication_traits`). - `tests/auth.rs` runs all of this against a real gn-auth instance (set `GN4_TEST_AUTH_TOKEN`/`GN4_TEST_RESOURCE_ID`; skips otherwise). ### Wired into routes: the `/datasets/{dataset}/probesets...` family Three endpoints are gated this way so far, all the same way: `GET /datasets/{dataset}/probesets/{probeset}`, its `/values` sibling, and the `GET /datasets/{dataset}/probesets` listing (checked once for the dataset as a whole via `authorise_dataset`, rather than once per probeset in the list): - An `AuthToken` is required unconditionally -- every caller needs a valid token, even for public data. - If the dataset is publicly readable (`is_publicly_readable`), the data is returned with no further gn-auth call. - Otherwise, `authorise_trait`/`authorise_dataset` with `"group:resource:view-resource"` is called; a `false` result (including the "never linked, so no privileges" 404 case above) is a 403, not a 404 -- the dataset (and probeset, where there is one) exist, the caller just isn't allowed to see them. - A failure to reach/parse gn-auth's answer is a 502, distinct from a 403 (denied) or a 500 (this service's own database error) -- see `tests/dataset_probeset_info.rs`, `tests/dataset_probeset_values.rs` and `tests/dataset_probesets.rs` for all of these cases exercised against a real gn-auth instance. `GET /groups/{group}/traits` and its singular siblings (`/traits/ {trait_id}` and its `/values` sibling) are gated the same way as `/traits/{dataset}/...` below: resolve the group's own `PublishFreeze` row (by the group's name -- `PublishXRef`/`PublishFreeze` aren't tied by a foreign key, so this is a name lookup, not a join) and run the same `is_publicly_readable`/`authorise_trait`/`authorise_dataset` check against it. If a group has published traits but no matching `PublishFreeze` row to check, that's treated as a deny rather than as ungated/public data. `/traits` (the listing) skips the round trip only when the group has zero published traits -- there's nothing to protect in an empty list, so no reason to spend a gn-auth call (or risk denying) on it. All three endpoints, and the two above them (`/datasets/{dataset}/probesets/{probeset}/values` and `/traits/ {dataset}/{trait}/values`), carry `#[gnauth(secured)]` (`gnauth-macros`): a compile-time tag, checked by that attribute macro, for handlers that can share actual trait/probeset *values*. It's a textual check only -- it can't see across function calls -- so it verifies the handler takes `AuthToken` and mentions `auth::Authorized`, the capability type only `auth::check` can produce (its field is private, so it can't be fabricated outside `src/auth.rs`). Metadata-only endpoints (`/traits/{dataset}`, `/datasets/{dataset}/probesets`, `/groups/{group}/traits` and their singular siblings) are gated the same way but don't carry the tag -- they don't share values, just whether a trait/dataset is visible at all. `gnauth-macros` also has the opposite tag, `#[gnauth(disabled)]`, for the handful of endpoints that intentionally take no `AuthToken` at all -- `hello_world` and `version` (`src/lib.rs`) carry it, marking "no token needed" as a deliberate decision rather than an oversight. It checks the converse of `secured`: the signature must *not* take `AuthToken`. (`routes::dev::get_test_token` also takes no token, but isn't tagged -- it's a conditionally-registered test utility, a different category from a normal public endpoint.) ### Wired into routes: `/traits/{dataset}/...`, spanning both categories `/traits/{dataset}`, `/traits/{dataset}/{trait}` and its `/values` sibling (`src/routes/traits.rs`) answer the same question for *either* category through one URL shape -- see the README's `/traits` section for the addressing scheme. Gating is identical to the `/datasets/{dataset}/probesets...` family above, just resolved against whichever table the dataset belongs to: `PublishFreeze.public`/ `confidentiality` feed `is_publicly_readable` for a Publish dataset the same way `ProbeSetFreeze.public`/`confidentiality` already do for a ProbeSet one, and `authorise_trait`/`authorise_dataset` is called with that dataset's own name either way -- gn-auth's `/auth/data/authorisation` already dispatches "Publish"-suffixed dataset names to `gn_auth/auth/authorisation/data/phenotypes.py`'s `resources_by_datasets_and_traits`, the exact mRNA mechanism this integration already relies on, so no gn-auth-side change was needed to extend gating to phenotype data through this endpoint. No non-public `PublishFreeze` exists in the database this was tested against, so the deny path for the Publish branch specifically wasn't exercised against real data -- only against the already-proven ProbeSet branch and gn-auth's own "unlinked dataset" 404-to-deny behavior, which this new branch calls unchanged. The same caveat applies to `/groups/{group}/traits...` above, which resolves to the same `PublishFreeze` check. ### Every endpoint requires a token Everything above was about *authorization* (is this specific resource public, does this user have a privilege on it) for the handful of endpoints that check it. Separately, and more broadly: every endpoint in this API now requires a valid token (`AuthToken`, checked by local JWKS signature verification as described above) just to be called at all -- `/`, `/version`, `/openapi.json` and `/get-test-token` itself are the only exceptions (a health check, a version check, the spec describing all this, and the endpoint that hands out a token in the first place would be circular if it needed one). This is a blanket requirement independent of whether an endpoint also checks resource-level authorization: `/species`, `/groups/{group}/strains`, `/datasets/{dataset}` and the like need a token now but don't check anything beyond that it's valid -- same as a `is_publicly_readable` dataset in the `dataset_probeset*`/`traits*` family skips the authorization call but still needs a valid token. `ApiDoc`'s `#[openapi(security(("bearer_token" = [])), ...)]` (`src/lib.rs`) makes this the OpenAPI document's default security requirement, so individual handlers don't each repeat `security(...)`; the three exceptions besides `/get-test-token` (`/`, `/version`) override it with their own empty `security()`. `error::Unauthorized` (a `#[derive(utoipa::IntoResponses)]` type wrapping `ApiError`) is the shared 401 response every gated handler references by name in its own `responses(...)` list instead of writing out the same tuple by hand. ### Getting a token for local testing: `POST /get-test-token` Every live test in this repo (`tests/*.rs`, `test/python/test_*.py`) needs a real bearer token, obtained by hand so far (e.g. `test-token2.txt` in the setup notes). genenetwork4 can hand one out itself now: `POST /get-test-token` (`src/routes/dev.rs`) calls gn-auth's own password grant for gn-auth's well-known development user (`test@development.user`, from `flask init-dev-clients`/ `gn_auth/wsgi.py`) and returns the result as the `data` of the usual success envelope (README, "Success responses"). See the README's "Development" section for who it answers (only a caller on this machine; nothing to enable) and gn-auth's own `AUTHLIB_INSECURE_TRANSPORT` requirement for accepting the grant over plain HTTP at all. This doesn't change anything about how genenetwork4 verifies or authorizes tokens (src/auth.rs) -- it is purely a convenience for obtaining one of the tokens that mechanism already accepts, limited to callers on the same machine by a per-request check of the connection's peer address (127.0.x.x or ::1, and no proxy header). That check sees a reverse proxy or a container's network gateway as the peer, so a deployed instance needs more than this -- ideally `GENENETWORK4_TEST_TOKEN=false`, which leaves the route out entirely; see the README's "Behind nginx or in a container". The test suite uses it automatically: `tests/common::auth_token()` (Rust) and `test/python/auth_helpers.auth_token()` (Python) both try `POST /get-test-token` first, falling back to a `GN4_TEST_AUTH_TOKEN` env var only if that fails. With a reachable gn-auth (`AUTHLIB_INSECURE_TRANSPORT` set) the whole suite -- every endpoint now needing a token -- runs with no token manually provisioned at all. ### Found wrong in gn-auth while building this Four bugs in gn-auth itself blocked this integration before they were fixed (see gn-auth's own `RELEASE_NOTES.md` and its `fix-fresh-deployment-bugs` branch): a JWKS cache that could stay empty for hours after a fresh deployment's first token was issued, a dev client whose secret was never hashed so it could never authenticate, `public_resources()` crashing on every fresh deployment, and `POST /auth/resource/authorisation` crashing (instead of degrading to the anonymous view) on an invalid rather than missing bearer token.