name: genenetwork description: Use when the user wants anything from GeneNetwork (genenetwork.org): BXD and other mouse/rat genetic reference populations, traits ("get trait 20010"), per-strain values, QTL mapping with R/qtl or GEMMA, LOD/Manhattan plots, PheWAS, genotypes, or searching published phenotypes. Talks to GeneNetwork through its HTTP interface at https://vesuvio.pu.ac.ke/gn4.
GeneNetwork
The user is talking to GeneNetwork, and you are their way in. Talk about GeneNetwork in their terms: traits, groups, strains, mapping, plots, PheWAS. Do not talk about "the API", endpoints or HTTP unless they ask. Everything below is how you do it; what you say should be what GeneNetwork did.
What the user can do in GeneNetwork
When the request is open ("what can I do?", or you finish a step), offer the next natural option from this list:
| Ask for | What GeneNetwork gives | Section |
|---|---|---|
| a trait ("get trait 20010") | metadata (description, peak locus, LOD, n, source) and the per-strain values | 3 |
| a QTL map | R/qtl scan (LOD) or GEMMA LMM with LOCO kinship (Manhattan), run on the server | 3 |
| a plot | an SVG LOD / Manhattan plot, drawn by GeneNetwork | 3 |
| PheWAS | every published trait of the group peaking near a position (e.g. the peak you just mapped) | 3 |
| a search | published phenotypes by free text, group, tissue, gene symbol, chromosome, year... | 5 |
| populations and data | species, groups (BXD...), strains, case attributes, genotype versions, expression datasets and probesets, publications by PubMed id, GN accessions | 4 |
A natural session is: trait, values, R/qtl, plot, GEMMA, plot, PheWAS at the
peak — and /compute/fitqtl at that peak when the user asks how much it
matters (with a covariate, if one could confound it). Whenever the group
records metadata that could confound the trait (sex, age, epoch, batch,
sequencing depth), the same mappings again with that covariate fitted, to see
the peak survive it. After each step, suggest the next one.
0. Do this first
- Check you can reach GeneNetwork (no token needed):
curl -s https://vesuvio.pu.ac.ke/gn4/versionand, for the docs repo,git ls-remote https://git.genenetwork.org/genenetwork4 HEAD. A sandbox proxy blocks any host that is not on the person's allowlist (connect_rejected, orCONNECT tunnel failed, 403from git). Ask them to allow both domains in one go, not one at a time:
To work with GeneNetwork I need two domains allowed for this session:
vesuvio.pu.ac.ke(the data and mapping service) andgit.genenetwork.org(the docs, so I can reload this skill when it changes). Please add them under claude.ai > Settings > Capabilities (admins: Organization settings > Capabilities), then start a new message or session.
If only the repo is blocked, GeneNetwork itself still works: carry on, and
tell the user the skill file cannot be refreshed (section 7). Say all this once
and stop probing other routes.
2. Token. Look for one already given to you in the task context. If there
is none, stop and ask (section 6). Never guess, invent, reuse a token that
was not given to you, or look for another way in: POST /gn4/get-test-token
is a deliberate 404, and there is no sign-up or login. A token carries no
rights of its own; what you may read is decided per dataset.
3. Note the build (section 7): remember .data.git from /version as
your baseline; if it changes later in the session, suggest reloading
docs/www/SKILL.md.
4. Read the spec, not only this page. GET /gn4/openapi.json lists every
operation and is what the server really runs; this page can lag. Use
jq -r '.paths | keys[]' before saying something is not available. The
descriptions inside it are the real reference: every request option is
documented in components.schemas (for example
jq '.components.schemas.RqtlRequest.properties' openapi.json).
5. Assume, don't ask. A bare numeric trait id (20010) is a published
phenotype of group BXD (dataset BXDPublish). Try it; ask only on 404.
Base URL for everything: https://vesuvio.pu.ac.ke/gn4. /gn4 without the
trailing slash redirects there; anything outside /gn4/ is a 404 from the
front end, not from GeneNetwork. Only /version and /openapi.json need no
token; everything else answers 401 (no token) or 404 (not routed).
1. Read the replies: GeneNetwork tells you what to do next
Every JSON reply is {"data","meta","notes","next"} and sometimes "actions".
notes: read them. They say how many strains were matched, what was run, and caveats.next: GET links to related things. Eachhrefalready includes/gn4: resolve against the origin (https://vesuvio.pu.ac.ke$href).actions: ready-to-send POSTs. On a trait's/valuesthey are the R/qtl and GEMMA calls with the body already filled in; on a compute reply they are the matching plot call (href with?trait=already set; post the reply back to it). Use them instead of building calls by hand. Select by href, not position:jq '.actions[] | select(.href|endswith("/compute/rqtl")) | .body'.- Every compute reply also has
next: a PheWAS link at that run's peak marker. Readnextandactionsof each reply before deciding what to do. meta:state(complete/partial),took_ms,request_id, pluscolumns/num_rowson tabular payloads.meta.state: partialmeans usable but incomplete; the reason is innotes. Do not read anullenrichment field as "no such value in GeneNetwork"; read it as "ask again later".- Do not join an
hrefto the current URL's directory: that double-prefixes it (.../gn4/gn4/...).
Always offer the next step
End every answer with one sentence saying what to do next, built from that
reply's own next/actions — never from an idea of what a mapping pipeline
"usually" does. The server already computed the options; the sentence is you
relaying them.
- One sentence, after every job — a compute run, a lookup, a plot, an
upload, a basket edit. Not a list, not a section: "Next: plot that scan
(
POST /plot/rqtlwith the reply) or look at the PheWAS at rsm10000000888 (GET /groups/BXD/phewas?locus=rsm10000000888)." - Prefer the reply's
actionshref over a hand-built call, and say which one you are offering, so the user can paste it or ask for it. - A refusal is an answer too. When the reply is an error, its
nextor its detail names the alternative (/compute/bulklmmfor a GEMMA permutation,/compute/gemmafor a covariate fit): offer that as the sentence. - Do not stack suggestions. One
next, plus one alternative at most, and only if they lead somewhere genuinely different. - The user can switch it off. "Skip the suggestions", "just the result", "no next steps" — then stop offering them for the rest of the session (or until they say otherwise), and give the answer alone. Do not re-add a suggestion in later sentences; it is the sentence that is disabled, not the content.
- Never invent a step the API cannot do. If nothing in
next/actionsfits, say so ("nothing further from this reply") rather than proposing a path that does not exist.
2. Workflow: map a trait and plot it
API=https://vesuvio.pu.ac.ke/gn4; AUTH="Authorization: Bearer $TOKEN"; D=$(mktemp -d)
curl -s -H "$AUTH" $API/groups/BXD/traits/20010 # metadata
curl -s -H "$AUTH" $API/groups/BXD/traits/20010/values > $D/v.json # values + actions
jq '.actions[]|select(.href|endswith("/compute/rqtl"))|.body' $D/v.json > $D/body.json
# R/qtl. Save the FULL reply to a file (never pipe through head), then plot it.
curl -s -H "$AUTH" -H 'Content-Type: application/json' --data @$D/body.json \
$API/compute/rqtl > $D/rqtl.json
curl -s -H "$AUTH" -H 'Content-Type: application/json' --data @$D/rqtl.json \
-D $D/h.txt -o rqtl.svg "$API/plot/rqtl?trait=BXD::20010"; grep -i x-plot-summary $D/h.txt
# GEMMA: same shape. Use /compute/gemma/ravanan -- it is the default runner
# (see "Which GEMMA endpoint" below).
jq '.actions[]|select(.href|endswith("/compute/gemma"))|.body' $D/v.json > $D/gbody.json
curl -s -H "$AUTH" -H 'Content-Type: application/json' --data @$D/gbody.json \
"$API/compute/gemma/ravanan?id=t20010" > $D/gemma.json
curl -s -H "$AUTH" -H 'Content-Type: application/json' --data @$D/gemma.json \
-o gemma.svg "$API/plot/gemma?trait=BXD::20010"
# PheWAS at the peak: follow the compute reply's own `next` (10 kb window), or widen it
jq -r '.next[0].href' $D/rqtl.json
curl -s -H "$AUTH" --get --data-urlencode locus=<peak marker> --data window=1000000 \
$API/groups/BXD/phewas # window = half-width in bp
Give the user the SVGs (they render directly) and quote the x-plot-summary
header (peak, and the reference lines drawn when you asked for them).
Short API tokens (gn4ai_...)
A second credential format beside gn-auth's JWT, for agents and scripts that
should not carry a user's password or a long-lived JWT. Opaque: nothing is
stored inside it, and everything about it lives in the server's registry (one
SQLite database, tokens.sqlite).
- Administer them from this machine only.
scripts/gn4-token.py create|list|revoke|who(routes/tokens,/tokens/{id},/tokens/who). Those routes answer only a direct caller on this machine — the server checks the TCP peer address, not a header — and a request that arrives through a proxy is refused, so this can never be driven from the network. - The token is shown exactly once, by
create. Only its SHA-256 is stored; there is no way to recover or enumerate one.listshows id, subject, scope, status, prefix, expiry and last use. - Send it as
Authorization: Bearer gn4ai_...— never as a query parameter, which would land in nginx's log and innextlinks. - Public data only. It works on every token-only endpoint (search, compute, plots, genotype metadata and frequencies, baskets, uploads, matrices) and on secured endpoints for public rows; a confidential dataset answers 403 telling the caller to use a gn-auth JWT, because gn-auth cannot validate an opaque token.
- Revocation takes effect on the next request, with no restart: it is a row in the registry. Expiry is fixed when minted (30 days by default).
- 64 characters,
gn4ai_+ 56 body characters + a 2-character checksum, from an alphabet with no look-alikes (0/O,1/l/I) — a typo is refused before the registry is asked. - In the log, one JSON line per request names the person and the credential:
{"event":"request","ts":…,"user":"zachs","scheme":"opaque","token":"agent-1"}— never the token or its hash.scripts/gn4-log.py -f LOGrenders that as a table with the handle resolved against the registry, so it shows the subject as the registry has it now and the token's status today ([revoked],[expired],[unknown]);--jsonenriches the records forjq, and--summarycounts requests per caller and credential.
Which GEMMA endpoint: ravanan by default
Two endpoints run the same computation, and the reply shape is identical:
| endpoint | runs |
|---|---|
/compute/gemma/ravanan |
gemma itself, one invocation per chromosome, scattered by ravanan (Slurm) |
/compute/gemma |
gemma-wrapper (kinship, then GWA, both --loco) as a server subprocess, no cluster |
Use /compute/gemma/ravanan by default. Reach for /compute/gemma only
when:
- Slurm is down or job submission fails (
502/ravanan_failed, or a job that never leavesPENDING) -- the in-process runner needs no cluster; - the user asks for it, or the request needs something only it supports
(covariates, normalization -- ravanan refuses those with a
422); - you are testing or verifying results. The wrapper endpoint is the
reference: send the same body to both and compare rows.
scripts/gemma-compare.py --base URL --body FILEdoes exactly that, prints both wall-clock times, and exits non-zero if anyrsdisagrees onbeta,se,l_mleorp_lrt.
Both are warm-cached and both are cheap to repeat: a repeat of an identical
request comes back from cache (ravanan's store, or gemma-wrapper's
--cache-dir) in ~1-2 s against ~12-15 s for a first run, so re-running to
verify costs almost nothing.
3. GeneNetwork functionality, in detail
Mapping and plots (POST, verified working)
/compute/rqtl: R/qtlscanone(Haley-Knott) over the group's genotypes. Columnsname, chr, pos, lod.nperm=0by default, so no permutation lines./compute/gemma: GEMMA LMM with leave-one-chromosome-out kinship. Columnschr, rs, ps, n_miss, allele1, allele0, af, beta, se, l_mle, additive, p_lrt. Pass?id=and poll/compute/gemma/status/{id}for long runs. Per-marker p-values only; the plot draws no reference line unless asked —?bonferroni=truefor GEMMA's own,?thresholds=for another engine's. Effect size is in the row, next to the score:beta(GEMMA's effect per maternal-allele copy) withse, andadditive=−beta/2, which is the number GeneNetwork's own tables and CSV show. Quoteadditivewith its sign ("each maternal-allele copy lowers the trait by X" / the other way) and remember it is this run on these samples, not theadditivein trait info, which came from the published mapping. A note also carries GEMMA's own variance component for the null model (-pve): under LOCO there is one per chromosome, and the note lists every one —chr1 0.6674±0.3451, chr2 0.8426±0.3668, …— with the range and the two extremes (plus theirvg/ve) spelled out. Quote per-chromosome values, not an average: they are twenty different kinship matrices.pveis the share of variance the kinship explains — the same quantity as/compute/bulklmm'sh2, per chromosome instead of genome-wide — so it is the honest way to check the two engines against each other. It is scale-free: comparepvewithh2, nevervgwithσ²_e. Apveof exactly 0 or 1 is a boundary (the standard error then reads "not available"), a direction rather than a proportion.- Choosing individuals: every group-mapping endpoint (
/compute/gemma,/compute/gemma/ravanan,/compute/rqtl,/compute/qtl2,/compute/fitqtl,/compute/bulklmm) excludes parents and F1s by default.include_parents_f1: truekeeps them all;individuals: {include: [names], exclude: [names]}selects by name,excludewinning, with an unknown name a 422 that lists the panel. Remember including them changes the kinship, not the fit — GEMMA and R/qtl compute kinship without phenotypes, so parents/F1s shape relatedness while the association uses only animals with values; the reply's Genotype note gives both counts. /compute/bnw: a Bayesian network over a set of traits (BNW). Givegroup+traits, orvalues(one entry per variable with per-sample values). Columnsfrom, to, posterior, in_structure; one row per candidate edge, strongest first. A posterior is not a p-value — it is the share of the k-best networks containing the edge, i.e. how often the search drew that arrow in this sample, so quote it as a structural claim and never as a threshold on a QTL. Parameters default to BNW's own (maxparents 4, k 1, tiers 3, threshold 0.5) and the reply names the ones used. At most 40 variables, at least 3 samples complete across every variable.POST /plot/bnwdraws it from the posted reply — the network by default,?kind=posteriorfor the matrix of every candidate edge (arrow width and label are the posterior; dotted lines are edges that did not clear the threshold).POST /compute/bnw/featuresturns the network into traits: each node with parents becomes a module (node + parents) scored by the first principal component of its standardised members;?module=NAMEreturns that module's{name, value}scores, which go straight into/compute/rqtlor/compute/gemmaassamples.GET /compute/bnw/exampleslists BNW's own worked examples, and/compute/bnw/examples/{name}returns one as a matrix you can post to/compute/bnw— use them to try a real network before inventing data./compute/fitqtl: how big an effect is, not just where. Give it the marker names a scan found (its peaks) and it fits R/qtl'sfitqtlat each: columnsmarker, chr, pos, lod, pct_var, additive, se, t, p. Use it after/compute/rqtlwhenever the user asks "how much does this QTL matter" —additiveis half the difference between the two homozygote classes (the same convention GeneNetwork's published additive uses),pct_varthe variance explained, andpthe model's p-value. At most 100 markers per call; markers that cannot be fitted are absent, not zero. It fits covariates too —covariates: [{"case_attribute": "Sex"}], the same shape and the same coding as/compute/rqtland/compute/gemma— so theadditiveit reports can be the effect after conditioning on a confounder, which is usually the number worth quoting. The reply names any covariate column it had to leave out because it does not vary among the phenotyped individuals (R/qtl would otherwise stop with "system is exactly singular"). Without covariates itslodat a marker equals/compute/rqtl's LOD at that marker; with one fitted it is a different model, so the two no longer match — quote the scan for "where", this for "how much"./compute/qtl2with"rqtl2_cross": "do": a multiparent scan — 8 founder alleles, an LMM with LOCO kinship. Only a cross that ships founder genotypes can do this (a.genois a biallelic collapse);dois the one this deployment has. Passsamplesby name; the LOD means "some founder differs", not "this strain differs" — per-founder effects are not returned yet. Minutes of genotype probabilities the first time, cached per cross./compute/qtl2: the same panel through R/qtl2 — genotype probabilities and an LMM withlocokinship ("kinship": "none"turns it into a Haley-Knott scan on probabilities). Rows aremarker, chr, pos, lod, so/plot/rqtldraws them unchanged. Reach for it when a scan's peaks depend on relatedness, or when the panel is multiparent; it is slower than/compute/rqtl(a DO cross costs minutes of genotype probabilities, cached per panel). Autosomes only, andnotessays what was left out.- Need a GEMMA threshold? This API refuses
npermon GEMMA (one LOCO scan per permutation, one GEMMA slot). Two routes:/compute/bulklmmwithnperm(rotation null, seconds), or run GEMMA's own--permutatelocally on the files the reply names (exact GEMMA null, hours). R/qtl'snpermis a sanity check only — its null ignores relatedness. - Need a threshold, not just a LOD?
POST /compute/bulklmm(BulkLMM.jl) fits an LMM per marker with a global kinship matrix and permutes by rotation, so"nperm": 200is cheap and its thresholds belong to the model that made the LODs (noteslist alpha 0.10/0.05/0.01 and the seed). It is not GEMMA's LOCO model — say so when quoting a p-value — andh2_nullnear zero means it collapsed to a fixed-effects fit. One trait at a time is serial in Julia (the package threads only its many-trait scan); blocks of markers are scanned in parallel by the driver, so a reply'snotessay how many ran. - Genotypes at a peak (the only genotypes the API returns):
GET /groups/{g}/traits/{id}/peak-genotypes?marker=<marker>re-runs the scan, confirms the marker is at the peak (within?window=Mb, LOD ≥?min_lod=) and returns one call per animal that has a value —422 not_the_peak/not_a_hitotherwise. Plot it by posting that reply'sgenotypeswith the trait's values toPOST /plot/genotypes(R/qtl book Fig 4.1: genotype class across, value up). .../markers/{marker}/classes?trait=<id>summarises a trait's values per genotype class (n,mean,sd,min,max) at one marker — how you see that a peak rests on two animals, or that a marker is nearly monomorphic after a conversion dropsH. Genotypes never leave the API./genes/{symbol}/aliasesreturns Wikidata's alias list for a gene (cached seven days ingene-aliases.lmdb;?species=humanfor the ortholog's labels). A"stale": trueanswer means Wikidata was unreachable and this is the last value -- not that the gene has no aliases./probesets/{name}finds every dataset carrying a probeset id (indexed; suffixed_Avariants included,namesays which matched) and/genes/{symbol}/traitslists a symbol's expression traits across datasets with a real total,?sort=lod,?min_lod=,?group=. Neither returns values — a token is enough for the metadata, but values stay per-dataset: follownextto/datasets/{ds}/probesets/{name}/values./groups/{group}/genotypes/{version}/markerslists marker names only;.../markers/{marker}/locationgives one marker'sname/chr/cM/Mb. Use?chr=&page=&limit=./groups/{group}/genotypes/{version}/frequencies(and.../markers/{marker}/frequenciesfor one locus; for a file no group lists,/groups/unlisted/genotypes/{name}/frequencies): how the panel is actually called —{"B": 0.49, "D": 0.49, "H": 0.02}. Use it before trusting a flat scan: a marker that is 99 % missing produces the same flat LOD as real data. Probability-valued files are binned 0.05 wide (?bin=to change); a file over 256 MB is refused (413) rather than read.- Compute replies carry the class-count hint at their own peak
(
.../markers/{peak}/classes?trait={trait_id}): filling in the trait id is how you check a peak before reporting it, and/compute/qtl2'snotesexplains what ariselfconversion did toH. - Uploading new traits/covariates:
POST /matriceswith{"name","genotype_version","contents":"id,trait1,cov1\nBXD1,1.2,3\n…", "kinds":{"cov1":"covariate"}}— a delimited table whose first column names the individuals. It is checked against that genotype version's panel (same individuals, each named once) and refused with the difference otherwise. Each column becomes a basket<matrix>.<column>: map it with{"basket": "<matrix>.<column>"}on any compute endpoint.GET /matriceslists yours,GET /matrices/{name}/valuesreads the table back.scripts/example-matrix.pywrites a valid example from a genotype file. - Uploading genotypes:
POST /uploads/genotypeswith{"name":"<user>-GENONAME","base_version":"BXD.4","contents":"<tsv>"}— tab-delimited, header = individuals from column 4, then marker/chr/pos. It updates a population: ≤50 new individuals, at least one overlapping, ≤200 000 markers, and the values must be the same kind as the base's (letters, whole numbers, probabilities). The reply's Every compute endpoint takes it by name —{"genotype_upload": "<name>"}on/compute/{rqtl,gemma,qtl2,fitqtl}— withsamples(or abasket) matched by name; the server writes the form each engine reads. Uploading the same name again replaces it, and the digest in the reply says which version ran. A one-chromosome upload scans flat or empty under LOCO kinship — use"kinship": "overall"if that is all you have.scripts/example-genotype-upload.pymakes a valid one from any.geno. - Changing values: there is no write endpoint for traits, and there never
will be — but a user's own column can be stored and mapped.
POST /basketswith{"name", "values":[{name,value}…]}(values may be numbers, strings like"NA", or null),PATCH /baskets/{name}to edit (a new revision, a new digest), thenPOST /compute/qtl2with"basket": "<name>"instead ofsamples. Baskets are private to the token's user, live in their own SQLite file, and a mapping reply records the revision and digest it used — quote them when reporting a result. Each edit is a new revision with its own timestamp (GET /baskets/{name}/revisions), old revisions stay readable (?revision=N), and"basket": "name@2"maps a pinned one — quotename@revisionwhen a result must be reproducible. All four compute endpoints takebasket. Baskets live in one file per user underBASKET_DIR, named for the user, so a new token still finds them.GET /basketslists them,DELETEremoves one. - Follow
nextfreely: every link is aGETthat returns 200, and the API offers the whole graph from any response (genotypes -> frequencies -> markers -> a marker's location/tally -> PheWAS; trait -> values -> genotypes;/version-> every listing).actionsholds thePOSTs, which are not linkable. /plot/rqtl,/plot/gemma: post the compute reply back, receive animage/svg+xmlplot. Do not draw your own.- Options (all optional, add to the posted body; full list in
components.schemas.RqtlRequest/GemmaRequest): - R/qtl:
nperm(permutations; 100 takes about 14 s and addssignificant/suggestivethresholds that/plot/rqtlthen draws),method(hkdefault,ehk,mr,em,imp),model(normaldefault,np),control(a marker to condition on),genotype_version,include_parents_f1,covariates. - GEMMA:
lmm(default 9),genotype_version,include_parents_f1,covariates,trait_name("BXD::20010"— optional, but it is what the reply reports and what/plot/gemmatitles the picture with; without it the title is<group>::gemma).npermis refused (422 permutations_not_supported): for significance send the same values to/compute/rqtlwithnperm. - Normalizing before mapping (both engines, same three fields):
normalize=none|log2|log10|sqrt|negate|zscore|robust_zscore|rank(GN2's "Quantile"/qnorm; note it is a rank-based inverse-normal transform, not array quantile normalisation), pluswinsor_quantile(e.g.0.05for the 5th/95th percentile) orwinsor_fences(1.5= GN2's outlier rule,Q1-1.5*IQR). They run transform → winsorise → standardise, and the reply'snotesname the pipeline and any clipped/outlying values.GET …/values?normalize=does the same for the values endpoint and carries the fields into the suggested mapping bodies, so use that when you want the fetched numbers to be the mapped ones.zscoreis the population sd, matching genenetwork.org; effects (beta/additive) come out on the transformed scale, so say which scale you are quoting. - Covariates (
/compute/rqtl,/compute/gemmaand/compute/fitqtl):covariates: [{"case_attribute": "<name>"}]. Worth trying when the study's own metadata could confound the trait — sex, age at death, batch/epoch, cohort, sequencing depth — because they are fitted alongside the marker, and a peak that survives them is the stronger result. Read.data.descriptionsbefore choosing one, not just.data.names: the names do not explain themselves, and the descriptions do (BXD:SeqCvge= "Depth of DNA sequencing for 1 male of each strain in 2016",Epoch= BXD family subgroups,Epoch1(BXD1-32),Epoch2-6 (BXD33-220),Status=Live= Available at JAX, Cryo=Cryopreserved only, Extinct,RRID= a SciCrunch identifier). BXD has noSex; an unknown name is422 covariate_unavailablelisting the ones the group does have.- Then read the covariate line in
notes. It reports the coding and the complete cases, e.g.Covariate Status (Live=…, Cryo=…): categorical coded Cryo=0, Cyro=1, Ext=2, Live=3, UTHSC=4, UTHSC2=5, 90 of 93 samples have a value, 34 with both the trait and the covariate; 6 levels. Report that n next to any LOD/p change — a peak that shifts because 60 samples left the fit is not a discovery. Note the data's own typo:CyroandCryoare two levels. - Categorical vs numeric. Numeric text passes through as numbers;
anything else with 2–20 distinct values among the strains being
mapped is coded 0..k−1 in sorted order; more than 20, a single level,
or a
mapthat misses a value is a422asking for an explicitmap. Measured on BXD (trait 10001,BXD.4):RRIDis refused —422 … 82 distinct values, too many to code automatically— whileStatus(6 levels) andEpochare accepted. - Compound values are a trap.
SeqCvgemixes numbers with two-library values (40+31L) and carries 30 distinct values across the group: if few enough fall inside the mapped panel it is accepted, but as a categorical — and the GEMMA run then fails (502 gemma_failed) rather than telling you why. The same is true of any mostly-numeric attribute with a few odd entries. Send the coding yourself:{"case_attribute": "SeqCvge", "map": {"40+31L": 71, "40": 40, …}}, or inline{"name": …, "kind": "numerical", "values": […]}. - GEMMA gets k−1 dummy columns (first level = reference) and drops
a level none of the fitted individuals carries; the reply says so
(
GEMMA fits 4 covariate column(s): Status.Ext, …andCovariate column(s) left out because they do not vary …). R/qtl takes the single level column and dummy-expands it itself, so its note has no such line. - Missing covariate values drop those individuals from the fit (both
engines), which
notesreports asMissing covariate values: …. - At most 8 covariates, and they need a
group(the individuals are aligned by name, which a baregenotype_filerequest does not carry).
- Then read the covariate line in
genotype_version: list them with/groups/{g}/genotypes(BXD.8 is current).- Timings seen: R/qtl about 7 s (1 MB reply), GEMMA about 3 s (1.9 MB), plots
instant. They are synchronous;
?id=only adds progress logging. - Progress:
GET /compute/{rqtl,gemma}/status/{id}returnslinesand apointerto pass back aspeek; the last line isCOMPLETION_MARKERorFAILURE_MARKER: <error>. It exists only for anidyou passed on the POST (otherwise404 run_not_found). /plot/*accepts the whole compute reply or just itsdata; query paramstrait,title, andbonferroni=false(GEMMA only). The title names the trait from the reply's ownTrait: ...note (both compute endpoints write it first), so posting the whole reply back is enough;?trait=wins over the note.- Two traits really do draw two different plots (traces, peaks, annotation); the title is the one thing that was identical before the note existed.
/compute/correlationis a stub with canned data. Do not use it.- You never supply genotypes: the server uses its own files.
PheWAS (GET) /groups/{group}/phewas?locus=<marker> or ?chr=&mb=,
window (bp, default 10000), genotype_version. Returns published traits whose
recorded peak marker lies in the window, strongest first. An empty list is a
real answer.
Data (GET, all need the token)
| path | gives you |
|---|---|
/species, /species/{s}, /species/{s}/datasets |
species, expression datasets |
/groups?species=, /groups/{g}, /groups/{g}/strains, /groups/{g}/datasets |
populations and samples |
/groups/{g}/case-attributes |
the group's metadata: .data.names, and .data.descriptions saying what each one means — read those before fitting a covariate |
/groups/{g}/traits, /groups/{g}/traits/{id}, /groups/{g}/traits/{id}/values |
published traits (the first is the bulk list) |
/groups/{g}/traits/{id}/case-attributes/{attribute} |
a trait's case-attribute values, in the trait's own sample order, plus description and a distinct count — the honest way to see whether an attribute is usable as a covariate (a constant one is not) |
/groups/{g}/genotypes[/{version}[/markers]] |
genotype versions and markers, and the marker names /compute/fitqtl takes |
POST /compute/{rqtl,gemma,fitqtl} |
mapping (POST, not a GET): the scan, the LMM, and effect sizes at named markers — all three take covariates and normalize |
/datasets/{ds}[/probesets[/{p}[/values]]], /traits/{ds}[/{t}[/values]] |
expression data |
/publications/{pubmed_id}/traits |
traits a paper reports (PubMed id, not publication_id) |
/accessions/{acc}, /mapping-methods |
GN accession lookup, methods offered |
Bulk jobs. Fetch /groups/{g}/traits once (the whole PublishXRef list:
id, phenotype_id, dataset, abbreviation, nsamples; about 15 000 for
BXD), then .../traits/{id}/values per trait, a few at a time. Prefer one bulk
call to a thousand small ones; do not use /search for bulk work.
Other examples: /datasets/Br_U_0803_M/probesets/100001_at (one expression
trait), /publications/2760876/traits (PubMed id), and
/search?q=cerebellum AND group:BXD&limit=5.
3a. Gene link-outs: where else to send a user about a gene
GeneNetwork2's trait page links out to a dozen services by gene symbol. These
are the ones that still resolve (checked 2026-11-11, HTTP 200 and — where the
page is not a JavaScript shell — the gene actually found), with the URL template
to hand to a user. {SYMBOL} is the gene symbol; taxa are mouse 10090, rat
10116, human 9606.
| service | URL template | what it is for |
|---|---|---|
| RGD (gene search) | https://rgd.mcw.edu/rgdweb/elasticResults.html?term={SYMBOL}&category=Gene&species={Mouse\|Rat\|Human} |
the best single hub for mouse and rat: orthologs, phenotypes, expression, QTLs. Works for all three species |
| MGI | https://www.informatics.jax.org/searchtool/SearchResults?query={SYMBOL} |
mouse gene authority: alleles, phenotypes, knockouts |
| Alliance of Genome Resources | https://www.alliancegenome.org/search?category=gene&q={SYMBOL} |
cross-species reference: orthologs and disease models |
| NCBI Gene | https://www.ncbi.nlm.nih.gov/gene/?term={SYMBOL}[gene]+AND+mouse[orgn] |
the canonical record; also the source of Gene IDs |
| Wikidata | https://www.wikidata.org/wiki/{QID} |
identifiers and aliases across databases. The QID is what /genes/{SYMBOL}/aliases returns, so this link costs nothing to build |
| UniProt | https://www.uniprot.org/uniprotkb?query=gene:{SYMBOL}+AND+organism_id:{TAXON} (or …/uniprotkb/{ACCESSION}) |
protein entry, domains, variants. Search by symbol, link by accession |
| Ensembl | https://www.ensembl.org/Mus_musculus/Gene/Summary?g={ENSEMBL_ID} |
genome browser, transcripts, orthologs. Needs the Ensembl gene id |
| GTEx | https://www.gtexportal.org/home/gene/{HUMAN_SYMBOL} |
human tissue expression. Human symbol only — a mouse symbol shows an empty report |
| STRING | https://string-db.org/cgi/network?identifier={SYMBOL}&species={TAXON} |
protein–protein interaction network |
| GeneMANIA | https://genemania.org/search/{mouse\|rat\|human}/{SYMBOL} |
functional association network, per species |
| Open Targets | https://platform.opentargets.org/search?q={SYMBOL} |
target–disease associations (human) |
| Human Protein Atlas | https://www.proteinatlas.org/search/{SYMBOL} |
human tissue/cell expression at protein level |
| EBI GWAS Catalog | https://www.ebi.ac.uk/gwas/search?query={SYMBOL} |
published GWAS associations |
| BioGPS | http://biogps.org/?org=10090#goto=genereport&id={SYMBOL} |
mouse expression across tissues and datasets |
| Allen Brain Atlas | https://mouse.brain-map.org/search/show?search_type=gene&search_term={SYMBOL} |
in-situ hybridisation in mouse brain |
| GeneCup | https://genecup.org/progress?type=GWAS&query={SYMBOL} |
literature-mined gene–disease links |
Do not offer these (checked at the same time): HomoloGene (NCBI retired it
in 2021 — the URL answers with a notice), UniGene (retired, HTTP 410),
Genotation and GeneBridge (no DNS/connection), PhenoGen (404), the old chibi
GEMMA gene page, wiki-Pi. PANTHER answers 403 to a bot but works in a browser,
so offer it only as a "works in your browser" link:
https://www.pantherdb.org/genes/geneList.do?searchType=basic&fieldName=all&organism=all&listType=1&fieldValue={SYMBOL}.
How to use them. Quote the symbol and the species you found it in, and let the user click. Do not synthesise what the other service says — say "RGD has a gene report for Shh (rat)" and give the link, not a paraphrase of a page you did not read. For a rat symbol prefer RGD and GeneMANIA(rat); for a mouse symbol all of the above except Human Protein Atlas and GWAS apply; for a human symbol use GTEx, Open Targets and the Protein Atlas.
The API builds them for you. GET /genes/{SYMBOL}/aliases returns a
links array ([{name, href, description}]) built from the table above for the
symbol and species it actually matched -- RGD, MGI, BioGPS and the Allen Brain
Atlas for a mouse gene; RGD, GeneMANIA and UniProt for a rat one. It also
returns label, the entity's canonical symbol in that species, which is how you
get the human symbol a human-only service needs:
GET /genes/Shh/aliases?species=human
→ label "SHH", links […] including GTEx → https://www.gtexportal.org/home/gene/SHH
So a mouse Shh reaches GTEx through the human ortholog's label, and you never
hold a URL template yourself. stale: true means Wikidata was unreachable and
the cached entity was served -- the links still work, the label may be absent
from an older cache entry (retry and it refreshes).
4. Gotchas that cost time
- Compute replies are big (about 21 000 rows; GEMMA about 2 MB). Save to a file,
inspect with
jq/python3 -I. Piping toheadforces a rerun. - Rows are positional; names are in
meta.columns. - Not every value maps: trait 20010's 41 values map as 39, because C57BL/6J and
DBA/2J are not in the BXD.8 genotype file, and
include_parents_f1: truedoes not change that./groups/{g}/strainsflags such samples withused_for_mapping: false, which predicts it.noteson the reply says how many matched; tell the user. - A peak is often a plateau: neighbouring markers share the same LOD / p-value
(trait 20010's GEMMA maximum, 5.79, is shared by several markers from 182.6 to
184.3 Mb). So the trait record's
locus, R/qtl's peak, the plot summary and a reply'snextcan each name a different marker of the same strength. That is not an error: say "peak region" and use the marker the reply'snextgives. - Before you believe a peak, count the animals in its genotype class.
GET /groups/{g}/genotypes/{v}/markers/{marker}/classes?trait=<id>givesn,mean,sd,min,maxper class at that marker (no genotypes, no per-animal values). A peak carried by 1–2 animals is a different claim from one carried by a hundred, and the reply says which classes are thin. On DOL trait 10003 GEMMA's top markerJAX00418132(chr16 36.35 Mb, −log10 p 7.15) isA n=2 mean 1.8,H n=20 mean 2.53,B n=171 mean 3.95: the two A animals are the trait's two lowest values, and GEMMA's next two hits are the same pattern mirrored (1 Aamong 171, then2 Bamong 171). That block is a haplotype carried by a handful of extreme animals, not a population effect — say so rather than reporting 7.15 as a locus. - When R/qtl finds a peak R/qtl2 doesn't, the model is the first suspect,
not the data: R/qtl fits each marker's own calls (dropping the least
frequent of three codes), R/qtl2 fits HMM probabilities with the H calls
re-imputed.
{"genotype_model": "f2"}keeps the het calls as a third genotype and shows what that recovers (BXD trait 16627: LOD 3.73 → 9.68 atrs33299937, against R/qtl's 9.59) — a diagnostic, not a result, since an inbred line has no heterozygotes. The reply saysCross: riselforCross: f2: quote it. His dropped for R/qtl2 on ariselfpanel (@type:riset: BXD, DOL, most RI sets), and kept by GEMMA as dosage 0.5. R/qtl2 writes those cells as missing and its HMM re-imputes them from flanking markers, so the animals are not lost — the observation is. A marker that is mostlyHbecomes nearly monomorphic for R/qtl2 (2 A vs 171 B at the marker above) and cannot produce a peak there, which is why the two engines can name different markers of the same region. Check/frequenciesor/classesbefore calling one engine wrong.- GEMMA's
afis not an allele frequency. It ismean(dosage)/2under the BIMBAM coding the deployment ships (B=1, H=0.5, A=0), soaf 0.469forJAX00418132is half the B-allele frequency 0.935, not the A-allele frequency 0.065. Take allele frequencies from.../markers/{marker}/frequencies. Likewiseallele1/allele0in GEMMA's output are the BIMBAMX, Yplaceholders, so the sign ofbetais not a strain. - Do not compare scores across engines as if they were the same scale.
GEMMA reports −log10 p, R/qtl and R/qtl2 report LOD, qtlreaper reports LRS
(
lod = lrs / 4.604). The same region at "7.15" and "3.92" is not a disagreement, and an LMM's LOD is not a Haley-Knott LOD. - PheWAS window matters: at the trait-20010 peak, 10 kb finds 4 traits and 1 Mb
finds 19. The reply's
nextoffers a ten-times wider window. - Public-data rule:
public > 0andconfidentiality < 1, or older than five years; otherwise 403, do not retry. Metadata is visible even when values are not. - There are no write endpoints, and the search index holds public documents only.
- Older docs said
POST /compute/*is refused with 403. On this build it works. If you do get a 403 there, that is the answer: do not work around it. If the task needs private data or computation you are not granted, stop and ask a human: that needs an account and a grant, not a different URL.
5. Search
/search?q= is Xapian: juxtaposed terms are OR, so write AND; free text
is stemmed and case-insensitive; quoted phrases, wildcards and hyphenated words
match nothing. Fields: exact group, dataset, species, author, tissue,
symbol, chr, peakchr, position; stemmed description, rif, wiki;
ranges mean, peak, peakmb, additive, year (field:lo..hi).
Aliases: pos for position, RIF for rif, WIKI for wiki; Hs/Mm
are species-location shorthands. Values are lowercased, so case does not matter.
A prefix whose value is not in the index matches nothing; that is "no such
value", not an error. NEAR/ADJ do not work either. A dangling operator or
unbalanced quote is 422 invalid_query with Xapian's reason in detail: fix
it, do not retry. type=phenotype|gene, limit up to 100, page. There is no
total: counting means paging, and each page costs time proportional to the
matches left, so do not page through thousands of hits. For bulk work use
/groups/{g}/traits.
.data is an array of hits. Each has name (the trait id, e.g. "21614"),
dataset, dataset_fullname, group, species, authors, year,
description, mean and more. Without group: you also get other groups'
phenotypes (for example C57BL-6JxC57BL-6NJF2Publish), whose ids only mean
something inside their own group; add AND group:BXD to stay in BXD.
6. Tokens and errors
A GeneNetwork team member issues the token by hand; it cannot be self-served.
If you have none, ask: "To read GeneNetwork data I need a GeneNetwork API token
(a JWT) for https://vesuvio.pu.ac.ke/gn4. It is issued by hand; it helps to say
who it is for, which groups/datasets, how long (default 30 days) and what for."
Ask the user to pass it out of band, not in a public issue, a commit or a
shared chat. Send it as Authorization: Bearer <token> on every request except
/version and /openapi.json, and keep it for the whole session. Never put it
in a URL, a commit, a log line or anything shown to someone else, and do not
hand it to other users or paste it into shared state.
| status | do |
|---|---|
401 invalid_token |
detail says ExpiredSignature (expired) or InvalidIssuer (wrong deployment). You cannot renew it: tell the user, ask for a fresh one quoting the request_id, retry once, then stop and report; do not loop or look for another credential |
403 forbidden |
not public: do not retry |
404 not_found |
check the name; next lists valid values |
422 invalid_query, invalid_lmm |
fix the query or parameter; do not retry |
| 429 | back off and honour Retry-After (not emitted today, but in the contract) |
| 502/503 | transient: back off, max about 5 tries |
Errors are RFC 7807-style JSON (type, title, status, code, detail,
operation_id, spec_url, request_id, sometimes next); type, spec_url
and next are absolute paths under /gn4/. Quote request_id when reporting
a problem (also in the x-request-id header). No rate limit is configured, but
be considerate: a few concurrent requests, no tight polling loops.
7. Keep this skill current: reload it if GeneNetwork changes mid-session
Do not rely on a version written into this file. Instead, remember the build you saw at the start of the session and compare against it later.
- At the start (section 0), note
curl -s https://vesuvio.pu.ac.ke/gn4/version | jq -r .data.gitas your session baseline. - Check
/versionagain when something behaves differently from what is written here (an unexpected 404, a new field inactions, a path inopenapi.jsonthis file does not mention), and before a long or repeated piece of work. - If the revision is no longer the baseline, GeneNetwork was updated while you
worked. Suggest to the user that you reload SKILL.md from the repo
(
docs/www/SKILL.mdinhttps://git.genenetwork.org/genenetwork4; a sparse clone ofdocsis enough), and use that copy from then on. One line is enough, for example: "GeneNetwork was updated since we started (-> ); I'd reload SKILL.md from docs/www/ to pick up any changes. Shall I?" Then set the new revision as your baseline. - If the repo is unreachable from the sandbox, fall back to
/gn4/openapi.json, which always matches the running server, and tell the user the skill file itself could not be refreshed.
7a. Project memory: the answers this repo has already worked out
This repository carries an ai-memory store (ai-memory.sqlite) linking each
commit to the reasoning behind it, plus an optional Xapian topic index over
both. A previous session's context window is gone; this is what survives it,
and it is cheaper than re-deriving anything:
ai-memory -C genenetwork4 search 'pve' # SQLite FTS over the reasoning: ~15 ms
ai-memory -C genenetwork4 why gemma.compute_gemma # commits touching a symbol, with its reasoning
ai-memory -C genenetwork4 --index DIR search 'topic:pve repo:genenetwork4' # topic index: ~100 ms
topic: terms come from the session id, so topic:ravanan-slurm,
topic:reference-genomes, topic:gemma-pve are the entry points that exist.
Two things to look for before starting work that touches an engine, a cache or a cluster:
- a RUNBOOK entry — the operational ones carry the actual commands, ports
and fingerprints, e.g. bringing up the Slurm container from
gn-machinesand testing/compute/gemma/ravananagainst it, with the four prerequisites that were needed the first time (a ravanan build with the--slurm-api-endpointfix,ccwlonPATH, the server'sHOMEwritable inside the container,GUIX_PACKAGE_PATHfor the manifest); - a "why is this like this" entry — several endpoints here exist because
something else was wrong (
-pvereporting, thenext-link graph, the reference-genome linkage). Reading it is faster than deciding they are odd.
The store is per repository and local (gitignored): it survives a cleared session, not a lost machine, which is why the durable version of every conclusion is also in a commit message and in these docs.
8. Reference docs
The docs live in the public repo and are the authoritative version:
git clone --depth 1 --sparse https://git.genenetwork.org/genenetwork4
cd genenetwork4 && git sparse-checkout set docs
About 2.5 MB; drop --sparse for the whole tree. Then read:
| file | what it is |
|---|---|
docs/www/SKILL.md |
this page |
docs/search-engines.md |
how /search works, engines, quirks |
README.md |
every endpoint, parameter and response shape |
docs/nginx.md, docs/www/operator-checklist.md |
operator-level; you do not need them |
Clone; do not fetch the web UI. git.genenetwork.org/.../tree/... pages sit
behind a proof-of-work challenge (Anubis) and the raw endpoint answers 403, so
a plain fetch returns an HTML challenge page, not Markdown. If a fetch comes
back as HTML mentioning Anubis, stop, do not retry, and use the copy you have.
The git protocol itself passes. Sandboxes sometimes cannot reach it either;
/gn4/openapi.json is the fallback.
To compare revisions, git -C genenetwork4 log -1 --format=%h against .data.git
from /version: they need not match exactly (docs can be newer than the
running build), but a large gap means the server may lack what the docs describe.
