# What is missing from genenetwork4, ranked by effort A survey of `genenetwork1`, `genenetwork2` and `genenetwork3` (all cloned side by side) for functionality that genenetwork4 does not have, limited to what would fit a JSON API rather than a rendered page. Ranked by the effort it would take to migrate, cheapest first, because that -- not which repo it came from -- is the real cost driver. Date: 2026-11-15. Row counts and table locations were measured read-only against the production `db_webqtl`; the GN1/GN2/GN3 columns were established by reading the code in each clone. ## How to read the columns yes the feature exists in that repo, found by reading its code part only partly there, or a stub, or only the write/UI half no no implementation found in that repo `GN4` is not a column: every row in these tables is a gap in genenetwork4, and the "why" column says what the gap consists of. Effort is S (hours to a day), M (days), L (a week or two), XL (a new concept -- write paths, permissions, a service of its own). ## Tier S -- hours to a day The data is already in the database genenetwork4 reads, and the shape already exists in the API. +--------------------------------------+------+-----+------+--------+-----------------------------------------------------------+ | feature | GN1 | GN2 | GN3 | effort | why | +--------------------------------------+------+-----+------+--------+-----------------------------------------------------------+ | correlation matrix | yes | yes | no | S-M | decisions recorded; N^2 calls to the live correlate_named | | limit_to / ids_only / names_only | no | yes | no | S | optional query params on existing listings | | tissue correlation | yes | yes | yes | S-M | TissueProbeSetData is in the DB; the maths already exists | | literature correlation | yes | yes | yes | S-M | LCorrRamin3 is a lookup table, no statistics involved | | whole-dataset sample matrix export | yes | yes | yes | S-M | one SQL sweep; the only choice is streaming/CSV vs JSON | | genotype file exports (bimbam/rqtl2) | part | yes | no | S | BIMBAM is already written internally for GEMMA | | menu / dropdown tree | yes | yes | yes | S | SQL only; the gap is the nested shape, not the data | | probeset annotation enrichment | yes | yes | yes | S-M | GeneList and db_GeneOntology are present | | mapping-result store | yes | no | part | S-M | runs live in /tmp caches today; durable results are new | +--------------------------------------+------+-----+------+--------+-----------------------------------------------------------+ ## Tier M -- days New data inside MariaDB, or new maths, but no new service or storage. +-----------------------------------+------+-----+------+--------+-------------------------------------------------------------+ | feature | GN1 | GN2 | GN3 | effort | why | +-----------------------------------+------+-----+------+--------+-------------------------------------------------------------+ | partial correlations | part | yes | yes | M | needs a genotype/covariate matrix plus a job shape | | clustered heatmap | yes | yes | yes | M | clustering is cheap; plotting infrastructure already exists | | SNP / variant browser | yes | yes | no | M-L | SnpAll is 79.5 million rows; window queries first, UI later | | region / interval queries | yes | no | part | M | locations exist everywhere, a range endpoint does not | | GeneRIF / GeneWiki read | yes | yes | yes | M | small tables, but a gene-centric resource the model lacks | | compare traits / corr. annotation | yes | yes | no | M | two-trait scatter plus stats, assembled from what exists | | QTLminer / qtlreaper re-analysis | yes | yes | yes | M-L | external R/Python, job shape like /compute/rqtl | | metadata about / history | no | yes | yes | M-L | a second data source: Virtuoso + the gn-docs git tree | +-----------------------------------+------+-----+------+--------+-------------------------------------------------------------+ ## Tier L and XL -- weeks, or a new concept Write paths, permissions, visualisation services, or a different storage model. None of these is a data-read problem. +------------------------------------+-----+-----+------+--------+------------------------------------------------------------+ | feature | GN1 | GN2 | GN3 | effort | why | +------------------------------------+-----+-----+------+--------+------------------------------------------------------------+ | case-attribute write / moderation | no | no | yes | L | genenetwork4 is read-only; needs scopes and an audit trail | | WGCNA | no | yes | yes | L | R package, heavy compute, result storage | | network graph | yes | yes | no | L | visualisation; overlaps the BNW structure work | | collections / data sharing | yes | yes | no | L-XL | baskets exist, a sharing and permission model does not | | submit / update trait (data entry) | yes | yes | part | XL | write + moderation + validation + auth, all new | | genome graph / JBrowse | yes | no | no | XL | a browser service, not an API resource | | LLM / GN-QA | no | no | yes | XL | external LLM dependency, unrelated to the data model | | LMDB sample-data backend | no | no | yes | XL | a storage migration question, not an endpoint | +------------------------------------+-----+-----+------+--------+------------------------------------------------------------+ ## What genenetwork4 already covers So the tables above are not misread as a general backlog: species; groups with strains, datasets, traits, genotypes, case attributes, parents-and-F1s and phewas; datasets with probesets; traits with values, per-trait case attributes and peak genotypes; genotype files with markers, frequencies, classes and locations; uploads; probesets; genes with traits and aliases; publications leading to traits; search; mapping methods; tokens; baskets; matrices; plots; and the compute family -- GEMMA (direct and via ravanan), R/qtl, R/qtl2, fitqtl, BulkLMM, BNW, and the sample-correlation row. ## Evidence for the cost estimates ProbeSetData 5 314 694 464 rows trait x individual x value PublishData 3 288 638 rows phenotype values LCorrRamin3 127 368 780 rows the only stored correlations SnpAll 79 534 772 rows db_webqtl IndelAll 142 894 rows db_webqtl GeneList 133 471 rows db_webqtl (87 688 in db_webqtl_s) GeneRIF / GeneRIFXRef 8 425 / 9 387 rows db_webqtl TissueProbeSetData 90 562 DataIds, 26-32 values each (avg 29.0) The pattern that matters: every *value* feature is a query-design problem because of the first four rows, while every *metadata* feature is small -- which is why the S tier is almost entirely metadata. ## Recommended order 1. correlation matrix decisions are already recorded, and the row endpoint it builds on is live 2. tissue + literature corr. completes the three kinds GN2 offers on the trait page 3. pagination / dropdown tree unblocks any client that has to list 4. sample matrix + genotypes the "get my data out" gap 5. partial correlations the last GN2 correlation still missing ## Repo locations, for whoever picks a row up GN1 web/webqtl/{correlation,correlationMatrix,intervalMapping, markerRegression,pairScan,snpBrowser,heatmap,geneWiki, collection,dataSharing,qtlminer,submitTrait,updateTrait} GN2 gn2/wqflask/{api,correlation,correlation_matrix,heatmap,wgcna, network_graph}; templates/show_trait_calculate_correlations.html GN3 gn3/api/{correlation,heatmaps,wgcna,search,metadata,menu, sampledata,case_attributes,data_entry,rqtl,rqtl2,gemma}.py