diff options
| author | Pjotr Prins <pjotr.public01@thebird.nl> | 2026-10-03 13:32:53 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-10-03 13:32:53 +0000 |
| commit | 5bcab4cbcf7bea03223161dd29d81d93f71975ce (patch) | |
| tree | f2895db3b83b77391a2a7d4c5428c5b8d5dcd4bb /docs | |
| parent | 04347014fd571ff99b4343c1ba26822c2de990bd (diff) | |
| download | genenetwork4-5bcab4cbcf7bea03223161dd29d81d93f71975ce.tar.gz | |
doc: Add docs/genenetwork_datatypes.md, Species section.
* docs/genenetwork_datatypes.md: New file. Species table field semantics (sourced from factory/docs/gn-gemtext-threads/topics/ gn-uploader/adding-species.gmi and this repo's own docs/mariadb_db-webqtl.md schema dump), its relationships to InbredSet, Strain, Chr_Length and GeneIDXRef, its role in GN's authorization and RDF classification scheme, known quirks (mouse-centric legacy genotype columns, undocumented FamilyOrderId/ OrderId, no UI for adding a species), and a survey of existing Species-related implementations across GeneNetwork2, GeneNetwork3, gn-guile and this repo.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/genenetwork_datatypes.md | 99 |
1 files changed, 99 insertions, 0 deletions
diff --git a/docs/genenetwork_datatypes.md b/docs/genenetwork_datatypes.md new file mode 100644 index 0000000..f666f50 --- /dev/null +++ b/docs/genenetwork_datatypes.md @@ -0,0 +1,99 @@ +# GeneNetwork data types + +Notes on the core data types GeneNetwork is built around, compiled from +`factory/docs/gn-gemtext-threads`, `factory/code/genenetwork2`, +`factory/code/genenetwork3`, `factory/code/gn-guile`, and this repo's own +`docs/mariadb_db-webqtl.md` schema dump. + +## Species + +The `Species` table (`db_webqtl`; full `CREATE TABLE` in +`docs/mariadb_db-webqtl.md`) is the root of GeneNetwork's data hierarchy: +every `InbredSet` (a group/population, e.g. BXD), every `Strain`, and +every chromosome length record (`Chr_Length`) is scoped to a species via +a `SpeciesId` foreign key. + +### Fields + +Field semantics, from `topics/gn-uploader/adding-species.gmi` (written +while manually adding C. elegans to the database) and cross-checked +against the live schema: + +| Field | Meaning | +|-----------------|---------| +| `Id` | Internal GeneNetwork species identifier (auto-increment primary key). | +| `SpeciesId` | Set equal to `Id` after insert (see "Known quirks" below); other tables' `SpeciesId` foreign keys reference this, not `Id` directly, though in practice the two are always equal. | +| `SpeciesName` | A layman's name, e.g. "Mouse", "Roundworm", "Barley". | +| `Name` | Mostly the same as `SpeciesName`, lowercased, e.g. "mouse". This is the short name used in URLs and API lookups. | +| `MenuName` | Display name shown in the UI, typically combining `SpeciesName` and `FullName`/assembly, e.g. "Mouse (Mus musculus, mm10)". | +| `FullName` | The species' binomial (scientific) name, e.g. "Mus musculus", "Caenorhabditis elegans". | +| `Family` | A loose grouping across species, e.g. "Vertebrates", "Nematodes". | +| `FamilyOrderId` | Sort order within `Family` for menu display. Exact derivation undocumented (`adding-species.gmi` flags this with a TODO). | +| `TaxonomyId` | The NCBI Taxonomy ID (https://www.ncbi.nlm.nih.gov/), e.g. 10090 for mouse, 6239 for C. elegans. | +| `OrderId` | Sort order for display. Exact derivation undocumented, same TODO as `FamilyOrderId`. | + +### Relationships + +- `InbredSet.SpeciesId` -> `Species` -- every group/population (e.g. + BXD) belongs to exactly one species. +- `Strain.SpeciesId` -> `Species` -- every strain belongs to exactly one + species (`genenetwork2/gn2/base/species.py`'s `TheSpecies`/ + `Chromosomes` classes join on this). +- `Chr_Length.SpeciesId` -> `Species` -- chromosome names and lengths + are per species (used by `gn3.db.species.get_chromosome` and GN2's + `Chromosomes.chromosomes`, both selecting from `Chr_Length` joined to + either `Species` directly by name, or indirectly via `InbredSet` for + cases where only a group name is known). +- `GeneIDXRef` cross-references gene IDs between species. GeneNetwork's + internal gene identifiers are mouse-centric by convention: GN3's + `gn3.db.species.translate_to_mouse_gene_id` translates a rat or human + gene ID to its mouse equivalent via this table (returns the ID + unchanged if the species is already "mouse"). +- `InbredSet.SpeciesId` also scopes authorization: GN3's + `gn3/api/case_attributes.py` resolves a species ID from `InbredSet` to + build the resource ID used to check a user's access with `gn-auth` + (https://github.com/genenetwork/gn-auth), so species is part of GN's + permission hierarchy, not just a data-organization concept. +- In the RDF/SPARQL metadata graph (`topics/next-gen-databases/gn-classification-scheme.gmi`), + species is one of the three levels of GN's resource classification + scheme (`gnc:DatasetType`, `gnc:Set`, `gn:Species`), alongside + `InbredSet` ("Set") and `DatasetType`. Every member of the `Species` + table has a corresponding `gn:Species`-classified RDF resource; + `gn-guile`'s `gn/data/species.scm` builds this metadata from SPARQL, + separately from the MariaDB `Species` table. + +### Known quirks + +- GeneNetwork's genotype tables were originally built mouse-only and + still carry that legacy: `topics/gn-uploader/genotypes-assemblies-markers-and-genenetwork.gmi` + notes the `Geno` table has assembly-specific, mouse-named columns + (`Mb_mm8`, `Chr_mm8`) that don't generalize to other species' genome + assemblies. +- There is no UI for adding a new species; it's done by hand with + `INSERT INTO Species(...)` followed by `UPDATE Species SET + SpeciesId=Id WHERE ...` (see `adding-species.gmi` for the full + worked example, adding C. elegans). +- `FamilyOrderId` and `OrderId` remain undocumented upstream (flagged + TODO in `adding-species.gmi`); their exact intended derivation isn't + recorded anywhere found in these repos. + +### Existing implementations + +- **GeneNetwork2** (`gn2/wqflask/api/router.py`): `GET + /api/v_pre1/species` (all species, columns `Id`/`Name`/`FullName`/ + `TaxonomyId`) and `GET /api/v_pre1/species/<species_name>` (one + species, matched against `Name`, `FullName` or `SpeciesName`, same + 4-column subset). `gn2/base/species.py`'s `TheSpecies` class also + exposes a species' chromosomes via `Chr_Length`. +- **GeneNetwork3**: `gn3/db/species.py` has `get_all_species`, + `get_chromosome`, `species_name` (species for an `InbredSet` group) + and `translate_to_mouse_gene_id`; none of these are exposed as a + `Species`-table REST endpoint. `gn3/api/metadata.py`'s `GET /species` + and `GET /species/<name>` are SPARQL-based RDF metadata lookups + instead, a different data source from the MariaDB `Species` table. +- **gn-guile** (`gn/data/species.scm`): SPARQL/RDF metadata only, same + category as GN3's `/species` above, not backed by `db_webqtl`. +- **GeneNetwork4** (this repo): `GET /species` (all species, mirrors + GN3's `get_all_species` query/ordering) and `GET /species/{species}` + (one species, every column, same match logic as GN2's + `get_species_info` -- see `README.md`). |
