about summary refs log tree commit diff
path: root/scripts
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-11-14 03:45:38 +0000
committerPjotr <wrk@bosa.lan>2026-11-14 03:45:38 +0000
commitf8b34ca82b40f9eda65fa3abab580a0ca8e66f49 (patch)
tree3c4acb2d5cf4d8694c0851441a0ba51f6707b180 /scripts
parent5c6f36f6d0155d73e3fb8d24aee1a6827a77ae97 (diff)
downloadgenenetwork4-f8b34ca82b40f9eda65fa3abab580a0ca8e66f49.tar.gz
groups: the design in /groups/{group}, and a drift check against GN2's ParInfo
Two follow-ups to the parental-design endpoint.

/groups/{group} now carries the headline in its notes -- "Parental design:
C57BL/6J (maternal) x DBA/2J (paternal), F1s B6D2F1 / D2B6F1; a mapping excludes
the parents and F1s by default" -- plus a next link to the full design. "Which
cross is this?" is two strain names, and needing a second request to see them
was the only reason the endpoint had to be called first. A group with no
metadata says so in the same note, pointing at parents_and_f1s.json and at
`individuals.exclude`, rather than leaving the reader to wonder why nothing was
dropped.

The metadata is the same fact in two places (GN2's ParInfo, our JSON) and it
drifted once already, silently: the paternal/maternal labels were swapped
against GN2's unpacking, and nothing noticed because the exclusion rule takes
both strains regardless of role. scripts/check-parental-metadata.py parses
ParInfo with ast (no gn2 import) and compares it group by group, reporting
disagreements per role and treating groups the JSON knows but GN2 does not as
informational. tests/parental_metadata.rs runs it while a GN2 checkout is
present (GN2_CHECKOUT, or a sibling path) and skips once GN2 is retired -- at
which point the JSON is the only copy and there is nothing to drift from. It
reports 23 groups reproduced, 1 extra (BXD-Longevity).
Diffstat (limited to 'scripts')
-rwxr-xr-xscripts/check-parental-metadata.py106
1 files changed, 106 insertions, 0 deletions
diff --git a/scripts/check-parental-metadata.py b/scripts/check-parental-metadata.py
new file mode 100755
index 0000000..16dd72c
--- /dev/null
+++ b/scripts/check-parental-metadata.py
@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+"""Compare the deployment's parents-and-f1s.json against GN2's ParInfo.
+
+`parents_and_f1s.json` is GN2's `ParInfo` as data (see
+`gn2/utility/webqtlUtil.py`). Two copies of one fact drift, and this one already
+did once: the JSON's `paternal` and `maternal` labels were swapped against the
+way GN2 unpacks the tuple (`f1, f12, maternal, paternal`). Nothing mapped wrongly
+-- the exclusion rule takes both strains regardless of role -- but the metadata
+said the opposite of the truth, and anything that later reads "maternal" would
+have been backwards.
+
+So while GN2 exists, check the two agree:
+
+    python3 scripts/check-parental-metadata.py \\
+        --parinfo  ../genenetwork2/gn2/utility/webqtlUtil.py \\
+        --json     /path/to/genotype-files/parents_and_f1s.json
+
+Exit 0 when every group GN2 knows is reproduced, 1 otherwise (with the
+differences on stdout), 2 when a file cannot be read.
+
+Extra groups in the JSON are reported but are not a failure: the deployment may
+know a family GN2 never did (BXD-Longevity), and that is metadata of its own.
+"""
+
+import argparse
+import ast
+import json
+import sys
+
+
+def read_parinfo(path):
+    """`ParInfo` as a dict, parsed rather than imported (no gn2 runtime)."""
+    source = open(path, encoding="utf-8").read()
+    for node in ast.parse(source).body:
+        if isinstance(node, ast.Assign):
+            targets = [t.id for t in node.targets if isinstance(t, ast.Name)]
+            if "ParInfo" in targets:
+                return ast.literal_eval(node.value)
+    raise SystemExit(f"{path}: no ParInfo assignment found")
+
+
+def roles_from_json(entry):
+    """(f1s, maternal, paternal) as the JSON spells them, empty names dropped."""
+    f1s = [f["strain"] for f in entry.get("f1s") or [] if f.get("strain")]
+    def strain(key):
+        role = entry.get(key) or {}
+        return role.get("strain") or None
+    return f1s, strain("maternal"), strain("paternal")
+
+
+def roles_from_parinfo(value):
+    f1, f12, maternal, paternal = value
+    return [name for name in (f1, f12) if name], (maternal or None), (paternal or None)
+
+
+def main(argv):
+    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    parser.add_argument("--parinfo", required=True,
+                        help="path to gn2/utility/webqtlUtil.py")
+    parser.add_argument("--json", required=True,
+                        help="path to parents_and_f1s.json")
+    args = parser.parse_args(argv)
+
+    try:
+        parinfo = read_parinfo(args.parinfo)
+        document = json.load(open(args.json, encoding="utf-8"))
+    except OSError as error:
+        print(f"cannot read: {error}", file=sys.stderr)
+        return 2
+
+    problems = 0
+    for group, value in sorted(parinfo.items()):
+        if group not in document:
+            print(f"{group}: missing from the JSON (GN2 has it)")
+            problems += 1
+            continue
+        expected = roles_from_parinfo(value)
+        found = roles_from_json(document[group])
+        if expected == found:
+            continue
+        problems += 1
+        print(f"{group}: differs")
+        for label, want, got in (
+            ("F1s", expected[0], found[0]),
+            ("maternal", expected[1], found[1]),
+            ("paternal", expected[2], found[2]),
+        ):
+            if want != got:
+                print(f"    {label}: GN2 {want!r}, JSON {got!r}")
+
+    extra = sorted(set(document) - set(parinfo))
+    if extra:
+        print(f"groups in the JSON that GN2 does not list (fine, not a failure): "
+              f"{', '.join(extra)}")
+
+    if problems:
+        print(f"\n{problems} group(s) disagree; parents_and_f1s.json should be "
+              f"regenerated from ParInfo")
+        return 1
+    print(f"OK: {len(parinfo)} GN2 groups reproduced"
+          + (f", {len(extra)} extra in the JSON" if extra else ""))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))