about summary refs log tree commit diff
path: root/docs
diff options
context:
space:
mode:
authorpi coding agent <pi@localhost>2026-11-05 19:05:35 +0000
committerFrederick Muriuki Muriithi <fredmanglis@gmail.com>2026-11-06 07:06:52 -0600
commit3a7ee65224db66aac63ea7f76a538e5948d64625 (patch)
tree52a94185ead8d1851c6ced95efa62ad45584cf07 /docs
parent9128371f1e5252aa46a83aa75d2d6df077e57573 (diff)
downloadgn-uploader-3a7ee65224db66aac63ea7f76a538e5948d64625.tar.gz
docs: update tasks and todos
- mark Stage 01 hand-off and missing-value symbols done
- update Stage 02/03 and job-chain tasks
- add untranspose tasks/considerations
Diffstat (limited to 'docs')
-rw-r--r--docs/WIP/geno-uploads-tasks-and-todos.md35
1 files changed, 26 insertions, 9 deletions
diff --git a/docs/WIP/geno-uploads-tasks-and-todos.md b/docs/WIP/geno-uploads-tasks-and-todos.md
index f23125a..ca5a256 100644
--- a/docs/WIP/geno-uploads-tasks-and-todos.md
+++ b/docs/WIP/geno-uploads-tasks-and-todos.md
@@ -35,7 +35,8 @@ implementation notes.
   - ✅ Run the collector as a `gn_libs.jobs` background job (job type
     `genotype-file-stats`); the job status/summary pages show its output and
     errors.
-  - ⬜ Persist/return the resulting `FileStats` for the next stage.
+  - ✅ Persist/return the resulting `FileStats` for the next stage (Stage 02
+    reads it from the collector job).
 
 ## Stage 02 — Study design & genotype encoding
 
@@ -48,19 +49,25 @@ implementation notes.
     defaults).
   - ✅ `@`-derived labels are a starting point only; the user can change them.
     Presence is not validation.
-- ⬜ Allow the user to specify missing-value symbol(s).
+- ✅ Allow the user to specify missing-value symbol(s).
+- ✅ Seed `dot_geno_fields.unk` into the missing-value symbols.
+- ⬜ Revisit the Stage 02 UI's three-character symbol limit: symbols may be
+  longer and need not be single characters (low priority).
 - ✅ Capture the confirmed encoding/study design for downstream stages (stored
   in the collector job metadata, key `stage02`).
 
 ## Stage 03 — Final QC
 
-- ⬜ Perform value-level QC parameterised by Stage 01/02 results.
-- ⬜ Restrict checks to the symbols present in the file.
-- ⬜ Verify/handle the ordering of samples/individuals/cases (their order
-  matters in the generated `.geno` file).
-- ⬜ Verify markers/records are ordered by chromosome (ascending) and then by
-  position (ascending) within each chromosome; decide how to handle overlapping
-  positions (detection/handling TBD).
+- ✅ Perform value-level QC parameterised by Stage 01/02 results: every sample
+  value must be an encoding or missing-value symbol; empty values are errors
+  and matching is case-sensitive.
+- ✅ Restrict checks to the symbols present in the file (checks use the
+  confirmed encoding/missing symbols; absent symbols are not required).
+- ⬜ Apply the sample/individual/case ordering rules and present the order for
+  user review and reordering (see design notes).
+- ⬜ Sort/verify records by chromosome (ascending), then cM (ascending), then Mb
+  (ascending) — placeholder ordering with a warning/TODO until the appropriate
+  ordering is known.
 
 ## Stages 04–07 — Conversion & ingest
 
@@ -71,6 +78,14 @@ implementation notes.
   Stage 03).
 - ⬜ Stage 07: pre-compute mapping(s).
 
+## Untransposing transposed files
+
+- 💭 Decide where to untranspose transposed genotype files (before Stage 01,
+  within Stage 01, or before Stage 04). See design notes.
+- 💭 Adapt the existing transpose helper (or write a genotype-aware one): keep
+  `#` comments and `@` directives, handle ragged rows, and avoid overwriting
+  the original upload; record the canonical file path in job metadata.
+
 ## Refactoring / technical debt
 
 - 💭 **Deferred:** once `add_genotype_records` grows to cover the later stages,
@@ -85,6 +100,8 @@ implementation notes.
   the phenotypes `load_data_to_database`/`load_phenotypes_success_handler`
   pattern): proceed on success, and stop/report diagnostics on failure (see
   design notes).
+- ✅ Schedule the Stage 03 QC job from Stage 02 and record `stage03_job_id` for
+  idempotency.
 - ⬜ Make stage transitions idempotent: revisiting a completed job's status URL
   must not re-run a non-idempotent stage; present the previously entered
   information instead (see design notes).