diff options
| author | pi coding agent <pi@localhost> | 2026-11-05 19:05:35 +0000 |
|---|---|---|
| committer | Frederick Muriuki Muriithi <fredmanglis@gmail.com> | 2026-11-06 07:06:52 -0600 |
| commit | 3a7ee65224db66aac63ea7f76a538e5948d64625 (patch) | |
| tree | 52a94185ead8d1851c6ced95efa62ad45584cf07 /docs | |
| parent | 9128371f1e5252aa46a83aa75d2d6df077e57573 (diff) | |
| download | gn-uploader-3a7ee65224db66aac63ea7f76a538e5948d64625.tar.gz | |
docs: update tasks and todos
- mark Stage 01 hand-off and missing-value symbols done - update Stage 02/03 and job-chain tasks - add untranspose tasks/considerations
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/WIP/geno-uploads-tasks-and-todos.md | 35 |
1 files changed, 26 insertions, 9 deletions
diff --git a/docs/WIP/geno-uploads-tasks-and-todos.md b/docs/WIP/geno-uploads-tasks-and-todos.md index f23125a..ca5a256 100644 --- a/docs/WIP/geno-uploads-tasks-and-todos.md +++ b/docs/WIP/geno-uploads-tasks-and-todos.md @@ -35,7 +35,8 @@ implementation notes. - ✅ Run the collector as a `gn_libs.jobs` background job (job type `genotype-file-stats`); the job status/summary pages show its output and errors. - - ⬜ Persist/return the resulting `FileStats` for the next stage. + - ✅ Persist/return the resulting `FileStats` for the next stage (Stage 02 + reads it from the collector job). ## Stage 02 — Study design & genotype encoding @@ -48,19 +49,25 @@ implementation notes. defaults). - ✅ `@`-derived labels are a starting point only; the user can change them. Presence is not validation. -- ⬜ Allow the user to specify missing-value symbol(s). +- ✅ Allow the user to specify missing-value symbol(s). +- ✅ Seed `dot_geno_fields.unk` into the missing-value symbols. +- ⬜ Revisit the Stage 02 UI's three-character symbol limit: symbols may be + longer and need not be single characters (low priority). - ✅ Capture the confirmed encoding/study design for downstream stages (stored in the collector job metadata, key `stage02`). ## Stage 03 — Final QC -- ⬜ Perform value-level QC parameterised by Stage 01/02 results. -- ⬜ Restrict checks to the symbols present in the file. -- ⬜ Verify/handle the ordering of samples/individuals/cases (their order - matters in the generated `.geno` file). -- ⬜ Verify markers/records are ordered by chromosome (ascending) and then by - position (ascending) within each chromosome; decide how to handle overlapping - positions (detection/handling TBD). +- ✅ Perform value-level QC parameterised by Stage 01/02 results: every sample + value must be an encoding or missing-value symbol; empty values are errors + and matching is case-sensitive. +- ✅ Restrict checks to the symbols present in the file (checks use the + confirmed encoding/missing symbols; absent symbols are not required). +- ⬜ Apply the sample/individual/case ordering rules and present the order for + user review and reordering (see design notes). +- ⬜ Sort/verify records by chromosome (ascending), then cM (ascending), then Mb + (ascending) — placeholder ordering with a warning/TODO until the appropriate + ordering is known. ## Stages 04–07 — Conversion & ingest @@ -71,6 +78,14 @@ implementation notes. Stage 03). - ⬜ Stage 07: pre-compute mapping(s). +## Untransposing transposed files + +- 💭 Decide where to untranspose transposed genotype files (before Stage 01, + within Stage 01, or before Stage 04). See design notes. +- 💭 Adapt the existing transpose helper (or write a genotype-aware one): keep + `#` comments and `@` directives, handle ragged rows, and avoid overwriting + the original upload; record the canonical file path in job metadata. + ## Refactoring / technical debt - 💭 **Deferred:** once `add_genotype_records` grows to cover the later stages, @@ -85,6 +100,8 @@ implementation notes. the phenotypes `load_data_to_database`/`load_phenotypes_success_handler` pattern): proceed on success, and stop/report diagnostics on failure (see design notes). +- ✅ Schedule the Stage 03 QC job from Stage 02 and record `stage03_job_id` for + idempotency. - ⬜ Make stage transitions idempotent: revisiting a completed job's status URL must not re-run a non-idempotent stage; present the previously entered information instead (see design notes). |
