| Age | Commit message (Collapse) | Author |
|
Carry the file's sample list in the QC job metadata, redirect the QC success
handler to a sample-order review page, and validate the submitted order as a
permutation of the file's samples. On acceptance, launch the next-stage job
(type genotype-stage04) with the accepted order and the preceding stage state in
its metadata; re-submitting reuses an existing Stage 04 job.
Add a minimal Stage 04 placeholder worker for the hand-off point.
Co-authored-by: pi coding agent <pi@pi.dev>
|
|
Require cM/Mb values to be finite, non-negative numbers (integers are accepted
since they convert to floats without loss) in both orientations, alongside the
existing empty/missing-value checks.
Co-authored-by: pi coding agent <pi@pi.dev>
|
|
Flag empty or missing-value headings (untransposed header row, transposed row
labels and marker headings), empty or missing-value marker/Chr/cM/Mb values, and
empty sample values during Stage 01 in both orientations. Empty strings are
excluded from `geno_symbols`. Genuine missing genotype values must use the
configured `na_strings`.
Co-authored-by: pi coding agent <pi@pi.dev>
|
|
Add `uploader.genotypes.normalise`, which writes a canonical untransposed
matrix (`<stem>__work<suffix>`) next to the original, dropping comments and
`@` directives and leaving the original untouched. Ragged or empty matrices
raise `NormalisationError`, and `qc.check_file` now handles only the normal
orientation.
Stage 02 now puts `stage02`, `genotype_collector_job_id`, `file` and `config`
in the QC job's metadata at launch, and uses `jobs_by_metadata` to reuse an
existing QC job. The completed collector job's metadata is no longer modified.
The Stage 03 worker normalises and QCs the working file from its own metadata.
Change the placeholder marker ordering to chromosome, then Mb, then cM.
Co-authored-by: pi coding agent <pi@pi.dev>
|
|
Only comments before the data matrix start feed `FileStats.description`:
before the header row for untransposed files, and before the first data
(markers) row for transposed files. Later comments are still counted but not
used, since they relate to adjacent rows and would be taken out of context by
untransposing.
Track `matrix_started` per chunk and drop later chunks' comments once the
matrix has started, so the multi-processed path matches the single-threaded
one.
Co-authored-by: pi coding agent <pi@pi.dev>
|
|
If errors exist, exit with "GENERALERROR" category rather than with
the "OK" category.
|
|
- Store the genotype file path and collector config in the collector job
metadata so later stages can re-read the file.
- Seed `dot_geno_fields.unk` into the Stage 02 missing-value symbols default.
- Add `uploader.genotypes.qc` with value checks (empty and unknown symbols)
and a placeholder marker-ordering helper (chromosome, cM, Mb) that warns.
- Add the `uploader.scripts.genotypes.qc` worker that reads the collector job,
runs the checks and exits non-zero when errors are found.
- On Stage 02 save, schedule the QC job and record its id for idempotency.
|
|
Move `uploader/scripts/genotypes/types.py` to `uploader/genotypes/types.py` so
application code can use `FileStats` and friends without importing from
`uploader.scripts`. Update the collector script and tests to the new location.
|
|
Add a `description` field to `FileStats`, built by cleaning comment lines
(strip the configured comment character and surrounding whitespace) and joining
them with newlines. Detect comments using the configured comment character
rather than a hard-coded '#', and preserve comment order when merging chunks.
|
|
|
|
The `scripts` package now lives under `uploader`, so update every reference
from `scripts.<...>` to `uploader.scripts.<...>`: Python imports, `-m` command
module strings, logging module names, test imports, CLI/gunicorn entry points,
lint/guix paths and the WIP documentation.
Note: the untracked `run-dev-flask.sh` and `run-dev-gunicorn.sh` helper scripts
still need their entry-point module updated manually.
|
|
Move the top-level `scripts` package to `uploader/scripts` so that it becomes
the `uploader.scripts` package. All files are moved unchanged; references are
updated in a follow-up commit.
|