about summary refs log tree commit diff
path: root/uploader/genotypes
AgeCommit message (Collapse)Author
5 daysAdd the Stage 03 sample-order review and hand-offFrederick Muriuki Muriithi
Carry the file's sample list in the QC job metadata, redirect the QC success handler to a sample-order review page, and validate the submitted order as a permutation of the file's samples. On acceptance, launch the next-stage job (type genotype-stage04) with the accepted order and the preceding stage state in its metadata; re-submitting reuses an existing Stage 04 job. Add a minimal Stage 04 placeholder worker for the hand-off point. Co-authored-by: pi coding agent <pi@pi.dev>
5 daysValidate submitted sample ordersFrederick Muriuki Muriithi
Add `is_valid_order`, which checks that a submitted order is a permutation of the file's samples (catching missing, extra and duplicated samples), with tests. Co-authored-by: pi coding agent <pi@pi.dev>
5 daysAdd genotype sample order proposal helpersFrederick Muriuki Muriithi
Add `proposed_order`, which keeps an existing population order and appends missing samples alphabetically (or sorts alphabetically without an existing order), and `population_sample_order`, which reads a population's order from `StrainXRef.OrderId`. Co-authored-by: pi coding agent <pi@pi.dev>
5 daysReject duplicate marker names within a chromosomeFrederick Muriuki Muriithi
Track marker names per chromosome during Stage 03 QC and report an error for duplicates within the same chromosome. Marker names may still be reused on different chromosomes. Co-authored-by: pi coding agent <pi@pi.dev>
5 daysRegister genotype job error handlersFrederick Muriuki Muriithi
Set `error_handler` on the genotype statistics collection and QC jobs so failures render the genotype upload error page. Structured file errors are listed; other failures fall back to stdout/stderr and recovery guidance. Co-authored-by: pi coding agent <pi@pi.dev>
5 daysAdd helpers and handlers for genotype job errorsFrederick Muriuki Muriithi
Add `job_errors.errors_from_job`, which normalises the structured errors a genotype job records in its stdout (collector `FileStats.errors` and QC result errors), and collector/QC error handler functions that render them. The HTML template and job-metadata registration follow in separate commits. Co-authored-by: pi coding agent <pi@pi.dev>
6 daysUpdate genotype/phenotype/sample job call sites for keyword metadataFrederick Muriuki Muriithi
`initialise_job` now takes metadata as keyword arguments, so replace the `extra_meta={...}` arguments with `**{...}` at the gn-libs job call sites. Co-authored-by: pi coding agent <pi@pi.dev>
6 daysNormalise genotype files and move Stage 02 state into the QC jobFrederick Muriuki Muriithi
Add `uploader.genotypes.normalise`, which writes a canonical untransposed matrix (`<stem>__work<suffix>`) next to the original, dropping comments and `@` directives and leaving the original untouched. Ragged or empty matrices raise `NormalisationError`, and `qc.check_file` now handles only the normal orientation. Stage 02 now puts `stage02`, `genotype_collector_job_id`, `file` and `config` in the QC job's metadata at launch, and uses `jobs_by_metadata` to reuse an existing QC job. The completed collector job's metadata is no longer modified. The Stage 03 worker normalises and QCs the working file from its own metadata. Change the placeholder marker ordering to chromosome, then Mb, then cM. Co-authored-by: pi coding agent <pi@pi.dev>
12 daysAdd Stage 03 genotype value QC and schedule it from Stage 02pi coding agent
- Store the genotype file path and collector config in the collector job metadata so later stages can re-read the file. - Seed `dot_geno_fields.unk` into the Stage 02 missing-value symbols default. - Add `uploader.genotypes.qc` with value checks (empty and unknown symbols) and a placeholder marker-ordering helper (chromosome, cM, Mb) that warns. - Add the `uploader.scripts.genotypes.qc` worker that reads the collector job, runs the checks and exits non-zero when errors are found. - On Stage 02 save, schedule the QC job and record its id for idempotency.
12 daysUnify missing-value symbol parsing in r_qtl.utilspi coding agent
Move `parse_missing_values_symbols` from the genotypes views into `r_qtl.utils` and use it in the genotypes Stage 01 collector config and the phenotypes individual-file uploads, replacing their divergent inline splits with the more robust whitespace/comma parsing. Move the tests to `tests/r_qtl/test_utils.py`.
12 daysAdd missing-value symbols to Stage 02pi coding agent
Add a missing-value symbol(s) input to the Stage 02 encoding form and a `parse_missing_values_symbols` helper in the genotypes views. Parse the input (whitespace/comma-separated, unique symbols) and store it in the collector job's `stage02` metadata alongside the description and encoding; prefill the input when the page is revisited.
13 daysAdd the Stage 02 study design/encoding review pagepi coding agent
Redirect the collector job's success handler to the new Stage 02 route, carrying the collector job id. The `study_design` view loads the job, validates it and parses the FileStats via `collector_job.file_stats`, then renders the collected description, the genotype symbols, and the study-design/encoding inputs. POST stores the confirmed description and encoding in the collector job's metadata (`stage02`) and redirects back to the review page (post/redirect/get), which makes revisiting and re-submitting idempotent. Bind the study-design and encoding fragments to the saved values.
13 daysAdd a helper to read FileStats from a collector jobpi coding agent
Validate that a job is a completed genotype-stats job for the requested species/population/dataset and parse its FileStats from the job's recorded stdout.
13 daysMove genotype data types into the uploader.genotypes packagepi coding agent
Move `uploader/scripts/genotypes/types.py` to `uploader/genotypes/types.py` so application code can use `FileStats` and friends without importing from `uploader.scripts`. Update the collector script and tests to the new location.
13 daysSchedule the genotype statistics collector as a background jobpi coding agent
Add `collector_config.command` to build the collector CLI command from the collector config. Replace the `_collect_geno_csv_stats_` stub in `add_genotype_records` with a `gn_libs.jobs` background job that runs the collector. The standard background-job status/summary pages surface the collector's stdout and stderr, so failures show the available diagnostics. JSON requests receive a `redirect-to` for the job status page; HTML requests are redirected there too. Set the collector job's `success_handler` and add `collect_genotype_stats_success_handler`, which currently redirects to the job summary page (dumping the job details). It will be replaced by the Stage 02 study-design/encoding flow.
13 daysBuild the collector config from the upload formpi coding agent
Add `uploader.genotypes.collector_config.from_form` to map the upload form's metadata fields to the statistics collector's config keys (`markers`, `chromosome`, `cm`, `mb`, `transposed`, `separator`, `comment.char`, `na_strings`), falling back to the collector's defaults and normalising the missing-value strings. Use it in `add_genotype_records` instead of the previous `sep`/`na.strings` stub metadata.
2026-10-22Use the shared resolver for genotype uploadspi coding agent
Wire `add_genotype_records` to `uploader.files.resolve_uploaded_file` for both the resumable and direct upload paths, replacing the incorrect `form["uploaded-file"]` lookup. On failure, return a JSON error for JSON requests, or flash and redirect back to the add-records page.
2026-10-01Fix type errors.Frederick Muriuki Muriithi
2026-10-01Improve function documentation.Frederick Muriuki Muriithi
Use python's official documentation format, specified by PEP 287 for documenting the function's parameters, return and exceptions.
2026-10-01Fetch genotype dataset by species and population only.Frederick Muriuki Muriithi
2026-10-01Have consistent interface for "dataset_by_id" functions.Frederick Muriuki Muriithi
2026-10-01Fix pylint issues to get a clean working state.Frederick Muriuki Muriithi
2026-09-28BugFix: Myriad of minor bugfixes.Frederick Muriuki Muriithi
2026-09-17Add outline of conversion of file to R/qtl2 format.Frederick Muriuki Muriithi
2026-09-17Use 'Accept' header in ajax requests for content negotiation.Frederick Muriuki Muriithi
The endpoint can return either HTML or JSON depending on the 'Accept' header.
2026-09-14Init "Add Genotypes Records" endpoint.Frederick Muriuki Muriithi
2026-09-14Check for zero records before attempting operations.Frederick Muriuki Muriithi
2026-09-14Use Python's logging rather than the app logger.Frederick Muriuki Muriithi
2026-09-10Include centiMorgans.Frederick Muriuki Muriithi
2026-09-10Enable use of DataTables for UI.Frederick Muriuki Muriithi
2026-09-10Bugfix: Update links and argument after previous commits.Frederick Muriuki Muriithi
2026-09-10PoC: UI for genotype records.Frederick Muriuki Muriithi
Implement a proof-of-concept (PoC) UI for genotype records.
2026-09-10Fetch genotype records.Frederick Muriuki Muriithi
2026-09-04Fetch genetic markers by both species and population.Frederick Muriuki Muriithi
A species can have more markers than are considered in a particular population. This update changes the code to filter the markers by both species and population to ensure only the markers referenced in the population actually show up.
2026-04-13Retrieve ALL markers for a species regardless of data.Frederick Muriuki Muriithi
Retrieve all the genetic markers that exist for a particular species regardless of whether or not a particular marker has corresponding samples allele data.
2026-04-13Rework `list_markers`: return list of markers and basic metadata.Frederick Muriuki Muriithi
2026-04-13Get possibly filtered set of markers and count of total records.Frederick Muriuki Muriithi
Get a possibly filtered set of markers ensuring the markers are specific to the dataset. Also return the total number of markers in the dataset, even if we do not return the entire list.
2026-03-27genotypes: Refactor to consistently use monads.Frederick Muriuki Muriithi
2026-03-27genotypes: Move "GET" handling out of context manager.Frederick Muriuki Muriithi
2026-03-27genotypes: Add module-level logger.Frederick Muriuki Muriithi
2026-03-25genotypes: Remove obsolete endpoints.Frederick Muriuki Muriithi
2025-12-10Feature Flags: Generically deal with HTTP-based feature flags.Frederick Muriuki Muriithi
* Define a default `FEATURE_FLAGS_HTTP` configuration variable that's an empty list to help defining http-based feature flags that can be used to turn on/off features * Build macro to include hidden fields for feature flags where necessary. * Extend flask's `url_for` function to deal with defined feature flags in a mostly transparent way
2025-02-18Extract common pattern into generic function.Frederick Muriuki Muriithi
2025-02-18Use new searchable list tables with off-ramps for data creation.Frederick Muriuki Muriithi
2024-12-10Retain population for "List Markers" pageFrederick Muriuki Muriithi
For UI consistency, and as a convenience to the users, retain the population details for the "List Markers" page to help them navigate more easily using the breadcrumbs.
2024-11-22Use gn-libs code for db connection.Frederick Muriuki Muriithi
Use the code in gn-libs to connect to the database, rather than a local module.
2024-09-27Extract common functionality into reusable function.Frederick Muriuki Muriithi
2024-09-26Send the dataset ID. Fix bugs.Frederick Muriuki Muriithi
2024-09-25Setup auth with dataset creation.Frederick Muriuki Muriithi
2024-09-25Lint: Fix myriad of linting issuesFrederick Muriuki Muriithi