aboutsummaryrefslogtreecommitdiff
path: root/quality_control
AgeCommit message (Collapse)Author
2024-02-20Track filename in the errorsFrederick Muriuki Muriithi
R/qtl2 bundles can contain more than one file, of the same type. When errors are encountered in any of the files, we need to be able to inform the user which file it is, in addition to the line and column number.
2024-02-12Refactor: Use new decimal places checker.Frederick Muriuki Muriithi
2024-02-12Build generic decimal places checker.Frederick Muriuki Muriithi
2024-01-20Fetch sample/case names from databaseFrederick Muriuki Muriithi
Fetch the sample/case names from the database rather than from a static file in the repository. Issue: https://issues.genenetwork.org/issues/quality-control/read-samples-from-database-by-species
2024-01-15Update imports: `take` was moved to different package.Frederick Muriuki Muriithi
2024-01-15Extract common functional tools to separate package.Frederick Muriuki Muriithi
2024-01-05Revert "QC: Check for only one decimal place."Frederick Muriuki Muriithi
This reverts commit c213b0010c3ddc8d3215adab65bd489a9b884e30. After a larger discussion with Arthur, Rob and Pjotr present, it was verified that the strict checks are important. This reverts the commit that allowed for looser checks.
2023-11-24QC: Check for only one decimal place.Frederick Muriuki Muriithi
2022-11-16qc: Allow whole numbers or numbers where the decimals are all zeroesFrederick Muriuki Muriithi
2022-07-19Save standard error data. Fix linting and typing errors.Frederick Muriuki Muriithi
2022-07-19Implement data insertion (averages/means)Frederick Muriuki Muriithi
* quality_control/file_utils.py (new file): extract common file utilities. * quality_control/parsing.py (refactor): extract common file utilities. * scripts/insert_data.py: Implement data insertion for averages/means
2022-06-28Check for inconsistent columnsFrederick Muriuki Muriithi
The number columns in each contents line should be equal to the nember of columns in the header line.
2022-06-15Fix linting and type errorsFrederick Muriuki Muriithi
2022-06-10Enable upload of zipfilesFrederick Muriuki Muriithi
2022-06-02Enable user abortion of file parsingFrederick Muriuki Muriithi
Enable the user to abort the background parsing of the file.
2022-06-02Provide missing argumentFrederick Muriuki Muriithi
2022-05-27Accept zeroesFrederick Muriuki Muriithi
2022-05-27Revert "Accept exponential notation for standard-error files"Frederick Muriuki Muriithi
This reverts commit 960c1a5b831d8761a3e1716f86ded4cc5b67eea0. After meeting with Arthur, it was confirmed that the CSV file should not have values in the exponential notation.
2022-05-20Accept exponential notation for standard-error filesFrederick Muriuki Muriithi
2022-05-19Extract progress indication from the parsingFrederick Muriuki Muriithi
Since progress indication is not part of the parsing, this commit extracts the progress indication into functions with well defined input arguments that hide the progress indication logic from the parsing function.
2022-05-18Remove obsoleted code. Fix linting errors.Frederick Muriuki Muriithi
Remove the old code that relied on exceptions to parse errors in the uploaded files.
2022-05-18Simplify slot names and fix namedtuple type nameFrederick Muriuki Muriithi
- replace `line_number` with `line` and `column_number` with `column` to save on slot name lengths. - Fix the type name for the `DuplicateHeading` namedtuple type
2022-05-18Parse files with new non-exception functionsFrederick Muriuki Muriithi
Parse the files with the new functions that return error objects instead of raising exceptions
2022-05-18Return errors when found or None otherwiseFrederick Muriuki Muriithi
This commit adds a number of functions that return the error object when an error is found, or `None` otherwise. It avoids the use of exceptions as control flow constructs.
2022-05-02Optimise strain names parsingFrederick Muriuki Muriithi
- Use a way faster way of parsing the strains file
2022-04-27Implement command-line interface for QC of filesFrederick Muriuki Muriithi
2022-04-26Queue file parsing jobsFrederick Muriuki Muriithi
Enable the queuing of file parsing jobs, since the files could be really large and take a long time to parse and present results. * etc/default_config.py: Add default config for redis server * manifest.scm: Add redis, and rq as dependencies * qc_app/__init__.py * qc_app/jobs.py: module to hold utilities for management of the jobs * qc_app/parse.py: Enqueue the job - extract file-parsing code to callable function * qc_app/templates/base.html: Enable addition of extra meta tags * qc_app/templates/job_progress.html: template to display job progress * qc_app/templates/no_such_job.html: template to indicate when a job id is invalid * quality_control/parsing.py: Add the total size parsed so far
2022-04-25Call parsing function. Fix a few issues.Frederick Muriuki Muriithi
* qc_app/entry.py: Pass filetype onward to parsing endpoint * qc_app/parse.py: Call the function(s) necessary to parse a file * quality_control/errors.py: Fix argument passing to super class
2022-04-25Advance seek_position on errorFrederick Muriuki Muriithi
Advance the seek position, once we have yielded up an error to causing an infinite loop in certain conditions, where the `parse_errors` function ends up resuming the gile in the same position once it experiences an error.
2022-04-25`take`: function to select a few items from an iterableFrederick Muriuki Muriithi
To avoid processing all the items in an iterable, the `take` function is added in this commit. It realised a limited number (specified at call time) of items from the iterable given.
2022-04-21Collect all the errorsFrederick Muriuki Muriithi
Build a function to collect all the parsing errors into a "sequence" of dict objects containing the issues found.
2022-04-20Update parameters for testsFrederick Muriuki Muriithi
Derive a "correct" sample file from an existing sample file with errors for testing with large files. Fix issue caught by test.
2022-04-20Fix linting issuesFrederick Muriuki Muriithi
2022-04-13Implement remaining file parsing testsFrederick Muriuki Muriithi
* Implement remaining file parsing tests and some helpers functions needed for ensuring the tests work.
2022-04-13Check for `DuplicateHeader` errors in `parse_file` functionFrederick Muriuki Muriithi
2022-04-13Update `valid_header` to pass check for duplicated headersFrederick Muriuki Muriithi
2022-04-13Add test for duplicate headersFrederick Muriuki Muriithi
2022-04-13Implement test for parsing that failsFrederick Muriuki Muriithi
* Improve tests that ensure parsing fails in case the file has errors * Add strains.csv file * Implement minimum viable functionality that passes the implemented tests
2022-04-13Add tests outline for file parsingFrederick Muriuki Muriithi
Add dummy failing tests and a stub for the parsing of the files
2022-04-13Update Exception nameFrederick Muriuki Muriithi
Change the exception name to be more descriptive.
2022-04-11Implement MVI for headers validity checking functionFrederick Muriuki Muriithi
Add a minimum viable implementation that passes the tests for the function that checks for the validity of the headers
2022-04-11Add tests for headersFrederick Muriuki Muriithi
* Add tests to check for validity of the headers * Add stubs for the tests
2022-04-11Implement MVP for valid_value functionsFrederick Muriuki Muriithi
* Implement the minimum viable functions for the average and standard error `valid_value` functions.
2022-04-11Implement tests and stubs for functions under testFrederick Muriuki Muriithi