aboutsummaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)Author
2024-08-06Pass logger on to inner functionsFrederick Muriuki Muriithi
Pass the logger forward to inner functions to help with debugging things.
2024-07-25Fix typing and linting errors.Frederick Muriuki Muriithi
2024-07-25Rename module: qc_app --> uploaderFrederick Muriuki Muriithi
2024-07-05bug: Return a hashable key, not a dict.Frederick Muriuki Muriithi
2024-07-02Call correct method.Frederick Muriuki Muriithi
2024-07-02Ensure no duplicated values for the query.Frederick Muriuki Muriithi
2024-07-01Check for genotype samples in the databaseFrederick Muriuki Muriithi
Check for genotype samples in both the R/qtl2 file and in the database.
2024-06-27Fix bug with the logging setup.Frederick Muriuki Muriithi
2024-04-12Move entry-point module to scripts package.Frederick Muriuki Muriithi
This ensures the entry-point script/module is actually installed together with the rest of the code.
2024-04-08Fix pylint and mypy errors.Frederick Muriuki Muriithi
2024-04-04Remove unused database connection.Frederick Muriuki Muriithi
2024-04-03Reduce size of data inserted per queryFrederick Muriuki Muriithi
Reduce the size of data inserted per query since MariDB allows a packet with a maximum size of 1GB. This should hopefully resolve the …OperationalError: (2006, 'Server has gone away') error.
2024-03-29Quiet linter.Frederick Muriuki Muriithi
2024-03-22Notify user if identifiers are not consistent.Frederick Muriuki Muriithi
2024-03-22Map names in files to names in database.Frederick Muriuki Muriithi
2024-03-22Fix linting issue.Frederick Muriuki Muriithi
2024-03-20Fix bug: correctly merge standard-error values in file to data in dbFrederick Muriuki Muriithi
`read_datavalues(…)` function returns a dict of the form: ``` { ProbeSetName01: ({…}, …), ProbeSetName02: ({…}, …), ︙ } ``` Previously, the generator would thus try to index into the keys of the datavalues, which were strings, leading to an error. This commit changes the generator to return the values of the datavalues dict as a flattened list of values.
2024-02-28Fix bug: fetch from cursor, not return from cursor.execute(…)load-raw-data-no-parsingFrederick Muriuki Muriithi
2024-02-21Check that samples/cases are consistentFrederick Muriuki Muriithi
Ensure that **ALL** samples/cases/individuals mentioned in any of the pheno files actually exist in at least one of the geno files.
2024-02-21Pass 'filename' value to error checker function.Frederick Muriuki Muriithi
2024-02-20Track filename in the errorsFrederick Muriuki Muriithi
R/qtl2 bundles can contain more than one file, of the same type. When errors are encountered in any of the files, we need to be able to inform the user which file it is, in addition to the line and column number.
2024-02-15Filter out NULL valuesFrederick Muriuki Muriithi
2024-02-15Only log out information if the check is actually run.Frederick Muriuki Muriithi
2024-02-13Add some parallelism to the QC ChecksFrederick Muriuki Muriithi
2024-02-13Provide nice UI progress indicators.Frederick Muriuki Muriithi
2024-02-12Check for errors in the 'phenose' file.Frederick Muriuki Muriithi
2024-02-12Check for errors in the 'pheno' file.Frederick Muriuki Muriithi
2024-02-12Collect and display errors on 'geno' files in the bundle.Frederick Muriuki Muriithi
2024-02-12Add scaffolding for running individual files' QC checksFrederick Muriuki Muriithi
2024-02-12Extract missing files check to external functionFrederick Muriuki Muriithi
2024-02-12Retrive and display generic bundle errorsFrederick Muriuki Muriithi
Implement the first QC check for generic errors e.g. missing files in bundle, etc. Display retrieved errors on UI.
2024-02-12R/qtl2 QC: Set up scaffolding for QC UIFrederick Muriuki Muriithi
Set up the scaffolding for the flows and UI that will be used when running QC against the uploaded R/qtl2 bundle. This will be fleshed out later, and the UI is likely to change somewhat, down the line.
2024-02-12Provide better error messagingFrederick Muriuki Muriithi
Provide a better error message that can help the user figure out what they did wrong and fix it for themselves instead of the older, cryptic and extremely unhelpful error message.
2024-02-05Run QC on the 'geno' file in R/qtl2 bundle.Frederick Muriuki Muriithi
2024-02-05Add a QC step: Check all listed files existFrederick Muriuki Muriithi
Add a QC step to the R/qtl2 bundle processing - check that all files that are listed in the control file also exist in the bundle.
2024-02-05Fix linting and type errors.Frederick Muriuki Muriithi
2024-01-24Checks: Update code and tests to ensure all checks pass.Frederick Muriuki Muriithi
2024-01-24redis-prefix: Update file validation codeFrederick Muriuki Muriithi
Update the file validation script and routes to use the redis prefix for jobs.
2024-01-23Update scripts to use redis prefix.Frederick Muriuki Muriithi
2024-01-22scripts: Pass in logger to get detailed updates.Frederick Muriuki Muriithi
2024-01-22UI: Show progress of R/qtl2 bundle processing.Frederick Muriuki Muriithi
2024-01-20Fetch sample/case names from databaseFrederick Muriuki Muriithi
Fetch the sample/case names from the database rather than from a static file in the repository. Issue: https://issues.genenetwork.org/issues/quality-control/read-samples-from-database-by-species
2024-01-19scripts: Process R/qtl2 bundleFrederick Muriuki Muriithi
Build script to start the processing of the R/qtl2 bundle.
2024-01-16Filter out items with values of `None`.Frederick Muriuki Muriithi
The ProbeSetData table does not allow `NULL` for the `value` column for good reason - if there is no value, just don't have a record for that particular sample.
2024-01-15Update imports: `take` was moved to different package.Frederick Muriuki Muriithi
2024-01-15Process the `pheno` files.Frederick Muriuki Muriithi
Process the data in the pheno files and insert it into the database.
2024-01-15Initialise `install_phenos` script.Frederick Muriuki Muriithi
2024-01-15Extract common structure into separate modules.Frederick Muriuki Muriithi
2024-01-11Update pmap data in the database.Frederick Muriuki Muriithi
2024-01-11Update gmap data in the databaseFrederick Muriuki Muriithi
Update the genetic positions (in centiMorgans) in the database.