| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 14 days | Refactor `qc_pheno_file` and reuse it for different file types. | Frederick Muriuki Muriithi | |
| The QC/QA steps taken by the `qc_pheno_file` function are very similar across the "pheno", "phenose" and "phenonum" files. This commit makes the `qc_pheno_file` function a higher-order function and we pass the file-type specific check(s) as a callable (function) to be used for the QC/QA process. | |||
| 14 days | Check for errors in `pheno` files. | Frederick Muriuki Muriithi | |
| 2024-10-21 | Check `phenocovar` files for errors. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Cleanup: Delete all extracted files after processing. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Leave TODO notes for what needs to be done. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Fetch samples from database | Frederick Muriuki Muriithi | |
| Fetch the samples from the database. These will be used to verify that the samples in the phenotype files already exist in the database and are valid. | |||
| 2024-10-17 | Undo transpose for any transposed files | Frederick Muriuki Muriithi | |
| To reduce the complexity involved in the processing of the files, we undo any transposition of the CSV files for those files that are marked as transposed. | |||
| 2024-10-17 | Pass new arguments to QC function. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Add `speciesid` and `populationid` arguments to the script. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Add the working directory argument to the script. | Frederick Muriuki Muriithi | |
| Add a `--working-dir` argument to allow passing a directory where the script process the files within. If not provided, it defaults to a directory within the systems temporary directory. | |||
| 2024-10-17 | Extract the R/qtl2 bundle for processing. | Frederick Muriuki Muriithi | |
| To enable processing of the files individually, this commit will enable the extraction of the files into a known working directory in which all further processing will take place. | |||
| 2024-10-17 | Extract common functions. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Save errors for each file in lists. Parallelise error checking. | Frederick Muriuki Muriithi | |
| * Save the errors for each file in a redis list for that file. * Make error checking parallel, i.e. ensure every file of a particular type is checked completely independent of other files of the same type. | |||
| 2024-10-17 | Rewrite the QC code for R/qtl2 | Frederick Muriuki Muriithi | |
| 2024-10-14 | Initialise background script for running QC on phenotype bundles. | Frederick Muriuki Muriithi | |
| 2024-10-14 | BugFix: Use provided prefix | Frederick Muriuki Muriithi | |
| Use the provided prefix rather than calling `jobs.jobsnamespace()` function that depends of an app context existing. | |||
| 2024-10-14 | Make addition of arguments independent of each other. | Frederick Muriuki Muriithi | |
| 2024-09-09 | Enable samples uploads. | Frederick Muriuki Muriithi | |
| 2024-09-03 | Initialise the populations package and update references. | Frederick Muriuki Muriithi | |
| 2024-08-28 | Move code handling expression data upload into new module. | Frederick Muriuki Muriithi | |
| 2024-08-16 | Log out correct parameters. | Frederick Muriuki Muriithi | |
| 2024-08-13 | Bug: cross reference with NULL cM when "gmap" file is absent | Frederick Muriuki Muriithi | |
| The "gmap" file might not exist in some bundles. In those instances, cross-reference the data without including the genotypes' physical positions (cM). | |||
| 2024-08-12 | Rename module: Module contains exceptions classes. | Frederick Muriuki Muriithi | |
| 2024-08-08 | Fix bugs and pass in logger to functions. | Frederick Muriuki Muriithi | |
| 2024-08-06 | Pass logger on to inner functions | Frederick Muriuki Muriithi | |
| Pass the logger forward to inner functions to help with debugging things. | |||
| 2024-07-25 | Fix typing and linting errors. | Frederick Muriuki Muriithi | |
| 2024-07-25 | Rename module: qc_app --> uploader | Frederick Muriuki Muriithi | |
| 2024-07-05 | bug: Return a hashable key, not a dict. | Frederick Muriuki Muriithi | |
| 2024-07-02 | Call correct method. | Frederick Muriuki Muriithi | |
| 2024-07-02 | Ensure no duplicated values for the query. | Frederick Muriuki Muriithi | |
| 2024-07-01 | Check for genotype samples in the database | Frederick Muriuki Muriithi | |
| Check for genotype samples in both the R/qtl2 file and in the database. | |||
| 2024-06-27 | Fix bug with the logging setup. | Frederick Muriuki Muriithi | |
| 2024-04-12 | Move entry-point module to scripts package. | Frederick Muriuki Muriithi | |
| This ensures the entry-point script/module is actually installed together with the rest of the code. | |||
| 2024-04-08 | Fix pylint and mypy errors. | Frederick Muriuki Muriithi | |
| 2024-04-04 | Remove unused database connection. | Frederick Muriuki Muriithi | |
| 2024-04-03 | Reduce size of data inserted per query | Frederick Muriuki Muriithi | |
| Reduce the size of data inserted per query since MariDB allows a packet with a maximum size of 1GB. This should hopefully resolve the …OperationalError: (2006, 'Server has gone away') error. | |||
| 2024-03-29 | Quiet linter. | Frederick Muriuki Muriithi | |
| 2024-03-22 | Notify user if identifiers are not consistent. | Frederick Muriuki Muriithi | |
| 2024-03-22 | Map names in files to names in database. | Frederick Muriuki Muriithi | |
| 2024-03-22 | Fix linting issue. | Frederick Muriuki Muriithi | |
| 2024-03-20 | Fix bug: correctly merge standard-error values in file to data in db | Frederick Muriuki Muriithi | |
| `read_datavalues(…)` function returns a dict of the form: ``` { ProbeSetName01: ({…}, …), ProbeSetName02: ({…}, …), ︙ } ``` Previously, the generator would thus try to index into the keys of the datavalues, which were strings, leading to an error. This commit changes the generator to return the values of the datavalues dict as a flattened list of values. | |||
| 2024-02-28 | Fix bug: fetch from cursor, not return from cursor.execute(…)load-raw-data-no-parsing | Frederick Muriuki Muriithi | |
| 2024-02-21 | Check that samples/cases are consistent | Frederick Muriuki Muriithi | |
| Ensure that **ALL** samples/cases/individuals mentioned in any of the pheno files actually exist in at least one of the geno files. | |||
| 2024-02-21 | Pass 'filename' value to error checker function. | Frederick Muriuki Muriithi | |
| 2024-02-20 | Track filename in the errors | Frederick Muriuki Muriithi | |
| R/qtl2 bundles can contain more than one file, of the same type. When errors are encountered in any of the files, we need to be able to inform the user which file it is, in addition to the line and column number. | |||
| 2024-02-15 | Filter out NULL values | Frederick Muriuki Muriithi | |
| 2024-02-15 | Only log out information if the check is actually run. | Frederick Muriuki Muriithi | |
| 2024-02-13 | Add some parallelism to the QC Checks | Frederick Muriuki Muriithi | |
| 2024-02-13 | Provide nice UI progress indicators. | Frederick Muriuki Muriithi | |
| 2024-02-12 | Check for errors in the 'phenose' file. | Frederick Muriuki Muriithi | |
 |
index : gn-uploader | |
| gn-uploader: Data uploader for GeneNetwork |
| aboutsummaryrefslogtreecommitdiff |