| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2024-10-24 | Reduce number of local variables created in function. | Frederick Muriuki Muriithi | |
| Use the values in the argparse.Namespace object directly to avoid creating many local variables. | |||
| 2024-10-24 | Add `jobid` to extraction directory | Frederick Muriuki Muriithi | |
| Add the job id to help "group" files that belong to the same job. | |||
| 2024-10-24 | Check pheno, phenose, and phenonum files with logging | Frederick Muriuki Muriithi | |
| Add logging to the `qc_pheno_file` function so that we get the messages pushed to redis for every file that is being checked for errors. | |||
| 2024-10-24 | Check phenocovar files with logging | Frederick Muriuki Muriithi | |
| Build a function-scope logger using the new `scripts.redis_logger.RedisMessageListHandler` log handler to log-out the messages for each phenocovar file being processed in a more thread-safe way. | |||
| 2024-10-24 | Fix linting and typing errors. | Frederick Muriuki Muriithi | |
| 2024-10-24 | Add a `--loglevel` argument to select loglevel in scripts | Frederick Muriuki Muriithi | |
| To enable selection of log levels within scripts, this commit adds the `--loglevel` argument to the list of arguments that can be passed to scripts. | |||
| 2024-10-24 | Parse entire namespace object rather than individual values. | Frederick Muriuki Muriithi | |
| 2024-10-24 | Move logger creation to `build_main` function | Frederick Muriuki Muriithi | |
| Since the module-level loggers are built mostly the same, move the creation of the logger to the more general function to reduce repetition. | |||
| 2024-10-24 | Add handler to send log messages to specific Redis list | Frederick Muriuki Muriithi | |
| Class `RedisMessageListHandler` builds a log handler that pushes any messages logged out to the redis list specified by its `fullyqualifiedkey` argument. | |||
| 2024-10-22 | Refactor `qc_pheno_file` and reuse it for different file types. | Frederick Muriuki Muriithi | |
| The QC/QA steps taken by the `qc_pheno_file` function are very similar across the "pheno", "phenose" and "phenonum" files. This commit makes the `qc_pheno_file` function a higher-order function and we pass the file-type specific check(s) as a callable (function) to be used for the QC/QA process. | |||
| 2024-10-22 | Check for errors in `pheno` files. | Frederick Muriuki Muriithi | |
| 2024-10-21 | Check `phenocovar` files for errors. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Cleanup: Delete all extracted files after processing. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Leave TODO notes for what needs to be done. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Fetch samples from database | Frederick Muriuki Muriithi | |
| Fetch the samples from the database. These will be used to verify that the samples in the phenotype files already exist in the database and are valid. | |||
| 2024-10-17 | Undo transpose for any transposed files | Frederick Muriuki Muriithi | |
| To reduce the complexity involved in the processing of the files, we undo any transposition of the CSV files for those files that are marked as transposed. | |||
| 2024-10-17 | Pass new arguments to QC function. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Add `speciesid` and `populationid` arguments to the script. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Add the working directory argument to the script. | Frederick Muriuki Muriithi | |
| Add a `--working-dir` argument to allow passing a directory where the script process the files within. If not provided, it defaults to a directory within the systems temporary directory. | |||
| 2024-10-17 | Extract the R/qtl2 bundle for processing. | Frederick Muriuki Muriithi | |
| To enable processing of the files individually, this commit will enable the extraction of the files into a known working directory in which all further processing will take place. | |||
| 2024-10-17 | Extract common functions. | Frederick Muriuki Muriithi | |
| 2024-10-17 | Save errors for each file in lists. Parallelise error checking. | Frederick Muriuki Muriithi | |
| * Save the errors for each file in a redis list for that file. * Make error checking parallel, i.e. ensure every file of a particular type is checked completely independent of other files of the same type. | |||
| 2024-10-17 | Rewrite the QC code for R/qtl2 | Frederick Muriuki Muriithi | |
| 2024-10-14 | Initialise background script for running QC on phenotype bundles. | Frederick Muriuki Muriithi | |
| 2024-10-14 | BugFix: Use provided prefix | Frederick Muriuki Muriithi | |
| Use the provided prefix rather than calling `jobs.jobsnamespace()` function that depends of an app context existing. | |||
| 2024-10-14 | Make addition of arguments independent of each other. | Frederick Muriuki Muriithi | |
| 2024-09-09 | Enable samples uploads. | Frederick Muriuki Muriithi | |
| 2024-09-03 | Initialise the populations package and update references. | Frederick Muriuki Muriithi | |
| 2024-08-28 | Move code handling expression data upload into new module. | Frederick Muriuki Muriithi | |
| 2024-08-16 | Log out correct parameters. | Frederick Muriuki Muriithi | |
| 2024-08-13 | Bug: cross reference with NULL cM when "gmap" file is absent | Frederick Muriuki Muriithi | |
| The "gmap" file might not exist in some bundles. In those instances, cross-reference the data without including the genotypes' physical positions (cM). | |||
| 2024-08-12 | Rename module: Module contains exceptions classes. | Frederick Muriuki Muriithi | |
| 2024-08-08 | Fix bugs and pass in logger to functions. | Frederick Muriuki Muriithi | |
| 2024-08-06 | Pass logger on to inner functions | Frederick Muriuki Muriithi | |
| Pass the logger forward to inner functions to help with debugging things. | |||
| 2024-07-25 | Fix typing and linting errors. | Frederick Muriuki Muriithi | |
| 2024-07-25 | Rename module: qc_app --> uploader | Frederick Muriuki Muriithi | |
| 2024-07-05 | bug: Return a hashable key, not a dict. | Frederick Muriuki Muriithi | |
| 2024-07-02 | Call correct method. | Frederick Muriuki Muriithi | |
| 2024-07-02 | Ensure no duplicated values for the query. | Frederick Muriuki Muriithi | |
| 2024-07-01 | Check for genotype samples in the database | Frederick Muriuki Muriithi | |
| Check for genotype samples in both the R/qtl2 file and in the database. | |||
| 2024-06-27 | Fix bug with the logging setup. | Frederick Muriuki Muriithi | |
| 2024-04-12 | Move entry-point module to scripts package. | Frederick Muriuki Muriithi | |
| This ensures the entry-point script/module is actually installed together with the rest of the code. | |||
| 2024-04-08 | Fix pylint and mypy errors. | Frederick Muriuki Muriithi | |
| 2024-04-04 | Remove unused database connection. | Frederick Muriuki Muriithi | |
| 2024-04-03 | Reduce size of data inserted per query | Frederick Muriuki Muriithi | |
| Reduce the size of data inserted per query since MariDB allows a packet with a maximum size of 1GB. This should hopefully resolve the …OperationalError: (2006, 'Server has gone away') error. | |||
| 2024-03-29 | Quiet linter. | Frederick Muriuki Muriithi | |
| 2024-03-22 | Notify user if identifiers are not consistent. | Frederick Muriuki Muriithi | |
| 2024-03-22 | Map names in files to names in database. | Frederick Muriuki Muriithi | |
| 2024-03-22 | Fix linting issue. | Frederick Muriuki Muriithi | |
| 2024-03-20 | Fix bug: correctly merge standard-error values in file to data in db | Frederick Muriuki Muriithi | |
| `read_datavalues(…)` function returns a dict of the form: ``` { ProbeSetName01: ({…}, …), ProbeSetName02: ({…}, …), ︙ } ``` Previously, the generator would thus try to index into the keys of the datavalues, which were strings, leading to an error. This commit changes the generator to return the values of the datavalues dict as a flattened list of values. | |||
 |
index : gn-uploader | |
| gn-uploader: Data uploader for GeneNetwork |
| aboutsummaryrefslogtreecommitdiff |