| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2024-01-15 | Process the `pheno` files. | Frederick Muriuki Muriithi | |
| Process the data in the pheno files and insert it into the database. | |||
| 2024-01-15 | Initialise `install_phenos` script. | Frederick Muriuki Muriithi | |
| 2024-01-15 | Extract common structure into separate modules. | Frederick Muriuki Muriithi | |
| 2024-01-11 | Update pmap data in the database. | Frederick Muriuki Muriithi | |
| 2024-01-11 | Update gmap data in the database | Frederick Muriuki Muriithi | |
| Update the genetic positions (in centiMorgans) in the database. | |||
| 2024-01-11 | Cleanup linting and typing errors. | Frederick Muriuki Muriithi | |
| 2024-01-11 | CLI: Require datasetid as an argument. | Frederick Muriuki Muriithi | |
| 2024-01-10 | Provide convenience functions to avoid subtle call errors | Frederick Muriuki Muriithi | |
| 2024-01-10 | Cross-reference genotype data to the dataset. | Frederick Muriuki Muriithi | |
| 2024-01-10 | Insert genotype data. | Frederick Muriuki Muriithi | |
| 2024-01-10 | Cross-reference individuals to populations. | Frederick Muriuki Muriithi | |
| 2024-01-10 | Insert any new individuals/samples into the database. | Frederick Muriuki Muriithi | |
| 2024-01-10 | Insert any new markers | Frederick Muriuki Muriithi | |
| Insert any new markers found into the database. | |||
| 2024-01-09 | WIP: Install genotypes from R/qtl2 bundle | Frederick Muriuki Muriithi | |
| Load the genotype information from the R/qtl2 bundle and insert it into the database. | |||
| 2024-01-09 | Fix linting errors | Frederick Muriuki Muriithi | |
| 2024-01-09 | scripts: Log to redis | Frederick Muriuki Muriithi | |
| Add a logger that will log output to a redis list. This will be useful to retrieve the output(s) of the running script to help display some form of progress to the user. | |||
| 2024-01-08 | Use extracted functions and fix bugs | Frederick Muriuki Muriithi | |
| 2023-12-20 | Fix ID column key | Frederick Muriuki Muriithi | |
| Fix the wrong assumption that the ID key will always be "ProbeSetID". The key for the ID column could change from study to study or dataset to dataset. | |||
| 2023-12-18 | Samples: Hook-up external async script to upload the samples. | Frederick Muriuki Muriithi | |
| 2023-12-14 | samples: Create external script and fix some bugs. | Frederick Muriuki Muriithi | |
| 2023-11-30 | Bug: Fix insertion error | Frederick Muriuki Muriithi | |
| Fix a bug that was causing the data insertion to fail. | |||
| 2023-11-30 | Fix errors caught by pylint and mypy. | Frederick Muriuki Muriithi | |
| 2023-11-15 | Log out DB errors even when handled | Frederick Muriuki Muriithi | |
| Log out any DB errors that occur to help with tracking and debugging failures. | |||
| 2023-11-15 | Remove debugging statement. | Frederick Muriuki Muriithi | |
| 2023-11-15 | Bug: Insert ProbeSets before average data and cross-refs | Frederick Muriuki Muriithi | |
| Insert the ProbeSets first before inserting the average data and cross-referencing it. | |||
| 2023-11-15 | mypy: Fix typing errors. | Frederick Muriuki Muriithi | |
| 2023-11-15 | Linting: Fix linting errors. | Frederick Muriuki Muriithi | |
| 2023-10-30 | Prevent overwrite. Avoid inserting same probeset multiple times. | Frederick Muriuki Muriithi | |
| * Avoid overwriting existing data by adding a random string to the name of the ProbeSet * Read ProbeSet names separately from the means for the strains, to avoid entering the same ProbeSet record over and over with varying names. | |||
| 2023-10-18 | Link to `ProbeSet.Id` in XRef Table | Frederick Muriuki Muriithi | |
| The "ProbeSetId" value in the file is actually stored in the database in the `ProbeSet.Name` field, whereas the `ProbeSetXRef` table actually uses the `ProbeSet.Id` value to define the relationships. This commit fixes that issue. | |||
| 2023-10-16 | Fix bug: Insert ProbeSets if they do not exist. | Frederick Muriuki Muriithi | |
| 2023-10-16 | Fix bug: Provide the platform_id to `insert_means` | Frederick Muriuki Muriithi | |
| 2023-10-16 | Fix bug: parsing fields from query. | Frederick Muriuki Muriithi | |
| 2022-08-05 | Update job status. Display stdout and stderr outputs | Frederick Muriuki Muriithi | |
| * Display the status of the job, as it is running * Display STDERR output if an error occurs * Display STDOUT output as job is running and on successful completion of the job | |||
| 2022-08-04 | Implement data insertion | Frederick Muriuki Muriithi | |
| - Hook up external data insertion script to webserver code - Provide rudimentary status indication - Generalise some job creation details | |||
| 2022-08-04 | Handle stderr output | Frederick Muriuki Muriithi | |
| * Fix bug where stderr was not being updated on redis appropriately | |||
| 2022-08-04 | Format queries for output. Get rid of extraneous output. | Frederick Muriuki Muriithi | |
| 2022-07-19 | Save standard error data. Fix linting and typing errors. | Frederick Muriuki Muriithi | |
| 2022-07-19 | Implement data insertion (averages/means) | Frederick Muriuki Muriithi | |
| * quality_control/file_utils.py (new file): extract common file utilities. * quality_control/parsing.py (refactor): extract common file utilities. * scripts/insert_data.py: Implement data insertion for averages/means | |||
| 2022-07-19 | Check connections before launching | Frederick Muriuki Muriithi | |
| * qc_app/__init__.py (refactor): Check connection before launching the application * qc_app/check_connections.py (new file): Add code to check connections * qc_app/db_utils.py (refactor): enable passing the database uri as an argument to the connection creation function. * scripts/worker.py (refactor): Use new code to check for redis connection. | |||
| 2022-07-19 | Add generic worker script | Frederick Muriuki Muriithi | |
| Add a generic worker script, whose purpose is: - to launch the specific worker script - to capture both stdin and stderr streams and put them on redis In this way, we can launch redis-aware and redis-unaware workers and capture their outputs or errors for later processing. | |||
| 2022-07-19 | Rename file validation script | Frederick Muriuki Muriithi | |
| As preparation for building a new generic worker script, this commit renames the file validation script from 'worker.py' to 'validate_file.py' so as to ensure the name conforms better to what the script does. | |||
| 2022-07-07 | Replace 'magic' with builtin 'mimetypes' | Frederick Muriuki Muriithi | |
| Use the builtin mimetypes which gives better results | |||
| 2022-06-28 | Check for inconsistent columns | Frederick Muriuki Muriithi | |
| The number columns in each contents line should be equal to the nember of columns in the header line. | |||
| 2022-06-15 | Fix linting and type errors | Frederick Muriuki Muriithi | |
| 2022-06-10 | Enable upload of zipfiles | Frederick Muriuki Muriithi | |
| 2022-06-02 | Enable user abortion of file parsing | Frederick Muriuki Muriithi | |
| Enable the user to abort the background parsing of the file. | |||
| 2022-06-02 | Add an error display to the progress status report | Frederick Muriuki Muriithi | |
| Enable the progress status page to show all the errors found at any point during the processing of the file. | |||
| 2022-05-19 | Update Web-UI: use new error collection paradigm | Frederick Muriuki Muriithi | |
| - README.org: document how to run scripts manually - manifest.scm: remove python-rq as a dependency - qc_app/jobs.py: rework job launching and processing - qc_app/parse.py: use reworked job processing - qc_app/templates/job_progress.html: display progress correctly - qc_app/templates/parse_results.html: display final results - scripts/worker.py: new worker script | |||
| 2022-05-19 | Extract progress indication from the parsing | Frederick Muriuki Muriithi | |
| Since progress indication is not part of the parsing, this commit extracts the progress indication into functions with well defined input arguments that hide the progress indication logic from the parsing function. | |||
| 2022-05-18 | Update CLI: use new error collection code | Frederick Muriuki Muriithi | |
| Use the new code that collects the errors in the CLI client. | |||
 |
index : gn-uploader | |
| gn-uploader: Data uploader for GeneNetwork |
| aboutsummaryrefslogtreecommitdiff |