aboutsummaryrefslogtreecommitdiff
path: root/qc_app/jobs.py
AgeCommit message (Expand)Author
2024-01-24Checks: Update code and tests to ensure all checks pass.Frederick Muriuki Muriithi
2024-01-24redis-prefix: Update file validation code...Update the file validation script and routes to use the redis prefix for jobs. Frederick Muriuki Muriithi
2024-01-23Update scripts to use redis prefix.Frederick Muriuki Muriithi
2024-01-23Add prefix to redis keys...Since the GNQC application might find itself sharing the same redis server and/or database with other applications, add a prefix to its keys. Frederick Muriuki Muriithi
2024-01-20Fetch sample/case names from database...Fetch the sample/case names from the database rather than from a static file in the repository. Issue: https://issues.genenetwork.org/issues/quality-control/read-samples-from-database-by-species Frederick Muriuki Muriithi
2024-01-19scripts: Process R/qtl2 bundle...Build script to start the processing of the R/qtl2 bundle. Frederick Muriuki Muriithi
2023-12-18Samples: Hook-up external async script to upload the samples.Frederick Muriuki Muriithi
2023-11-15Linting: Fix linting errors.Frederick Muriuki Muriithi
2023-10-16Indicate progress for data insert.Frederick Muriuki Muriithi
2022-09-08Use sys.executable and setup default environment...Use the sys.executable to get the correct python binary to run the external processes, and setup the environment to point to the correct PYTHONPATH. Frederick Muriuki Muriithi
2022-08-04Implement data insertion...- Hook up external data insertion script to webserver code - Provide rudimentary status indication - Generalise some job creation details Frederick Muriuki Muriithi
2022-07-19Rework: Use generic worker script to launch process...Use the generic worker script as the interface for launching external processes. Frederick Muriuki Muriithi
2022-07-19Rename file validation script...As preparation for building a new generic worker script, this commit renames the file validation script from 'worker.py' to 'validate_file.py' so as to ensure the name conforms better to what the script does. Frederick Muriuki Muriithi
2022-07-07Add filetype details to database update code...The filetype determines the queries to be run to update the database, therefore, this commit adds filetype information. Frederick Muriuki Muriithi
2022-06-15Fix linting and type errorsFrederick Muriuki Muriithi
2022-06-02Expire the jobs in 14 days by defaultFrederick Muriuki Muriithi
2022-05-19Update Web-UI: use new error collection paradigm...- README.org: document how to run scripts manually - manifest.scm: remove python-rq as a dependency - qc_app/jobs.py: rework job launching and processing - qc_app/parse.py: use reworked job processing - qc_app/templates/job_progress.html: display progress correctly - qc_app/templates/parse_results.html: display final results - scripts/worker.py: new worker script Frederick Muriuki Muriithi
2022-04-28Use sqlite3 to save the jobs metadata...* Use sqlite to save the jobs metadata and enable UI update of the progress for large files Frederick Muriuki Muriithi
2022-04-28Update queuing and display results of file parsing...* Make the 'worker' functions free from needing the application context by passing all the details they need as arguments. * Enable the display of parsing results. Frederick Muriuki Muriithi
2022-04-26Fix issues caught processing the jobs...* Create and push the application context for the worker functions * Fix the update of meta fields Frederick Muriuki Muriithi
2022-04-26Queue file parsing jobs...Enable the queuing of file parsing jobs, since the files could be really large and take a long time to parse and present results. * etc/default_config.py: Add default config for redis server * manifest.scm: Add redis, and rq as dependencies * qc_app/__init__.py * qc_app/jobs.py: module to hold utilities for management of the jobs * qc_app/parse.py: Enqueue the job - extract file-parsing code to callable function * qc_app/templates/base.html: Enable addition of extra meta tags * qc_app/templates/job_progress.html: template to display job progress * qc_app/templates/no_such_job.html: template to indicate when a job id is invalid * quality_control/parsing.py: Add the total size parsed so far Frederick Muriuki Muriithi