gn-uploader - gn-uploader: Data uploader for GeneNetwork

Age	Commit message (Collapse)	Author
2024-01-19	scripts: Process R/qtl2 bundle	Frederick Muriuki Muriithi
	Build script to start the processing of the R/qtl2 bundle.
2023-12-18	Samples: Hook-up external async script to upload the samples.	Frederick Muriuki Muriithi

2023-11-15	Linting: Fix linting errors.	Frederick Muriuki Muriithi

2023-10-16	Indicate progress for data insert.	Frederick Muriuki Muriithi

2022-09-08	Use sys.executable and setup default environment	Frederick Muriuki Muriithi
	Use the sys.executable to get the correct python binary to run the external processes, and setup the environment to point to the correct PYTHONPATH.
2022-08-04	Implement data insertion	Frederick Muriuki Muriithi
	- Hook up external data insertion script to webserver code - Provide rudimentary status indication - Generalise some job creation details
2022-07-19	Rework: Use generic worker script to launch process	Frederick Muriuki Muriithi
	Use the generic worker script as the interface for launching external processes.
2022-07-19	Rename file validation script	Frederick Muriuki Muriithi
	As preparation for building a new generic worker script, this commit renames the file validation script from 'worker.py' to 'validate_file.py' so as to ensure the name conforms better to what the script does.
2022-07-07	Add filetype details to database update code	Frederick Muriuki Muriithi
	The filetype determines the queries to be run to update the database, therefore, this commit adds filetype information.
2022-06-15	Fix linting and type errors	Frederick Muriuki Muriithi

2022-06-02	Expire the jobs in 14 days by default	Frederick Muriuki Muriithi

2022-05-19	Update Web-UI: use new error collection paradigm	Frederick Muriuki Muriithi
	- README.org: document how to run scripts manually - manifest.scm: remove python-rq as a dependency - qc_app/jobs.py: rework job launching and processing - qc_app/parse.py: use reworked job processing - qc_app/templates/job_progress.html: display progress correctly - qc_app/templates/parse_results.html: display final results - scripts/worker.py: new worker script
2022-04-28	Use sqlite3 to save the jobs metadata	Frederick Muriuki Muriithi
	* Use sqlite to save the jobs metadata and enable UI update of the progress for large files
2022-04-28	Update queuing and display results of file parsing	Frederick Muriuki Muriithi
	* Make the 'worker' functions free from needing the application context by passing all the details they need as arguments. * Enable the display of parsing results.
2022-04-26	Fix issues caught processing the jobs	Frederick Muriuki Muriithi
	* Create and push the application context for the worker functions * Fix the update of meta fields
2022-04-26	Queue file parsing jobs	Frederick Muriuki Muriithi
	Enable the queuing of file parsing jobs, since the files could be really large and take a long time to parse and present results. * etc/default_config.py: Add default config for redis server * manifest.scm: Add redis, and rq as dependencies * qc_app/__init__.py * qc_app/jobs.py: module to hold utilities for management of the jobs * qc_app/parse.py: Enqueue the job - extract file-parsing code to callable function * qc_app/templates/base.html: Enable addition of extra meta tags * qc_app/templates/job_progress.html: template to display job progress * qc_app/templates/no_such_job.html: template to indicate when a job id is invalid * quality_control/parsing.py: Add the total size parsed so far