gn-uploader - gn-uploader: Data uploader for GeneNetwork

Age	Commit message (Collapse)	Author
2023-10-16	Fix bug: Provide the platform_id to `insert_means`	Frederick Muriuki Muriithi

2023-10-16	Fix bug: parsing fields from query.	Frederick Muriuki Muriithi

2022-08-05	Update job status. Display stdout and stderr outputs	Frederick Muriuki Muriithi
	* Display the status of the job, as it is running * Display STDERR output if an error occurs * Display STDOUT output as job is running and on successful completion of the job
2022-08-04	Implement data insertion	Frederick Muriuki Muriithi
	- Hook up external data insertion script to webserver code - Provide rudimentary status indication - Generalise some job creation details
2022-08-04	Handle stderr output	Frederick Muriuki Muriithi
	* Fix bug where stderr was not being updated on redis appropriately
2022-08-04	Format queries for output. Get rid of extraneous output.	Frederick Muriuki Muriithi

2022-07-19	Save standard error data. Fix linting and typing errors.	Frederick Muriuki Muriithi

2022-07-19	Implement data insertion (averages/means)	Frederick Muriuki Muriithi
	* quality_control/file_utils.py (new file): extract common file utilities. * quality_control/parsing.py (refactor): extract common file utilities. * scripts/insert_data.py: Implement data insertion for averages/means
2022-07-19	Check connections before launching	Frederick Muriuki Muriithi
	* qc_app/__init__.py (refactor): Check connection before launching the application * qc_app/check_connections.py (new file): Add code to check connections * qc_app/db_utils.py (refactor): enable passing the database uri as an argument to the connection creation function. * scripts/worker.py (refactor): Use new code to check for redis connection.
2022-07-19	Add generic worker script	Frederick Muriuki Muriithi
	Add a generic worker script, whose purpose is: - to launch the specific worker script - to capture both stdin and stderr streams and put them on redis In this way, we can launch redis-aware and redis-unaware workers and capture their outputs or errors for later processing.
2022-07-19	Rename file validation script	Frederick Muriuki Muriithi
	As preparation for building a new generic worker script, this commit renames the file validation script from 'worker.py' to 'validate_file.py' so as to ensure the name conforms better to what the script does.
2022-07-07	Replace 'magic' with builtin 'mimetypes'	Frederick Muriuki Muriithi
	Use the builtin mimetypes which gives better results
2022-06-28	Check for inconsistent columns	Frederick Muriuki Muriithi
	The number columns in each contents line should be equal to the nember of columns in the header line.
2022-06-15	Fix linting and type errors	Frederick Muriuki Muriithi

2022-06-10	Enable upload of zipfiles	Frederick Muriuki Muriithi

2022-06-02	Enable user abortion of file parsing	Frederick Muriuki Muriithi
	Enable the user to abort the background parsing of the file.
2022-06-02	Add an error display to the progress status report	Frederick Muriuki Muriithi
	Enable the progress status page to show all the errors found at any point during the processing of the file.
2022-05-19	Update Web-UI: use new error collection paradigm	Frederick Muriuki Muriithi
	- README.org: document how to run scripts manually - manifest.scm: remove python-rq as a dependency - qc_app/jobs.py: rework job launching and processing - qc_app/parse.py: use reworked job processing - qc_app/templates/job_progress.html: display progress correctly - qc_app/templates/parse_results.html: display final results - scripts/worker.py: new worker script
2022-05-19	Extract progress indication from the parsing	Frederick Muriuki Muriithi
	Since progress indication is not part of the parsing, this commit extracts the progress indication into functions with well defined input arguments that hide the progress indication logic from the parsing function.
2022-05-18	Update CLI: use new error collection code	Frederick Muriuki Muriithi
	Use the new code that collects the errors in the CLI client.
2022-05-11	Tag string as a python f-string to fix error display bug	Frederick Muriuki Muriithi

2022-05-02	Optimise strain names parsing	Frederick Muriuki Muriithi
	- Use a way faster way of parsing the strains file
2022-04-27	Enable managing app via setup.py	Frederick Muriuki Muriithi
	While the application is developed with GNU Guix, the end user might not be using it, and therefore, this commit provides a way for the user to install the application with the usual python package management systems.