| Age | Commit message (Collapse) | Author |
|---|
|
Use the sys.executable to get the correct python binary to run the
external processes, and setup the environment to point to the correct
PYTHONPATH.
|
|
- Hook up external data insertion script to webserver code
- Provide rudimentary status indication
- Generalise some job creation details
|
|
Use the generic worker script as the interface for launching external
processes.
|
|
As preparation for building a new generic worker script, this commit
renames the file validation script from 'worker.py' to
'validate_file.py' so as to ensure the name conforms better to what
the script does.
|
|
The filetype determines the queries to be run to update the database,
therefore, this commit adds filetype information.
|
|
|
|
|
|
- README.org: document how to run scripts manually
- manifest.scm: remove python-rq as a dependency
- qc_app/jobs.py: rework job launching and processing
- qc_app/parse.py: use reworked job processing
- qc_app/templates/job_progress.html: display progress correctly
- qc_app/templates/parse_results.html: display final results
- scripts/worker.py: new worker script
|
|
* Use sqlite to save the jobs metadata and enable UI update of the
progress for large files
|
|
* Make the 'worker' functions free from needing the application
context by passing all the details they need as arguments.
* Enable the display of parsing results.
|
|
* Create and push the application context for the worker functions
* Fix the update of meta fields
|
|
Enable the queuing of file parsing jobs, since the files could be
really large and take a long time to parse and present results.
* etc/default_config.py: Add default config for redis server
* manifest.scm: Add redis, and rq as dependencies
* qc_app/__init__.py
* qc_app/jobs.py: module to hold utilities for management of the jobs
* qc_app/parse.py: Enqueue the job - extract file-parsing code to
callable function
* qc_app/templates/base.html: Enable addition of extra meta tags
* qc_app/templates/job_progress.html: template to display job progress
* qc_app/templates/no_such_job.html: template to indicate when a job
id is invalid
* quality_control/parsing.py: Add the total size parsed so far
|
