aboutsummaryrefslogtreecommitdiff
path: root/sheepdog/worker.py
AgeCommit message (Expand)Author
2022-05-24Fix some linting issuesFrederick Muriuki Muriithi
2022-05-16Enable running the worker in "one-shot" mode...Enable the running of the worker script in one-shot mode. Frederick Muriuki Muriithi
2022-03-03Do incremental backoff if there are no jobs on the queue...Since the worker polls a queue for jobs, it can get into a busy poll. This was the reason that there was a delay of 0.1 seconds between each poll instance. This commit takes this a little further by doing an incremental backoff, where it waits longer and longer after each poll where it does not find a job available, up to an arbitrary maximum. Frederick Muriuki Muriithi
2022-03-03Run partial correlations in an external process...Run the partial correlations code in an external python process decoupling it from the server and making it asynchronous. Summary of changes: * gn3/api/correlation.py: - Remove response processing code - Queue partial corrs processing - Create new endpoint to get results * gn3/commands.py - Compose the pcorrs command to be run in an external process - Enable running of subprocess commands with list args * gn3/responses/__init__.py: new module indicator file * gn3/responses/pcorrs_responses.py: Hold response processing code extracted from ~gn3.api.correlations.py~ file * scripts/partial_correlations.py: CLI script to process the pcorrs * sheepdog/worker.py: - Add the *genenetwork3* path at the beginning of the ~sys.path~ list to override any GN3 in the site-packages - Add any environment variables to be set for the command to be run Frederick Muriuki Muriithi
2021-03-10Use the correct id to fetch cmd statusBonfaceKilz
2021-03-10Fix encoding issues with workerBonfaceKilz
2021-03-10Use the correct redis cmd when updating variablesBonfaceKilz
2021-03-10Use correct queue name in workerBonfaceKilz
2021-02-16Add basic worker for processing commands...This will later be replaced with GNU Guile BonfaceKilz