genenetwork3/sheepdog, branch auth/implement-authorization-code-flow GeneNetwork3 REST API for data science and machine learning http://git.genenetwork.org/genenetwork3/atom?h=auth%2Fimplement-authorization-code-flow 2023-04-17T11:42:05+00:00 Hook up code to use external search script for phenotypes 2023-04-17T11:42:05+00:00 Frederick Muriuki Muriithi 2023-04-17T11:42:05+00:00 urn:sha1:8af8105444522c2c71b5ddd36a550e964cddffbf Fix some linting issues 2022-05-24T02:28:35+00:00 Frederick Muriuki Muriithi 2022-05-24T02:28:35+00:00 urn:sha1:bbc4b165761b359115fd1a249ea22e64d55db384 Enable running the worker in "one-shot" mode 2022-05-16T07:37:23+00:00 Frederick Muriuki Muriithi 2022-05-16T07:37:23+00:00 urn:sha1:a657b502e6ed46ea0887b5febb89a7408f163820 Enable the running of the worker script in one-shot mode. Do incremental backoff if there are no jobs on the queue 2022-03-03T07:20:04+00:00 Frederick Muriuki Muriithi 2022-02-25T09:22:19+00:00 urn:sha1:7de9fea87dc2c9bcb242fd7ffda11af63dbf4268 Since the worker polls a queue for jobs, it can get into a busy poll. This was the reason that there was a delay of 0.1 seconds between each poll instance. This commit takes this a little further by doing an incremental backoff, where it waits longer and longer after each poll where it does not find a job available, up to an arbitrary maximum. Run partial correlations in an external process 2022-03-03T07:20:04+00:00 Frederick Muriuki Muriithi 2022-02-23T11:51:47+00:00 urn:sha1:6d39c92fbc9a7b82cd8eef60c62cd5d83acb49a1 Run the partial correlations code in an external python process decoupling it from the server and making it asynchronous. Summary of changes: * gn3/api/correlation.py: - Remove response processing code - Queue partial corrs processing - Create new endpoint to get results * gn3/commands.py - Compose the pcorrs command to be run in an external process - Enable running of subprocess commands with list args * gn3/responses/__init__.py: new module indicator file * gn3/responses/pcorrs_responses.py: Hold response processing code extracted from ~gn3.api.correlations.py~ file * scripts/partial_correlations.py: CLI script to process the pcorrs * sheepdog/worker.py: - Add the *genenetwork3* path at the beginning of the ~sys.path~ list to override any GN3 in the site-packages - Add any environment variables to be set for the command to be run Use the correct id to fetch cmd status 2021-03-10T12:16:52+00:00 BonfaceKilz 2021-03-10T12:16:52+00:00 urn:sha1:f6fb8fc46aea76e9c4011a091b9a86306b72d42d Fix encoding issues with worker 2021-03-10T11:26:01+00:00 BonfaceKilz 2021-03-10T11:26:01+00:00 urn:sha1:9170e293e896d1d283d3b19d79d673c9a92724d2 Use the correct redis cmd when updating variables 2021-03-10T10:36:24+00:00 BonfaceKilz 2021-03-10T10:36:24+00:00 urn:sha1:a81b01836886fc6e15ee23647f26b6bfd563c4cf Use correct queue name in worker 2021-03-10T10:20:49+00:00 BonfaceKilz 2021-03-10T10:20:49+00:00 urn:sha1:e3a871e023d66495574f9b096036ae6d438629fd Add basic worker for processing commands 2021-02-16T13:22:11+00:00 BonfaceKilz 2021-02-16T13:19:05+00:00 urn:sha1:ec0f9f611796ab67ce816d07f807701cb688cbeb This will later be replaced with GNU Guile