aboutsummaryrefslogtreecommitdiff
path: root/gn3/api
AgeCommit message (Collapse)Author
2022-03-08Fix tests, and issues caught by testsFrederick Muriuki Muriithi
Fix some issues caught by tests due to changes introducing the hand-off of the partial correlations computations to an external process Fix some issues due to the changes that introduce context managers for database connections Update some tests to take the above two changes into consideration
2022-03-08Create database connections within context managersFrederick Muriuki Muriithi
Use the `with` context manager to open database connections, so as to ensure that those connections are closed once the call is completed. This hopefully avoids the 'too many connections' error
2022-03-04Automatically decode Redis stringsFrederick Muriuki Muriithi
2022-03-03Add endpoint for checking state of external processesFrederick Muriuki Muriithi
Long-running computations are handed off to external processes. This avoids timeouts in the webserver, and also reduces chances of instability of the webserver. The results of these long-running computations are needed eventually, so this commit provides a way to check for the state of the computation, and the results if any.
2022-03-03Run partial correlations in an external processFrederick Muriuki Muriithi
Run the partial correlations code in an external python process decoupling it from the server and making it asynchronous. Summary of changes: * gn3/api/correlation.py: - Remove response processing code - Queue partial corrs processing - Create new endpoint to get results * gn3/commands.py - Compose the pcorrs command to be run in an external process - Enable running of subprocess commands with list args * gn3/responses/__init__.py: new module indicator file * gn3/responses/pcorrs_responses.py: Hold response processing code extracted from ~gn3.api.correlations.py~ file * scripts/partial_correlations.py: CLI script to process the pcorrs * sheepdog/worker.py: - Add the *genenetwork3* path at the beginning of the ~sys.path~ list to override any GN3 in the site-packages - Add any environment variables to be set for the command to be run
2022-02-21Fix a myriad of linter issuesFrederick Muriuki Muriithi
* Use `with` in place of plain `open` * Use f-strings in place of `str.format()` * Remove string interpolation from queries - provide data as query parameters * other minor fixes
2022-02-18Test partial correlations endpoint with non-existent primary traitsFrederick Muriuki Muriithi
Test that the partial correlations endpoint responds with an appropriate "not-found" message and the corresponding 404 status code in the case where a request is made and the primary trait requested for does not exist in the database. Summary of the changes in each file: * gn3/api/correlation.py: generalise the building of the response * gn3/computations/partial_correlations.py: return with a "not-found" if the primary trait does not exist in the database * gn3/db/partial_correlations.py: Fix a number of bugs that led to exceptions in the case that the primary trait did not exist * pytest.ini: register a `slow` pytest marker * tests/integration/test_partial_correlations.py: Add a new test to check for an appropriate 404 response in case of a primary trait that does not exist in the database.
2022-02-17Test partial correlations endpoint with missing data in POST requestFrederick Muriuki Muriithi
Add a test for the partial correlations endpoint, with: - no data in the request - missing items in the data Fix the bugs caught by the test
2022-02-02response object error fixAlexander Kabui
2022-02-02pep8 formattingAlexander Kabui
2022-02-02return 401 on request failAlexander Kabui
2022-02-02pep8 formattingAlexander Kabui
2022-02-02new line fixAlexander Kabui
2022-01-22generate required json data for ctl apiAlexander Kabui
2022-01-22add endpoint for ctlAlexander Kabui
2022-01-10Convert NaN to NoneFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi Comment: https://github.com/genenetwork/genenetwork3/pull/67#issuecomment-1000828159 * Convert NaN values to None to avoid possible bugs with the string replace method used before.
2021-12-24Replace `NaN` with `null` in JSON stringFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * `NaN` is not a valid JSON value, and leads to errors in the code. This commit replaces all `NaN` values with `null`.
2021-12-24Encode the data to JSON and set the status codeFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Encode bytes objects to string * Encode NaN values to "null" * gn3/api/correlation.py:
2021-12-24Add API endpoint for partial correlationsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Add an API endpoint for the partial correlation. * gn3/api/correlation.py:
2021-12-17Add API endpoint for partial correlationsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Add an API endpoint for the partial correlation.
2021-12-02Implement dataset metadata API endpoint.Arun Isaac
* guix.scm: Import (gnu packages rdf). (genenetwork3)[propagated-inputs]: Add python-sparqlwrapper. * gn3/settings.py (SPARQL_ENDPOINT): New variable. * gn3/api/general.py: Import datasets from gn3.db. (dataset_metadata): New API endpoint. * gn3/db/datasets.py: Import re, Template from string, Dict and Optional from typing, JSON and SPARQLWrapper from SPARQLWrapper, SPARQL_ENDPOINT from gn3.settings. (sparql_query, dataset_metadata): New functions.
2021-10-19Enable vertical and horizontal heatmapsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/non-clustered-heatmaps-and-flipping.gmi * Update the request endpoint, so that it produces a vertical or horizontal heatmap depending on the user's request.
2021-10-12Merge branch 'main' of https://github.com/genenetwork/genenetwork3 into ↵zsloan
bug/fix_rqtl_covariates
2021-09-27fix merge conflictsAlexander Kabui
2021-09-27Remove unnecessary variable.Frederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Fix issue according to review https://github.com/genenetwork/genenetwork3/pull/37#discussion_r714549781
2021-09-23minor fixes for endpointAlexander Kabui
2021-09-22Fix pylint errorsFrederick Muriuki Muriithi
* Add missing function and module docstrings * Remove unused imports * Fix import order * Rework some code sections to fix issues * Disable some pylint errors.
2021-09-22Update check: Heatmaps need at least 2 itemsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Update the check to look for at least 2 traits before trying to generate the heatmap.
2021-09-22Return serialized plotly figureFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/api/heatmaps.py: Serialize the figure to JSON * gn3/heatmaps.py: Return the figure object Serialize the Plotly figure into JSON, and return that, so that it can be used on the client to display the image.
2021-09-22jsonify resultsAlexander Kabui
2021-09-20Enable Cross-Origin Resource SharingFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/api/heatmaps.py: Fix bugs in data parsing * gn3/app.py: enable CORS * gn3/settings.py: add flask-cors configurations * guix.scm: Add flask-cors dependency For easier testing of the heatmaps generation feature, this commit activates the cross-origin resource sharing for all "localhost" origins.
2021-09-20Return only the dataFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/api/heatmaps.py: Parse incoming data to build up correct trait names and respond with only the computed heatmap data. * gn3/heatmaps.py: Return only the computed data for heatmaps and clustering. Since GN3 is supposed to handle only the data, and db-access, this commit ensures that GN3 responds to the client with only the computed heatmap data, and does not try to generate the heatmaps themselves. The generation of the heatmaps will be delegated to the UI clients, such as GeneNetwork2.
2021-09-17Return path to generated filename for nowFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * To help with demonstrating that the code is producing the expected output, for now, we return the path to the generated html file that displays the interactive heatmap. At this point, it is mostly useful in the development environment. Moving forward, we might have to actually stream the raw html, or if we can get the Kaleido library packaged for GNU Guix, stream the images binary data instead.
2021-09-16pass user input to call scriptAlexander Kabui
2021-09-16Intergrate the heatmap generation with the APIFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Intergrate the heatmap generation code on the /api/heatmaps/clustered endpoint. The endpoint should take a json query of the form: {"traits_names": [ ... ] } where the "traits_name" value is a list of the full names of traits. A sample query to the endpoint could be something like the following: curl -i -X POST "http://localhost:8080/api/heatmaps/clustered" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -d '{ "traits_names": [ "UCLA_BXDBXH_CARTILAGE_V2::ILM103710672", "UCLA_BXDBXH_CARTILAGE_V2::ILM2260338", "UCLA_BXDBXH_CARTILAGE_V2::ILM3140576", "UCLA_BXDBXH_CARTILAGE_V2::ILM5670577", "UCLA_BXDBXH_CARTILAGE_V2::ILM2070121", "UCLA_BXDBXH_CARTILAGE_V2::ILM103990541", "UCLA_BXDBXH_CARTILAGE_V2::ILM1190722", "UCLA_BXDBXH_CARTILAGE_V2::ILM6590722", "UCLA_BXDBXH_CARTILAGE_V2::ILM4200064", "UCLA_BXDBXH_CARTILAGE_V2::ILM3140463" ] }' which should respond with a json response containing the raw binary string for the png format and possibly another for the svg format.
2021-09-16add initial endpoint for wgcnaAlexander Kabui
2021-09-03Add covarstruct as a possible keyword for the rqtl_wrapper.R script (not ↵zsloan
integrated into the script yet though)
2021-08-18Fix obvious linting errorsMuriithi Frederick Muriuki
* Fix linting errors that do not change the function of the code.
2021-08-17Fix obvious linting errorsMuriithi Frederick Muriuki
* Fix linting errors that do not change the function of the code.
2021-08-11use normal function for correlation (#34)Alexander Kabui
* use normal function for correlation + rename functions * update test for sample correlation * use normal function for tissue correlation + rename functions
2021-08-07Add gunicorn support for productionPjotr Prins
2021-06-18Fixed pylint issueszsloan
2021-06-18Fixed issue where all bool kwargs were always being passed to ↵zsloan
generate_rqtl_cmd and also made code check if output file already exists (so caching works)
2021-05-25Include code that processes rqtl output files and returns actual results ↵zsloan
instead of just the output filename
2021-05-18Account for boolean kwargs in compute() by storing them in a list, since ↵zsloan
they don't have corresponding values
2021-05-18Fixed variety of issues detected by pylintzsloan
2021-05-17Read in kwargs fromrequest and pass the command, tmpdir, and kwargs to ↵zsloan
generate_rqtl_cmd which returns the actual command and output path
2021-05-17Add rqtl.py for rqtl endpointszsloan
2021-05-13Add end-point for running an rQTL programBonfaceKilz
* gn3/api/general.py (run_r_qtl): New function. * gn3/settings.py: New variable.
2021-05-13Rename file_utils to fs_helpersBonfaceKilz
Generally avoid naming things with a "utils" prefix/ suffix since it encourages contributors to dump any new functions there; and over time, as the code grows, things get messy...
generated by cgit v1.2.3 (git 2.44.0) at 2024-11-30 03:00:10 +0000