aboutsummaryrefslogtreecommitdiff
AgeCommit message (Expand)Author
2021-08-07Add gunicorn support for productionPjotr Prins
2021-08-07Update info on running guix containerPjotr Prins
2021-08-05db: traits: Return unique values when fetching sample dataBonfaceKilz
2021-08-05Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_decom...Muriithi Frederick Muriuki
2021-08-05Reorganise the database code...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Reorganise the code to separate the datasets from the traits, and to more closely conform to the same flow as that in GN1 Muriithi Frederick Muriuki
2021-08-05Build up trait_name items from full name...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * The full name of the traits from search contains multiple parts to it, and as such, we use it to retrieve the appropriate data and set it up in the final trait_info dictionary that is produced. Muriithi Frederick Muriuki
2021-08-04Fix issues caught by pylint...* gn3/computations/slink.py: remove unused imports * gn3/db/traits.py: remove unnecessary `else` clauses * tests/unit/db/test_traits.py: add docstrings for functions Muriithi Frederick Muriuki
2021-08-04Retrieve the RISet and RISet ID values...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Retrieve the RISet and RISet ID values from the database. Muriithi Frederick Muriuki
2021-08-04Add tests for post-processing functions...Issues: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add missing tests for some post-processing functions Muriithi Frederick Muriuki
2021-08-04sql: Add documentation extracted from within the database....It turns out there is documentation of the database within the database itself. I have dumped that it into an org file. * sql/schema-from-in-db-documentation.org: New file. Arun Isaac
2021-08-04Update tests...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Update the tests to deal with changes in the code. Muriithi Frederick Muriuki
2021-08-04Avoid string interpolation: use prepared statement...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Following Arun's comment at https://github.com/genenetwork/genenetwork3/pull/31#issuecomment-890915813 this commit eliminates string interpolation, and adds a map of tables for the various types of traits dataset names Muriithi Frederick Muriuki
2021-08-04Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_decom...Muriithi Frederick Muriuki
2021-08-03guix.scm: Add csvdiff as a propagated inputBonfaceKilz
2021-08-02sql: Document the table Strain....* sql/schema.org (Strain): New section. Arun Isaac
2021-07-31sql: Add schema description....* sql/schema.org: New file. Arun Isaac
2021-07-30Rework db functions to enable postprocessing...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Rework the database functions to return a dict of key-value pairs, which eases the postprocessing of the trait information. The postprocessing is mainly to try an maintain data compatibility with the code that is at the following locations: https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/base/webqtlTrait.py https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/base/webqtlDataset.py https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/Heatmap.py This was mainly a proof-of-concept, and the functions do not have testing added for them: there is therefore need to add testing for the new functions, and probably even rework them if they are found to be complicated. Muriithi Frederick Muriuki
2021-07-30Remove extra space...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Remove extra space that was causing test to fail. Muriithi Frederick Muriuki
2021-07-30Add module for common utilities...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/function_helpers.py: new file Provides a new module to hold common programming utilities that are generic enough that they will find use across the entire application. The first utility function provided in this commit is the `compose` function, whose purpose, as indicated by its name, is to take a number of functions and compose them into a single function, which when called, will return the same result that would have been got had the user called the functions in a chain from right to left. Muriithi Frederick Muriuki
2021-07-30Return dict from query functions...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: return dicts rather than tuples/list * tests/unit/db/test_traits.py: Update tests Return dicts with the key-value pairs set up so as to ease with the data manipulation down the pipeline. This is also useful to help with the retrieval of all other extra information that was left out in the first iteration. This commit also updates the tests by ensuring they expect dicts rather than tuples. Muriithi Frederick Muriuki
2021-07-30Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_decom......Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi Fix merge conflicts in: * gn3/db/traits.py * tests/unit/db/test_traits.py Muriithi Frederick Muriuki
2021-07-29Merge pull request #30 from genenetwork/Feature/Update-db-from-csv-data...Feature/update db from csv dataBonfaceKilz
2021-07-29Merge branch 'main' into Feature/Update-db-from-csv-dataBonfaceKilz
2021-07-29Delete "update_raw" and it's test-casesBonfaceKilz
2021-07-29Add method for updating values from a sample dataset...* gn3/db/traits.py (update_sample_data): New function. * tests/unit/db/test_traits.py: New test cases for ^^. BonfaceKilz
2021-07-29Add partial type annotations for slink module...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add some type annotations for the `nearest` function. * Leave some comments regarding the issues experienced when trying to add some typing annotations to the function to help with future endeavours of the same. Muriithi Frederick Muriuki
2021-07-29Add type annotations to the function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add some type annotations to the functions to reduce the chances of bugs creeping into the code. Muriithi Frederick Muriuki
2021-07-29Retrieve trait information...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: add functions to retrieve traits information * tests/unit/db/test_traits.py: add tests for new function Add functions to retrieve traits information as is done in genenetwork1 https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/base/webqtlTrait.py#L397-L456 At this point, the data retrieval functions are probably incomplete, as there is more of the `retrieveInfo` function in GN1 that has not been considered as of this commit. Muriithi Frederick Muriuki
2021-07-29Make name retrieval more general...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: make function more general * tests/unit/db/test_traits.py: parametrize the tests Make the name retrieval more general for the different types of traits by changing the column specification and table as appropriate. Muriithi Frederick Muriuki
2021-07-29Retrieve 'ProbeSet' trait name...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: new function (retrieve_probeset_trait_name) * tests/unit/db/test_traits.py: test(s) for new function Add a function to retrieve the name of a 'ProbeSet' trait in a manner similar to genenetwork1's retrieval of the same, as implemented here https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/base/webqtlDataset.py#L140-154 Unlike in genenetwork1, we do not mutate an object, instead, we return the values as retrieved from the database, and the caller will deal with the returned values as appropriate. Muriithi Frederick Muriuki
2021-07-29Add partial type annotations for slink module...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add some type annotations for the `nearest` function. * Leave some comments regarding the issues experienced when trying to add some typing annotations to the function to help with future endeavours of the same. Muriithi Frederick Muriuki
2021-07-29Add type annotations to the function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add some type annotations to the functions to reduce the chances of bugs creeping into the code. Muriithi Frederick Muriuki
2021-07-29db: traits: Remove publishdata columnBonfaceKilz
2021-07-28Retrieve trait information...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: add functions to retrieve traits information * tests/unit/db/test_traits.py: add tests for new function Add functions to retrieve traits information as is done in genenetwork1 https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/base/webqtlTrait.py#L397-L456 At this point, the data retrieval functions are probably incomplete, as there is more of the `retrieveInfo` function in GN1 that has not been considered as of this commit. Muriithi Frederick Muriuki
2021-07-28Make name retrieval more general...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: make function more general * tests/unit/db/test_traits.py: parametrize the tests Make the name retrieval more general for the different types of traits by changing the column specification and table as appropriate. Muriithi Frederick Muriuki
2021-07-28Retrieve 'ProbeSet' trait name...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/traits.py: new function (retrieve_probeset_trait_name) * tests/unit/db/test_traits.py: test(s) for new function Add a function to retrieve the name of a 'ProbeSet' trait in a manner similar to genenetwork1's retrieval of the same, as implemented here https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/base/webqtlDataset.py#L140-154 Unlike in genenetwork1, we do not mutate an object, instead, we return the values as retrieved from the database, and the caller will deal with the returned values as appropriate. Muriithi Frederick Muriuki
2021-07-26tests: test_db: Add test case for "update_raw"BonfaceKilz
2021-07-26gn3: db: Create a raw update query...* gn3/db/__init__.py (update_raw): New function. BonfaceKilz
2021-07-26db: traits: Fetch sample_data from a trait in csv formBonfaceKilz
2021-07-26Merge branch 'main' of github.com:genenetwork/genenetwork3Muriithi Frederick Muriuki
2021-07-26Fix issues caught by pylint...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Fix a myriad of issues caught by pylint to ensure the code passes all tests. Muriithi Frederick Muriuki
2021-07-26db: traits: Remove unused functionsBonfaceKilz
2021-07-26Check if corr_coefficient is NaN, since apparently it's stored as NaN instead...zsloan
2021-07-23sql: metadata_audit: Make char-set encoding explicit(utf8mb4)...See: https://www.eversql.com/mysql-utf8-vs-utf8mb4-whats-the-difference-between-utf8-and-utf8mb4/ BonfaceKilz
2021-07-23Add data examples for `slink`. Implement function....Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/slink.py: Copy the function, mostly verbatim from genenetwork1. See: https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/slink.py#L107-L138 * tests/unit/computations/test_slink.py: Add a test with some example data to test that the implementation gives the same results as that in genenetwork1 Muriithi Frederick Muriuki
2021-07-23Add more test data...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi Muriithi Frederick Muriuki
2021-07-23Iterate through all valid pairs...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/slink.py: Fix the iteration construct. Given two lists of member coordinates, such as [0, 1] and [3, 5], the initial code would iterate over the pairs [0, 3] and [1, 5]. This commit fixes the iteration constructs such that the new code iterates over the pairs [0, 3], [0, 5], [1, 3] and [1, 5]. Muriithi Frederick Muriuki
2021-07-23Extract function to flatten list of lists...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/slink.py: Extract the `__flatten_list_of_lists` function since it is used in more than one place. Muriithi Frederick Muriuki
2021-07-23Fix issue caught in `nearest` while testing `slink`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * While running tests for slink, to try and understand what it is doing in order to write the appropriate tests for it, an issue arose that pointed a blindspot in the former understanding of now `nearest` should work. This commit fixes the issue found in both the expected data, and the code. Muriithi Frederick Muriuki
2021-07-23New function (`slink`): return [] on exception...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/slink.py: Add minimum code to pass new test * tests/unit/computations/test_slink.py: new test Add test to ensure that the new `slink` function return an empty list in case and exception is raised. Add the new `slink` function with minimum amount of code needed to pass the test. Muriithi Frederick Muriuki