aboutsummaryrefslogtreecommitdiff
path: root/gn3/db
AgeCommit message (Expand)Author
2021-11-29db: traits: Support additions and deletions from csv fileBonfaceKilz
2021-11-29db: traits: Remove trailing ".0" in int valuesBonfaceKilz
2021-11-29db: traits: Remove "\n\n" when generating csv file...In excel, "\n\n" is replaced with ",,,," during upload. BonfaceKilz
2021-11-23Fix a myriad of linting errors...* Fix linting errors like: - Unused variables - Undeclared variable errors (mostly caused by typos, and wrong names) - Missing documentation strings for functions etc. Frederick Muriuki Muriithi
2021-11-23Update documentation for functions...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Document functions for posterity. Frederick Muriuki Muriithi
2021-11-22Add test to query builders...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Add some tests for the query builders to ensure that the queries are built up correctly. Frederick Muriuki Muriithi
2021-11-22Migrate `web.webqtl.correlation.CorrelationPage.fetchAllDatabaseData` function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Migrate the `web.webqtl.correlation.CorrelationPage.fetchAllDatabaseData` function from GN1 to GN3. Frederick Muriuki Muriithi
2021-11-22Make the DB connection argument the first...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * To make the code more composable down the line, make the database connection argument the first argument for functions that access the database, since they will always require the connection. Frederick Muriuki Muriithi
2021-11-04Add some condition checking functions...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Add the `check_for_literature_info` and `check_symbol_for_tissue_correlation` functions to check for the presence of specific data. Frederick Muriuki Muriithi
2021-11-04Fix linting and typing errors...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi Frederick Muriuki Muriithi
2021-11-04Rework sorting: remove `compare_tissue_correlation_absolute_values`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/db/correlations.py: Remove the `compare_tissue_correlation_absolute_values` function which is no longer needed. Frederick Muriuki Muriithi
2021-11-04Complete `build_temporary_tissue_correlations_table`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/computations/partial_correlations.py: Remove comments after updating usage of the function at call point * gn3/db/correlations.py: Complete the implementation of the `build_temporary_tissue_correlations_table` function Frederick Muriuki Muriithi
2021-11-04Move `correlations_of_all_tissue_traits`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/computations/partial_correlations.py: new function (`correlations_of_all_tissue_traits`). * gn3/db/correlations.py: delete function (`correlations_of_all_tissue_traits`). Move the function to `gn3.computations.partial_correlations` module and comment out the db-access code. Rework it to receive, as arguments, the data it previously fetched from the database, and add comments on future rework to get the function working again. Frederick Muriuki Muriithi
2021-11-04Add missing comma...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi Frederick Muriuki Muriithi
2021-11-04Move the function to computations module...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * The function `batch_computed_tissue_correlation` is a pure computations function with no expressions accessing the database, as far as I can tell, therefore, this commit moves the function over to the gn3.computations.partial_correlations module that holds the pure computation functions. Frederick Muriuki Muriithi
2021-11-04Remove if clauses: replace with dict...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Remove the if clauses to simplify the code flow: use a dictionary of queries and select the appropriate query from the dictionary instead. Frederick Muriuki Muriithi
2021-11-04Stub out `batch_computed_tissue_correlation` function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Stub out `batch_computed_tissue_correlation` function to be used in implementing the function down the line. Frederick Muriuki Muriithi
2021-11-04Complete `correlations_of_all_tissue_traits`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Complete the implementation of the `correlations_of_all_tissue_traits` function by providing a call to a non-implemented function. Frederick Muriuki Muriithi
2021-11-04Implement `fetch_gene_symbol_tissue_value_dict_for_trait`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Implement `fetch_gene_symbol_tissue_value_dict_for_trait` function which is a migration of the `web.webqtl.correlation.correlationFunction.getGeneSymbolTissueValueDictForTrait` function in GeneNetwork1. Frederick Muriuki Muriithi
2021-10-25Start implementation of `fetch_tissue_correlations` and dependencies...* compare_tissue_correlation_absolute_values: New function. Complete. Used for sorting of tissue correlation values * fetch_symbol_value_pair_dict: New function. Complete. Maps gene symbols to tissue expression data * fetch_gene_symbol_tissue_value_dict: New function. Complete. Wrapper for `gn3.db.correlations.fetch_symbol_value_pair_dict` function * fetch_tissue_probeset_xref_info: New function. Complete. Retrieves the Probeset XRef information for tissues from the database. * correlations_of_all_tissue_traits: Stub. Dependencies not completed yet. * build_temporary_tissue_correlations_table: Stub. Dependencies not completed yet. * fetch_tissue_correlations: New function. Incomplete. This function calls (a) stub(s) function(s) which is/are under development still. Frederick Muriuki Muriithi
2021-10-25Implement `fetch_literature_correlations` and depedencies...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Migrate: * `web.webqtl.correlation.CorrelationPage.getTempLiteratureTable` * `web.webqtl.correlation.CorrelationPage.fetchLitCorrelations` from GeneNetwork1. The first function creates and populates a temporary table with the literature correlations data. The second function uses the data in the newly created temporary table to link the trait with the correlation value. Frederick Muriuki Muriithi
2021-10-25Implement `get_filename` for correlations...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Implement `get_filename` for the correlations, to be used to determine whether to do fast or normal correlations. This is a migration of the `web.webqtl.correlation.CorrelationPage.getFileName` function in GN1 Frederick Muriuki Muriithi
2021-10-21Implement `translate_to_mouse_gene_id` function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Migrate the `web.webqtl.correlation/CorrelationPage.translateToMouseGeneID` function in GN1 to GN3. This is a function that retrieves data from the database, and therefore uses a system outside of our code, therefore, the function does not have a corresponding unit test. This kind of function will probably need to be tested at the integration or system tests level, where we test that our code interacts correcly with any and all external systems that it should. Frederick Muriuki Muriithi
2021-10-18Move `export_informative` function to `gn3.db.traits` module...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/db/traits.py: Move `export_informative` function here * gn3/partial_correlations.py: Remove `export_informative` function * tests/unit/db/test_traits.py: Move `export_informative` function tests here * tests/unit/test_partial_correlations.py: Remove `export_informative` function tests The `export_informative` function relates more to the traits than to the partial correlations, and could find use in more than just the partial correlations stuff. This commit moves the function to the more traits-specific `gn3.db.traits` module. Frederick Muriuki Muriithi
2021-10-18Move 'export_trait_data' to 'gn3.db.traits' module...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/db/traits.py: Move function `export_trait_data` here * gn3/heatmaps.py: Remove function `export_trait_data` * tests/unit/db/test_traits.py: Move function `export_trait_data` tests here * tests/unit/test_heatmaps.py: Remove function `export_trait_data` here Function `export_trait_data` more closely corresponds to the traits and is used in more than just the `gn3.heatmaps` module. This commit moves the relevant code over to the `gn3.db.traits` module and also moves the tests to the corresponding tests modules. Frederick Muriuki Muriithi
2021-09-27Update terminology: `riset` to `group`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Update terminology to use the appropriate domain terminology according to Zachary's direction at https://github.com/genenetwork/genenetwork3/pull/37#issuecomment-926041744 Frederick Muriuki Muriithi
2021-09-27Update terminology: `strain` to `sample`...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Update the terminology used: use `sample` in place of `strain` according to Zachary's direction at https://github.com/genenetwork/genenetwork3/pull/37#issuecomment-926043306 Frederick Muriuki Muriithi
2021-09-22Fix more pylint errorsFrederick Muriuki Muriithi
2021-09-22Fix typing issues...* Ignore some errors * Update typing definitions for some portions of code * Add missing imports Frederick Muriuki Muriithi
2021-09-22Fix pylint errors...* Add missing function and module docstrings * Remove unused imports * Fix import order * Rework some code sections to fix issues * Disable some pylint errors. Frederick Muriuki Muriithi
2021-09-17Fix a number of linting issues...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi Frederick Muriuki Muriithi
2021-09-01Fix linting and typing issues...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi Muriithi Frederick Muriuki
2021-09-01Built top-level genotype file parsing function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: parse genotype files * tests/unit/db/test_genotypes.py: test parsing is correct Add the overall genotype files parsing function and tests to check that the parsing works as expected. Muriithi Frederick Muriuki
2021-09-01Parse data lines into markers...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: parse data lines in file to genetic markers. * tests/unit/db/test_genotypes.py: test that parsing works. Add some tests to check that the parsing of the markers works as expected, and add the code to actually parse the markers. Muriithi Frederick Muriuki
2021-09-01Parse the genotype file's data header...* gn3/db/genotypes.py: parse data header * tests/unit/db/test_genotypes.py: check that header's parse works correctly. Add tests to check that the parser works as expected. Add code to implement the parsing and pass the tests. Muriithi Frederick Muriuki
2021-09-01Implement parsing of genotype labels...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: parse genotype labels * tests/unit/db/test_genotypes.py: test that genotype labels are parsed correctly As part of parsing the genotype files into usable python data structures, this commit adds a function to parse the label lines (beginning with "@") into the appropriate values. Muriithi Frederick Muriuki
2021-08-31Fix linting errors, minor bugs and reorganise code...* Fix some linting errors and some minor bugs caught by the linter. Move the `random_string` function to separate module for use in multiple places in the code. Muriithi Frederick Muriuki
2021-08-31Update `heatmap_data` function: remove extraneous data...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/heatmap.py: update function * gn3/db/traits.py: new function Remove extraneous data and arguments from the function. - Load the genotype file - Generate traits file - Provide both raw traits data, and exported traits data in return Muriithi Frederick Muriuki
2021-08-31Provide utilities for genotype files...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: New module * gn3/settings.py: Add new configuration variable * qtlfilesexport.py: Test out new code Add a module containing functions dealing with the genotype files. Add a configuration variable to point to the location of the genotype files. Muriithi Frederick Muriuki
2021-08-17Provide top-level `riset` key-value pair...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Provide the expected, top-level `riset` key-value pair and eliminate the redundant key-value pair. Muriithi Frederick Muriuki
2021-08-17Fix errors: add in missing parenthesis...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Call the `cursor.fetchone()` function to get results. Without the parenthesis, the code was trying to use the function itself as the results, which was a bug, and would lead to failure. Muriithi Frederick Muriuki
2021-08-09Set up the trait dataset type correctly...* gn3/db/traits.py: setup `trait_dataset_type` * tests/unit/db/test_traits.py: fix tests The type ('Temp', 'Geno', 'Publish', and 'ProbeSet') relate to a trait's dataset, and not the trait itself. This commit updates the code to take this into consideration. The dataset type is also set up from a trait's full name, therefore this commit removes the `trait_type` argument from the `retrieve_trait_info` function. Muriithi Frederick Muriuki
2021-08-09Retrieve the trait data...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add functions to retrieve the `value`, `variance`, and `ndata` values for any given trait. Muriithi Frederick Muriuki
2021-08-09Add missing arguments. Fix typo....Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Fix minor bugs in the code. Muriithi Frederick Muriuki
2021-08-09Fix linting errors...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add module, class and function docstrings * Deactivate some irrelevant pylint errors * Fix indentations and line-lengths Muriithi Frederick Muriuki
2021-08-08Only load extra data if the traits have basic info...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Only load the extra trait data if the basic trait information is found. Muriithi Frederick Muriuki
2021-08-08Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_decom...Muriithi Frederick Muriuki
2021-08-05db: traits: Return unique values when fetching sample dataBonfaceKilz
2021-08-05Reorganise the database code...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Reorganise the code to separate the datasets from the traits, and to more closely conform to the same flow as that in GN1 Muriithi Frederick Muriuki
2021-08-05Build up trait_name items from full name...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * The full name of the traits from search contains multiple parts to it, and as such, we use it to retrieve the appropriate data and set it up in the final trait_info dictionary that is produced. Muriithi Frederick Muriuki