aboutsummaryrefslogtreecommitdiff
path: root/gn3/db
AgeCommit message (Collapse)Author
2022-01-10Rework database functions to fetch multiple itemsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * In an attempt to optimise the performance of the partial correlations feature, this commit reworks some database access functions to fetch multiple items from the database, per query, unlike their original forms which would fetch a single item per query. This reduces queries to the database, and should hopefully improve the responsiveness of the partial correlations feature.
2022-01-05Merge branch 'main' into fix/check-for-duplicates-before-deletions-or-insertionsBonfaceKilz
2022-01-04traits.py: Don't use "value" to check if a record existsBonfaceKilz
The problem with using the "value" record is that it's a floating point number. See <https://www.bonfacemunyoki.com/post/2021-10-21-comparing-floating-point-numbers/> on why comparing floating point numbers can be an issue.
2022-01-04traits.py: Return early during an insert if the give record existsBonfaceKilz
Sometimes, a user will try to insert data twice, on in some instances, 2 different users will attempt the same inserts of the same records separately. In such cases, ignore the insert, and return early.
2022-01-04traits.py Delete a record only if it existsBonfaceKilz
In the case when the user tries to delete the same data twice, prior to this commit, an error was being generated. This commit remedies this by checking if a record exists prior to deleting it.
2021-12-23db: Assist mypy with explicit type declaration.Arun Isaac
* gn3/db/datasets.py (dataset_metadata): Assist mypy by explicitly declaring type of return dictionary.
2021-12-23db: Fix sparql_query return type.Arun Isaac
* gn3/db/datasets.py: Import List from typing. (sparql_query): Set return type to List[Dict[str, Any]].
2021-12-23db: Fix wrong continued indentation.Arun Isaac
* gn3/db/datasets.py (dataset_metadata): Fix wrong continued indentation.
2021-12-14linting: Fix obvious linting issuesFrederick Muriuki Muriithi
2021-12-14Fix bug: expects int, gets object - Fixed.Frederick Muriuki Muriithi
2021-12-14mypy: ignore some imports and errorsFrederick Muriuki Muriithi
* Ignore some missing library stubs * Ignore some typing errors * Fix obvious typing errors
2021-12-14Set explicit formatting for sequences in queriesFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * the mysqlclient is sometimes finicky with the format specifiers, especially in the `WHERE ... IN ...` clauses where a sequence of values is provided. This commit explicitly sets the format specifiers for such clauses so as to avoid sporadic failures due to differences in the form of data from one query to the next.
2021-12-13Fix errors with queryFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Use the correct format string paradigm: put f before opening quote. * Provide missing space just before 'FROM' clause
2021-12-10Fix parametersFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Provide parameters as a dict of items (which works for MySQL), rather than as keywork arguments (as works in PostgreSQL).
2021-12-09Prevent error on no result. Fix indexingFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * If the dataset name is not found, don't cause an exception, instead, return the provided search name. * Use the correct inner object
2021-12-09Provide clearer error messageFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * To help with debugging error conditions, provide the original name to help with figuring out why the error was raised
2021-12-06Merge branch 'main' of github.com:genenetwork/genenetwork3Frederick Muriuki Muriithi
2021-12-02Implement dataset metadata API endpoint.Arun Isaac
* guix.scm: Import (gnu packages rdf). (genenetwork3)[propagated-inputs]: Add python-sparqlwrapper. * gn3/settings.py (SPARQL_ENDPOINT): New variable. * gn3/api/general.py: Import datasets from gn3.db. (dataset_metadata): New API endpoint. * gn3/db/datasets.py: Import re, Template from string, Dict and Optional from typing, JSON and SPARQLWrapper from SPARQLWrapper, SPARQL_ENDPOINT from gn3.settings. (sparql_query, dataset_metadata): New functions.
2021-11-29Merge branch 'main' into partial-correlationsMuriithi Frederick Muriuki
2021-11-29Fix linting errorsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
2021-11-29Provide entry-point function for the partial correlationsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Provide the entry-point function to the partial correlation feature. This is the function that ochestrates the fetching of the data, and processing it for output by the API endpoint (to be implemented).
2021-11-29Fix bugs in dataFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Process the db_name and db_type values. * Return data correctly
2021-11-29Update return typeFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Return the complete filename when found, or the boolean value False, when it is not found.
2021-11-29Return only valuesFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Return the values from the database, not the tuples.
2021-11-29Update typing notations on functionsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
2021-11-29Fix query parametrisationFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Pass parameters to the query the way the MySQL driver expects.
2021-11-29Retrieve the species name given the groupFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Migrate the `web.webqtl.dbFunction.webqtlDatabaseFunction.retrieveSpecies` in GeneNetwork1 to `gn3.db.species.species_name` in GeneNetwork3 to enable the retrieval of the species name, given the group name (formerly RISet).
2021-11-29db: traits: Support additions and deletions from csv fileBonfaceKilz
2021-11-29db: traits: Remove trailing ".0" in int valuesBonfaceKilz
2021-11-29db: traits: Remove "\n\n" when generating csv fileBonfaceKilz
In excel, "\n\n" is replaced with ",,,," during upload.
2021-11-25db: traits: Support additions and deletions from csv fileBonfaceKilz
2021-11-24db: traits: Remove trailing ".0" in int valuesBonfaceKilz
2021-11-23db: traits: Remove "\n\n" when generating csv fileBonfaceKilz
In excel, "\n\n" is replaced with ",,,," during upload.
2021-11-23Fix a myriad of linting errorsFrederick Muriuki Muriithi
* Fix linting errors like: - Unused variables - Undeclared variable errors (mostly caused by typos, and wrong names) - Missing documentation strings for functions etc.
2021-11-23Update documentation for functionsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Document functions for posterity.
2021-11-22Add test to query buildersFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Add some tests for the query builders to ensure that the queries are built up correctly.
2021-11-22Migrate `web.webqtl.correlation.CorrelationPage.fetchAllDatabaseData` functionFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Migrate the `web.webqtl.correlation.CorrelationPage.fetchAllDatabaseData` function from GN1 to GN3.
2021-11-22Make the DB connection argument the firstFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * To make the code more composable down the line, make the database connection argument the first argument for functions that access the database, since they will always require the connection.
2021-11-04Add some condition checking functionsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Add the `check_for_literature_info` and `check_symbol_for_tissue_correlation` functions to check for the presence of specific data.
2021-11-04Fix linting and typing errorsFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
2021-11-04Rework sorting: remove `compare_tissue_correlation_absolute_values`Frederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/db/correlations.py: Remove the `compare_tissue_correlation_absolute_values` function which is no longer needed.
2021-11-04Complete `build_temporary_tissue_correlations_table`Frederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/computations/partial_correlations.py: Remove comments after updating usage of the function at call point * gn3/db/correlations.py: Complete the implementation of the `build_temporary_tissue_correlations_table` function
2021-11-04Move `correlations_of_all_tissue_traits`Frederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * gn3/computations/partial_correlations.py: new function (`correlations_of_all_tissue_traits`). * gn3/db/correlations.py: delete function (`correlations_of_all_tissue_traits`). Move the function to `gn3.computations.partial_correlations` module and comment out the db-access code. Rework it to receive, as arguments, the data it previously fetched from the database, and add comments on future rework to get the function working again.
2021-11-04Add missing commaFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
2021-11-04Move the function to computations moduleFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * The function `batch_computed_tissue_correlation` is a pure computations function with no expressions accessing the database, as far as I can tell, therefore, this commit moves the function over to the gn3.computations.partial_correlations module that holds the pure computation functions.
2021-11-04Remove if clauses: replace with dictFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Remove the if clauses to simplify the code flow: use a dictionary of queries and select the appropriate query from the dictionary instead.
2021-11-04Stub out `batch_computed_tissue_correlation` functionFrederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Stub out `batch_computed_tissue_correlation` function to be used in implementing the function down the line.
2021-11-04Complete `correlations_of_all_tissue_traits`Frederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Complete the implementation of the `correlations_of_all_tissue_traits` function by providing a call to a non-implemented function.
2021-11-04Implement `fetch_gene_symbol_tissue_value_dict_for_trait`Frederick Muriuki Muriithi
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi * Implement `fetch_gene_symbol_tissue_value_dict_for_trait` function which is a migration of the `web.webqtl.correlation.correlationFunction.getGeneSymbolTissueValueDictForTrait` function in GeneNetwork1.
2021-10-25Start implementation of `fetch_tissue_correlations` and dependenciesFrederick Muriuki Muriithi
* compare_tissue_correlation_absolute_values: New function. Complete. Used for sorting of tissue correlation values * fetch_symbol_value_pair_dict: New function. Complete. Maps gene symbols to tissue expression data * fetch_gene_symbol_tissue_value_dict: New function. Complete. Wrapper for `gn3.db.correlations.fetch_symbol_value_pair_dict` function * fetch_tissue_probeset_xref_info: New function. Complete. Retrieves the Probeset XRef information for tissues from the database. * correlations_of_all_tissue_traits: Stub. Dependencies not completed yet. * build_temporary_tissue_correlations_table: Stub. Dependencies not completed yet. * fetch_tissue_correlations: New function. Incomplete. This function calls (a) stub(s) function(s) which is/are under development still.
generated by cgit v1.2.3 (git 2.44.0) at 2024-10-19 16:44:54 +0000