aboutsummaryrefslogtreecommitdiff
path: root/wqflask/base
AgeCommit message (Collapse)Author
2022-09-22Provide database cursor as argument for TheSpecies and ChromosomesFrederick Muriuki Muriithi
Provide the database cursor as an argument to the methods of the two classes: * wqflask.base.species.TheSpecies * wqflask.base.species.Chromosomes Also update dependent code to initialise and pass the cursor where these classes are used.
2022-09-21Added Markers/HumanMarkers import to datasetgroup.py and removed it fromzsloan
__init__.py Previously it threw an error when it tried to use Markers/HumanMarkers in datasetgroup.py; since Markers/HumanMarkers doesn't seem to be used anywhere else, I think it's okay for the import to only be in datasetgroup.py
2022-09-21Refactor: pass redis and db connections as argumentFrederick Muriuki Muriithi
Pass the redis and database connection/cursors as arguments to the class methods, and do not retain a copy of the connections/cursors. This allows us to do the connection management in the context managers elsewhere - ideally, at the top-level. For now the context manager is within the `create_dataset` function, but this should be moved out to a higher level once the lower levels are verified to be working as expected.
2022-09-20Remove global `Dataset_Getter` variableFrederick Muriuki Muriithi
Since the `__post_init__` method in the 'DatasetType' class checks whether data is in redis before hitting the database, it does not matter whether there is a global object of the type, as long as we make sure to call the object correctly. This commit makes that happen.
2022-09-20Revert "Use member variable `self.conn` to avoid test failure"Frederick Muriuki Muriithi
This reverts commit 4fafdfd8487ddf61105b2a5ca92534c8014c1db4. While using `self.conn.cursor()` works in the tests, it will not work in the application since the point of initialisation could be very different from the point where the 'property' is called.
2022-09-20Use member variable `self.conn` to avoid test failureFrederick Muriuki Muriithi
Use the `self.conn` variable to prevent test from failing.
2022-09-20Decompose file into separate modulesrefactor-data_set_py-20220920Frederick Muriuki Muriithi
To ease future refactors on the code, decompose the file into a module with multiple modules that can be refactored semi-independently.
2022-09-12Fix query: Add comma to separate columns.Frederick Muriuki Muriithi
2022-09-12Fix query: remove wrongly-placed commaFrederick Muriuki Muriithi
2022-09-12Check db resultset before indexing into itFrederick Muriuki Muriithi
The assumption that the resultset will never be empty was leading to a lot of errors.
2022-09-09Fix bug in retrieve_other_nameszsloan
After a recent change, it always tries to unpack 6 values from the query, but it should only be unpacking 4 values if not type ProbeSet (so it was throwing an error for anything not ProbeSet)
2022-09-09tissue:fetch for query formattingAlexander_Kabui
2022-09-09Retain only a single import of the same moduleFrederick Muriuki Muriithi
2022-09-09Merge pull request #732 from BonfaceKilz/bug/fix-broken-correlationsMuriithi Frederick Muriuki
Bug/fix broken correlations
2022-09-09Fetch sample values correctlyMunyoki Kilyungi
* wqflask/base/data_set.py (MrnaAssayDataSet.retrieve_sample_data): Fix indentation.
2022-09-09Add missing importMunyoki Kilyungi
* wqflask/base/trait.py: Import utility.hmac
2022-09-09Import missing module.Frederick Muriuki Muriithi
2022-09-09Filter out Samples without valuesFrederick Muriuki Muriithi
2022-09-09Use `database_connection()`Frederick Muriuki Muriithi
Call the `database_connection()` function to get a connection to the database.
2022-09-09Replace `g.db.execute` with `cursor.execute`Frederick Muriuki Muriithi
2022-09-08Simplify how chromosomes are fetchedMunyoki Kilyungi
* wqflask/base/species.py: Import "Any" and "Union". (TheSpecies): Add type hints. Get rid of redundant "if ... else" statement. (IndChromosome.mb_length): Add type hints. (Chromosomes): Inject conn and add type hints.
2022-09-08Remove redundant cursor.fetchone()Munyoki Kilyungi
* wqflask/base/data_set.py (DataSet.retrieve_other_names): Call cursor.fetchone() only once.
2022-09-08Use "%s" in sql queryMunyoki Kilyungi
* wqflask/base/mrna_assay_tissue_data.py: Delete db_tools. (MrnaAssayTissueData.get_symbol_values_pairs): Re-format query. * wqflask/tests/unit/base/test_mrna_assay_tissue_data.py: (test_get_trait_symbol_and_tissue_values): Add test for above.
2022-09-08Inject database connection to mrna_assay_tissue_data classMunyoki Kilyungi
* wqflask/base/mrna_assay_tissue_data.py: Imports: Delete database_connection, escape, and database_connector. (MrnaAssayTissueData): Inject conn. Re-format queries. Rework 'if ... else' logic. Re-work how results are assigned to 'self.data[symbol]' - remove dot-notation. (MrnaAssayTissueData.get_symbol_values_pairs): Move box-comments to doc-string. Rework how results are assigned to 'symbol_values_dict' - remove dot-notation. * wqflask/tests/unit/base/test_mrna_assay_tissue_data.py (test_mrna_assay_tissue_data_initialisation): New test. * wqflask/wqflask/correlation/correlation_functions.py: Import database_connection. (get_trait_symbol_and_tissue_values): Inject conn object.
2022-09-08Fix sql queriesMunyoki Kilyungi
"%s" should only be used outside table names and column names otherwise a string literal will be inserted thereby leading to errors in the sql statements. * wqflask/base/data_set.py (geno_mrna_confidentiality): Use f-strings for table/columns/clause. * wqflask/base/trait.py (retrieve_trait_info): Ditto. * wqflask/wqflask/gsearch.py (GSearch.__init__): Ditto. * wqflask/wqflask/interval_analyst/GeneUtil.py (loadGenes): Ditto. * wqflask/wqflask/snp_browser/snp_browser.py (SnpBrowser.get_browser_results): Ditto.
2022-09-08Add missing "," in cursor.execute()Munyoki Kilyungi
* wqflask/base/data_set.py (DatasetGroup.get_mapping_methods): Fix failing query by adding a missing ",".
2022-09-08Remove intermediate variables that are only used onceMunyoki Kilyungi
* wqflask/base/data_set.py (DatasetGroup.__init__): Remove "query" variable. (DataSet.chunk_dataset): Fix indentation. * wqflask/wqflask/search_results.py (get_GO_symbols): Remove "this_term" variable. * wqflask/wqflask/views.py (search_page): Remove "the_search" variable.
2022-09-08Delete dead importsMunyoki Kilyungi
* wqflask/base/data_set.py: Remove un-used imports. * wqflask/base/trait.py: Ditto. wqflask/tests/wqflask/show_trait/test_show_trait.py: Ditto. * wqflask/wqflask/show_trait/show_trait.py: Ditto. * wqflask/wqflask/views: Ditto.
2022-09-08Replace g.db with database_connection() context managerMunyoki Kilyungi
* wqflask/base/data_set.py (DatasetType.set_dataset_key): Replace g.db with "database_connection()". (DatasetGroup.get_mapping_methods): Ditto. (DataSet.get_accession_id): Ditto. (DataSet.retrieve_other_names): Rework "if ... else" logic to get rid of "query" and "_vars" variables. Also, nest "try ... except" block within "database_connection". (DataSet.chunk_dataset): Replace g.db with "database_connection()". (DataSet.get_probeset_data): Ditto. (DataSet.get_trait_data): Ditto. (PhenotypeDataSet.get_trait_info): Ditto. (PhenotypeDataSet.retrieve_sample_data): Ditto. (GenotypeDataSet.retrieve_sample_data): Ditto. (MrnaAssayDataSet.get_trait_info): Ditto. (MrnaAssayDataSet.retrieve_sample_data): Ditto. (MrnaAssayDataSet.retrieve_genes): Ditto. (TempDataSet): Ditto. (geno_mrna_confidentiality): Ditto. (query_table_timestamp): Ditto. * wqflask/wqflask/api/router.py: Replace "flask.g" with database_connection import. (get_species_list): Replace g.db with "database_connection()". (get_species_info): Ditto. (get_groups_list): Ditto. (get_group_info): Ditto. (get_datasets_for_group): Ditto. (get_dataset_info): Ditto. (fetch_traits): Ditto. (all_sample_data): Ditto. (trait_sample_data): Ditto. (get_trait_info): Ditto. (get_dataset_trait_ids): Ditto. (get_samplelist): Ditto. (get_group_id_from_dataset): Ditto. (get_group_id): Ditto.
2022-09-08Delete logger importMunyoki Kilyungi
* wqflask/base/trait.py: Remove utility.logger
2022-09-08Replace assertion with Value Error for non-existent datasetMunyoki Kilyungi
* wqflask/base/trait.py (retrieve_trait_info): Wrap error message in ValueError.
2022-09-08Replace g.db with database_connection() context managerMunyoki Kilyungi
* wqflask/base/mrna_assay_tissue_data.py: Replace "flask.g" with database_connection. (MrnaAssayTissueData.__init__): Use database_connection. (MrnaAssayTissueData.get_symbol_values_pairs): Ditto. * wqflask/base/species.py: Replace "Flask.g" imports with "database_connection". (Chromosomes.chromosomes): Use database_connection. * wqflask/base/trait.py: Import database_connection. (retrieve_trait_info): Use database_connection. * wqflask/utility/authentication_tools.py: Replace "flask.g" with database_connection. (get_group_code): Use database_connection. * wqflask/utility/helper_functions.py: Replace "flask.g" with "database_connection". (get_species_groups): Use database_connection. * wqflask/wqflask/db_info.py: Replace "Flask" and "g" with "database_connection". (InfoPage.get_info): Use database_connection. * wqflask/wqflask/do_search.py (DoSearch.execute): Use database_connection(). * wqflask/wqflask/external_tools/send_to_geneweaver.py: Replace "Flask" and "g" import with database_connection. (test_chip): Use database_connection. * wqflask/wqflask/external_tools/send_to_webgestalt.py: Replace "Flask, g" imports with database_connection. (test_chip): Use database_connection. * wqflask/wqflask/gsearch.py: Replace "Flask" and "g" import with database_connection. (GSearch.__init__): Use database_connection. * wqflask/wqflask/interval_analyst/GeneUtil.py (loadGenes): Use database_connection(). * wqflask/wqflask/show_trait/SampleList.py: Replace "flask.g import" with database_connection. (SampleList.get_attributes): Use database_connection. (SampleList.get_extra_attribute_values): Ditto. * wqflask/wqflask/show_trait/show_trait.py: Replace "Flask" and "g" import with database_connection. (ShowTrait.__init__): Use database_connection. (ShowTrait.get_external_links): Ditto. (get_nearest_marker): Ditto.
2022-09-08Merge pull request #727 from Alexanderlacuna/feature/use-textfilesAlexander Kabui
use text files for Probeset
2022-09-07catch general errors this will recreate the fileAlexander_Kabui
2022-09-06Fix error with dataset menu query in data_set.pyzsloan
I'm only using g.db.execute here because using cursor is throwing an error and I don't want to change the way this query is currently written since I didn't write it (I believe it was initially written using g.db.execute and whoever wrote it just forgot to put the g.db.execute, since previously it just called "fetchall")
2022-09-06Merge branch 'testing' into feature/generalize_tableszsloan
2022-09-06Set up the self.* variables from query resultsFrederick Muriuki Muriithi
Set up the appropriate self.* variables from the results of running the appropriate query.
2022-09-06Use dataset name if dataset group name is not set.Frederick Muriuki Muriithi
2022-08-31Delete unused modulesMunyoki Kilyungi
* wqflask/base/data_set.py: Delete "menu_main" import. * wqflask/db/call.py: Delete it. * wqflask/db/gn_server.py: Ditto. * wqflask/wqflask/submit_bnw.py: Ditto.
2022-08-31Replace fetchall, fetchone, fetch1 with database_connectionMunyoki Kilyungi
* wqflask/base/data_set.py: Replace "db.call" import with "database_connection". (create_datasets_list): Use "database_connection" to fetch data. (DatasetGroup.__init__): Ditto. (DataSet.retrieve_other_names): Ditto. (PhenotypeDataSet.setup): Remove query escaping in string and format the string. (GenotypeDataSet.setup): Ditto. (MrnaAssayDataSet.setup): Ditto. * wqflask/db/webqtlDatabaseFunction.py: Remove db.call import. (retrieve_species): Use database_connection() to fetch data. (retrieve_species_id): Ditto.
2022-08-31Remove "with Bench ..." callsMunyoki Kilyungi
"with Bench" instruments how long a function takes and generates time reports on as INFO logs. This should be done on a developer server. Should the log level be low enough, this bench marks will generate a lot of noise. Instrumentation should be done during development. * wqflask/base/data_set.py (create_datasets_list): Remove "with Bench...". * wqflask/db/call.py (fetchone): Ditto. (fetchall): Ditto. (gn_server): Ditto. * wqflask/wqflask/gsearch.py (GSearch.__init__): Ditto. * wqflask/wqflask/marker_regression/display_mapping_results.py (DisplayMappingResults.__init__): Ditto. * wqflask/wqflask/marker_regression/run_mapping.py (RunMapping.__init__): Ditto. * wqflask/wqflask/update_search_results.py (GSearch.__init__): Ditto. * wqflask/wqflask/views.py (search_page): Ditto. (heatmap_page): Ditto. (mapping_results_page): Ditto.
2022-08-31Remove usage of "logger" and un-necessary comments wrt the sameMunyoki Kilyungi
Logging is used to introspect variables or notify the commencement of a given operation. Logging should only be used to log errors. Also, most of the logging is either "logger.debug" or "logger.info"; and this won't show up in production/testing since we need a logging level above "WARNING" for them to show up. * wqflask/base/data_set.py (create_datasets_list): Remove logger. (Markers.add_pvalues): Ditto. (DataSet.retrieve_other_names): Ditto. * wqflask/base/mrna_assay_tissue_data.py: Ditto. * wqflask/base/webqtlCaseData.py: Ditto. * wqflask/db/call.py (fetch1): Ditto. (gn_server): Ditto. * wqflask/db/gn_server.py: Ditto. * wqflask/maintenance/set_resource_defaults.py: Ditto. * wqflask/utility/Plot.py (find_outliers): Ditto. * wqflask/utility/gen_geno_ob.py: Ditto. * wqflask/utility/helper_functions.py: Ditto. * wqflask/utility/pillow_utils.py: Ditto. * wqflask/utility/redis_tools.py: Ditto. * wqflask/wqflask/api/gen_menu.py (get_groups): Ditto. * wqflask/wqflask/api/mapping.py: Ditto. * wqflask/wqflask/api/router.py (get_dataset_info): Ditto. * wqflask/wqflask/collect.py (report_change): Ditto. * wqflask/wqflask/correlation/corr_scatter_plot.py: Ditto. * wqflask/wqflask/ctl/ctl_analysis.py (CTL): Ditto. (CTL.__init__): Ditto. (CTL.run_analysis): Ditto. (CTL.process_results): Ditto. * wqflask/wqflask/db_info.py: Ditto. * wqflask/wqflask/do_search.py (DoSearch.execute): Ditto. (DoSearch.mescape): Ditto. (DoSearch.get_search): Ditto. (MrnaAssaySearch.run_combined): Ditto. (MrnaAssaySearch.run): Ditto. (PhenotypeSearch.run_combined): Ditto. (GenotypeSearch.get_where_clause): Ditto. (LrsSearch.get_where_clause): Ditto. (MeanSearch.run): Ditto. (RangeSearch.get_where_clause): Ditto. (PvalueSearch.run): Ditto. * wqflask/wqflask/docs.py: Ditto. * wqflask/wqflask/export_traits.py: Ditto. * wqflask/wqflask/external_tools/send_to_bnw.py: Ditto. * wqflask/wqflask/external_tools/send_to_geneweaver.py: Ditto. * wqflask/wqflask/external_tools/send_to_webgestalt.py: Ditto. * wqflask/wqflask/gsearch.py (GSearch.__init__): Ditto. * wqflask/wqflask/heatmap/heatmap.py: Ditto. * wqflask/wqflask/marker_regression/display_mapping_results.py (DisplayMappingResults): Ditto. * wqflask/wqflask/marker_regression/gemma_mapping.py: Ditto. * wqflask/wqflask/marker_regression/plink_mapping.py (run_plink): Ditto. * wqflask/wqflask/marker_regression/qtlreaper_mapping.py (run_reaper): Ditto. * wqflask/wqflask/marker_regression/rqtl_mapping.py: Ditto. * wqflask/wqflask/marker_regression/run_mapping.py (RunMapping.__init__): Ditto. * wqflask/wqflask/parser.py (parse): Ditto. * wqflask/wqflask/search_results.py (SearchResultPage.__init__): Ditto. * wqflask/wqflask/update_search_results.py (GSearch.__init__): Ditto. * wqflask/wqflask/user_login.py (send_email): Ditto. (logout): Ditto. (forgot_password_submit): Ditto. (password_reset): Ditto. (password_reset_step2): Ditto. (register): Ditto. * wqflask/wqflask/user_session.py (create_signed_cookie): Ditto.
2022-08-31add new environment variable:TEXTDIRAlexander_Kabui
2022-07-12Include genotypes for F1s/reference/non-reference strains when fetchingzsloan
sample data for genotype traits
2022-06-23Change jsonable in GeneralTrait so that it passes all necessary tablezsloan
information
2022-06-23Convert trait.view to string for conversion to JSONzsloan
It's originally a boolean, which causes an error when passed to the JS code as JSON
2022-06-13Check for chromosome of 'Un' and print location as 'Not available' in such caseszsloan
2022-04-14Add group code to loading pages when applicablezsloan
This adds the group code to phenotype traits on loading pages, and also sets the group code as an attribute of the dataset.group class.
2022-03-25Fix issues that prevented genotype traits from being added to collectionszsloan
Some of this was caused by heatmaps supporting code; that code should probably pass the traits differently than the way it does in the "trait_info_str" function
2022-02-21Fix incorrect dataset trait data cachingzsloan
Trait data caching wasn't working correctly because it didn't account for the samplelist, causing caching to work incorrect in any situation where the target dataset's samplelist wasn't the same as that of the trait being correlated against. Trait data is stored as a dictionary where the keys are trait IDs and values are *lists* of sample values. This means that the caching needs to account for the exact same set of samples; otherwise you'll end up with samples being mismatched (since "the third sample with a value" for one dataset's trait might not be the same as "the third sample with a value" for another dataset's trait). To fix this, I added the samplelist to the functions that generate and fetch the hash file. This will require more cache files, though, so this should probably be reexamined later to make the code work with only a single cache file for each dataset.
generated by cgit v1.2.3 (git 2.44.0) at 2024-11-23 18:27:03 +0000