aboutsummaryrefslogtreecommitdiff
path: root/wqflask/base
AgeCommit message (Collapse)Author
2022-10-12Use SQL to fetch trait metadata instead of gn-proxy, since it's dramatically ↵zsloan
faster
2022-10-12Include chr/mb values in jsonable traitzsloan
2022-10-04Add more information to exception to help in debuggingFrederick Muriuki Muriithi
2022-09-26Remove unused database_connection import from species.pyzsloan
2022-09-25Fix issue where the display name was used in the hmac for phenotype traitszsloan
2022-09-22Provide database cursor as argument for TheSpecies and ChromosomesFrederick Muriuki Muriithi
Provide the database cursor as an argument to the methods of the two classes: * wqflask.base.species.TheSpecies * wqflask.base.species.Chromosomes Also update dependent code to initialise and pass the cursor where these classes are used.
2022-09-21Added Markers/HumanMarkers import to datasetgroup.py and removed it fromzsloan
__init__.py Previously it threw an error when it tried to use Markers/HumanMarkers in datasetgroup.py; since Markers/HumanMarkers doesn't seem to be used anywhere else, I think it's okay for the import to only be in datasetgroup.py
2022-09-21Refactor: pass redis and db connections as argumentFrederick Muriuki Muriithi
Pass the redis and database connection/cursors as arguments to the class methods, and do not retain a copy of the connections/cursors. This allows us to do the connection management in the context managers elsewhere - ideally, at the top-level. For now the context manager is within the `create_dataset` function, but this should be moved out to a higher level once the lower levels are verified to be working as expected.
2022-09-20Remove global `Dataset_Getter` variableFrederick Muriuki Muriithi
Since the `__post_init__` method in the 'DatasetType' class checks whether data is in redis before hitting the database, it does not matter whether there is a global object of the type, as long as we make sure to call the object correctly. This commit makes that happen.
2022-09-20Revert "Use member variable `self.conn` to avoid test failure"Frederick Muriuki Muriithi
This reverts commit 4fafdfd8487ddf61105b2a5ca92534c8014c1db4. While using `self.conn.cursor()` works in the tests, it will not work in the application since the point of initialisation could be very different from the point where the 'property' is called.
2022-09-20Use member variable `self.conn` to avoid test failureFrederick Muriuki Muriithi
Use the `self.conn` variable to prevent test from failing.
2022-09-20Decompose file into separate modulesrefactor-data_set_py-20220920Frederick Muriuki Muriithi
To ease future refactors on the code, decompose the file into a module with multiple modules that can be refactored semi-independently.
2022-09-12Fix query: Add comma to separate columns.Frederick Muriuki Muriithi
2022-09-12Fix query: remove wrongly-placed commaFrederick Muriuki Muriithi
2022-09-12Check db resultset before indexing into itFrederick Muriuki Muriithi
The assumption that the resultset will never be empty was leading to a lot of errors.
2022-09-09Fix bug in retrieve_other_nameszsloan
After a recent change, it always tries to unpack 6 values from the query, but it should only be unpacking 4 values if not type ProbeSet (so it was throwing an error for anything not ProbeSet)
2022-09-09tissue:fetch for query formattingAlexander_Kabui
2022-09-09Retain only a single import of the same moduleFrederick Muriuki Muriithi
2022-09-09Merge pull request #732 from BonfaceKilz/bug/fix-broken-correlationsMuriithi Frederick Muriuki
Bug/fix broken correlations
2022-09-09Fetch sample values correctlyMunyoki Kilyungi
* wqflask/base/data_set.py (MrnaAssayDataSet.retrieve_sample_data): Fix indentation.
2022-09-09Add missing importMunyoki Kilyungi
* wqflask/base/trait.py: Import utility.hmac
2022-09-09Import missing module.Frederick Muriuki Muriithi
2022-09-09Filter out Samples without valuesFrederick Muriuki Muriithi
2022-09-09Use `database_connection()`Frederick Muriuki Muriithi
Call the `database_connection()` function to get a connection to the database.
2022-09-09Replace `g.db.execute` with `cursor.execute`Frederick Muriuki Muriithi
2022-09-08Simplify how chromosomes are fetchedMunyoki Kilyungi
* wqflask/base/species.py: Import "Any" and "Union". (TheSpecies): Add type hints. Get rid of redundant "if ... else" statement. (IndChromosome.mb_length): Add type hints. (Chromosomes): Inject conn and add type hints.
2022-09-08Remove redundant cursor.fetchone()Munyoki Kilyungi
* wqflask/base/data_set.py (DataSet.retrieve_other_names): Call cursor.fetchone() only once.
2022-09-08Use "%s" in sql queryMunyoki Kilyungi
* wqflask/base/mrna_assay_tissue_data.py: Delete db_tools. (MrnaAssayTissueData.get_symbol_values_pairs): Re-format query. * wqflask/tests/unit/base/test_mrna_assay_tissue_data.py: (test_get_trait_symbol_and_tissue_values): Add test for above.
2022-09-08Inject database connection to mrna_assay_tissue_data classMunyoki Kilyungi
* wqflask/base/mrna_assay_tissue_data.py: Imports: Delete database_connection, escape, and database_connector. (MrnaAssayTissueData): Inject conn. Re-format queries. Rework 'if ... else' logic. Re-work how results are assigned to 'self.data[symbol]' - remove dot-notation. (MrnaAssayTissueData.get_symbol_values_pairs): Move box-comments to doc-string. Rework how results are assigned to 'symbol_values_dict' - remove dot-notation. * wqflask/tests/unit/base/test_mrna_assay_tissue_data.py (test_mrna_assay_tissue_data_initialisation): New test. * wqflask/wqflask/correlation/correlation_functions.py: Import database_connection. (get_trait_symbol_and_tissue_values): Inject conn object.
2022-09-08Fix sql queriesMunyoki Kilyungi
"%s" should only be used outside table names and column names otherwise a string literal will be inserted thereby leading to errors in the sql statements. * wqflask/base/data_set.py (geno_mrna_confidentiality): Use f-strings for table/columns/clause. * wqflask/base/trait.py (retrieve_trait_info): Ditto. * wqflask/wqflask/gsearch.py (GSearch.__init__): Ditto. * wqflask/wqflask/interval_analyst/GeneUtil.py (loadGenes): Ditto. * wqflask/wqflask/snp_browser/snp_browser.py (SnpBrowser.get_browser_results): Ditto.
2022-09-08Add missing "," in cursor.execute()Munyoki Kilyungi
* wqflask/base/data_set.py (DatasetGroup.get_mapping_methods): Fix failing query by adding a missing ",".
2022-09-08Remove intermediate variables that are only used onceMunyoki Kilyungi
* wqflask/base/data_set.py (DatasetGroup.__init__): Remove "query" variable. (DataSet.chunk_dataset): Fix indentation. * wqflask/wqflask/search_results.py (get_GO_symbols): Remove "this_term" variable. * wqflask/wqflask/views.py (search_page): Remove "the_search" variable.
2022-09-08Delete dead importsMunyoki Kilyungi
* wqflask/base/data_set.py: Remove un-used imports. * wqflask/base/trait.py: Ditto. wqflask/tests/wqflask/show_trait/test_show_trait.py: Ditto. * wqflask/wqflask/show_trait/show_trait.py: Ditto. * wqflask/wqflask/views: Ditto.
2022-09-08Replace g.db with database_connection() context managerMunyoki Kilyungi
* wqflask/base/data_set.py (DatasetType.set_dataset_key): Replace g.db with "database_connection()". (DatasetGroup.get_mapping_methods): Ditto. (DataSet.get_accession_id): Ditto. (DataSet.retrieve_other_names): Rework "if ... else" logic to get rid of "query" and "_vars" variables. Also, nest "try ... except" block within "database_connection". (DataSet.chunk_dataset): Replace g.db with "database_connection()". (DataSet.get_probeset_data): Ditto. (DataSet.get_trait_data): Ditto. (PhenotypeDataSet.get_trait_info): Ditto. (PhenotypeDataSet.retrieve_sample_data): Ditto. (GenotypeDataSet.retrieve_sample_data): Ditto. (MrnaAssayDataSet.get_trait_info): Ditto. (MrnaAssayDataSet.retrieve_sample_data): Ditto. (MrnaAssayDataSet.retrieve_genes): Ditto. (TempDataSet): Ditto. (geno_mrna_confidentiality): Ditto. (query_table_timestamp): Ditto. * wqflask/wqflask/api/router.py: Replace "flask.g" with database_connection import. (get_species_list): Replace g.db with "database_connection()". (get_species_info): Ditto. (get_groups_list): Ditto. (get_group_info): Ditto. (get_datasets_for_group): Ditto. (get_dataset_info): Ditto. (fetch_traits): Ditto. (all_sample_data): Ditto. (trait_sample_data): Ditto. (get_trait_info): Ditto. (get_dataset_trait_ids): Ditto. (get_samplelist): Ditto. (get_group_id_from_dataset): Ditto. (get_group_id): Ditto.
2022-09-08Delete logger importMunyoki Kilyungi
* wqflask/base/trait.py: Remove utility.logger
2022-09-08Replace assertion with Value Error for non-existent datasetMunyoki Kilyungi
* wqflask/base/trait.py (retrieve_trait_info): Wrap error message in ValueError.
2022-09-08Replace g.db with database_connection() context managerMunyoki Kilyungi
* wqflask/base/mrna_assay_tissue_data.py: Replace "flask.g" with database_connection. (MrnaAssayTissueData.__init__): Use database_connection. (MrnaAssayTissueData.get_symbol_values_pairs): Ditto. * wqflask/base/species.py: Replace "Flask.g" imports with "database_connection". (Chromosomes.chromosomes): Use database_connection. * wqflask/base/trait.py: Import database_connection. (retrieve_trait_info): Use database_connection. * wqflask/utility/authentication_tools.py: Replace "flask.g" with database_connection. (get_group_code): Use database_connection. * wqflask/utility/helper_functions.py: Replace "flask.g" with "database_connection". (get_species_groups): Use database_connection. * wqflask/wqflask/db_info.py: Replace "Flask" and "g" with "database_connection". (InfoPage.get_info): Use database_connection. * wqflask/wqflask/do_search.py (DoSearch.execute): Use database_connection(). * wqflask/wqflask/external_tools/send_to_geneweaver.py: Replace "Flask" and "g" import with database_connection. (test_chip): Use database_connection. * wqflask/wqflask/external_tools/send_to_webgestalt.py: Replace "Flask, g" imports with database_connection. (test_chip): Use database_connection. * wqflask/wqflask/gsearch.py: Replace "Flask" and "g" import with database_connection. (GSearch.__init__): Use database_connection. * wqflask/wqflask/interval_analyst/GeneUtil.py (loadGenes): Use database_connection(). * wqflask/wqflask/show_trait/SampleList.py: Replace "flask.g import" with database_connection. (SampleList.get_attributes): Use database_connection. (SampleList.get_extra_attribute_values): Ditto. * wqflask/wqflask/show_trait/show_trait.py: Replace "Flask" and "g" import with database_connection. (ShowTrait.__init__): Use database_connection. (ShowTrait.get_external_links): Ditto. (get_nearest_marker): Ditto.
2022-09-08Merge pull request #727 from Alexanderlacuna/feature/use-textfilesAlexander Kabui
use text files for Probeset
2022-09-07catch general errors this will recreate the fileAlexander_Kabui
2022-09-06Fix error with dataset menu query in data_set.pyzsloan
I'm only using g.db.execute here because using cursor is throwing an error and I don't want to change the way this query is currently written since I didn't write it (I believe it was initially written using g.db.execute and whoever wrote it just forgot to put the g.db.execute, since previously it just called "fetchall")
2022-09-06Merge branch 'testing' into feature/generalize_tableszsloan
2022-09-06Set up the self.* variables from query resultsFrederick Muriuki Muriithi
Set up the appropriate self.* variables from the results of running the appropriate query.
2022-09-06Use dataset name if dataset group name is not set.Frederick Muriuki Muriithi
2022-08-31Delete unused modulesMunyoki Kilyungi
* wqflask/base/data_set.py: Delete "menu_main" import. * wqflask/db/call.py: Delete it. * wqflask/db/gn_server.py: Ditto. * wqflask/wqflask/submit_bnw.py: Ditto.
2022-08-31Replace fetchall, fetchone, fetch1 with database_connectionMunyoki Kilyungi
* wqflask/base/data_set.py: Replace "db.call" import with "database_connection". (create_datasets_list): Use "database_connection" to fetch data. (DatasetGroup.__init__): Ditto. (DataSet.retrieve_other_names): Ditto. (PhenotypeDataSet.setup): Remove query escaping in string and format the string. (GenotypeDataSet.setup): Ditto. (MrnaAssayDataSet.setup): Ditto. * wqflask/db/webqtlDatabaseFunction.py: Remove db.call import. (retrieve_species): Use database_connection() to fetch data. (retrieve_species_id): Ditto.
2022-08-31Remove "with Bench ..." callsMunyoki Kilyungi
"with Bench" instruments how long a function takes and generates time reports on as INFO logs. This should be done on a developer server. Should the log level be low enough, this bench marks will generate a lot of noise. Instrumentation should be done during development. * wqflask/base/data_set.py (create_datasets_list): Remove "with Bench...". * wqflask/db/call.py (fetchone): Ditto. (fetchall): Ditto. (gn_server): Ditto. * wqflask/wqflask/gsearch.py (GSearch.__init__): Ditto. * wqflask/wqflask/marker_regression/display_mapping_results.py (DisplayMappingResults.__init__): Ditto. * wqflask/wqflask/marker_regression/run_mapping.py (RunMapping.__init__): Ditto. * wqflask/wqflask/update_search_results.py (GSearch.__init__): Ditto. * wqflask/wqflask/views.py (search_page): Ditto. (heatmap_page): Ditto. (mapping_results_page): Ditto.
2022-08-31Remove usage of "logger" and un-necessary comments wrt the sameMunyoki Kilyungi
Logging is used to introspect variables or notify the commencement of a given operation. Logging should only be used to log errors. Also, most of the logging is either "logger.debug" or "logger.info"; and this won't show up in production/testing since we need a logging level above "WARNING" for them to show up. * wqflask/base/data_set.py (create_datasets_list): Remove logger. (Markers.add_pvalues): Ditto. (DataSet.retrieve_other_names): Ditto. * wqflask/base/mrna_assay_tissue_data.py: Ditto. * wqflask/base/webqtlCaseData.py: Ditto. * wqflask/db/call.py (fetch1): Ditto. (gn_server): Ditto. * wqflask/db/gn_server.py: Ditto. * wqflask/maintenance/set_resource_defaults.py: Ditto. * wqflask/utility/Plot.py (find_outliers): Ditto. * wqflask/utility/gen_geno_ob.py: Ditto. * wqflask/utility/helper_functions.py: Ditto. * wqflask/utility/pillow_utils.py: Ditto. * wqflask/utility/redis_tools.py: Ditto. * wqflask/wqflask/api/gen_menu.py (get_groups): Ditto. * wqflask/wqflask/api/mapping.py: Ditto. * wqflask/wqflask/api/router.py (get_dataset_info): Ditto. * wqflask/wqflask/collect.py (report_change): Ditto. * wqflask/wqflask/correlation/corr_scatter_plot.py: Ditto. * wqflask/wqflask/ctl/ctl_analysis.py (CTL): Ditto. (CTL.__init__): Ditto. (CTL.run_analysis): Ditto. (CTL.process_results): Ditto. * wqflask/wqflask/db_info.py: Ditto. * wqflask/wqflask/do_search.py (DoSearch.execute): Ditto. (DoSearch.mescape): Ditto. (DoSearch.get_search): Ditto. (MrnaAssaySearch.run_combined): Ditto. (MrnaAssaySearch.run): Ditto. (PhenotypeSearch.run_combined): Ditto. (GenotypeSearch.get_where_clause): Ditto. (LrsSearch.get_where_clause): Ditto. (MeanSearch.run): Ditto. (RangeSearch.get_where_clause): Ditto. (PvalueSearch.run): Ditto. * wqflask/wqflask/docs.py: Ditto. * wqflask/wqflask/export_traits.py: Ditto. * wqflask/wqflask/external_tools/send_to_bnw.py: Ditto. * wqflask/wqflask/external_tools/send_to_geneweaver.py: Ditto. * wqflask/wqflask/external_tools/send_to_webgestalt.py: Ditto. * wqflask/wqflask/gsearch.py (GSearch.__init__): Ditto. * wqflask/wqflask/heatmap/heatmap.py: Ditto. * wqflask/wqflask/marker_regression/display_mapping_results.py (DisplayMappingResults): Ditto. * wqflask/wqflask/marker_regression/gemma_mapping.py: Ditto. * wqflask/wqflask/marker_regression/plink_mapping.py (run_plink): Ditto. * wqflask/wqflask/marker_regression/qtlreaper_mapping.py (run_reaper): Ditto. * wqflask/wqflask/marker_regression/rqtl_mapping.py: Ditto. * wqflask/wqflask/marker_regression/run_mapping.py (RunMapping.__init__): Ditto. * wqflask/wqflask/parser.py (parse): Ditto. * wqflask/wqflask/search_results.py (SearchResultPage.__init__): Ditto. * wqflask/wqflask/update_search_results.py (GSearch.__init__): Ditto. * wqflask/wqflask/user_login.py (send_email): Ditto. (logout): Ditto. (forgot_password_submit): Ditto. (password_reset): Ditto. (password_reset_step2): Ditto. (register): Ditto. * wqflask/wqflask/user_session.py (create_signed_cookie): Ditto.
2022-08-31add new environment variable:TEXTDIRAlexander_Kabui
2022-07-12Include genotypes for F1s/reference/non-reference strains when fetchingzsloan
sample data for genotype traits
2022-06-23Change jsonable in GeneralTrait so that it passes all necessary tablezsloan
information
generated by cgit v1.2.3 (git 2.44.0) at 2024-10-19 14:38:20 +0000