diff options
| author | Frederick Muriuki Muriithi | 2022-10-19 11:18:52 +0300 |
|---|---|---|
| committer | Frederick Muriuki Muriithi | 2022-10-19 11:20:59 +0300 |
| commit | 21f33beec687e71bef2b33238a14b4e907aa8ec3 (patch) | |
| tree | 55170923c2bce926b52ccea8ef21f7fc001094f9 | |
| parent | 4e093c9e3ae7f49c07b3bc97b9c5141d8ed7cf45 (diff) | |
| download | gn-gemtext-21f33beec687e71bef2b33238a14b4e907aa8ec3.tar.gz | |
Issues (materialised-views-for-correlations): Update issue
* issues/materialised-views-for-correlations.gmi: Update issue with
notes on the different types of traits, and also some candidate
queries that could be materialized.
| -rw-r--r-- | issues/materialised-views-for-correlations.gmi | 22 |
1 files changed, 22 insertions, 0 deletions
diff --git a/issues/materialised-views-for-correlations.gmi b/issues/materialised-views-for-correlations.gmi index 855a143..10b878d 100644 --- a/issues/materialised-views-for-correlations.gmi +++ b/issues/materialised-views-for-correlations.gmi @@ -22,6 +22,12 @@ There is some work on => /topics/genotype-database the genotype database that should allow intermediate materialised views to be stored in lmdb +There might need to be multiple materialised views for the different types of traits, i.e. +* Phenotypes (Publish) +* Genotypes (Geno) +* mRNA (ProbeSet) +* Temp (Not sure how this would be handled) + ### Implementation Concerns The implementation of the materialised views will need to concern itself with the following issues: @@ -36,3 +42,19 @@ Maybe, if we are going down the materialised views road, we should also have mig The reason for the concern above is, for example, The developers (alex, arun, bonz, fredm, etc) can implement whatever schema changes, triggers and views locally, but they do not (and probably should not) have access to production. As such, there should be provided a sort of CLI endpoint for the deployment system (or person) to run to implement the changes on the production (and CI/CD) database to get similar results. The problem here, is that the migration might be moot, if the data is then moved out of the database, as is being planned. + +### Queries to Materialise + +Possible candidate queries for materialisation are: + +=> https://github.com/genenetwork/genenetwork2/blob/a2b837801d479ed2fb06ca33c07de9c271532c46/wqflask/base/data_set/dataset.py#L200-L292 + +The method above is doing way too much - it should probably be split into separate methods for each class, to simplify the code a little and make it clearer what each part does before reworking the queries for the materialized view. + + +=> https://github.com/genenetwork/genenetwork2/blob/a2b837801d479ed2fb06ca33c07de9c271532c46/wqflask/base/trait.py#L386-L610 + +The method above is also doing way too much. + + +Both methods above do not have the metadata, so probably also have a look at adding the metadata to the materialized views |