diff options
| author | Pjotr Prins | 2024-02-12 17:24:40 +0100 |
|---|---|---|
| committer | Pjotr Prins | 2024-02-12 17:24:40 +0100 |
| commit | 25be3b7cfdbe0ec777ca5e33803e26c0114b802d (patch) | |
| tree | ee26afcae7e7f331dd807b52f4702b05ae634b6b | |
| parent | 334f8708c60dfc7631cbfbf3a1a9d365b1e387aa (diff) | |
| download | gn-gemtext-25be3b7cfdbe0ec777ca5e33803e26c0114b802d.tar.gz | |
Restarted on precompute
| -rw-r--r-- | topics/systems/mariadb/precompute-mapping-input-data.gmi | 13 |
1 files changed, 11 insertions, 2 deletions
diff --git a/topics/systems/mariadb/precompute-mapping-input-data.gmi b/topics/systems/mariadb/precompute-mapping-input-data.gmi index 968277f..d8ebe15 100644 --- a/topics/systems/mariadb/precompute-mapping-input-data.gmi +++ b/topics/systems/mariadb/precompute-mapping-input-data.gmi @@ -18,12 +18,21 @@ GN relies on precomputed mapping scores for search and other functionality. Here Above is the quick win for plugging in GEMMA values. We will make sure not to recompute the values that are already up to date. This is achieved by naming the input and output files as a hash on their DB inputs. -Next: +Next for running the full batch: * [X] Store all GEMMA values efficiently +* [ ] Include metadata record in lmdb and as JSON file +* [ ] Remove junk from tarball +* [ ] List significant markers as metadata +* [ ] Reread below info +* [ ] Submit jobs to PBS +* [ ] Report results to mariadb + +And after: + * [ ] Track metadata of computed datasets (in RDF?) * [ ] Compute significance with GEMMA or other LMM (bulkLMM?) -* [ ] Store signficance and significant values for processing +* [ ] Store significance and significant values for processing * [ ] Update search & correlations to use these * [ ] Further optimize computations so they can run continuously in the background |