diff options
Diffstat (limited to 'issues/rdf')
-rw-r--r-- | issues/rdf/automate-rdf-generation-and-ingress.gmi | 37 | ||||
-rw-r--r-- | issues/rdf/hash-rdf-graph.gmi | 9 |
2 files changed, 46 insertions, 0 deletions
diff --git a/issues/rdf/automate-rdf-generation-and-ingress.gmi b/issues/rdf/automate-rdf-generation-and-ingress.gmi new file mode 100644 index 0000000..ef4ba9f --- /dev/null +++ b/issues/rdf/automate-rdf-generation-and-ingress.gmi @@ -0,0 +1,37 @@ +# Update RDF Generation and Ingress to Virtuoso + +## Tags + +* assigned: bonfacem +* priority: high +* tags: in-progress +* deadline: 2024-10-23 Wed + +We need to update Virtuoso in production. At the moment this is done manually. For the current set-up, we need to update the recent modified RIF+WIKI models: + + +``` +# Generate the RDF triples +time guix shell guile-dbi guile-hashing -m manifest.scm -- ./pre-inst-env ./examples/generif.scm --settings conf.scm --output /home/bonfacem/ttl-files/generif-metadata-new.ttl --documentation ./docs/generif-metadata.md + +# Make sure they are valid +guix shell -m manifest.scm -- rapper --input turtle --count /home/bonfacem/ttl-files/generif-metadata-new.ttl + +# Copy the files over to the exposed virtuoso path +cp /home/bonfacem/ttl-files/generif-metadata-new.ttl </some/dir/> + +# Get into Virtuoso (with a password) +guix shell virtuoso-ose -- isql <port-number> + +# Load the files to be loaded +# Assuming that '/var/lib/data' is where the files are +ld_dir('/var/lib/data', 'generif-metadata-new.ttl', 'http://genenetwork.org'); + +# Load the files +rdf_loader_run(); +CHECKPOINT; +``` + +Above steps should be automated and tested in CD before roll-out in production. Key considerations: + +- Pick latest important changes from git, so that we can pick what files to run instead of generating all the ttl files all the time. diff --git a/issues/rdf/hash-rdf-graph.gmi b/issues/rdf/hash-rdf-graph.gmi index c896218..2863108 100644 --- a/issues/rdf/hash-rdf-graph.gmi +++ b/issues/rdf/hash-rdf-graph.gmi @@ -5,3 +5,12 @@ ## Description Building the index is an expesive operation. Hash the graph and store the metadata in xapian, and similarly in the RDF store. The mcron-job should check whether this has changed, and if there's any difference, go ahead and re-build the index. + +Resolution: + +=> https://github.com/genenetwork/genenetwork3/pull/171 Improve Sharing Memory Across Processes. +=> https://github.com/genenetwork/genenetwork3/pull/172 Check whether table names were stored in xapian. +=> https://github.com/genenetwork/genenetwork3/pull/174 Wikidata index. +=> https://github.com/genenetwork/genenetwork3/pull/175 Refactor how the generif md5 sum is calculated and stored in XAPIAN. + +* closed |