Instructions for virtuoso

2 files changed, 117 insertions, 0 deletions

diff --git a/topics/systems/shepherd.gmi b/topics/systems/shepherd.gmi
new file mode 100644
index 0000000..5f9ca4f
--- /dev/null
+++ b/topics/systems/shepherd.gmi
@@ -0,0 +1,20 @@
+# Shepherd
+
+On Debian based systems we run shepherd as a shepherd user. The service gets started up through systemd:
+
+```
+[Unit]
+Description=Launch user shepherd daemon
+Wants=guix-daemon.service
+
+[Service]
+Type=oneshot
+ExecStart=/var/guix/profiles/per-user/shepherd/guix-profile/bin/shepherd --socket=/home/shepherd/.config/shepherd/shepherd.sock
+User=shepherd
+Group=shepherd
+
+[Install]
+WantedBy=multi-user.target
+```
+
+=> https://git.genenetwork.org/efraim/shepherd-services Running shepherd services
diff --git a/topics/systems/virtuoso.gmi b/topics/systems/virtuoso.gmi
new file mode 100644
index 0000000..d6b8b02
--- /dev/null
+++ b/topics/systems/virtuoso.gmi
@@ -0,0 +1,97 @@
+# Running Virtuoso
+
+We run instances of Virtuoso for our graph databases. Virtuoso is a remarkable software and running some really large databases, including Uniprot
+
+=> https://github.com/openlink/virtuoso-opensource
+=> https://www.uniprot.org/sparql/
+
+On Penguin2 virtuoso is running by default as a shepherd servive, see
+
+=> ./shepherd.gmi
+
+```
+guix environment --ad-hoc virtuoso-ose -- virtuoso-t -f
+```
+
+=> https://git.genenetwork.org/efraim/shepherd-services/src/branch/master/shepherd/init.d/virtuoso.scm penguin2:/home/shepherd/shepherd-services/shepherd/init.d/virtuoso.scm
+
+The database is initialized from 'penguin2:/export/virtuoso/var/lib/virtuoso/db/virtuoso.ini'
+
+## Running a new instance
+
+For testing we want to run a temporary instance. Let's start from the virtuoso.ini file:
+
+```
+mkdir -p ~/services/virtuoso
+cd services/virtuoso
+cp /export/virtuoso/var/lib/virtuoso/db/virtuoso.ini .
+```
+
+and edit it to change paths and ports - use non-priviliged ports(!). A full diff is below. Start the server in a screen or tmux (it may ask for creating ./db):
+
+```
+penguin2:~/services/virtuoso$ ~/.config/guix/current/bin/guix environment --ad-hoc virtuoso-ose glibc-locales
+
+penguin2:~/services/virtuoso$ /gnu/store/9aqd4jmkafhkdm095hnmxpxzws3ym3wd-virtuoso-ose-7.2.5/bin/virtuoso-t +foreground +configfile virtuoso.ini
+
+03:34:50 HTTP/WebDAV server online at 28890
+03:34:50 Server online at 21111 (pid 57078)
+```
+
+Now the server should respond to
+
+```
+curl localhost:28890/sparql
+```
+
+and the admin interface on
+
+```
+curl localhost:28890/conductor
+```
+
+To use the service from your remote machine use ssh tunnels:
+
+```
+ssh -L 28890:127.0.0.1:28890 -f -N myname@penguin2.genenetwork.org
+```
+
+and surf to http://localhost:28890/conductor. A good time to change the default password (dba:dba)!
+
+## Uploading data with CURL
+
+To upload RDF I use rapper to validate the data. First delete the existing graph with something like
+
+```
+curl -v --digest --user dba:password --verbose --url -G http://localhost:28890/sparql-graph-crud-auth --data-urlencode graph=https://BioHackrXiv.org/graph -X DELETE
+```
+
+Next update the graph with
+
+```
+curl -v -X PUT --digest -u dba:a-h9tlenHTill -H Content-Type:text/turtle -T test/data/biohackrxiv.ttl -G http://localhost:28890/sparql-graph-crud-auth --data-urlencode graph=https://BioHackrXiv.org/graph
+```
+
+Where BioHackrXiv is the name of the graph (in this example). A python version can be found in
+
+=> https://github.com/pubseq/bh20-seq-resource/blob/master/scripts/update_virtuoso/check_for_updates.py
+
+## Virtuoso.ini
+
+What changed in $HOME/services/virtuoso/virtuoso.ini
+
+```
++DatabaseFile                   = $HOME/services/virtuoso/db/virtuoso.db
++ErrorLogFile                   = $HOME/services/virtuoso/db/virtuoso.log
++LockFile                       = $HOME/services/virtuoso/db/virtuoso.lck
++TransactionFile                = $HOME/services/virtuoso/db/virtuoso.trx
++xa_persistent_file             = $HOME/services/virtuoso/db/virtuoso.pxa
++DatabaseFile                   = $HOME/services/virtuoso/db/virtuoso-temp.db
++TransactionFile                = $HOME/services/virtuoso/db/virtuoso-temp.trx
+-ServerPort                     = 1111
++ServerPort                     = 21111
++NumberOfBuffers                = 340000
++MaxDirtyBuffers                = 250000
++ServerPort                     = 28890
++ServerRoot                     = $HOME/services/virtuoso/vsp
+```