# Running Virtuoso

## Tags

* type: documentation
* status: unclear

## Description

We run instances of Virtuoso for our graph databases. Virtuoso is a remarkable software and running some really large databases, including Uniprot

=> https://github.com/openlink/virtuoso-opensource
=> https://www.uniprot.org/sparql/

On Penguin2 virtuoso is running by default as a shepherd servive, see

=> ./shepherd.gmi

```
guix environment --ad-hoc virtuoso-ose -- virtuoso-t -f
```

=> https://git.genenetwork.org/efraim/shepherd-services/src/branch/master/shepherd/init.d/virtuoso.scm penguin2:/home/shepherd/shepherd-services/shepherd/init.d/virtuoso.scm

The database is initialized from 'penguin2:/export/virtuoso/var/lib/virtuoso/db/virtuoso.ini'

### Running virtuoso in a guix system container

We have a Guix virtuoso service in the guix-bioinformatics channel. The easiest way to run virtuoso is to use the virtuoso service to run it in a guix system container. The only downside of this method is that, since guix system containers require root privileges to start up, you will need root priviliges on the machine you are running this on.

Here is a basic guix system configuration that runs virtuoso listening on port 8891, and with its HTTP server listening on port 8892. Among other things, the HTTP server provides a SPARQL endpoint to interact with.
```
(use-modules (gnu)
             (gn services databases))

(operating-system
  (host-name "virtuoso")
  (timezone "UTC")
  (locale "en_US.utf8")
  (bootloader (bootloader-configuration
               (bootloader grub-bootloader)
               (targets (list "/dev/sdX"))))
  (file-systems (cons (file-system
                        (device "root")
                        (mount-point "/")
                        (type "ext4"))
                      %base-file-systems))
  (users %base-user-accounts)
  (packages %base-packages)
  (services (cons (service virtuoso-service-type
                           (virtuoso-configuration
                            (server-port 8891)
                            (http-server-port 8892)))
                  %base-services)))
```

You can write the above configuration to a file, say virtuoso-os.scm, build a container with it, and run it with the command below. Everything inside the container is ephemeral and vanishes when the container is stopped. In order to persist the database, we mount a host directory /tmp/virtuoso-state at /var/lib/virtuoso in the container. /var/lib/virtuoso is the default state directory used by the Guix virtuoso service.
```
sudo $(guix system container --network --share=/tmp/virtuoso-state=/var/lib/virtuoso virtuoso-os.scm)
```

### Running virtuoso by invoking it on the command line

You may also choose to run virtuoso the traditional way by invoking it on the command line. Managing long-running instances started from the command line is messy. So, this method works best for temporary instances. Let's start from the virtuoso.ini file:

```
mkdir -p ~/services/virtuoso
cd services/virtuoso
cp /export/virtuoso/var/lib/virtuoso/db/virtuoso.ini .
```

and edit it to change paths and ports - use non-priviliged ports(!). A full diff is below. Start the server in a screen or tmux (it may ask for creating ./db):

```
penguin2:~/services/virtuoso$ ~/.config/guix/current/bin/guix environment --ad-hoc virtuoso-ose glibc-locales

penguin2:~/services/virtuoso$ /gnu/store/9aqd4jmkafhkdm095hnmxpxzws3ym3wd-virtuoso-ose-7.2.5/bin/virtuoso-t +foreground +configfile virtuoso.ini

03:34:50 HTTP/WebDAV server online at 28890
03:34:50 Server online at 21111 (pid 57078)
```

Now the server should respond to

```
curl localhost:28890/sparql
```

and the admin interface on

```
curl localhost:28890/conductor
```

To use the service from your remote machine use ssh tunnels:

```
ssh -L 28890:127.0.0.1:28890 -f -N myname@penguin2.genenetwork.org
```

and surf to http://localhost:28890/conductor. A good time to change the default password (dba:dba)!

### Uploading data with CURL

To upload RDF I use rapper to validate the data. First delete the existing graph with something like

```
curl -v --digest --user dba:password --verbose --url -G http://localhost:28890/sparql-graph-crud-auth --data-urlencode graph=https://BioHackrXiv.org/graph -X DELETE
```

Next update the graph with

```
curl -v -X PUT --digest -u dba:password -H Content-Type:text/turtle -T test/data/biohackrxiv.ttl -G http://localhost:28890/sparql-graph-crud-auth --data-urlencode graph=https://BioHackrXiv.org/graph
```

Where BioHackrXiv is the name of the graph (in this example). A python version can be found in

=> https://github.com/pubseq/bh20-seq-resource/blob/master/scripts/update_virtuoso/check_for_updates.py

### Virtuoso.ini

What changed in $HOME/services/virtuoso/virtuoso.ini

```
+DatabaseFile                   = $HOME/services/virtuoso/db/virtuoso.db
+ErrorLogFile                   = $HOME/services/virtuoso/db/virtuoso.log
+LockFile                       = $HOME/services/virtuoso/db/virtuoso.lck
+TransactionFile                = $HOME/services/virtuoso/db/virtuoso.trx
+xa_persistent_file             = $HOME/services/virtuoso/db/virtuoso.pxa
+DatabaseFile                   = $HOME/services/virtuoso/db/virtuoso-temp.db
+TransactionFile                = $HOME/services/virtuoso/db/virtuoso-temp.trx
-ServerPort                     = 1111
+ServerPort                     = 21111
+NumberOfBuffers                = 340000
+MaxDirtyBuffers                = 250000
+ServerPort                     = 28890
+ServerRoot                     = $HOME/services/virtuoso/vsp
```