# Setting up Local Development Database You need to set up a quick local database for development without polluting your environment. ## Method 1 (Using Guix system containers) Setting up mariadb in a Guix container is the preferred and easier method. But, you need root access to run the container. The genenetwork2 repo comes with a guix system container definition to run MariaDB and Redis. From the genenetwork2 repo, you can build and run the container using: ``` $ sudo $(./containers/db-container.sh) ``` You should now be able to connect to the database using ``` $ mysql --protocol tcp -u root ``` Create a database db_webqtl_s ``` MariaDB [mysql]> CREATE DATABASE db_webqtl_s; ``` Load the small database dump into the database. You may find the small database either on space or tux02 at /home/aruni/gn2.sql.lz ``` $ lzip -cd gn2.sql.lz | mysql --protocol tcp -u root db_webqtl_s ``` Since this is a develpoment server accessible only from localhost, it is ok to use the root user with no password. Configure your development instance of genenetwork2 with the following SQL_URI. ``` SQL_URI="mysql://root@127.0.0.1:3306/db_webqtl_s" ``` ## Method 2 (running a local container) A local container runs as a normal user. It requires some juggling of dirs to load an existing database. Make sure to not run on the same dirs as another mariadb instance(!) ``` cd ~/tmp/mariadb mkdir var ~/tmp/mariadb$ ~/opt/guix-pull/bin/guix shell -C -N coreutils sed mariadb --share=var=/var --share=/export2/tmp=/tmp ``` inside the container ``` mkdir -p /var/lib/data mkdir -p /var/lib/mysql mkdir /var/run ``` Initialize with ``` mysql_install_db ``` and run ``` export TMPDIR=/tmp mysqld_safe --datadir='./database' --port=3307 --user=$USER --nowatch --socket=/var/run/mysqld/mysqld.sock ``` Now from outside the container you should be able to connect with ``` stromboli:~/tmp/mariadb$ mysql --socket=var/run/mysqld/mysqld.sock ``` and ``` Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 3 Server version: 10.5.12-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ 4 rows in set (0.001 sec) ``` If you need to tweak the server configuration you can load the my.cnf file with the `--defaults-file=var/my.cnf` inside and outside the container. ## Method 3 (Manual method without Guix) /This is not recommended/ * An assumption is made that the GeneNetwork2 profile is in ~/opt/gn_profiles/gn2_latest for the purposes of this documentation. Please replace as appropriate. * We install the database files under ~/genenetwork/mariadb. Change as appropriate. Set up directories ``` mkdir -pv ~/genenetwork/mariadb/var/run mkdir -pv ~/genenetwork/mariadb/var/lib/data mkdir -pv ~/genenetwork/mariadb/var/lib/mysql ``` Set up default my.cnf ``` cat < ~/genenetwork/mariadb/my.cnf [client-server] socket=~/genenetwork/mariadb/var/run/mysqld/mysqld.sock port=3307 [server] user=$(whoami) socket=~/genenetwork/mariadb/var/run/mysqld/mysqld.sock basedir=~/opt/gn_profiles/gn2_latest datadir=~/genenetwork/mariadb/var/lib/data ft_min_word_len=3 EOF ``` Install the database ``` ~/opt/gn_profiles/gn2_latest/bin/mysql_install_db \ --defaults-file=~/genenetwork/mariadb/my.cnf ``` Running the daemon: ``` ~/opt/gn_profiles/gn2_latest/bin/mysqld_safe \ --defaults-file=~/genenetwork/mariadb/my.cnf ``` Connect to daemon ``` ~/opt/gn_profiles/gn2_latest/bin/mysql \ --defaults-file=~/genenetwork/mariadb/my.cnf ``` Set up password for user ``` MariaDB [(none)]> USE mysql; MariaDB [mysql]> ALTER USER ''@'localhost' IDENTIFIED BY ''; MariaDB [mysql]> FLUSH PRIVILEGES; ``` Now logout and login again with ``` $ ~/opt/gn_profiles/gn2_latest/bin/mysql \ --defaults-file=~/genenetwork/mariadb/my.cnf --password mysql ``` enter the newly set password and voila, you are logged in and your user has the password set up. Now, set up a new user, say webqtlout, and a default database they can connect to ``` MariaDB [mysql]> CREATE DATABASE webqtlout; MariaDB [mysql]> CREATE USER 'webqtlout'@'localhost' IDENTIFIED BY ''; MariaDB [mysql]> GRANT ALL PRIVILEGES ON webqtlout.* TO 'webqtlout'@'localhost'; ``` Now logout, and log back in as the new webqtlout user: ``` ~/opt/gn_profiles/gn2_latest/bin/mysql \ --defaults-file=~/genenetwork/mariadb/my.cnf \ --user=webqtlout --host=localhost --password webqtlout ``` and enter the password you provided. ### Setting up the Small Database Download the database from => http://ipfs.genenetwork.org/ipfs/QmRUmYu6ogxEdzZeE8PuXMGCDa8M3y2uFcfo4zqQRbpxtk Say you downloaded the file in ~/Downloads, you can now add the database to your server. First stop the server: ``` $ ps aux | grep mysqld # get the process ids $ kill -s SIGTERM ``` Now extract the database archive in the mysql data directory: ``` $ cd ~/genenetwork/mariadb/var/lib/data $ p7zip -k -d ~/Downloads/db_webqtl_s.7z ``` Now restart the server: ``` ~/opt/gn_profiles/gn2_latest/bin/mysqld_safe \ --defaults-file=~/genenetwork/mariadb/my.cnf ``` Then update the databases ``` $ ~/opt/gn_profiles/gn2_latest/bin/mysql_upgrade \ --defaults-file=~/genenetwork/mariadb/my.cnf \ --user=frederick --password --force ``` and login as the administrative user: ``` $ ~/opt/gn_profiles/gn2_latest/bin/mysql \ --defaults-file=~/genenetwork/mariadb/my.cnf \ --user=$(whoami) --password ``` and grant the privileges to your normal user: ``` MariaDB [mysql]> GRANT ALL PRIVILEGES ON db_webqtl_s.* TO 'webqtlout'@'localhost'; ``` now logout as the administrative user and log back in as the normal user ``` ~/opt/gn_profiles/gn2_latest/bin/mysql \ --defaults-file=~/genenetwork/mariadb/my.cnf \ --user=webqtlout --host=localhost --password db_webqtlout_s MariaDB [db_webqtlout_s]> SELECT * FROM ProbeSetData LIMIT 20; ``` verify you see some data. ### A Note on Connection to the Server So far, we have been connecting to the server by specifying --defaults-file option, e.g. ``` ~/opt/gn_profiles/gn2_latest/bin/mysql \ --defaults-file=~/genenetwork/mariadb/my.cnf \ --user=webqtlout --host=localhost --password db_webqtlout_s ``` which allows connection via the unix socket. We could drop that specification and connect via the port with: ``` ~/opt/gn_profiles/gn2_latest/bin/mysql \ --user=webqtlout --host=127.0.0.1 --port=3307 --password db_webqtlout_s ``` In this version, the host specification was changed from ``` --host=localhost ``` to ``` --host=127.0.0.1 ``` whereas, the **--defaults-file** file specification was dropped and a new **--port** specification was added.