# Restore backup ## Borg We use borg backups. Load a recent borg, for example from my profile ``` source /home/wrk/opt/borg/etc/profile ``` The backup passphrase needs to be set. ``` source /home/ibackup/.borg-pass ``` Now run against a repo on 'tux02' ``` borg list /export3/backup/tux01/borg/borg-tux01/ borg-backup-mariadb-20221004-03:13-Tue Mon, 2022-10-03 22:13:11 ``` The last backup on 'tux02' is from October 2022 - after I did a reinstall. That is not great. According to sheepdog the drops are happening to 'space' and 'epysode', but 'tux02' is missing: => https://rabbit.genenetwork.org/sheepdog/index.html ## Mariadb The first step is to find where the current mariadb is running. This will be a symlink in /var/lib/mysql -> some dir. ``` mysql -> /export2/mysql/borg-backup-mariadb-20211024-03:09-Sun ``` We typically run the database on an nvme partition. Check if there is enough space(!). It may be you need to remove the old database after making a backup (if it is not in borg). Next, find the backup. Typically it is symlinked from /export/backup. On ``` tux02:/export5/backup/bacchus/drop/tux01 ``` ### Unpack from backup We need a place to unpack the DB. On tux02 export5 is large, so let's try that and pick the last backup ``` cd /export5/mysql tux02:/export5/mysql# borg list /export5/backup/bacchus/drop/tux01/borg-tux01 borg extract --progress /export5/backup/bacchus/drop/tux01/borg-tux01::borg-backup-mariadb-20230207-02:31-Tue ``` As it happened, this was an incomplete backup! Check original size with ``` borg info --last 3 /export5/backup/bacchus/drop/tux01/borg-tux01 ``` so I went with ``` borg extract --progress /export5/backup/bacchus/drop/tux01/borg-tux01::borg-backup-mariadb-20230205-03:38-Sun ``` Mariadb's own ibdata1 is still at 100Gb. Too large, and we'll muck it out at some point. The total mariadb database at this point is 430Gb. ### Move DB in place To move the new DB in place we first have to stop mariadb, move the old database out of the way, move the new one in, and restart the DB after making sure permissions and symlink are right. Just to be sure we copy the old DB somewhere. ``` systemctl stop mariadb cd /export2/mysql cp -vau /export2/mysql/borg-backup-mariadb-20211024-03:09-Sun . root@tux02:/export2/mysql# cp -vau /export5/mysql/home/backup/tux01_mariadb_new/latest/* borg-backup-mariadb-20230205-03:38-Sun cd /var/lib rm mysql ln -s /export2/mysql/borg-backup-mariadb-20230205-03\:38-Sun mysql systemctl start mariadb systemctl status mariadb ``` Oops. Can't create test file /var/lib/mysql. We need to set permissions. ``` root@tux02:/var/lib# chown mysql.mysql mysql ``` And we are running. Next stop update genotype files. ## Files GN has a file repo for genotype files, lmdb and such, next to MariaDB. Typically ``` root@tux02:/home/gn2/production/genotype_files# du -sh . 38G . ``` the backup in drop lists, for example ``` root@tux02:/export5/backup/bacchus/drop/tux01# borg list borg-genenetwork/ borg-GN2-home-20230207-04:00-Tue Mon, 2023-02-06 22:00:02 borg-GN1-home-20230207-04:40-Tue Mon, 2023-02-06 22:40:01 borg-GN3-home-20230207-04:41-Tue Mon, 2023-02-06 22:41:27 borg-GN2s-home-20230207-04:41-Tue Mon, 2023-02-06 22:41:45 borg-ZACH-home-20230207-04:42-Tue Mon, 2023-02-06 22:42:01 borg-datafiles-home-20230207-04:42-Tue Mon, 2023-02-06 22:42:32 ``` At this poing ZACH-home is the main dir, so this fetches all files ``` borg extract --verbose borg-genenetwork::borg-ZACH-home-20230207-04:42-Tue home/zas1024/gn2-zach/genotype_files ``` and we copy it to /home/gn2/production. We also have a separate data files 'warehouse', but that is not linked up with the webserver right now.