diff options
Diffstat (limited to 'issues')
| -rw-r--r-- | issues/systems/gn2-time-machines.gmi | 102 |
1 files changed, 102 insertions, 0 deletions
diff --git a/issues/systems/gn2-time-machines.gmi b/issues/systems/gn2-time-machines.gmi new file mode 100644 index 0000000..efea479 --- /dev/null +++ b/issues/systems/gn2-time-machines.gmi @@ -0,0 +1,102 @@ +# GN2 Time Machines + +GN1 time machines are pretty straightforward. With GN2 the complexity has increased a lot because of interacting services and a larger dependency graph. + +Here I track what it takes today to install an instance of GN2 that is 'frozen' in time. + +## INFO + +### Setting up Guix + +We tend to install software in a guix profile. E.g. + +``` +guix pull -p ~/opt/guix-pull +. /home/wrk/opt/guix-pull/etc/profile +guix package -i mariadb -p /usr/local/guix-profiles/mariadb +``` + +### Mariadb (est. 1-2 hours) + +Set up a global Mariadb + +``` +guix package -i mariadb -p /usr/local/guix-profiles/mariadb +``` + +Usually I use the Debian version to set up defaults + +``` +apt-get install mariadb +cd /etc/systemd/system +cp /lib/systemd/system/mariadb.service . +systemctl disable mariadb +``` + +Add to systemd + +```diff ++Type=simple ++CapabilityBoundingSet=CAP_IPC_LOCK CAP_DAC_OVERRIDE CAP_AUDIT_WRITE ++PrivateDevices=false ++ProtectHome=false ++ExecStart=/usr/local/guix-profiles/mariadb/bin/mariadbd --pid-file=/var/run/mysqld/mariadb.pid $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION ++PIDFile=/usr/local/mysql/data/mysqld.pid ++# ExecStartPost=/bin/sh -c "systemctl unset-environment _WSREP_START_POSITION" +-ExecStartPost=/etc/mysql/debian-start ++RestartSec=15s ++TimeoutStartSec=infinity ++TimeoutStopSec=infinity +``` + +comment out the galera ExecStart too. + +``` +systemctl enable mariadb-guix.service +``` + +Make sure all symlinks point to our configuration file. + +Before starting systemd you may want to make sure the database is running. + +``` +/usr/local/guix-profiles/mariadb/bin/mariadbd --pid-file=/var/run/mysqld/mariadb.pid --verbose (--help) +``` + +as root you should be able to login with + +``` +mysql -e 'show databases' +``` + +### Mariadb database from backup (est. 2 hours) + +We have daily incremental backups on P2, Tux02 and Epysode. First restore the files with + +``` +. ~/.borg-pass +cd /export2/tux01-restore +borg extract --progress /export2/backup/tux01/borg-tux01::borg-backup-mariadb-20220815-03:13-Mon +``` + +Extracting 430Gb takes about 90 minutes. + +Now make sure mariadb is stopped. Copy the database to fast storage and symlink the DB dir. + +``` +root@epysode:/export/tux01-mirror# +cp -vau /export2/tux01-restore/home/backup/tux01_mariadb_new . +systemctl stop mariadb +ln -s /export2/tux01-restore/home/backup/tux01_mariadb_new /var/lib/mysql +systemctl start mariadb +/usr/local/guix-profiles/guix-profiles/mariadb/bin/mysql_upgrade -u webqtlout -pwebqtlout +/export/backup/scripts/tux02/system_check.sh +``` + +``` + +In the process I discover that ibdata1 file has grown to 100GB. Not a problem yet, but we should purge that + +=> https://www.percona.com/blog/2013/08/20/why-is-the-ibdata1-file-continuously-growing-in-mysql/ + +(obviously we don't want to use mysqldump right now, but I'll need to do some future work). |