diff options
| author | Pjotr Prins | 2025-02-28 07:29:49 +0100 |
|---|---|---|
| committer | Pjotr Prins | 2025-02-28 07:29:49 +0100 |
| commit | b7d372b376b15b4bc987a91f5698b4187ed78905 (patch) | |
| tree | 2c255371b5840f03f1e2238df244638fa1f6fefa | |
| parent | 69e7f79919cf7f96bc91e28433fe845fed80233b (diff) | |
| download | gn-gemtext-b7d372b376b15b4bc987a91f5698b4187ed78905.tar.gz | |
On backups
| -rw-r--r-- | issues/systems/tux04-production.gmi | 32 | ||||
| -rw-r--r-- | topics/systems/backups-with-borg.gmi | 4 |
2 files changed, 34 insertions, 2 deletions
diff --git a/issues/systems/tux04-production.gmi b/issues/systems/tux04-production.gmi index 61804fa..01e1638 100644 --- a/issues/systems/tux04-production.gmi +++ b/issues/systems/tux04-production.gmi @@ -17,8 +17,9 @@ Luckily not too much is running on this machine and if we mount things again, mo * [X] fix groups and users * [X] get guix going * [X] get mariadb going -* [ ] fire up GN2 service -* [ ] fire up SPARQL service +* [X] fire up GN2 service +* [X] fire up SPARQL service +* [X] sheepdog * [ ] fix CRON jobs and backups * [ ] test full reboots @@ -245,3 +246,30 @@ Feb 23 14:04:31 genenetwork-production shepherd[1]: Service redis (PID 3977) exi ``` This is caused by a newer version of redis. This is odd because we are using the same version from the container?! + +Actually it turned out the redis DB was corrupted on the SSD! Same for some other databases (ugh). + +Fred copied all data to an enterprise level storage, and we rolled back to some older DBs, so hopefully we'll be OK for now. + +# Reinstating backups + +In the next step we need to restore backups as described in + +=> /topics/systems/backups-with-borg + +I already created an ibackup user. Next we test the backup script for mariadb. + +One important step is to check the database: + +``` +/usr/bin/mariadb-check -c -u user -p* db_webqtl +``` + +A successful mariadb backup consists of multiple steps + +``` +2025-02-27 11:48:28 +0000 (ibackup@tux04) SUCCESS 0 <32m43s> mariabackup-dump +2025-02-27 11:48:29 +0000 (ibackup@tux04) SUCCESS 0 <00m00s> mariabackup-make-consistent +2025-02-27 12:16:37 +0000 (ibackup@tux04) SUCCESS 0 <28m08s> borg-tux04-sql-backup +2025-02-27 12:16:46 +0000 (ibackup@tux04) SUCCESS 0 <00m07s> drop-rsync-balg01 +``` diff --git a/topics/systems/backups-with-borg.gmi b/topics/systems/backups-with-borg.gmi index 252e6a4..5cdb2a3 100644 --- a/topics/systems/backups-with-borg.gmi +++ b/topics/systems/backups-with-borg.gmi @@ -191,6 +191,10 @@ borg list genenetwork::borg-tux04-sql-20250209-01:43-Sun Make sure you not only see just a symlink. +# More backups + +Our production server runs databases and file stores that need to be backed up too. + # Drop backups Once backups work it is useful to copy them to a remote server, so when the machine stops functioning we have another chance at recovery. See |