From 2f84a722e5944bd4458abbd45fd637a9286bab04 Mon Sep 17 00:00:00 2001
From: Arun Isaac
Date: Fri, 1 Jul 2022 18:44:01 +0530
Subject: Move issues lost inside the topics directory.

---
 topics/systems/fallbacks-and-backups.gmi | 69 --------------------------------
 1 file changed, 69 deletions(-)
 delete mode 100644 topics/systems/fallbacks-and-backups.gmi

(limited to 'topics/systems/fallbacks-and-backups.gmi')

diff --git a/topics/systems/fallbacks-and-backups.gmi b/topics/systems/fallbacks-and-backups.gmi
deleted file mode 100644
index 1a22db9..0000000
--- a/topics/systems/fallbacks-and-backups.gmi
+++ /dev/null
@@ -1,69 +0,0 @@
-# Fallbacks and backups
-
-As a hurricane is barreling towards our machine room in Memphis we are checking our fallbacks and backups for GeneNetwork. For years we have been making backups on Amazon - both S3 and a running virtual machine. The latter was expensive, so I replaced it with a bare metal server which earns itself (if it hadn't been down for months, but that is a different story).
-
-## Tags
-
-* type: enhancement
-* assigned: pjotrp
-* keywords: systems, fallback, backup, deploy
-* status: in progress
-* priority: critical
-
-## Tasks
-
-* [.] backup ratspub, r/shiny, bnw, covid19, hegp, pluto services
-* [X] /etc /home/shepherd backups for Octopus
-* [X] /etc /home/shepherd backups for P2
-* [X] Get backups running again on fallback
-* [ ] fix redis queue for P2 - needs to be on rabbit
-* [ ] fix bacchus large backups
-* [ ] backup octopus01:/lizardfs/backup-pangenome on bacchus
-
-## Backup and restore
-
-We are using borg for backing up data. Borg is excellent at deduplication and compression of data and is pretty fast too. Incremental copies work with rsync - so that is fast. To restore the full MariaDB database from a local borg repo takes a few minutes:
-
-```
-wrk@epysode:/export/restore_tux01$ time borg extract -v /export2/backup/tux01/borg-tux01::BORG-TUX01-MARIADB-20210829-04:20-Sun
-real    17m32.498s
-user    8m49.877s
-sys     4m25.934s
-```
-
-This all contrasts heavily with restoring 300GB from Amazon S3.
-
-Next restore the GN2 home dir
-
-```
-root@epysode:/# borg extract export2/backup/tux01/borg-genenetwork::TUX01_BORG_GN2_HOME-20210830-04:00-Mon
-```
-
-## Get backups running on fallback
-
-Recently epysode was reinstated after hardware failure. I took the opportunity to reinstall the machine. The backups are described in the repo (genenetwork org members have access)
-
-=> https://github.com/genenetwork/gn-services/blob/master/services/backups.org
-
-As epysode was one of the main sheepdog messaging servers I need to reinstate:
-
-* [X] scripts for sheepdog
-* [X] enable trim
-* [X] reinstate monitoring web services
-* [X] reinstate daily backup from penguin2
-* [X] CRON
-* [X] make sure messaging works through redis
-* [X] fix and propagate GN1 backup
-* [X] fix and propagate IPFS and gitea backups
-* [X] add GN1 backup
-* [X] add IPFS backup
-* [X] other backups
-* [ ] email on fail
-
-Tux01 is backed up now. Need to make sure it propagates to
-
-* [X] P2
-* [X] epysode
-* [X] rabbit
-* [X] Tux02
-* [ ] bacchus
-- 
cgit v1.2.3