From 02a6d9ebc2e2c160874e2f52412cfc7be67b7231 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Fri, 21 Jun 2024 05:49:56 -0500 Subject: genecup: updated information - now running as a container --- issues/genecup-running-as-root.gmi | 6 ++++- .../per-service-profile-for-shepherd-services.gmi | 15 ++++++++--- topics/deploy/genecup.gmi | 30 ++++++++++++++++++++++ 3 files changed, 47 insertions(+), 4 deletions(-) create mode 100644 topics/deploy/genecup.gmi diff --git a/issues/genecup-running-as-root.gmi b/issues/genecup-running-as-root.gmi index c48924f..72d5704 100644 --- a/issues/genecup-running-as-root.gmi +++ b/issues/genecup-running-as-root.gmi @@ -1,5 +1,9 @@ # genecup is currently running as root on penguin2. +Is this still true for tux02? Fixed: genecup is now running as a container so we can close this. + +## Info + Currently when run using shepherd it fails like this: guix system: warning: Consider running 'guix pull' followed by @@ -32,7 +36,7 @@ A number of changes to different types of shepherd services (exec-command, make- * assigned: Efraim * type: failure * keywords: genecup, penguin2 -* status: unclear +* status: closed * priority: high ## Notes diff --git a/issues/per-service-profile-for-shepherd-services.gmi b/issues/per-service-profile-for-shepherd-services.gmi index 654a723..43c9118 100644 --- a/issues/per-service-profile-for-shepherd-services.gmi +++ b/issues/per-service-profile-for-shepherd-services.gmi @@ -1,11 +1,17 @@ # Each shepherd service needs its own guix profile. -Currently there are a bunch of shepherd services (on Penguin2, as user shepherd) who are using the global shepherd guix profile and loading guix-bioinformatics at service start time to start their services. This is bad because: +UPDATE: genecup is now running as a container on tux02. This is a model for getting rid of guix profiles. Note that we +do need to fixate guix-bioinformatics and guix-past channels *per* service. Otherwise updates are impossible. + +In time we may move to hosting these services in a guix system container (a VM). + +Currently there are a bunch of shepherd services (on tux02, as user shepherd) who are using the global shepherd guix profile and loading guix-bioinformatics at service start time to start their services. This is bad because: * we aren't using per-service guix-bioinformatics worktrees, so the branch changes between service refreshes * service start times are high because we have to rebuild the service (due to above) * services which are restarted are in an "untested" state until they work + ## Services which need to be migrated: * covid19-pubseq (uses shared (broken!) guix profile, uses shared config directory)(currently running by hand by Pjotr) @@ -22,6 +28,9 @@ Currently there are a bunch of shepherd services (on Penguin2, as user shepherd) ## Services which already have their own profile: +Note that the profiles essentially just contain the guix binary, acting like a time machine. It is +used to fire up a guix shell/environment: + * bnw * cronjob-gitea * cronjob-pubmed (genecup) @@ -34,8 +43,8 @@ Currently there are a bunch of shepherd services (on Penguin2, as user shepherd) ## Tags -* assigned: efraim +* assigned: efraim, pjotrp * type: bug -* status: unclear +* status: in progress * priority: high * keywords: shepherd, system administration diff --git a/topics/deploy/genecup.gmi b/topics/deploy/genecup.gmi new file mode 100644 index 0000000..2054844 --- /dev/null +++ b/topics/deploy/genecup.gmi @@ -0,0 +1,30 @@ +# Deploying genecup + +On Tux02 as shepherd user I can build: + +``` +./guix/bin/guix build -L ~/services/genecup/guix-past/modules -L ~/services/genecup/guix-bioinformatics genecup-latest-with-tensorflow-native -c 24 -M 8 +``` + +To run genecup in a container we have a trick to include the source code by cd'ing into the built version + +``` +export EDIRECT_PUBMED_MASTER=/export2/PubMed +export TMPDIR=/export/ratspub/tmp +export NLTK_DATA=/export2/PubMed/nltk_data + +# This version for the genecup guix profile which includes guix-bioinformatics, using genecup-channels.scm. +# shepherd@tux02:~/services/genecup$ ./guix/bin/guix build -L ~/services/genecup/guix-past/modules -L ~/services/genecup/guix-bioinformatics genecup-latest-with-tensorflow-native -c 24 -M 8 + +cd /home/shepherd/services/genecup +cd $(/home/shepherd/services/genecup/guix/bin/guix build -L /home/shepherd/services/genecup/guix-past/modules -L /home/shepherd/services/genecup/guix-bioinformatics genecup-latest-with-tensorflow-native -c 4) + +/home/shepherd/services/genecup/guix/bin/guix shell -C -N --expose=/etc/ssl/certs --expose=$EDIRECT_PUBMED_MASTER --share=$TMPDIR -L /home/shepherd/services/genecup/guix-past/modules -L /home/shepherd/services/genecup/guix-bioinformatics genecup-latest-with-tensorflow-native coreutils-minimal -- env TMPDIR=$TMPDIR EDIRECT_PUBMED_MASTER=$EDIRECT_PUBMED_MASTER NLTK_DATA=$NLTK_DATA ./server.py +``` + +That means you can also develop software running in a container with the last command by switching into a repo: + +``` +cd $repo +run last command using guix shell +``` -- cgit v1.2.3