7 files changed, 316 insertions, 17 deletions
diff --git a/topics/genenetwork/genenetwork-services.gmi b/topics/genenetwork/genenetwork-services.gmi
new file mode 100644
index 0000000..717fdd8
--- /dev/null
+++ b/topics/genenetwork/genenetwork-services.gmi
@@ -0,0 +1,122 @@
+# GeneNetwork Services
+
+## Tags
+
+* type: documentation
+* keywords: documentation, docs, doc, services, genenetwork services
+
+## GeneNetwork Core Services
+
+GeneNetwork is composed of a number of different services. This document attempts to document all the services that make up GeneNetwork and document what links give access to the services.
+
+### GeneNetwork2
+
+This is the main user-interface to the entire GeneNetwork system.
+
+#### Links
+
+=> https://github.com/genenetwork/genenetwork2 Repository
+=> https://genenetwork.org/ GN2 on production
+=> https://fallback.genenetwork.org/ GN2 on old production
+=> https://cd.genenetwork.org/ GN2 on CI/CD
+=> https://staging.genenetwork.org/ GN2 on staging
+
+### GeneNetwork3
+
+This is the main API server for GeneNetwork.
+
+#### Links
+
+=> https://github.com/genenetwork/genenetwork3 Repository
+=> https://genenetwork.org/api3/ GN3 on production
+=> https://fallback.genenetwork.org/api3/ GN3 on old production
+=> https://cd.genenetwork.org/api3/ GN3 on CI/CD
+=> https://staging.genenetwork.org/api3/ GN3 on staging
+
+### Sparql Service
+
+The SparQL service is served from a Virtuoso-OSE service.
+
+=> https://issues.genenetwork.org/topics/deploy/our-virtuoso-instances We have notes on our virtuoso instances here.
+
+
+#### Links
+
+=> https://github.com/genenetwork/genenetwork3 Repository
+=> https://sparql.genenetwork.org/sparql/ sparql-service on production
+* ??? sparql-service on old production
+* ??? sparql-service on CI/CD
+* ??? sparql-service on staging
+
+### GN-Auth
+
+This is the authorisation server for the GeneNetwork system.
+
+#### Links
+
+=> https://git.genenetwork.org/gn-auth/ Repository
+=> https://auth.genenetwork.org/ gn-auth on production
+=> https://fallback.genenetwork.org/gn-auth/ gn-auth on old production
+* ??? gn-auth on CI/CD
+=> https://staging-auth.genenetwork.org/ gn-auth on staging
+
+### GN-Uploader
+
+This service is to be used for uploading data to GeneNetwork. It is currently in development (best case, alpha).
+
+#### Links
+
+=> https://git.genenetwork.org/gn-uploader/ Repository
+* ??? gn-uploader on production
+* ??? gn-uploader on old production
+* ??? gn-uploader on CI/CD
+=> https://staging-uploader.genenetwork.org/ gn-uploader on staging
+
+### Aliases Server
+
+An extra server to respond with aliases for genetic (etc.) symbols.
+
+This is currently a project in racket, but we should probably pull in the features in this repository into one of the others (probably GeneNetwork3) and trash this repository.
+
+#### Links
+
+=> https://github.com/genenetwork/gn3 Repository
+=> https://genenetwork.org/gn3/ aliases-server on production
+=> https://fallback.genenetwork.org/gn3/ aliases-server on old production
+=> https://cd.genenetwork.org/gn3/ aliases-server on CI/CD
+=> https://staging.genenetwork.org/gn3/ aliases-server on staging
+
+### Markdown Editing Server
+
+#### Links
+
+=> https://git.genenetwork.org/gn-guile/ Repository
+=> https://genenetwork.org/facilities/ markdown-editing-server on production
+=> https://fallback.genenetwork.org/facilities/ markdown-editing-server on old production
+=> https://cd.genenetwork.org/facilities/ markdown-editing-server on CI/CD
+=> https://staging.genenetwork.org/facilities/ markdown-editing-server on staging
+
+## Support Services
+
+These are other services that support the development and maintenance of the core services.
+
+### Issue Tracker
+
+We use a text-based issue tracker that is accessible via
+=> https://issues.genenetwork.org/
+
+The repository for this service is at
+=> https://github.com/genenetwork/gn-gemtext-threads/
+
+### Repositories Server
+
+This is where a lot of the genenetwork repositories live. You can access it at
+=> https://git.genenetwork.org/
+
+### Continuous Integration Service
+
+…
+
+=> https://ci.genenetwork.org/
+
+### …
diff --git a/topics/guix/guix-profiles.gmi b/topics/guix/guix-profiles.gmi
index 578bb82..8cf41d8 100644
--- a/topics/guix/guix-profiles.gmi
+++ b/topics/guix/guix-profiles.gmi
@@ -16,7 +16,7 @@ Alternatively put the following into a channels.scm file.
 ```
 (list (channel
        (name 'gn-bioinformatics)
-       (url "https://gitlab.com/genenetwork/guix-bioinformatics")
+       (url "https://git.genenetwork.org/guix-bioinformatics")
        (branch "master")))
 ```
 Build a profile using
diff --git a/topics/gunicorn/deploying-app-under-url-prefix.gmi b/topics/gunicorn/deploying-app-under-url-prefix.gmi
new file mode 100644
index 0000000..b2e382f
--- /dev/null
+++ b/topics/gunicorn/deploying-app-under-url-prefix.gmi
@@ -0,0 +1,121 @@
+# Deploying Your Flask Application Under a URL Prefix With GUnicorn
+
+## TAGS
+
+* type: doc, documentation, docs
+* author: fredm, zachs
+* keywords: flask, gunicorn, SCRIPT_NAME, URL prefix
+
+## Introduction
+
+You have your application and are ready to deploy it, however, for some reason, you want to deploy it under a URL prefix, rather than at a top-level-domain.
+
+This short article details the things you need to set up.
+
+## Set up Your WebServer (Nginx)
+
+You need to tell your webserver to serve the application under a particular url prefix. You do this using that particular webserver's reverse-proxying configurations: For this article, we will use Nginx as the server.
+
+Normally, you'd simply do something like:
+
+```
+server {
+    server_name your.server.domain
+
+    ⋮
+
+    location /the-prefix/ {
+        proxy_pass    http://127.0.0.1:8080/;
+        proxy_set_header Host $host;
+        ⋮
+    }
+
+    ⋮
+}
+```
+
+Here, your top-level domain will be https://your.server.domain and you therefore want to access your shiny new application at https://your.server.domain/the-prefix/
+
+For a simple application, with no sessions or anything, this should work, somewhat, though you might run into trouble with things like static files (e.g. css, js, etc) if the application does not use the same ones as that one on the TLD.
+
+If you are using sessions, you might also run into an issue where there is an interaction in the session management of both applications, especially if the application on the TLD makes use of services from the application at the url prefix. This is mostly due to redirects from the url-prefix app getting lost and hitting the TLD app.
+
+To fix this, we change the configuration above to:
+
+```
+server {
+    server_name your.server.domain
+
+    ⋮
+
+    location /the-prefix/ {
+        proxy_pass    http://127.0.0.1:8080/the-prefix/;
+        proxy_set_header Host $host;
+        ⋮
+    }
+
+    ⋮
+}
+```
+
+but now, you get errors, since there is no endpoint in your shiny new app that in at the route /the-prefix/***.
+
+Enter Gunicorn!
+   
+
+## Setting up SCRIPT_NAME for GUnicorn
+
+### The "Hacky" Way
+
+At the point of invocation of GUnicorn, we set the SCRIPT_NAME environment variable to the value "/the-prefix" — note that there is no trailing slash; this is very important. You should now have something like:
+
+```
+$ export SCRIPT_NAME="/the-prefix"
+$ gunicorn --bind 0.0.0.0:8082 --workers …
+```
+
+The first line tells GUnicorn what the URL prefix is. It will use this to compute what URL to pass to the flask application.
+
+Example, say you try accessing the endpoint
+
+```
+https://your.server.domain/the-prefix/auth/authorise?response_type=code&client_id=some-id&redirect_uri=some-uri
+```
+
+Gunicorn will split that URL into 2 parts using the value of the SCRIPT_NAME environment variable, giving you:
+
+* https://your.server.domain
+* /auth/authorise?response_type=code&client_id=some-id&redirect_uri=some-uri
+
+It will then pass on the second part to flask. This is why the value of SCRIPT_NAME should not have a trailing slash.
+
+Note that using the SCRIPT_NAME environment variable is a convenience feature provided by GUnicorn, not a WSGI feature. If you ever change your WSGI server, there is no guarantee this fix will work.
+
+### Using WSGI Routing MiddleWare
+
+A better way is to make use of a WSGI routing middleware. You could do this by defining a separate WSGI entry point in your application's repository.
+
+```
+# wsgi_url_prefix.py
+from werkzeug.wrappers import Response
+from werkzeug.middleware.dispatcher import DispatcherMiddleware
+
+from app import create_app
+
+def init_prefixed_app(theapp):
+    theapp.wsgi_app = DispatcherMiddleware(
+        Response("Not Found", 404),
+        {
+            "/the-prefix": the_app.wsgi_app
+        })
+    return theapp
+
+
+app = init_prefixed_app(create_app())
+```
+
+## References
+
+=> https://docs.gunicorn.org/en/latest/faq.html#how-do-i-set-script-name
+=> https://dlukes.github.io/flask-wsgi-url-prefix.html
+=> https://www.reddit.com/r/Python/comments/juwj3x/comment/gchdsld/
diff --git a/topics/meetings/jnduli_bmunyoki.gmi b/topics/meetings/jnduli_bmunyoki.gmi
index c4832c6..8afe4f1 100644
--- a/topics/meetings/jnduli_bmunyoki.gmi
+++ b/topics/meetings/jnduli_bmunyoki.gmi
@@ -1,22 +1,39 @@
 # Meeting Notes
 
-## 2024-10-11
-* @priscilla @flisso: Try out API endpoints that don't require auth.
-* @jnduli Harden hook system for gn-auth.
+## 2024-10-18
+* @priscilla @flisso: Set up mariadb and virtuoso to test out some GN3 endpoints.
+* @priscilla @flisso @bmunyoki: Improve docs while hacking on the above.
 * @jnduli Remove gn-auth code from GN3.
-* @jnduli @bonfacem Finish up RIF Editing project.
-* @jnduli @alexm Create issue on describing the monitoring system.
-* @jnduli @alexm Create issue on prompt engineering in GN to improve what we already have.
-* @alex Work on R/Qtl.  NOTE: @jnduli/@bonfacem help out with this.
-* @alex: Review @bmunyoki's work on RIF/Indexing.
-* @flisso: Make sure we have C Elegans dataset and MIKK genotypes to production.
-* @flisso: Make sure we have HS Rats in testing stage.
-* @flisso: Make progress in learning back-end coding WRT GN.
+* @jnduli Resolve current issue with broken auth in gn-qa.
+* @jnduli @alexm Work on the R/Qtl design doc.
+* @flisso MIKK genotyping.
+* @flisso Make sure we have C Elegans and HS Rats dataset to testing, and have the genotyping pipeline working.
 * @shelbys: Modify existing Grant write-up for pangenomes.
-* @shelbys: Finish getting all the R2R scores from the first study.
+* @shelbys @bonfacem: Getting RDF into R2R.
 * @bonfacem RIF Indexing for RIF page in Xapian.
 * @bonfacem Work on properly containerizing gn-guile.
 
+## 2024-10-11
+* WIP @priscilla @flisso: Try out API endpoints that don't require auth.  NOTE: Priscilla got to set-up guix channels for gn3.  Felix ran into problems.  Priscilla set up the MySQL in her Ubuntu system.
+* NOT DONE: @jnduli Harden hook system for gn-auth.
+* WIP: @jnduli Remove gn-auth code from GN3.  NOTE: Sent latest patches to Fred.  Running issue, some patches may have caused gn-qa to fail.
+* DONE: @jnduli @bonfacem Finish up RIF Editing project.
+* NOT DONE: @jnduli @alexm Create issue on describing the monitoring system. 
+* NOT DONE: @jnduli @alexm Create issue on prompt engineering in GN to improve what we already have.
+* WIP: @alex Work on R/Qtl.  NOTE: @jnduli/@bonfacem help out with this.  NOTE: Finished writing the design doc for gn-qa.
+* DONE: Looked at documentation for R/Qtl.
+* NOT DONE: @alex: Review @bmunyoki's work on RIF/Indexing.
+* WIP: @flisso: Make sure we have C Elegans dataset and MIKK genotypes to production.   NOTE: Issues with data entry scripts.  Fred/Zach working to set up test environment.
+* WIP: @flisso: MIKK genotyping.  NOTE: Still testing the pipeline.  Halfway there.
+* NOT DONE: @flisso: Make sure we have HS Rats in testing stage.
+* WIP: @flisso: Make progress in learning back-end coding WRT GN.  NOTE: Issue setting up GN3.
+* WIP: @shelbys: Modify existing Grant write-up for pangenomes.  NOTE: Reviewed by Pj and Eric.  More mods based of feedback.  Paper got accepted by BioArxiv.  Added some docs to R2R evaluation code.
+* DONE: @shelbys: Finish getting all the R2R scores from the first study. NOTE: Got scores for all the scores from first papers using R2R instead of Fahamu.
+* NOT DONE: @bonfacem RIF Indexing for RIF page in Xapian.
+* WIP: @bonfacem Work on properly containerizing gn-guile.
+* DONE: @bonfacem Fix the gn-transform-database in CI.  Sent patches to Arun for review.
+* DONE: @bonfacem Fixed broken utf-8 characters in gn-gemtext.
+
 ## 2024-10-04
 * IN PROGRESS: @priscilla @bonfacem Setting up GN3.  @priscilla try out API endpoints that don't require auth. NOTE: @priscilla Able to set up guix as a package manager.  Trouble with Guix set-up with GN3.  @bonfacem good opportunity to improve docs in GN3.
 * IN PROGRESS: @jnduli Harden hook system for gn-auth.
diff --git a/topics/octopus/set-up-guix-for-new-users.gmi b/topics/octopus/set-up-guix-for-new-users.gmi
new file mode 100644
index 0000000..f459559
--- /dev/null
+++ b/topics/octopus/set-up-guix-for-new-users.gmi
@@ -0,0 +1,38 @@
+# Set up Guix for new users
+
+This document describes how to set up Guix for new users on a machine in which Guix is already installed (such as octopus01).
+
+## Create a per-user profile for yourself by running your first guix pull
+
+"Borrow" some other user's guix to run guix pull. In the example below, we use root's guix, but it might as well be any guix.
+```
+$ /var/guix/profiles/per-user/root/current-guix/bin/guix pull
+```
+This should create your very own Guix profile at ~/.config/guix/current. You may invoke guix from this profile as
+```
+$ ~/.config/guix/current/bin/guix ...
+```
+But, you'd normally want to make this more convenient. So, add ~/.config/guix/current/bin to your PATH. To do this, add the following to your ~/.profile
+```
+GUIX_PROFILE=~/.config/guix/current
+. $GUIX_PROFILE/etc/profile
+```
+Thereafter, you may run any guix command simply as
+```
+$ guix ...
+```
+
+## Pulling from a different channels.scm
+
+By default, guix pull pulls the latest commit of the main upstream Guix channel. You may want to pull from additional channels as well. Put the channels you want into ~/.config/guix/channels.scm, and then run guix pull. For example, here's a channels.scm if you want to use the guix-bioinformatics channel.
+```
+$ cat ~/.config/guix/channels.scm
+(list (channel
+       (name 'gn-bioinformatics)
+       (url "https://git.genenetwork.org/guix-bioinformatics")
+       (branch "master")))
+```
+And,
+```
+$ guix pull
+```
diff --git a/topics/octopus/slurm-upgrade.gmi b/topics/octopus/slurm-upgrade.gmi
index 1f09a79..822f68e 100644
--- a/topics/octopus/slurm-upgrade.gmi
+++ b/topics/octopus/slurm-upgrade.gmi
@@ -14,7 +14,7 @@ slurm can only be upgraded safely in small version increments. For example, it i
 
 Stop the slurmdbd, slurmctld and slurmd services.
 ```
-# systemctl stop slurmdbd slurmctld slurmd
+# systemctl stop slurmdbd slurmctld slurmd slurmrestd
 ```
 Backup the slurm StateSaveLocation (/var/spool/slurmd/ctld in our case) and the slurm configuration directory.
 ```
@@ -50,6 +50,7 @@ Reload the new systemd configuration files. Then, start the slurmdbd, slurmctld
 # systemctl start slurmdbd
 # systemctl start slurmctld
 # systemctl start slurmd
+# systemctl start slurmrestd
 ```
 
 ## Upgrade slurm on the worker nodes
@@ -74,14 +75,14 @@ Copy over any configuration file changes from octopus01. Then, reload the new sy
 
 It is a lot of typing to run the same command on all worker nodes. You could make this a little less cumbersome with the following bash for loop.
 ```
-for node in octopus02 octopus03 octopus05 octopus06 octopus07 octopus08 octopus09 octopus10 octopus11 tux06 tux07 tux08 tux09;
+for node in octopus02 octopus03 octopus05 octopus06 octopus07 octopus08 octopus09 octopus10 octopus11 tux05 tux06 tux07 tux08 tux09;
 do
     ssh $node your command
 done
 ```
 You can even do this for sudo commands using the -S flag of sudo that makes it read the password from stdin. Assuming your password is in the pass password manager, the bash for loop would then look like:
 ```
-for node in octopus02 octopus03 octopus05 octopus06 octopus07 octopus08 octopus09 octopus10 octopus11 tux06 tux07 tux08 tux09;
+for node in octopus02 octopus03 octopus05 octopus06 octopus07 octopus08 octopus09 octopus10 octopus11 tux05 tux06 tux07 tux08 tux09;
 do
     pass octopus | ssh $node sudo -S your command
 done
diff --git a/topics/systems/restore-backups.gmi b/topics/systems/restore-backups.gmi
index 518c56d..b97af2b 100644
--- a/topics/systems/restore-backups.gmi
+++ b/topics/systems/restore-backups.gmi
@@ -26,7 +26,7 @@ The last backup on 'tux02' is from October 2022 - after I did a reinstall. That
 
 According to sheepdog the drops are happening to 'space' and 'epysode', but 'tux02' is missing:
 
-=> https://rabbit.genenetwork.org/sheepdog/index.html
+=> http://sheepdog.genenetwork.org/sheepdog/status.html
 
 ## Mariadb