1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
|
# Debugging and developing code
Once we get to the stage of having a working system container it would be nice to develop code against it. The idea is to take an existing running system container and start modifying code *inside* the container by brining in an external path.
First build and start a guix system container as described in
=> /topics/guix/guix-system-containers-and-how-we-use-them.gmi
The idea is to do less `guix pull' and system container builds, so as to speed up development. The advantage of using an existing system container is that the full deployment is the same on our other running systems! No more path hacks, in other words.
# GN3 in system container
The easiest one is GN3 because it is meant to give a quick turnaround on debugging and testing. Login to the container using nsenter or equal. Doing a `ps xau` should show the gn3 config used, e.g.
```
guile --no-auto
-compile /gnu/store/zm5cxkhy0gx6b7vyyr54dh99gk8zbncn-gunicorn-genenetwork3-pola-wrapper --workers 20 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/8lmjj0vv0616cgwy2dx56pg30rkvgsj0-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --env HOME=/tmp gn3.app:create_app()
```
the config file may contain something like
```
AUTH_DB = "/export/data/genenetwork-sqlite/auth.db"
DATA_DIR = "/export/data/genenetwork"
SPARQL_ENDPOINT = "http://localhost:8892/sparql"
SQL_URI = "mysql://webqtlout:webqtlout@localhost/db_webqtl"
XAPIAN_DB_PATH = "/export/data/genenetwork-xapian"
```
When building the container you can add a source path that is shared with the host machine. We can use that to share the source directory for GN3 with the path in the guix deploy script:
```
--share=/export/source/fallback-debug
```
in that directory we clone the genenetwork3 repo and rebuild the machine. After restarting the machine the path should be visible. E.g.
```
nsenter -at 1359047 /run/current-system/profile/bin/bash --login
root@genenetwork /# ls /export/source/fallback-debug/
genenetwork3/
```
Another useful command is running a command from outside the VM:
```
nsenter -at 1359047 /run/current-system/profile/bin/tail -f /var/log/gunicorn-genenetwork3.log
```
Next, after making a note of the port and paths with `ps xau`, we stop the running GN3 instance with
```
herd stop gunicorn-genenetwork3
```
Now we can start GN3 properly. We can reuse the gunicorn setup above, but for debugging it may be better to run a single threaded flask server on the same port. The wrapper contains all paths and PYTHON modules, so let's reuse that:
```
/gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/zm5cxkhy0gx6b7vyyr54dh99gk8zbncn-gunicorn-genenetwork3-pola-wrapper --workers 1 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/8lmjj0vv0616cgwy2dx56pg30rkvgsj0-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --env HOME=/tmp "gn3.app:create_app()"
```
Note the added quotes. The command will fail with 'No module named gn3'. Good! Now to load the source dir we need to make it visible. We'll use $SOURCES for that.
Through above shepherd command find the profile in use by printing pola-wrapper. You can use a local Ruby command, such as
```
/gnu/store/7ri578qarmn1cj2inl243xar6p7j1vxh-ruby-3.1.4/bin/ruby -n -e '$_ =~ /(\"\S+-profile)/; p $1 if $1' < /gnu/store/d7idvpad0d49l7647ci046w3xiz58xym-gunicorn-genenetwork3-pola-wrapper
"\"/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile
```
and set the profile to match
```
export PROFILE=/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile
```
> Note: you can find ruby in /gnu/store in the VM as it shares with the host
This worked running gunicorn directly and loading the PYTHONPATH worker source path with `--chdir`!
```
herd stop gunicorn-genenetwork3
root@genenetwork: cd /export/source/fallback-debug/genenetwork3
PYTHONPATH=$PROFILE/lib/python3.10/site-packages /gnu/store/hhn20xg4vag4xiib2d7d4c1vkm09dcav-gunicorn-20.1.0/bin/gunicorn --workers 1 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/592bscjpr6xyz8asn743iqzgczg8l947-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --chdir /export/source/fallback-debug/genenetwork3 --log-level debug --reload --env HOME=/tmp gn3.app:create_app\(\)
```
Make sure you are loading gn3 code from your source dir (e.g. by introducing an error). The commit for sharing sources is at
=> https://git.genenetwork.org/gn-machines/commit/?id=0d551870499c886f900a5b87b2040db25e9a00cc
Anyway, at this stage I can edit GN3 code outside the container and it will update with gunicorn. It leads to solving the shared path issue. But first we want to also run GN2 inside the system container.
# GN2 in a system container
We clone genenetwork2 in the source path, next we take a hint from shepherd:
```
/gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/vg8q4kdnkzy7skv04z57ngm8rqn7kvhd-gunicorn-genenetwork2-pola-wrapper --workers 20 --timeout 1200 --bind 127.0.0.1:8892 --env GN2_PROFILE=/gnu/store/jl6x90wdbwbs7c7zxnyz2kfd0qx8bf5h-profile --env GN2_SETTINGS=/gnu/store/gn9pr6kvmf1zlaskd1bqn1dssx4sy5lw-gn2.conf --env HOME=/tmp gn2.wsgi
```
and we tell herd to stop genenetwork2.
```
herd stop gunicorn-genenetwork2
PYTHONPATH=/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile/lib/python3.10/site-packages /gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/vg8q4kdnkzy7skv04z57ngm8rqn7kvhd-gunicorn-genenetwork2-pola-wrapper --workers 20 --timeout 1200 --bind 127.0.0.1:8892 --env GN2_PROFILE=/gnu/store/jl6x90wdbwbs7c7zxnyz2kfd0qx8bf5h-profile --env GN2_SETTINGS=/gnu/store/gn9pr6kvmf1zlaskd1bqn1dssx4sy5lw-gn2.conf --chdir /export/source/fallback-debug/genenetwork2 --env HOME=/tmp gn2.wsgi
```
and we get an Error: can't chdir to '/export/source/fallback-debug/genenetwork2'. Now I banged my head against the wall for a while and realized, after a back-and-forth with Arun, that guile starts a container without that path being visible! So we run GN2 and GN3 as *containers* inside a guix system container (a VM). OK, that means, next to specifying the path in the build, we also have to specify the source path inside the container definition. The upside is being explicit. The downside may be performance - we'll have to look into that later. This link suggests running a container in a VM is 40% slower:
=> https://blog.nestybox.com/2020/09/23/perf-comparison.html
But, we'll have to look into such optimizations later.
After adding the source dir and changing the permissions of the secrets file I can
```
export PROFILE=/gnu/store/d77wrqsb11igma3ay5mykc57mnzwc76q-profile
/export/source/fallback-debug/genenetwork2# /gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/47vplgxkcwd7vk3r71qvvfkwr9rcqlsl-gunicorn-genenetwork2-pola-wrapper --workers 1 --timeout 1200 --bind 127.0.0.1:8892 --env GN2_PROFILE=$PROFILE --env GN2_SETTINGS=/gnu/store/gn9pr6kvmf1zlaskd1bqn1dssx4sy5lw-gn2.conf --chdir /export/source/fallback-debug/genenetwork2 --pythonpath=$PROFILE/lib/python3.10/site-packages --log-level debug --reload --env HOME=/tmp gn2.wsgi
```
Note that we need the --pythonpath. I pick up that profile from pola-wrapper, as well as the R path etc with
```
export PROFILE=/gnu/store/v1nv6nnfsmvsi5aangj580f46741nvx6-profile
root@genenetwork /export/source/fallback-debug/genenetwork3# PATH=$PATH:$PROFILE/bin R_LIBS_USER=$PROFILE/site-library PYTHONPATH=$PROFILE/lib/python3.10/site-packages /gnu/store/hhn20xg4vag4xiib2d7d4c1vkm09dcav-gunicorn-20.1.0/bin/gunicorn --workers 1 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/592bscjpr6xyz8asn743iqzgczg8l947-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --chdir /export/source/fallback-debug/genenetwork3 --log-level debug --reload --env HOME=/tmp gn3.app:create_app\(\)
```
To run the tests you can do something like
```
export PROFILE=/gnu/store/v1nv6nnfsmvsi5aangj580f46741nvx6-profile
export AUTHLIB_INSECURE_TRANSPORT=true
export OAUTH2_ACCESS_TOKEN_GENERATOR="tests.unit.auth.test_token.gen_token"
PATH=$PATH:$PROFILE/bin R_LIBS_USER=$PROFILE/site-library PYTHONPATH=$PROFILE/lib/python3.10/site-packages pytest
```
# Locating scripts and fonts
In Guix VMs we can not assume the start dir is the base of the source code - as we do in development. Paths may need to be written and that happens at the package level. E.g.
=> https://git.genenetwork.org/guix-bioinformatics/commit/?id=6303fe046b944a380704eb02144cca777fee689b
The great thing is that, once it is done, it will always work with Guix! It is also a security feature that the full store is hard coded.
For fonts we have
```
gn2/utility/Plot.py
44:VERDANA_FILE = "./gn2/wqflask/static/fonts/verdana.ttf"
45:COUR_FILE = "./gn2/wqflask/static/fonts/courbd.ttf"
```
and rewrite them with
=> https://git.genenetwork.org/guix-bioinformatics/commit/?id=1a6522f3a5a7d0990e50fe5dd91f286f62e083e7
# Fixing shared paths
By default both GN2 and GN3 run as containers. We pass temporary files through the file sytem, so let's try and fix that first. The good news is that they only have to share the TMPDIR. First we share a new directory under /var/tmp for the system container. Next we have to tell GN2 and GN3 system containers to use /var/tmp/gn2. This was done in commit
=> https://git.genenetwork.org/gn-machines/commit/?id=831cf86b4fbf7b054640fa46eede6040ad01340f
# Showing debug output
Flask typically runs in a gunicorn. To have debug output the simple thing is to print to stderr with
```
sys.stderr.write("Example error output shows in gunicorn log")
```
Once the flask app runs it has its own logger settings. What we can do is set the app logging locally
```
from flask import current_app
current_app.logger.setLevel(logging.DEBUG) # Force debug level since we assume we are using it!
current_app.logger.debug("%s: %s", title_vals, value)
```
I have forced that in gn3/debug.py for now. Putting __pk__ around rqtl_cmd it turned out the script was not defined. The file it should be calling is ./scripts/rqtl_wrapper.R. There are some confusing settings in GN3.
```
rqtl_wrapper = current_app.config["RQTL_WRAPPER"]
```
http://127.0.0.1:8893/api/menu/generate/json
# Speeding up channel builds
The routine of `guix pull` for channel builds is quite elaborate in a development phase. Also, for every change you'll have to update the remote guix-bioinformatics repository - and that is not so great either. One solution is to *remove* the guix-bioinformatics channel and use the -L switch on a local repo instead.
```
git clone https://git.genenetwork.org/guix-bioinformatics/
```
Now you need to remove guix-bioinformatics from `.config/guix/channels.scm` and now do a guix pull using the local repo:
```
guix pull -p ~/opt/guix-dev
```
To get a channel list from an existing guix try `guix describe -f channel`. That can be pasted into a channels file. Anyway, you should end up with something like this:
```
~/opt/guix-dev/bin/guix describe
Generation 1 Apr 02 2024 10:13:37 (current)
guix b0b988c
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: b0b988c41c9e0e591274495a1b2d6f27fcdae15a
guix-forge 6c622a6
repository URL: https://git.systemreboot.net/guix-forge/
branch: main
commit: 6c622a67051c22eeefe37eedb17d427fbb70c99b
guix-past 921f845
repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
branch: master
commit: 921f845dc0dec9f052dcda479a15e787f9fd5b0a
```
Now we can use guix bioinformatics with
```
~/opt/guix-dev/bin/guix search -L ~/guix-bioinformatics genenetwork
```
and it should show the genenetwork stack. Now you can build a machine using -L. See for example:
=> gndev-deploy
|
