1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
|
#+STARTUP: content
#+TITLE: Building Uploader Container (2024-03-21)
#+AUTHOR: Frederick Muriuki Muriithi
#+OPTIONS: ^:{}
* Log
Fix the [[https://git.genenetwork.org/gn-uploader/commit/?id=3958ce457f621f1b75fa8843ba859d8e6015a6c7][channel introduction documentation]] on =gn-uploader=
On =tux02= do:
#+BEGIN_SRC shell
$ guix pull -p ~/opt/guix -C channels.scm
#+END_SRC
where =channels.scm= has the following content
#+BEGIN_SRC scheme
(list
(channel
(name 'guix-bioinformatics)
(url "https://git.genenetwork.org/guix-bioinformatics/")
(branch "master"))
(channel
(name 'guix-forge)
(url "https://git.systemreboot.net/guix-forge/")
(branch "main")
(introduction
(make-channel-introduction
"0432e37b20dd678a02efee21adf0b9525a670310"
(openpgp-fingerprint
"7F73 0343 F2F0 9F3C 77BF 79D3 2E25 EE8B 6180 2BB3"))))
(channel
(name 'guix-past)
(url "https://gitlab.inria.fr/guix-hpc/guix-past")
(introduction
(make-channel-introduction
"0c119db2ea86a389769f4d2b9c6f5c41c027e336"
(openpgp-fingerprint
"3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5"))))
(channel
(name 'gn-auth)
(url "https://git.genenetwork.org/gn-auth")
(branch "main"))
(channel
(name 'gn-uploader)
(url "https://git.genenetwork.org/gn-uploader")
(branch "main")
(introduction
(make-channel-introduction
"93c77842315d304abbfc27d78d98b7d42da32a61"
(openpgp-fingerprint
"F370 F409 854B 90E3 52F3 AB01 362B 0BB8 B81D 5A42")))))
#+END_SRC
Pull was successful!
After that, we get the following on =tux02=:
#+BEGIN_SRC shell
$ ~/opt/guix/bin/guix describe --format=channels
hint: Consider installing the `glibc-locales' package and defining
`GUIX_LOCPATH', along these lines:
guix install glibc-locales
export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"
See the "Application Setup" section in the manual, for more info.
(list (channel
(name 'gn-uploader)
(url "https://git.genenetwork.org/gn-uploader")
(branch "main")
(commit
"3958ce457f621f1b75fa8843ba859d8e6015a6c7")
(introduction
(make-channel-introduction
"93c77842315d304abbfc27d78d98b7d42da32a61"
(openpgp-fingerprint
"F370 F409 854B 90E3 52F3 AB01 362B 0BB8 B81D 5A42"))))
(channel
(name 'gn-auth)
(url "https://git.genenetwork.org/gn-auth")
(branch "main")
(commit
"ade6c76208bd45c24cdd0e73bf3570e6a2bfa131"))
(channel
(name 'guix-forge)
(url "https://git.systemreboot.net/guix-forge/")
(branch "main")
(commit
"6c622a67051c22eeefe37eedb17d427fbb70c99b")
(introduction
(make-channel-introduction
"0432e37b20dd678a02efee21adf0b9525a670310"
(openpgp-fingerprint
"7F73 0343 F2F0 9F3C 77BF 79D3 2E25 EE8B 6180 2BB3"))))
(channel
(name 'guix-bioinformatics)
(url "https://git.genenetwork.org/guix-bioinformatics/")
(branch "master")
(commit
"2fdbf2445e5293ba9a12b154c58774e73dfeabe5"))
(channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git")
(branch "master")
(commit
"b0b988c41c9e0e591274495a1b2d6f27fcdae15a")
(introduction
(make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
"BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA"))))
(channel
(name 'guix-past)
(url "https://gitlab.inria.fr/guix-hpc/guix-past")
(branch "master")
(commit
"921f845dc0dec9f052dcda479a15e787f9fd5b0a")
(introduction
(make-channel-introduction
"0c119db2ea86a389769f4d2b9c6f5c41c027e336"
(openpgp-fingerprint
"3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5")))))
#+END_SRC
Now, build the container:
#+BEGIN_SRC shell
env PATH="/home/fredm/opt/guix/bin:${PATH}" ./uploader-deploy.sh
︙
successfully built /gnu/store/0fwp30li4ngfmradkiyc81cdmapm04bf-gn-uploader-0.0.1-a4ed5d82.drv
︙
/gnu/store/4rfwx4d376kbql9y31cfldf3prcfrvx2-run-container
︙
#+END_SRC
Restart the container
#+BEGIN_SRC shell
$ sudo systemctl stop genenetwork-uploader-container.service
$ sudo systemctl start genenetwork-uploader-container.service
$ sudo systemctl status genenetwork-uploader-container.service
#+END_SRC
Check logs in =/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log=
Still getting:
#+BEGIN_EXAMPLE
︙
2024-03-21 02:18:29 from gn2.wqflask.oauth2.checks import user_logged_in
2024-03-21 02:18:29 File "/gnu/store/2qhgkf5j649kv57nw4lzsbwlnb1790vq-profile/lib/python3.10/site-packages/gn2/wqflask/oauth2/checks.py", line 10, i
n <module>
2024-03-21 02:18:29 from .client import authserver_uri, oauth2_clientid, oauth2_clientsecret
2024-03-21 02:18:29 ImportError: cannot import name 'authserver_uri' from partially initialized module 'gn2.wqflask.oauth2.client' (most likely due to
a circular import) (/gnu/store/2qhgkf5j649kv57nw4lzsbwlnb1790vq-profile/lib/python3.10/site-packages/gn2/wqflask/oauth2/client.py)
2024-03-21 02:18:29 [2024-03-21 02:18:29 +0000] [9] [INFO] Worker exiting (pid: 9)
︙
#+END_EXAMPLE
When we verify where this comes from:
#+BEGIN_SRC shell
$ ls -l /gnu/store/2qhgkf5j649kv57nw4lzsbwlnb1790vq-profile/lib/python3.10/site-packages/ | grep gn2
lrwxrwxrwx 1 root root 104 Dec 31 1969 gn2 -> /gnu/store/vfa2zdla7ixbb2l5i7k6xnlh5d1rshz9-genenetwork2-3.11-4.1e81c74/lib/python3.10/site-packages/gn2
#+END_SRC
We see that the code being run is from [[https://github.com/genenetwork/genenetwork2/commit/1e81c74ee8752ad2913b4e610ba7575638755385][GeneNetwork2: commit 1e81c74]] from
05^{th} March 2024. In [[https://git.genenetwork.org/guix-bioinformatics/tree/gn/packages/genenetwork.scm?id=2fdbf2445e5293ba9a12b154c58774e73dfeabe5#n190][=guix-bioinformatics=, however]], we have specified a newer
[[https://github.com/genenetwork/genenetwork2/commit/b9e04ad79e0039edba25d58f8bc03e4d2a17583b][commit b9e04ad7]] from 07^{th} March 2024.
Stop uploader service and delete logs:
#+BEGIN_SRC shell
$ sudo systemctl stop genenetwork-uploader-container.service
$ sudo rm -fv /export2/guix-containers/genenetwork/uploader/var/log/gunicorn-g*
removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log'
removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork3.log'
removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-gn-auth.log'
removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-gn-uploader.log'
$ ls /export2/guix-containers/genenetwork/uploader/var/log/
debug gunicorn-genenetwork2.log mcron.log messages mysqld.log nscd.log secure.1.gz wtmp
guix-daemon.log maillog mcron.log.1.gz messages.1.gz nginx secure virtuoso.log
#+END_SRC
The file =/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log= is still showing up.
Stop all containers for good measure:
#+BEGIN_SRC shell
$ sudo systemctl stop genenetwork-uploader-container.service
$ sudo systemctl stop genenetwork-container.service
$ sudo systemctl stop genenetwork-development-container.service
$ sudo rm -fv /export2/guix-containers/genenetwork/uploader/var/log/gunicorn-g*
removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log'
$ ls /export2/guix-containers/genenetwork/uploader/var/log/
debug gunicorn-genenetwork2.log mcron.log messages mysqld.log nscd.log secure.1.gz wtmp
guix-daemon.log maillog mcron.log.1.gz messages.1.gz nginx secure virtuoso.log
#+END_SRC
The file =/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log= is still showing up.
Something else is running and recreating the file and I have no idea what it is.
Restart all containers:
#+BEGIN_SRC shell
$ sudo systemctl start genenetwork-uploader-container.service
$ sudo systemctl start genenetwork-container.service
$ sudo systemctl start genenetwork-development-container.service
#+END_SRC
* Log 2024-03-21T14:57+03:00UTC
** Reset profile
List installed packages
#+begin_src shell
/usr/local/guix-profiles/guix-daemon/bin/guix \
package -p ~/opt/guix --list-installed
#+end_src
which gave:
#+begin_example
guix-past 921f845 out /gnu/store/7sljixsgvsgdiafshqybdrl8m0f2j8v5-guix-past
guix b0b988c out /gnu/store/dhisx07fngb69qkj4gnlzz5iyakw25zy-guix-b0b988c41
guix-bioinformatics 2fdbf24 out /gnu/store/4vgrwfaxbvixh1jpsq9wqnm5jc7rhd0r-guix-bioinformatics
guix-forge 6c622a6 out /gnu/store/5bc8079iyd8z3n4l3np09yqgqhdzailv-guix-forge
gn-auth ade6c76 out /gnu/store/l52qdmn3c3liz0pywkndn9j49a9axh6i-gn-auth
gn-uploader 3958ce4 out /gnu/store/adqg6vxq9z2ds0y6a275yqaijv3vslpw-gn-uploader
#+end_example
remove all installed packages
#+begin_src shell
/usr/local/guix-profiles/guix-daemon/bin/guix \
package -p ~/opt/guix -r guix-past guix guix-bioinformatics guix-forge \
gn-auth gn-uploader
#+end_src
Verify everything was uninstalled by listing packages again as done previously.
Now list generations:
#+begin_src shell
/usr/local/guix-profiles/guix-daemon/bin/guix \
package -p ~/opt/guix --list-generations
#+end_src
which gave (truncated for readability):
#+begin_example
Generation 7 Nov 12 2023 20:45:25
guix-past 1e25b23 out /gnu/store/91yjvk23arknxlavd1m7rbva3qnrx6fp-guix-past
guix 330b94e out /gnu/store/rsb6nqychig8qdan23rj6w5m7q2kha22-guix-330b94e8b
︙
,- guix b0b988c out /gnu/store/8qrhzflvlp71ijsg6wv13fpcsxxq0n52-guix-b0b988c41
Generation 16 Mar 21 2024 07:04:36 (current)
,- gn-uploader 3958ce4 out /gnu/store/adqg6vxq9z2ds0y6a275yqaijv3vslpw-gn-uploader
,- gn-auth ade6c76 out /gnu/store/l52qdmn3c3liz0pywkndn9j49a9axh6i-gn-auth
,- guix-forge 6c622a6 out /gnu/store/5bc8079iyd8z3n4l3np09yqgqhdzailv-guix-forge
,- guix-bioinformatics 2fdbf24 out /gnu/store/4vgrwfaxbvixh1jpsq9wqnm5jc7rhd0r-guix-bioinformatics
,- guix b0b988c out /gnu/store/dhisx07fngb69qkj4gnlzz5iyakw25zy-guix-b0b988c41
,- guix-past 921f845 out /gnu/store/7sljixsgvsgdiafshqybdrl8m0f2j8v5-guix-past
#+end_example
Delete all generations except latest
#+begin_src shell
/usr/local/guix-profiles/guix-daemon/bin/guix \
package -p ~/opt/guix --delete-generations=7..16
#+end_src
giving
#+begin_example
guix package: warning: not removing generation 16, which is current
deleting /home/fredm/opt/guix-7-link
deleting /home/fredm/opt/guix-8-link
deleting /home/fredm/opt/guix-9-link
deleting /home/fredm/opt/guix-10-link
deleting /home/fredm/opt/guix-11-link
deleting /home/fredm/opt/guix-12-link
deleting /home/fredm/opt/guix-13-link
deleting /home/fredm/opt/guix-14-link
deleting /home/fredm/opt/guix-15-link
︙
#+end_example
Now delete symlinks to ensure next ~guix gc~ cleans up correctly
#+begin_src shell
rm -fv ~/opt/guix*
#+end_src
Now recreate a new profile with minimal channels:
#+begin_src shell
/usr/local/guix-profiles/guix-daemon/bin/guix \
pull -p ~/opt/guix -C minimal-channels.scm
#+end_src
Where =minimal-channels.scm= has the following content:
#+begin_src scheme
(list
(channel
(name 'guix-bioinformatics)
(url "https://git.genenetwork.org/guix-bioinformatics/")
(branch "master"))
(channel
(name 'guix-past)
(url "https://gitlab.inria.fr/guix-hpc/guix-past")
(introduction
(make-channel-introduction
"0c119db2ea86a389769f4d2b9c6f5c41c027e336"
(openpgp-fingerprint
"3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5")))))
#+end_src
Let's see what the new profile has:
#+begin_src shell
~/opt/guix/bin/guix describe
#+end_src
which gives:
#+begin_example
Generation 1 Mar 21 2024 07:34:58 (current)
guix-bioinformatics 2fdbf24
repository URL: https://git.genenetwork.org/guix-bioinformatics/
branch: master
commit: 2fdbf2445e5293ba9a12b154c58774e73dfeabe5
guix-past 921f845
repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
branch: master
commit: 921f845dc0dec9f052dcda479a15e787f9fd5b0a
guix b0b988c
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: b0b988c41c9e0e591274495a1b2d6f27fcdae15a
#+end_example
Awesome. Now, let's add the profile to the front of the path and build the container afresh:
#+begin_src shell
$ cd /home/fredm/gn-machines
$ export PATH="/home/fredm/opt/guix/bin:${PATH}"
$ ./uploader-deploy.sh
#+end_src
I get:
#+begin_example
guix system: error: failed to load 'uploader.scm':
ice-9/boot-9.scm:3330:6: In procedure resolve-interface:
no code for module (forge acme)
#+end_example
I need the =guix-forge= channel too. Add it to the channels:
#+begin_src scheme
;; minimal-channels.scm
(list
(channel
(name 'guix-bioinformatics)
(url "https://git.genenetwork.org/guix-bioinformatics/")
(branch "master"))
(channel
(name 'guix-past)
(url "https://gitlab.inria.fr/guix-hpc/guix-past")
(introduction
(make-channel-introduction
"0c119db2ea86a389769f4d2b9c6f5c41c027e336"
(openpgp-fingerprint
"3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5"))))
(channel
(name 'guix-forge)
(url "https://git.systemreboot.net/guix-forge/")
(branch "main")
(introduction
(make-channel-introduction
"0432e37b20dd678a02efee21adf0b9525a670310"
(openpgp-fingerprint
"7F73 0343 F2F0 9F3C 77BF 79D3 2E25 EE8B 6180 2BB3")))))
#+end_src
Pull guix with new =minimal-channels.scm=, and check to see what the profile contains:
#+begin_src shell
~/opt/guix/bin/guix describe
#+end_src
giving
#+begin_example
Generation 2 Mar 21 2024 07:49:01 (current)
guix-bioinformatics 2fdbf24
repository URL: https://git.genenetwork.org/guix-bioinformatics/
branch: master
commit: 2fdbf2445e5293ba9a12b154c58774e73dfeabe5
guix-past 921f845
repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
branch: master
commit: 921f845dc0dec9f052dcda479a15e787f9fd5b0a
guix-forge 6c622a6
repository URL: https://git.systemreboot.net/guix-forge/
branch: main
commit: 6c622a67051c22eeefe37eedb17d427fbb70c99b
guix b0b988c
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: b0b988c41c9e0e591274495a1b2d6f27fcdae15a
#+end_example
Attempt to rebuild the container:
#+begin_src shell
./uploader-deploy.sh
#+end_src
Success!!!
Stop the currently running container
#+begin_src shell
sudo systemctl stop genenetwork-uploader-container.service
#+end_src
Delete logs and check they are gone:
#+begin_src shell
$ sudo rm -fv /export2/guix-containers/genenetwork/uploader/var/log/gunicorn-g*.log
$ ls /export2/guix-containers/genenetwork/uploader/var/log/
#+end_src
= /export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log=
still rears its ugly head 😭.
Start the container for now:
#+begin_src shell
sudo systemctl start genenetwork-uploader-container.service
#+end_src
|
