summaryrefslogtreecommitdiff
path: root/issues/gn-uploader/build-uploader-container-20240321.org
blob: d1086df22eff5c36072682f2074589da01593e1f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
#+STARTUP: content
#+TITLE: Building Uploader Container (2024-03-21)
#+AUTHOR: Frederick Muriuki Muriithi
#+OPTIONS: ^:{}

* Log

Fix the [[https://git.genenetwork.org/gn-uploader/commit/?id=3958ce457f621f1b75fa8843ba859d8e6015a6c7][channel introduction documentation]] on =gn-uploader=

On =tux02= do:
#+BEGIN_SRC shell
$ guix pull -p ~/opt/guix -C channels.scm
#+END_SRC
where =channels.scm= has the following content

#+BEGIN_SRC scheme
  (list
   (channel
    (name 'guix-bioinformatics)
    (url "https://git.genenetwork.org/guix-bioinformatics/")
    (branch "master"))
   (channel
    (name 'guix-forge)
    (url "https://git.systemreboot.net/guix-forge/")
    (branch "main")
    (introduction
     (make-channel-introduction
      "0432e37b20dd678a02efee21adf0b9525a670310"
      (openpgp-fingerprint
       "7F73 0343 F2F0 9F3C 77BF  79D3 2E25 EE8B 6180 2BB3"))))
   (channel
    (name 'guix-past)
    (url "https://gitlab.inria.fr/guix-hpc/guix-past")
    (introduction
     (make-channel-introduction
      "0c119db2ea86a389769f4d2b9c6f5c41c027e336"
      (openpgp-fingerprint
       "3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5"))))
   (channel
    (name 'gn-auth)
    (url "https://git.genenetwork.org/gn-auth")
    (branch "main"))
   (channel
    (name 'gn-uploader)
    (url "https://git.genenetwork.org/gn-uploader")
    (branch "main")
    (introduction
     (make-channel-introduction
      "93c77842315d304abbfc27d78d98b7d42da32a61"
      (openpgp-fingerprint
       "F370 F409 854B 90E3 52F3  AB01 362B 0BB8 B81D 5A42")))))
#+END_SRC

Pull was successful!

After that, we get the following on =tux02=:
#+BEGIN_SRC shell
  $ ~/opt/guix/bin/guix describe --format=channels
  hint: Consider installing the `glibc-locales' package and defining
  `GUIX_LOCPATH', along these lines:
  
       guix install glibc-locales
       export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"
  
  See the "Application Setup" section in the manual, for more info.
  
  (list (channel
          (name 'gn-uploader)
          (url "https://git.genenetwork.org/gn-uploader")
          (branch "main")
          (commit
            "3958ce457f621f1b75fa8843ba859d8e6015a6c7")
          (introduction
            (make-channel-introduction
              "93c77842315d304abbfc27d78d98b7d42da32a61"
              (openpgp-fingerprint
                "F370 F409 854B 90E3 52F3  AB01 362B 0BB8 B81D 5A42"))))
        (channel
          (name 'gn-auth)
          (url "https://git.genenetwork.org/gn-auth")
          (branch "main")
          (commit
            "ade6c76208bd45c24cdd0e73bf3570e6a2bfa131"))
        (channel
          (name 'guix-forge)
          (url "https://git.systemreboot.net/guix-forge/")
          (branch "main")
          (commit
            "6c622a67051c22eeefe37eedb17d427fbb70c99b")
          (introduction
            (make-channel-introduction
              "0432e37b20dd678a02efee21adf0b9525a670310"
              (openpgp-fingerprint
                "7F73 0343 F2F0 9F3C 77BF  79D3 2E25 EE8B 6180 2BB3"))))
        (channel
          (name 'guix-bioinformatics)
          (url "https://git.genenetwork.org/guix-bioinformatics/")
          (branch "master")
          (commit
            "2fdbf2445e5293ba9a12b154c58774e73dfeabe5"))
        (channel
          (name 'guix)
          (url "https://git.savannah.gnu.org/git/guix.git")
          (branch "master")
          (commit
            "b0b988c41c9e0e591274495a1b2d6f27fcdae15a")
          (introduction
            (make-channel-introduction
              "9edb3f66fd807b096b48283debdcddccfea34bad"
              (openpgp-fingerprint
                "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"))))
        (channel
          (name 'guix-past)
          (url "https://gitlab.inria.fr/guix-hpc/guix-past")
          (branch "master")
          (commit
            "921f845dc0dec9f052dcda479a15e787f9fd5b0a")
          (introduction
            (make-channel-introduction
              "0c119db2ea86a389769f4d2b9c6f5c41c027e336"
              (openpgp-fingerprint
                "3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5")))))
#+END_SRC

Now, build the container:
#+BEGIN_SRC shell
  env PATH="/home/fredm/opt/guix/bin:${PATH}" ./uploader-deploy.sh
  ︙
  successfully built /gnu/store/0fwp30li4ngfmradkiyc81cdmapm04bf-gn-uploader-0.0.1-a4ed5d82.drv
  ︙
  /gnu/store/4rfwx4d376kbql9y31cfldf3prcfrvx2-run-container
  ︙
#+END_SRC

Restart the container
#+BEGIN_SRC shell
  $ sudo systemctl stop genenetwork-uploader-container.service
  $ sudo systemctl start genenetwork-uploader-container.service
  $ sudo systemctl status genenetwork-uploader-container.service
#+END_SRC

Check logs in =/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log=

Still getting:
#+BEGIN_EXAMPLE
︙
2024-03-21 02:18:29     from gn2.wqflask.oauth2.checks import user_logged_in
2024-03-21 02:18:29   File "/gnu/store/2qhgkf5j649kv57nw4lzsbwlnb1790vq-profile/lib/python3.10/site-packages/gn2/wqflask/oauth2/checks.py", line 10, i
n <module>
2024-03-21 02:18:29     from .client import authserver_uri, oauth2_clientid, oauth2_clientsecret
2024-03-21 02:18:29 ImportError: cannot import name 'authserver_uri' from partially initialized module 'gn2.wqflask.oauth2.client' (most likely due to
 a circular import) (/gnu/store/2qhgkf5j649kv57nw4lzsbwlnb1790vq-profile/lib/python3.10/site-packages/gn2/wqflask/oauth2/client.py)
2024-03-21 02:18:29 [2024-03-21 02:18:29 +0000] [9] [INFO] Worker exiting (pid: 9)
︙
#+END_EXAMPLE

When we verify where this comes from:
#+BEGIN_SRC shell
  $ ls -l /gnu/store/2qhgkf5j649kv57nw4lzsbwlnb1790vq-profile/lib/python3.10/site-packages/ | grep gn2
  lrwxrwxrwx 1 root root  104 Dec 31  1969 gn2 -> /gnu/store/vfa2zdla7ixbb2l5i7k6xnlh5d1rshz9-genenetwork2-3.11-4.1e81c74/lib/python3.10/site-packages/gn2
#+END_SRC

We see that the code being run is from [[https://github.com/genenetwork/genenetwork2/commit/1e81c74ee8752ad2913b4e610ba7575638755385][GeneNetwork2: commit 1e81c74]] from
05^{th} March 2024. In [[https://git.genenetwork.org/guix-bioinformatics/tree/gn/packages/genenetwork.scm?id=2fdbf2445e5293ba9a12b154c58774e73dfeabe5#n190][=guix-bioinformatics=, however]], we have specified a newer
[[https://github.com/genenetwork/genenetwork2/commit/b9e04ad79e0039edba25d58f8bc03e4d2a17583b][commit b9e04ad7]] from 07^{th} March 2024.

  Stop uploader service and delete logs:
#+BEGIN_SRC shell
  $ sudo systemctl stop genenetwork-uploader-container.service
  $ sudo rm -fv /export2/guix-containers/genenetwork/uploader/var/log/gunicorn-g*
  removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log'
  removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork3.log'
  removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-gn-auth.log'
  removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-gn-uploader.log'
  $ ls /export2/guix-containers/genenetwork/uploader/var/log/
  debug            gunicorn-genenetwork2.log  mcron.log       messages       mysqld.log  nscd.log  secure.1.gz   wtmp
  guix-daemon.log  maillog                    mcron.log.1.gz  messages.1.gz  nginx       secure    virtuoso.log
#+END_SRC

The file =/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log= is still showing up.

Stop all containers for good measure:
#+BEGIN_SRC shell
  $ sudo systemctl stop genenetwork-uploader-container.service
  $ sudo systemctl stop genenetwork-container.service
  $ sudo systemctl stop genenetwork-development-container.service
  $ sudo rm -fv /export2/guix-containers/genenetwork/uploader/var/log/gunicorn-g*
  removed '/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log'
  $ ls /export2/guix-containers/genenetwork/uploader/var/log/
  debug            gunicorn-genenetwork2.log  mcron.log       messages       mysqld.log  nscd.log  secure.1.gz   wtmp
  guix-daemon.log  maillog                    mcron.log.1.gz  messages.1.gz  nginx       secure    virtuoso.log
#+END_SRC

The file =/export2/guix-containers/genenetwork/uploader/var/log/gunicorn-genenetwork2.log= is still showing up.

Something else is running and recreating the file and I have no idea what it is.

Restart all containers:
#+BEGIN_SRC shell
  $ sudo systemctl start genenetwork-uploader-container.service
  $ sudo systemctl start genenetwork-container.service
  $ sudo systemctl start genenetwork-development-container.service
#+END_SRC