diff options
| -rw-r--r-- | issues/systems/fallback-errors.gmi | 51 | ||||
| -rw-r--r-- | topics/systems/debug-and-developing-code-with-genenetwork-system-container.gmi | 50 |
2 files changed, 97 insertions, 4 deletions
diff --git a/issues/systems/fallback-errors.gmi b/issues/systems/fallback-errors.gmi index dc424fb..e0111de 100644 --- a/issues/systems/fallback-errors.gmi +++ b/issues/systems/fallback-errors.gmi @@ -104,6 +104,21 @@ This also led to documenting how one should test GN2 and GN3 in a running VM: => /topics/guix/guix-system-containers-and-how-we-use-them.gmi +and fixing the shared TMPDIR. The next step however fails with + +``` +sh: line 1: Rscript: command not found +2024-04-01 07:33:13 File "/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile/lib/python3.10/site-packages/gn3/computations/rqtl.py", line 6 +5, in process_rqtl_mapping +(...) +2024-04-01 07:33:13 with open( +2024-04-01 07:33:13 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/output/d41d8cd98f00b204e9800998ecf8427ePr1axOhhSF5DMEMowkGu6AvGj6hf+TA2Ra7FIrlrT4Pw-output.csv' +``` + +So, for some reason Rscript is not in the path for GN3. So I added that to the propagated inputs. Now we still have the output error. There is no information the Rscript has actually been invoked and successfuly (or unsuccessfully) wrote the file. To fix this we mount the GN3 source directory in the container again. See + +=> /topics/systems/debug-and-developing-code-with-genenetwork-system-container.gmi + ## QTLReaper showing font problem Try HK on @@ -118,8 +133,44 @@ Try HK on OSError: cannot open resource ``` +This error in on gn2. An image file appears in /tmp/gn2/generated/. + +The fonts exist in the GN2 directory, e.g. +/gnu/store/mymfiz2jgkfc196az1c0vcc2digc84jy-genenetwork2-stable-stable-3.11-1.42b37bb/lib/python3.10/site-packages/gn2/wqflask/static/fonts/courbd.ttf, but are not found. In the code the path is set to + +``` +COUR_FILE = "./gn2/wqflask/static/fonts/courbd.ttf" +``` + +so most likely GN2 is not running from that actually source dir. + +## Pair scan runs, but does not show image + +Pair scan shows a table on the fallback, but not the image. On CD and production no pair scan results! + ## 413 Request Entity Too Large On mapping from => https://fallback.genenetwork.org/show_trait?trait_id=10002&dataset=HET3-ITPPublish + +## Purescript genome browser icons not showing + +## Files generated in /tmp + +Not unique file names appearing in TMPDIR: + +``` +-rw-r--r-- 1 gunicorn-genenetwork2 gunicorn-genenetwork2 337 Apr 2 07:03 175.png +-rw-r--r-- 1 gunicorn-genenetwork2 gunicorn-genenetwork2 287 Apr 2 07:03 75.png +-rw-r--r-- 1 gunicorn-genenetwork2 gunicorn-genenetwork2 303 Apr 2 07:03 25.png +-rw-r--r-- 1 gunicorn-genenetwork2 gunicorn-genenetwork2 348 Apr 2 07:03 125.png +-rw-r--r-- 1 gunicorn-genenetwork2 gunicorn-genenetwork2 835 Apr 2 07:03 -logP.png +``` + +Unique file names, but lingering on file system - note they are root because I created them outside a container + +``` +-rw-r--r-- 1 root root 10M Apr 2 07:03 4OJCzmGV.cross +-rw-r--r-- 1 root root 10M Apr 2 07:03 6cExgwkd.cross +``` diff --git a/topics/systems/debug-and-developing-code-with-genenetwork-system-container.gmi b/topics/systems/debug-and-developing-code-with-genenetwork-system-container.gmi index 3371067..d52090e 100644 --- a/topics/systems/debug-and-developing-code-with-genenetwork-system-container.gmi +++ b/topics/systems/debug-and-developing-code-with-genenetwork-system-container.gmi @@ -58,14 +58,16 @@ Note the added quotes. The command will fail with 'No module named gn3'. Good! N Through shepherd find the profile in use ``` -/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile/lib/python3.10/site-packages/ +export PROFILE=/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile/lib/python3.10/site-packages/ ``` This worked loading the PYTHONPATH worker source path with `--chdir`! ``` +herd stop gunicorn-genenetwork3 + root@genenetwork: cd /export/source/fallback-debug/genenetwork3 -root@genenetwork /export/source/fallback-debug/genenetwork3# PYTHONPATH=/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile/lib/python3.10/site-packages /gnu/store/hhn20xg4vag4xiib2d7d4c1vkm09dcav-gunicorn-20.1.0/bin/gunicorn --workers 1 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/592bscjpr6xyz8asn743iqzgczg8l947-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --chdir /export/source/fallback-debug/genenetwork3 --log-level debug --log-file gn3.log --reload --env HOME=/tmp gn3.app:create_app\(\) +PYTHONPATH=$PROFILE/lib/python3.10/site-packages /gnu/store/hhn20xg4vag4xiib2d7d4c1vkm09dcav-gunicorn-20.1.0/bin/gunicorn --workers 1 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/592bscjpr6xyz8asn743iqzgczg8l947-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --chdir /export/source/fallback-debug/genenetwork3 --log-level debug --reload --env HOME=/tmp gn3.app:create_app\(\) ``` Make sure you are loading gn3 code from your source dir (e.g. by introducing an error). The commit for sharing sources is at @@ -85,6 +87,7 @@ We clone genenetwork2 in the source path, next we take a hint from shepherd: and we tell herd to stop genenetwork2. ``` +herd stop gunicorn-genenetwork2 PYTHONPATH=/gnu/store/yi76sybwqql4ky60yahv91z57srb2fr0-profile/lib/python3.10/site-packages /gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/vg8q4kdnkzy7skv04z57ngm8rqn7kvhd-gunicorn-genenetwork2-pola-wrapper --workers 20 --timeout 1200 --bind 127.0.0.1:8892 --env GN2_PROFILE=/gnu/store/jl6x90wdbwbs7c7zxnyz2kfd0qx8bf5h-profile --env GN2_SETTINGS=/gnu/store/gn9pr6kvmf1zlaskd1bqn1dssx4sy5lw-gn2.conf --chdir /export/source/fallback-debug/genenetwork2 --env HOME=/tmp gn2.wsgi ``` @@ -97,13 +100,52 @@ But, we'll have to look into such optimizations later. After adding the source dir and changing the permissions of the secrets file I can ``` -/export/source/fallback-debug/genenetwork2# /gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/47vplgxkcwd7vk3r71qvvfkwr9rcqlsl-gunicorn-genenetwork2-pola-wrapper --workers 1 --timeout 1200 --bind 127.0.0.1:8892 --env GN2_PROFILE=/gnu/store/jl6x90wdbwbs7c7zxnyz2kfd0qx8bf5h-profile --env GN2_SETTINGS=/gnu/store/gn9pr6kvmf1zlaskd1bqn1dssx4sy5lw-gn2.conf --chdir /export/source/fallback-debug/genenetwork2 --pythonpath=/gnu/store/hvv0r5nzhbbsnd9s68cmx5q0sznjhnrp-profile/lib/python3.10/site-packages --log-level debug --log-file /tmp/gn2.log --reload --env HOME=/tmp gn2.wsgi +export PROFILE=/gnu/store/d77wrqsb11igma3ay5mykc57mnzwc76q-profile +/export/source/fallback-debug/genenetwork2# /gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile --no-auto-compile /gnu/store/47vplgxkcwd7vk3r71qvvfkwr9rcqlsl-gunicorn-genenetwork2-pola-wrapper --workers 1 --timeout 1200 --bind 127.0.0.1:8892 --env GN2_PROFILE=$PROFILE --env GN2_SETTINGS=/gnu/store/gn9pr6kvmf1zlaskd1bqn1dssx4sy5lw-gn2.conf --chdir /export/source/fallback-debug/genenetwork2 --pythonpath=$PROFILE/lib/python3.10/site-packages --log-level debug --reload --env HOME=/tmp gn2.wsgi +``` + +Note that we need the --pythonpath. I pick up that profile from pola-wrapper, as well as the R path etc with + +``` +export PROFILE=/gnu/store/v1nv6nnfsmvsi5aangj580f46741nvx6-profile +root@genenetwork /export/source/fallback-debug/genenetwork3# PATH=$PATH:$PROFILE/bin R_LIBS_USER=$PROFILE/site-library PYTHONPATH=$PROFILE/lib/python3.10/site-packages /gnu/store/hhn20xg4vag4xiib2d7d4c1vkm09dcav-gunicorn-20.1.0/bin/gunicorn --workers 1 --timeout 1200 --bind 127.0.0.1:8893 --env GN3_CONF=/gnu/store/592bscjpr6xyz8asn743iqzgczg8l947-gn3.conf --env GN3_SECRETS=/etc/genenetwork/gn3-secrets.py --chdir /export/source/fallback-debug/genenetwork3 --log-level debug --reload --env HOME=/tmp gn3.app:create_app\(\) ``` -Note that we need the --pythonpath. I pick up that profile from pola-wrapper. +To run the tests you can do something like + +``` +export PROFILE=/gnu/store/v1nv6nnfsmvsi5aangj580f46741nvx6-profile +export AUTHLIB_INSECURE_TRANSPORT=true +export OAUTH2_ACCESS_TOKEN_GENERATOR="tests.unit.auth.test_token.gen_token" +PATH=$PATH:$PROFILE/bin R_LIBS_USER=$PROFILE/site-library PYTHONPATH=$PROFILE/lib/python3.10/site-packages pytest +``` # Fixing shared paths By default both GN2 and GN3 run as containers. We pass temporary files through the file sytem, so let's try and fix that first. The good news is that they only have to share the TMPDIR. First we share a new directory under /var/tmp for the system container. Next we have to tell GN2 and GN3 system containers to use /var/tmp/gn2. This was done in commit => https://git.genenetwork.org/gn-machines/commit/?id=831cf86b4fbf7b054640fa46eede6040ad01340f + +# Showing debug output + +Flask typically runs in a gunicorn. To have debug output the simple thing is to print to stderr with + +``` +sys.stderr.write("Example error output shows in gunicorn log") +``` + +Once the flask app runs it has its own logger settings. What we can do is set the app logging locally + +``` +from flask import current_app +current_app.logger.setLevel(logging.DEBUG) # Force debug level since we assume we are using it! +current_app.logger.debug("%s: %s", title_vals, value) +``` + +I have forced that in gn3/debug.py for now. Putting __pk__ around rqtl_cmd it turned out the script was not defined. The file it should be calling is ./scripts/rqtl_wrapper.R. There are some confusing settings in GN3. + +``` +rqtl_wrapper = current_app.config["RQTL_WRAPPER"] +``` + +http://127.0.0.1:8893/api/menu/generate/json |