From 94bd8d5695d36f4fa5ef183823d5e9fc7a3beb7e Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Wed, 27 Oct 2021 11:20:33 +0200 Subject: Manual testing --- topics/testing/genenetwork-checklist.gmi | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 topics/testing/genenetwork-checklist.gmi diff --git a/topics/testing/genenetwork-checklist.gmi b/topics/testing/genenetwork-checklist.gmi new file mode 100644 index 0000000..703faa2 --- /dev/null +++ b/topics/testing/genenetwork-checklist.gmi @@ -0,0 +1,22 @@ +# GN Checklist + +This is the checklist to run through when testing GN2. + +## Tags + +* assigned: pjotrp + +## Tasks + +This is a check list for testing whether GN2 is completely deployed + +* [ ] wain page +=> https://luna.genenetwork.org/ +* [ ] simple search (e.g. shh in hippocampus mRNA) +=> https://luna.genenetwork.org/search?species=mouse&group=BXD&type=Hippocampus+mRNA&dataset=HC_M2_0606_P&search_terms_or=shh&search_terms_and=&FormID=searchResult +* [ ] try your own search term +* [ ] wild card seach (e.g. sh* in hippocampus mRNA) +=> https://luna.genenetwork.org/search?species=mouse&group=BXD&type=Hippocampus+mRNA&dataset=HC_M2_0606_P&search_terms_or=sh*&search_terms_and=&FormID=searchResult +* [ ] try your own wild-card search + +will continue... -- cgit v1.2.3 From 1c9077f85e7cc76e0752eb1d0f83a75dd23b1d3f Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Thu, 28 Oct 2021 11:07:54 +0300 Subject: Remove elastic search issues --- issues/remove-elastic-search.gmi | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 issues/remove-elastic-search.gmi diff --git a/issues/remove-elastic-search.gmi b/issues/remove-elastic-search.gmi new file mode 100644 index 0000000..af475d6 --- /dev/null +++ b/issues/remove-elastic-search.gmi @@ -0,0 +1,15 @@ +# Remove Elastic Search + +Elastic Search is no longer used, yet we have dead code referencing it +in gn2. + + +## Tasks + +* assigned: bonfacekilz +* keywords: chore, done + +## Notes + +While at it, "@depracated" was removed since it added extra noise to +the logs. \ No newline at end of file -- cgit v1.2.3 From 6751f9eebfba95a8f3fd2373a8eee12254ae9474 Mon Sep 17 00:00:00 2001 From: jgart Date: Thu, 28 Oct 2021 20:31:48 -0400 Subject: Add comment to add-linter issue --- issues/gnbug/add-linter.gmi | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/issues/gnbug/add-linter.gmi b/issues/gnbug/add-linter.gmi index 7992753..1a8a301 100644 --- a/issues/gnbug/add-linter.gmi +++ b/issues/gnbug/add-linter.gmi @@ -14,3 +14,11 @@ support. I am not sure. Pjotr is of the philosophy that we should keep the text as unstructured and unrestrictive as possible, and I agree with this philosophy. Linters are likely to annoy users more than help them. They might discourage casual use of the issue tracker. Especially with tags, we don't really want to restrict the user from coming up with new tags. WDYT? Am I missing some scenario where the linter could be essential? + +Nope, it was just something I thought could be convenient if we wanted to restrict +the format for tags, etc... + +If we'd like to "keep the text as unstructured and unrestrictive as possible" I'm +fine with that too. + +I was just worried about using "invalid" tags. Now, I know it's ok :) -- cgit v1.2.3 From c458a930930705ff34ef0a0fdf47925753c414f3 Mon Sep 17 00:00:00 2001 From: jgart Date: Thu, 28 Oct 2021 20:32:16 -0400 Subject: Add new feature request for gnbug subcommand Generating bug report templates might be a nice convenience. --- issues/gnbug/add-convenience-templates.gmi | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 issues/gnbug/add-convenience-templates.gmi diff --git a/issues/gnbug/add-convenience-templates.gmi b/issues/gnbug/add-convenience-templates.gmi new file mode 100644 index 0000000..41eb975 --- /dev/null +++ b/issues/gnbug/add-convenience-templates.gmi @@ -0,0 +1,29 @@ +# Add convenienve templates + +* feature-request + +WDYT of adding a subcommand called `report` to generate a template for a bug report? + +I was thinking of an interface like the following: + +``` +gnbug report feature-request "New feature: IPFS Downloads" +``` + +Or expressed as generic command parameters: + +``` +command subcommand tag(s) title +``` + +Which creates a file in issues/binderlite directory with a template like the +following: + + +``` +# New feature: IPFS Downloads + +* feature-request +``` + +Might be bloat but might be nice and convenient also. WDYT? -- cgit v1.2.3 From 61530ef4f3ce2c7ad6a617a683160ea081680a03 Mon Sep 17 00:00:00 2001 From: jgart Date: Thu, 28 Oct 2021 20:40:21 -0400 Subject: Add return-canned-text-if-notebook-description-returns-none issue --- ...d-text-if-notebook-description-returns-none.gmi | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 issues/binderlite/return-canned-text-if-notebook-description-returns-none.gmi diff --git a/issues/binderlite/return-canned-text-if-notebook-description-returns-none.gmi b/issues/binderlite/return-canned-text-if-notebook-description-returns-none.gmi new file mode 100644 index 0000000..49f5d3f --- /dev/null +++ b/issues/binderlite/return-canned-text-if-notebook-description-returns-none.gmi @@ -0,0 +1,25 @@ +# Return canned text if notebook description returns none + +* bug? +* assigned: jgart + +Notebooks that do not have a description will use python's `None` value as the +project's description. + +Should we return some other text instead? + +We could do something like the following idea: + +``` +if notebook.description == None: + return "Project does not have a description." +``` + +Or we could do: + +``` +if notebook.description == None: + return "Could not retrieve description for project." +``` + +WDYT? -- cgit v1.2.3 From f74813e97dee960379fdbffd733597dd7ed404d7 Mon Sep 17 00:00:00 2001 From: jgart Date: Thu, 28 Oct 2021 20:49:19 -0400 Subject: Add integrate-styling-with-gn2 issue --- issues/binderlite/integrate-styling-with-gn2.gmi | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 issues/binderlite/integrate-styling-with-gn2.gmi diff --git a/issues/binderlite/integrate-styling-with-gn2.gmi b/issues/binderlite/integrate-styling-with-gn2.gmi new file mode 100644 index 0000000..dc158fa --- /dev/null +++ b/issues/binderlite/integrate-styling-with-gn2.gmi @@ -0,0 +1,11 @@ +# Integrate styling of app with GN2 + +* ongoing +* assigned: jgart + +Currently using antiweb for styling with modifications. + +=> https://git.sr.ht/~sircmpwn/antiweb antiweb + +Copying nav bar and fonts from GN2 will make the app look more integrated with +GN2 UI. -- cgit v1.2.3 From 6c9b4acc6e9e210bf174967f1ed6294dd64a71d6 Mon Sep 17 00:00:00 2001 From: jgart Date: Thu, 28 Oct 2021 20:58:51 -0400 Subject: Adds ansi2html feature for binderlite issue --- .../ansi2html_terminal_user_feedback.gmi | 36 +++++++++++++++++++++ .../binderlite_ansi2html_terminal_diagram.png | Bin 0 -> 8784 bytes 2 files changed, 36 insertions(+) create mode 100644 issues/binderlite/ansi2html_terminal_user_feedback.gmi create mode 100644 issues/binderlite/binderlite_ansi2html_terminal_diagram.png diff --git a/issues/binderlite/ansi2html_terminal_user_feedback.gmi b/issues/binderlite/ansi2html_terminal_user_feedback.gmi new file mode 100644 index 0000000..969448a --- /dev/null +++ b/issues/binderlite/ansi2html_terminal_user_feedback.gmi @@ -0,0 +1,36 @@ +# ansi2html terminal user feedback + +* ongoing +* feature-request +* assigned: jgart + +https://builds.sr.ht/~sircmpwn/job/610711 + +https://builds.sr.ht/~sircmpwn/job/611028 + +This issue proposes an alternative way to generate terminal output to the browser +from a process that does not dependent on js libraries but still looks great. + +Drew Devault uses this method in builds.sr.ht, the continuous integration +system/app that is part of the SourceHut suite. + +builds.sr.ht generates terminal output using jinja templates and a relatively +small python library called ansi2html. It is already packaged by guix upstream +as python-ansi2html (It could use an update). + +https://github.com/pycontribs/ansi2html +https://pypi.org/project/ansi2html/ + +Here are two examples of it in action at builds.sr.ht: + +https://builds.sr.ht/~sircmpwn/job/610711 + +https://builds.sr.ht/~sircmpwn/job/611028 + +And, the relevant places in builds.sr.ht code base: + +https://git.sr.ht/~sircmpwn/builds.sr.ht/tree/master/item/buildsrht/templates/job.html#L117 + +https://git.sr.ht/~sircmpwn/builds.sr.ht/tree/master/item/buildsrht/blueprints/jobs.py#L333 + +I've added a diagram showing how it will integrate with binderlite. diff --git a/issues/binderlite/binderlite_ansi2html_terminal_diagram.png b/issues/binderlite/binderlite_ansi2html_terminal_diagram.png new file mode 100644 index 0000000..a4f16b4 Binary files /dev/null and b/issues/binderlite/binderlite_ansi2html_terminal_diagram.png differ -- cgit v1.2.3 From 9095249780591d55cf75b3b03c3940e879c6bf2b Mon Sep 17 00:00:00 2001 From: Efraim Flashner Date: Wed, 3 Nov 2021 14:14:30 +0200 Subject: track each shepherd service having its own profile --- .../per-service-profile-for-shepherd-services.gmi | 33 ++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 issues/per-service-profile-for-shepherd-services.gmi diff --git a/issues/per-service-profile-for-shepherd-services.gmi b/issues/per-service-profile-for-shepherd-services.gmi new file mode 100644 index 0000000..151675e --- /dev/null +++ b/issues/per-service-profile-for-shepherd-services.gmi @@ -0,0 +1,33 @@ +# Each shepherd service needs its own guix profile. + +Currently there are a bunch of shepherd services (on Penguin2, as user shepherd) who are using the global shepherd guix profile and loading guix-bioinformatics at service start time to start their services. This is bad because: + +* we aren't using per-service guix-bioinformatics worktrees, so the branch changes between service refreshes +* service start times are high because we have to rebuild the service (due to above) +* services which are restarted are in an "untested" state until they work + +## Services which need to be migrated: + +* covid19-pubseq (uses shared (broken!) guix profile, uses shared config directory) +* genome_browser (uses own guix profile, uses shared guix-bioinformatics, uses shared config directory) +* hrdp-project (uses shared (broken!) guix profile) +* power (uses shared (broken!) guix profile) +* ratspub (uses own guix profile, uses shared guix-bioinformatics)(deprecated(?)) +* rn6app (uses shared (broken!) guix profile) +* singlecell (uses shared (broken!) guix profile) +* pluto (uses own guix profile, uses shared config directory)(currently disabled) + +## Services which need to be monitored: + +* genenetwork1 (uses own guix profile, uses shared config directory) +* cronjob-pubmed (uses shared guix profile, should use genecup profile) +* mcron (uses shared guix profile, probably OK) +* virtuoso (uses shared guix profile, probably OK) + +## Services which already have their own profile: + +* bnw +* genecup +* gitea +* cronjob-gitea +* ipfs -- cgit v1.2.3 From de458a1fa86688f57b9fef46f799d60cd07e2388 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Thu, 4 Nov 2021 10:46:10 +0100 Subject: GN --- topics/cli/weechat.gmi | 1 + topics/testing/genenetwork-checklist.gmi | 39 +++++++++++++++++++++++++++++++- 2 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 topics/cli/weechat.gmi diff --git a/topics/cli/weechat.gmi b/topics/cli/weechat.gmi new file mode 100644 index 0000000..719c95e --- /dev/null +++ b/topics/cli/weechat.gmi @@ -0,0 +1 @@ +# Weechat diff --git a/topics/testing/genenetwork-checklist.gmi b/topics/testing/genenetwork-checklist.gmi index 703faa2..ba19733 100644 --- a/topics/testing/genenetwork-checklist.gmi +++ b/topics/testing/genenetwork-checklist.gmi @@ -10,7 +10,7 @@ This is the checklist to run through when testing GN2. This is a check list for testing whether GN2 is completely deployed -* [ ] wain page +* [ ] visit main page => https://luna.genenetwork.org/ * [ ] simple search (e.g. shh in hippocampus mRNA) => https://luna.genenetwork.org/search?species=mouse&group=BXD&type=Hippocampus+mRNA&dataset=HC_M2_0606_P&search_terms_or=shh&search_terms_and=&FormID=searchResult @@ -18,5 +18,42 @@ This is a check list for testing whether GN2 is completely deployed * [ ] wild card seach (e.g. sh* in hippocampus mRNA) => https://luna.genenetwork.org/search?species=mouse&group=BXD&type=Hippocampus+mRNA&dataset=HC_M2_0606_P&search_terms_or=sh*&search_terms_and=&FormID=searchResult * [ ] try your own wild-card search +* [ ] visit a trait page +=> https://luna.genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P + will continue... + + +curl -X POST -b 'anon_user_v1=4d2ce6d3-0d4c-4349-b491-a4e179fdc163:d5fdc040c737e2a775d2' https://luna.genenetwork.org/run_mapping + +https://luna.genenetwork.org/run_mapping +User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 +Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 +Accept-Language: en-US,en;q=0.5 +Accept-Encoding: gzip, deflate, br +Content-Type: application/x-www-form-urlencoded +Content-Length: 12345 +Origin: https://luna.genenetwork.org +Connection: keep-alive +Referer: https://luna.genenetwork.org/loading +Cookie: anon_user_v1=4d2ce6d3-0d4c-4349-b491-a4e179fdc163:d5fdc040c737e2a775d2; 404 Not Found: The requested URL=Wild-Type-Mouse.gif +Upgrade-Insecure-Requests: 1 +Sec-GPC: 1 + +https://luna.genenetwork.org/run_mapping +POST /run_mapping HTTP/1.1 +Host: luna.genenetwork.org +User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 +Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 +Accept-Language: en-US,en;q=0.5 +Accept-Encoding: gzip, deflate, br +Referer: https://luna.genenetwork.org/loading +Content-Type: application/x-www-form-urlencoded +Content-Length: 12345 +Origin: https://luna.genenetwork.org +Connection: keep-alive +Cookie: anon_user_v1=4d2ce6d3-0d4c-4349-b491-a4e179fdc163:d5fdc040c737e2a775d2; 404 Not Found: The requested URL=Wild-Type-Mouse.gif +Upgrade-Insecure-Requests: 1 +Sec-GPC: 1 +Cache-Control: max-age=0 -- cgit v1.2.3 From 5322de934b05658337a4109e8c6cb450d6f63c96 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Thu, 4 Nov 2021 11:39:45 +0100 Subject: Weechat and slack! --- topics/cli/weechat.gmi | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/topics/cli/weechat.gmi b/topics/cli/weechat.gmi index 719c95e..b9119b2 100644 --- a/topics/cli/weechat.gmi +++ b/topics/cli/weechat.gmi @@ -1 +1,22 @@ # Weechat + +Arun has packaged weechat for slack and matrix. Current install using a guix environment: + +``` +env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/opt/guix-latest/bin/guix environment --ad-hoc coreutils weechat weechat-matrix weechat-wee-slack nss-certs openssl python +echo $GUIX_ENVIRONMENT +/gnu/store/i4a3ynfjvq8r8cch14gkn2fwfs77wagj-profile/ +weechat +/python load /gnu/store/i4a3ynfjvq8r8cch14gkn2fwfs77wagj-profile/share/weechat/python/wee_slack.py +/slack register +``` + +So, capture the shell GUIX_ENVIRONMENT and use that to launch the wee-slack plugin. Next register a token through te Slack web interface. + +If you go through the motions you get a token and + +``` +/slack register 2654233192084.2682932906197.fd6d6e55c0038f-etc-052b4fa578ec497710b4b8770f1-etc +``` + +More instructions for https://github.com/wee-slack/wee-slack. -- cgit v1.2.3 From 04d79b5b657a50079f806221bc0a1cdab419b344 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Thu, 4 Nov 2021 11:55:52 +0100 Subject: weechat --- topics/cli/weechat.gmi | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/topics/cli/weechat.gmi b/topics/cli/weechat.gmi index b9119b2..2d9e33c 100644 --- a/topics/cli/weechat.gmi +++ b/topics/cli/weechat.gmi @@ -1,6 +1,8 @@ # Weechat -Arun has packaged weechat for slack and matrix. Current install using a guix environment: +Weechat is a very versatile terminal chat that can handle IRC, Matrix and Slack. + +Arun has packaged weechat for slack and matrix in GNU Guix. Current install using a guix environment: ``` env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/opt/guix-latest/bin/guix environment --ad-hoc coreutils weechat weechat-matrix weechat-wee-slack nss-certs openssl python @@ -17,6 +19,7 @@ If you go through the motions you get a token and ``` /slack register 2654233192084.2682932906197.fd6d6e55c0038f-etc-052b4fa578ec497710b4b8770f1-etc +/python reload slack ``` More instructions for https://github.com/wee-slack/wee-slack. -- cgit v1.2.3 From 1793406aa06b4320d4731cbc9bd5bb7b37f8393d Mon Sep 17 00:00:00 2001 From: jgart Date: Thu, 4 Nov 2021 22:49:00 -0400 Subject: issues: Open notebook directly if full path to notebook file is given --- .../open-files-directly-if-full-path-is-given.gmi | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 issues/binderlite/open-files-directly-if-full-path-is-given.gmi diff --git a/issues/binderlite/open-files-directly-if-full-path-is-given.gmi b/issues/binderlite/open-files-directly-if-full-path-is-given.gmi new file mode 100644 index 0000000..2fb4b2f --- /dev/null +++ b/issues/binderlite/open-files-directly-if-full-path-is-given.gmi @@ -0,0 +1,16 @@ +# Open notebook directly if full path to notebook file is given + +* feature-request +* assigned: jgart + +We'd like to be able launch a notebook directly if the user gives the fullpath to the notebook file. + +Should we allow urls like this? + +https://github.com/jgarte/guile-notebook-genenetwork-api/blob/master/genenetwork-api.ipynb + +The above is a valid github url that also points to the notebook file. + +Or only this? + +https://github.com/jgarte/guile-notebook-genenetwork-api/genenetwork-api.ipynb -- cgit v1.2.3 From bfaf199cf3a7ec29aeaaf9ec4e387ea8bc5f16db Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sat, 6 Nov 2021 11:56:49 +0100 Subject: Progress in machine room --- tasks/dana.gmi | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/tasks/dana.gmi b/tasks/dana.gmi index af053b7..51f33b6 100644 --- a/tasks/dana.gmi +++ b/tasks/dana.gmi @@ -6,9 +6,16 @@ ## Tasks -* [X] Tux02 https firewalling * [ ] Octopus 10Gbs switch + cables -* [ ] Mount 4TB SSDs for Tux01 and Tux02 + - a test shows Octopus08 is at 10Gbs on the second interface + - need to hook up new cables +* [ ] Add gateway on 10Gbs switch => ../topics/systems/decommission-machines.gmi +Completed: + +* [X] Tux02 https firewalling +* [X] Mount 4TB SSDs for Tux01 and Tux02 + + ## Info -- cgit v1.2.3 From 7bd19c5d1a27b786b1c125187e5c8e7f1d566d8e Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sun, 7 Nov 2021 09:09:50 +0100 Subject: weechat --- topics/cli/weechat.gmi | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/topics/cli/weechat.gmi b/topics/cli/weechat.gmi index 2d9e33c..b6fc062 100644 --- a/topics/cli/weechat.gmi +++ b/topics/cli/weechat.gmi @@ -2,6 +2,8 @@ Weechat is a very versatile terminal chat that can handle IRC, Matrix and Slack. +## Weechat install + slack + Arun has packaged weechat for slack and matrix in GNU Guix. Current install using a guix environment: ``` @@ -23,3 +25,42 @@ If you go through the motions you get a token and ``` More instructions for https://github.com/wee-slack/wee-slack. + +## Weechat IRC + +We have a GeneNetwork channel + +``` +/set irc.server_default.nicks pjotrp +/server add libera irc.libera.chat/6697 -ssl -autoconnect +/connect libera +/join #guix +/join #guix-risc-v +/join #guix-hpc +/join #genenetwork +``` + +## Weechat matrix + +The matrix plugin works similar to above Slack plugin. + +``` +/script load matrix.py +/matrix server add fosdem chat.fosdem.org +/set matrix.server.matrix_org.username pjotrp-linux +/set matrix.server.matrix_org.password **** +/set matrix.server.matrix_org.autoconnect on +/save +``` + +Only the matrix version is not so polished. For example, it is hard to find the buffers because they are named by a hash. In a buffer you can rename with + +``` +/buffer set short_name myname +``` + +Also, you need to enable every channel explicitly to exchange keys + +``` +/olm verify @user:matrix.org * +``` -- cgit v1.2.3 From 0efcd6579f9fb6005b7af50f31ad1c2dd9235b14 Mon Sep 17 00:00:00 2001 From: Efraim Flashner Date: Sun, 7 Nov 2021 11:00:10 +0200 Subject: update shepherd services issues --- issues/per-service-profile-for-shepherd-services.gmi | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/issues/per-service-profile-for-shepherd-services.gmi b/issues/per-service-profile-for-shepherd-services.gmi index 151675e..b22448b 100644 --- a/issues/per-service-profile-for-shepherd-services.gmi +++ b/issues/per-service-profile-for-shepherd-services.gmi @@ -8,26 +8,26 @@ Currently there are a bunch of shepherd services (on Penguin2, as user shepherd) ## Services which need to be migrated: -* covid19-pubseq (uses shared (broken!) guix profile, uses shared config directory) -* genome_browser (uses own guix profile, uses shared guix-bioinformatics, uses shared config directory) -* hrdp-project (uses shared (broken!) guix profile) -* power (uses shared (broken!) guix profile) +* covid19-pubseq (uses shared (broken!) guix profile, uses shared config directory)(currently running by hand by Pjotr) +* hrdp-project (uses shared (broken!) guix profile) (upstream source is gone) * ratspub (uses own guix profile, uses shared guix-bioinformatics)(deprecated(?)) -* rn6app (uses shared (broken!) guix profile) -* singlecell (uses shared (broken!) guix profile) * pluto (uses own guix profile, uses shared config directory)(currently disabled) ## Services which need to be monitored: * genenetwork1 (uses own guix profile, uses shared config directory) -* cronjob-pubmed (uses shared guix profile, should use genecup profile) +* genome_browser (uses own guix profile, uses shared config directory) * mcron (uses shared guix profile, probably OK) * virtuoso (uses shared guix profile, probably OK) ## Services which already have their own profile: * bnw +* cronjob-gitea +* cronjob-pubmed (genecup) * genecup * gitea -* cronjob-gitea * ipfs +* power +* rn6app +* singlecell -- cgit v1.2.3 From d3d1a31040f3f2aee176125ca7898e4f00167efe Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Mon, 8 Nov 2021 12:16:36 +0100 Subject: sheepdog --- topics/systems/sheepdog.gmi | 1 + 1 file changed, 1 insertion(+) diff --git a/topics/systems/sheepdog.gmi b/topics/systems/sheepdog.gmi index 478adf1..137a5dd 100644 --- a/topics/systems/sheepdog.gmi +++ b/topics/systems/sheepdog.gmi @@ -13,6 +13,7 @@ that need resolving at some point. ## Tasks * [X] add locking functionality for tags - added borg with-lock (test) +* [X] chgrp functionality in sheepdog_borg * [ ] check whether rsync dir exists, repo valid and/or no lock before proceeding * [ ] send digest E-mails * [ ] smart state E-mails on services going down -- cgit v1.2.3 From b4df0131b5c4cf9ba1e48d66f5c47b38aafdd394 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 10 Nov 2021 14:24:51 -0500 Subject: binderlite: Add server name configuration feature --- issues/binderlite/add-configuration-to-set-server-name.gmi | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 issues/binderlite/add-configuration-to-set-server-name.gmi diff --git a/issues/binderlite/add-configuration-to-set-server-name.gmi b/issues/binderlite/add-configuration-to-set-server-name.gmi new file mode 100644 index 0000000..d54997c --- /dev/null +++ b/issues/binderlite/add-configuration-to-set-server-name.gmi @@ -0,0 +1,11 @@ +# Add configuration to set server name + +* assigned: jgart + +* feature + +Let's use environment variables as configuration + +https://flask.palletsprojects.com/en/2.0.x/cli/?highlight=environment%20variables + +https://docs.python.org/3/library/os.html?highlight=os%20environ#os.environ -- cgit v1.2.3 From 2d7e532a4a231c743e313231ff4cfdb23e6de75f Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 10 Nov 2021 14:29:23 -0500 Subject: issue: binderlite: Close spinner for submit form --- issues/binderlite/add-spinner-for-submit-form.gmi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/issues/binderlite/add-spinner-for-submit-form.gmi b/issues/binderlite/add-spinner-for-submit-form.gmi index 0800fa8..de19c55 100644 --- a/issues/binderlite/add-spinner-for-submit-form.gmi +++ b/issues/binderlite/add-spinner-for-submit-form.gmi @@ -13,3 +13,15 @@ a container to be built? A spinning guix gnu logo could be cool. https://www.w3schools.com/howto/howto_css_loader.asp https://spin.js.org/https://spin.js.org/ + + +## Solution + +We're using ansi2html now. + +See the following issue: + +=> ./ansi2html_terminal_user_feedback.gmi + +* closed + -- cgit v1.2.3 From 640c2dccf8933be8de98352d45ccb4364fd9b986 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 10 Nov 2021 14:31:31 -0500 Subject: issues: binderlite: Close nginx forwarding error regarding static assets --- issues/binderlite/error-regarding-static-assets-after-forwarding.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/issues/binderlite/error-regarding-static-assets-after-forwarding.gmi b/issues/binderlite/error-regarding-static-assets-after-forwarding.gmi index 27e7bd1..bd39df9 100644 --- a/issues/binderlite/error-regarding-static-assets-after-forwarding.gmi +++ b/issues/binderlite/error-regarding-static-assets-after-forwarding.gmi @@ -50,3 +50,5 @@ http://localhost/49761/?token=f6269bbf48ee5c9887732f5bc76d32f37e1f25483195ae88 The next step is to get jupyter to accept the generated token on the first request without needing to input it manually first. + +* closed -- cgit v1.2.3 From 74ef08481243820503f543b103f3a6dbb8a3a61f Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 10 Nov 2021 14:33:11 -0500 Subject: issues: binderlite: Close open notebook directly feature --- issues/binderlite/open-files-directly-if-full-path-is-given.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/issues/binderlite/open-files-directly-if-full-path-is-given.gmi b/issues/binderlite/open-files-directly-if-full-path-is-given.gmi index 2fb4b2f..203e87a 100644 --- a/issues/binderlite/open-files-directly-if-full-path-is-given.gmi +++ b/issues/binderlite/open-files-directly-if-full-path-is-given.gmi @@ -14,3 +14,5 @@ The above is a valid github url that also points to the notebook file. Or only this? https://github.com/jgarte/guile-notebook-genenetwork-api/genenetwork-api.ipynb + +* closed -- cgit v1.2.3 From b1b742dd53786817c2ba87e6716e0b21c257de6d Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sun, 14 Nov 2021 08:16:34 +0100 Subject: Arvados --- topics/arvados/covid19.gmi | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 topics/arvados/covid19.gmi diff --git a/topics/arvados/covid19.gmi b/topics/arvados/covid19.gmi new file mode 100644 index 0000000..be96605 --- /dev/null +++ b/topics/arvados/covid19.gmi @@ -0,0 +1,7 @@ +# COVID-19 + +We are setting up a Cloud HPC for COVID-19 using Arvados as the underlying infrastructure. + +## Tasks + +=> https://github.com/yatisht/usher Fast phylogeny -- cgit v1.2.3 From 3787440fb80f3d0e0f6834aaf9fda07d6dd9b2e1 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sun, 14 Nov 2021 16:15:19 -0600 Subject: gemma-wrapper transactions --- .../gemma/gemma-wrapper-has-incomplete-files.gmi | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 issues/gemma/gemma-wrapper-has-incomplete-files.gmi diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi new file mode 100644 index 0000000..4bea71d --- /dev/null +++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi @@ -0,0 +1,22 @@ +# gemma-wrapper has incomplete files + +Gemma wrapper caches files - but it can happen a cached file is incomplete and never updated again. The problem appears when GNU parallel is invoked and hits an error. The task here is to make gemma-wrapper transactional. + +## Tags + +* assigned: pjotrp, zachs + +## Tasks + +* [ ] parse parallel job log for failed tasks and remove the output files. +* [ ] create a (global) lock file for gemma-wrapper + +## Info + +GNU parallel can fail, but does not tell how individual processes did. Need to check if it can return a thread (number). If not we have the option of checking the GEMMA status file and/or see if the output file is complete (by counting number of lines). + +Turns out GNU parallel can keep track of jobs in a job log - and even rerun the ones missing. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! + +There is another parallel issue (pun intended) where gemma-wrapper is invoked twice for the same job. This is quite possible when people get impatient waiting for a first job to finish. + +One solution is to write a lock file using the inputs as a hash. The lock file can contain a PID and we can check if that is still alive. I should do the same for sheepdog locks(!) -- cgit v1.2.3 From bae8d9f1f1db6eadd1d1424358156b03e1250d4a Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Tue, 16 Nov 2021 05:27:02 -0600 Subject: On GN documentation --- topics/documentation/gn-documentation.gmi | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 topics/documentation/gn-documentation.gmi diff --git a/topics/documentation/gn-documentation.gmi b/topics/documentation/gn-documentation.gmi new file mode 100644 index 0000000..617c8d2 --- /dev/null +++ b/topics/documentation/gn-documentation.gmi @@ -0,0 +1,17 @@ +# GeneNetwork documentation + +GeneNetwork requires an online 'wiki' and a place for documentation. Both of these can be maintained in markdown and/or gemini gemtext. We are using a repo: + +=> https://github.com/genenetwork/gn-docs + +What we need to do is render documents from markdown (as is done with the facilities page) + +=> https://github.com/genenetwork/gn-docs/blob/master/general/help/facilities.md + +the markdown is rendered on github, which makes it easy to edit for most users, so it should be the default. + +## Tags + +* assigned: pjotrp + +## Info -- cgit v1.2.3 From 50797e6b3edadf2ba2c93c635ca67fc589b7622b Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Tue, 16 Nov 2021 06:22:13 -0600 Subject: genewiki and global search --- issues/genenetwork/genewiki.gmi | 24 ++++++++++++++++++++++++ issues/genenetwork/global-search.gmi | 12 ++++++++++++ tasks/zsloan.gmi | 12 ++++++++++++ topics/documentation/gn-documentation.gmi | 4 ++++ 4 files changed, 52 insertions(+) create mode 100644 issues/genenetwork/genewiki.gmi create mode 100644 issues/genenetwork/global-search.gmi create mode 100644 tasks/zsloan.gmi diff --git a/issues/genenetwork/genewiki.gmi b/issues/genenetwork/genewiki.gmi new file mode 100644 index 0000000..e7096dc --- /dev/null +++ b/issues/genenetwork/genewiki.gmi @@ -0,0 +1,24 @@ +# Genewiki conversion + +GeneNetwork1 contains a genewiki: + +=> http://gn1.genenetwork.org/webqtl/main.py?FormID=geneWiki&symbol=BRCA2 + +We want to migrate it to markdown documents that can be fetched from + +=> https://github.com/genenetwork/gn-docs/genes + +So the steps are to (1) migrate the existing genewiki data in the database to named markdown documents in that repository and (2) create a rendered page that is found through + +=> https://genenetwork.org/doc/genes/BRCA2 + +with an edit button, similar to + +=> http://genenetwork.org/facilities/ + +## Tags + +* assigned: pjotrp, zsloan +* enhancement + +## Tasks diff --git a/issues/genenetwork/global-search.gmi b/issues/genenetwork/global-search.gmi new file mode 100644 index 0000000..01d9c0f --- /dev/null +++ b/issues/genenetwork/global-search.gmi @@ -0,0 +1,12 @@ +# Global search problems + +Global search is the top bar of GN2 + +## Tags + +* assigned: pjotrp, zsloan + +## Tasks + +* [ ] BRCA2 does not render results in table +* [ ] 'Brca2' with quotes gives a SQL error diff --git a/tasks/zsloan.gmi b/tasks/zsloan.gmi new file mode 100644 index 0000000..7e60240 --- /dev/null +++ b/tasks/zsloan.gmi @@ -0,0 +1,12 @@ +# Tasks for Zach + +## Tags + +* assigned: zsloan + +## Tasks + +* [ ] Fix global search +=> ../issues/genenetwork/global-search.gmi fix global search +* [ ] GN1 migrate genewiki pages +=> ../issues/genenetwork/genewiki.gmi diff --git a/topics/documentation/gn-documentation.gmi b/topics/documentation/gn-documentation.gmi index 617c8d2..653a1a4 100644 --- a/topics/documentation/gn-documentation.gmi +++ b/topics/documentation/gn-documentation.gmi @@ -14,4 +14,8 @@ the markdown is rendered on github, which makes it easy to edit for most users, * assigned: pjotrp +## Tasks + +=> storyboards + ## Info -- cgit v1.2.3 From 5300d6034fad3a57e45795a485f70b0aeef1f665 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 17 Nov 2021 01:13:38 -0500 Subject: add qc checks list --- topics/quality-control/qc-checks.gmi | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 topics/quality-control/qc-checks.gmi diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi new file mode 100644 index 0000000..55b4847 --- /dev/null +++ b/topics/quality-control/qc-checks.gmi @@ -0,0 +1,29 @@ +# Quality Control Checks + +1. Gene Symbols to ProbeSetId (Affymetrix format): + +AFFX-BkGr-GC03_st -> TCO500002136.mm.2 + +2. Inbred Strain names should prefer long form: + +B6 -> C57BL/6 +D2 -> DBA/2 + +3. Probeset IDs that don't have any values should be pruned: + +For example an Affymetrix data set might have ~28,000 entries and the data set that +is allowed into the GeneNetwork will be 22,000 entries. + +4. The standard error between male and female mice has to be computed. + +5. SE values have to be computed to 8 decimal places. + +6. The average between male and female mice has to be computed. + +7. AVG values have to be computed to only 3 decimal places. + +8. Datasets/studies having the same ProbeSetID should be grouped together. + +9. There should be no trailing spaces in data cells. + +10. Entries should have the same capitalization style. -- cgit v1.2.3 From eb2c81c4fdeab43f1d8b0b746638075562829040 Mon Sep 17 00:00:00 2001 From: jgart Date: Fri, 19 Nov 2021 01:43:21 -0500 Subject: Add more info to qc checks --- topics/quality-control/qc-checks.gmi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index 55b4847..f2e47fd 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -2,6 +2,18 @@ 1. Gene Symbols to ProbeSetId (Affymetrix format): +We favour using Illumina, Affimetrix, and other platform formats. + +Custom formats require a new annotation file to be created. + +We usually use Ensemble ID or Gene IDs. + +1.1 Ensemble transcript IDs usually have duplicates that need to be pruned. + +ENSMBL1234 + +## Example Gene Symbol to ProbeSetId + AFFX-BkGr-GC03_st -> TCO500002136.mm.2 2. Inbred Strain names should prefer long form: -- cgit v1.2.3 From 3a8cc55a68ff69c68540d60aef79d30198ff4994 Mon Sep 17 00:00:00 2001 From: jgart Date: Fri, 19 Nov 2021 01:48:21 -0500 Subject: Add enumerate QC checks issue --- issues/quality-control/enumerate-all-qc-checks.gmi | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 issues/quality-control/enumerate-all-qc-checks.gmi diff --git a/issues/quality-control/enumerate-all-qc-checks.gmi b/issues/quality-control/enumerate-all-qc-checks.gmi new file mode 100644 index 0000000..2f72f24 --- /dev/null +++ b/issues/quality-control/enumerate-all-qc-checks.gmi @@ -0,0 +1,7 @@ +# Enumerating all QC checks + +We need to enumerate all QC checks and platforms that we want to test against. + +* assigned: jgart, arthur + +* in progress -- cgit v1.2.3 From 3b96854df76b15239c6cbc7d1c2144b29a4b5745 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Fri, 19 Nov 2021 11:23:48 +0300 Subject: issue: authorisation: Add extra TODO items --- issues/authorisation.gmi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/issues/authorisation.gmi b/issues/authorisation.gmi index 044962d..c98509b 100644 --- a/issues/authorisation.gmi +++ b/issues/authorisation.gmi @@ -10,6 +10,18 @@ data up-loader, and it requires authorisation, this is a good time to clean up authorisation. +# TODO + +- [ ] Make user able to delete/ add extra value + +- [ ] Fetch complete list of samples from database and genotype file + instead of only fetching that list from the database. Look at trait + page for reference. + +- [ ] Extend idea of csv generation to probeset data. Get data from + the genotype file and work out how to update the values from the + database. + # Notes mRNA datasets ("ProbeSetFreeze" in the DB) contain traits, so when you -- cgit v1.2.3 From a687b59d7dab8fbf244d9b44327a1840ca1bf41f Mon Sep 17 00:00:00 2001 From: acenteno Date: Fri, 19 Nov 2021 13:47:33 -0600 Subject: Update enumerate-all-qc-checks.gmi --- issues/quality-control/enumerate-all-qc-checks.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/issues/quality-control/enumerate-all-qc-checks.gmi b/issues/quality-control/enumerate-all-qc-checks.gmi index 2f72f24..da2799a 100644 --- a/issues/quality-control/enumerate-all-qc-checks.gmi +++ b/issues/quality-control/enumerate-all-qc-checks.gmi @@ -5,3 +5,5 @@ We need to enumerate all QC checks and platforms that we want to test against. * assigned: jgart, arthur * in progress + +Arthur was here. -- cgit v1.2.3 From 35799349d17dfa98ca12afbc40f2d80cfccbccfa Mon Sep 17 00:00:00 2001 From: acenteno Date: Fri, 19 Nov 2021 13:49:02 -0600 Subject: Update qc.gmi --- topics/quality-control/qc.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/topics/quality-control/qc.gmi b/topics/quality-control/qc.gmi index 96c4604..05c3566 100644 --- a/topics/quality-control/qc.gmi +++ b/topics/quality-control/qc.gmi @@ -31,3 +31,5 @@ Note we could run QC through the REST API too. That would allow it to be run from R and Python and Jupyter notebooks. Make it part of GN3. The tricky part is still how the data is handled in escrow. + +Arthur was here also. -- cgit v1.2.3 From 78143cb46fb98b2152d9e40d5f885e5eea2414c3 Mon Sep 17 00:00:00 2001 From: acenteno Date: Fri, 19 Nov 2021 13:49:58 -0600 Subject: Update qc.gmi --- topics/quality-control/qc.gmi | 2 -- 1 file changed, 2 deletions(-) diff --git a/topics/quality-control/qc.gmi b/topics/quality-control/qc.gmi index 05c3566..96c4604 100644 --- a/topics/quality-control/qc.gmi +++ b/topics/quality-control/qc.gmi @@ -31,5 +31,3 @@ Note we could run QC through the REST API too. That would allow it to be run from R and Python and Jupyter notebooks. Make it part of GN3. The tricky part is still how the data is handled in escrow. - -Arthur was here also. -- cgit v1.2.3 From 1d1d0f4f448a2b0e15f10a3a7030d93f955cb421 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sat, 20 Nov 2021 12:45:16 +0100 Subject: gemma-wrapper: add lock support --- issues/gemma/gemma-wrapper-has-incomplete-files.gmi | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi index 4bea71d..3c9a7ad 100644 --- a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi +++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi @@ -17,6 +17,10 @@ GNU parallel can fail, but does not tell how individual processes did. Need to c Turns out GNU parallel can keep track of jobs in a job log - and even rerun the ones missing. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! +## Dealing with locks + There is another parallel issue (pun intended) where gemma-wrapper is invoked twice for the same job. This is quite possible when people get impatient waiting for a first job to finish. One solution is to write a lock file using the inputs as a hash. The lock file can contain a PID and we can check if that is still alive. I should do the same for sheepdog locks(!) + +=> https://github.com/genetics-statistics/gemma-wrapper/commit/e7e516ec5a6ffc5b398302fa204685a40e76e171 Added locking support -- cgit v1.2.3 From 722fc762c7749470b1f7a22cfd74e7acda8146a1 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sat, 20 Nov 2021 12:56:05 +0100 Subject: gemma-wrapper: add lock support --- issues/gemma/gemma-wrapper-has-incomplete-files.gmi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi index 3c9a7ad..7b8093a 100644 --- a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi +++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi @@ -9,7 +9,7 @@ Gemma wrapper caches files - but it can happen a cached file is incomplete and n ## Tasks * [ ] parse parallel job log for failed tasks and remove the output files. -* [ ] create a (global) lock file for gemma-wrapper +* [X] create a (global) lock file for gemma-wrapper ## Info -- cgit v1.2.3 From 2f09b7c24096ae7de0fef7df12de4f5f36d0514d Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sun, 21 Nov 2021 10:29:37 +0100 Subject: gemma musings --- issues/gemma/gemma-wrapper-has-incomplete-files.gmi | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi index 7b8093a..6441376 100644 --- a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi +++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi @@ -15,7 +15,13 @@ Gemma wrapper caches files - but it can happen a cached file is incomplete and n GNU parallel can fail, but does not tell how individual processes did. Need to check if it can return a thread (number). If not we have the option of checking the GEMMA status file and/or see if the output file is complete (by counting number of lines). -Turns out GNU parallel can keep track of jobs in a job log - and even rerun the ones missing. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! +The 'obvious' fix would be to create an error handler in GEMMA itself that would clean up output files on error exit. E.g. using + +=> https://www.cplusplus.com/reference/exception/set_terminate/ + +The problem is that it is NOT a catch all. If there is a hardware fault - a hanging CPU core, for example, which we see - or a problem in a library, such as openblas, there is no guarantee that the terminate handler will be called. Another complication is that a terminate handler needs to be aware of the files being output - i.e., we need to carry the state down somehow. I think we can probably address these issues as much is handled in the GEMMA PARAM class, but it is not worth the effort. + +It turns out that GNU parallel can keep track of jobs in a job log - and even rerun the ones missing. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! To me this is the obvious solution because 'parallel' is monitoring outside the GEMMA process and is a hardened piece of software. On failure it simply designates runs that way and we can clean up any (partly) produced files followed by a safe rerun. The lock routine below ascertains no processes are creating the same output at the same time. ## Dealing with locks -- cgit v1.2.3 From 1abf5bc1b5158d9a83ad61ef2ac27bdb62160ad5 Mon Sep 17 00:00:00 2001 From: jgart Date: Mon, 22 Nov 2021 20:51:32 -0500 Subject: Add ideas and references topic for gemini --- topics/gemini/ideas_and_references.gmi | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 topics/gemini/ideas_and_references.gmi diff --git a/topics/gemini/ideas_and_references.gmi b/topics/gemini/ideas_and_references.gmi new file mode 100644 index 0000000..2544594 --- /dev/null +++ b/topics/gemini/ideas_and_references.gmi @@ -0,0 +1,3 @@ +Ideas and References + +=> gemini://drewdevault.com/2021/11/15/Reframing-gemini.gmi Reframing the philosophy of Gemini \ No newline at end of file -- cgit v1.2.3 From 77153d053fee86f80c9bcf190ef1d303602c95a9 Mon Sep 17 00:00:00 2001 From: jgart Date: Mon, 22 Nov 2021 20:59:40 -0500 Subject: Adds gene symbols issue --- issues/quality-control/enumerate-all-qc-checks.gmi | 2 -- issues/quality-control/gene-symbols.gmi | 13 +++++++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) create mode 100644 issues/quality-control/gene-symbols.gmi diff --git a/issues/quality-control/enumerate-all-qc-checks.gmi b/issues/quality-control/enumerate-all-qc-checks.gmi index da2799a..2f72f24 100644 --- a/issues/quality-control/enumerate-all-qc-checks.gmi +++ b/issues/quality-control/enumerate-all-qc-checks.gmi @@ -5,5 +5,3 @@ We need to enumerate all QC checks and platforms that we want to test against. * assigned: jgart, arthur * in progress - -Arthur was here. diff --git a/issues/quality-control/gene-symbols.gmi b/issues/quality-control/gene-symbols.gmi new file mode 100644 index 0000000..63265ea --- /dev/null +++ b/issues/quality-control/gene-symbols.gmi @@ -0,0 +1,13 @@ +# Gene Symbol Mappings + +How can/should I lookup *all* the known mappings between different representations of symbol names? + +For example, B6 maps to C57BL/6J and D2 maps to DBA/2J. + +I'll need that mapping information for every strain, symbol etc... that has an alias in order to make the conversions. + +Ideally, I can have those mappings in some machine-readable format like csv, json, etc... + +* assigned: jgart, arthur + +* in progress -- cgit v1.2.3 From deca20da6e0a7b17b7222bb031a7da5fb43ede27 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Tue, 23 Nov 2021 10:36:38 +0100 Subject: GN1 and machine room --- issues/authorisation.gmi | 7 +++++-- issues/genenetwork1/gn1-symbols.gmi | 23 +++++++++++++++++++++++ tasks/pjotrp.gmi | 3 ++- topics/systems/machine-room.gmi | 7 ++++--- 4 files changed, 34 insertions(+), 6 deletions(-) create mode 100644 issues/genenetwork1/gn1-symbols.gmi diff --git a/issues/authorisation.gmi b/issues/authorisation.gmi index c98509b..751ec86 100644 --- a/issues/authorisation.gmi +++ b/issues/authorisation.gmi @@ -2,13 +2,16 @@ Look at Chris' authorisation code: -=> https://github.com/chfi/gn-proxy +=> https://github.com/genenetwork/gn-proxy Atm genenetwork2's handling of authorisation is a bit clunky; Since @Bonface is working on the data data up-loader, and it requires authorisation, this is a good time to clean up authorisation. +# Tags + +* assigned: bonfacem, pjotrp, zachs # TODO @@ -123,4 +126,4 @@ On the UI(quote from Zach): to the show trait page. Key point is that I changed the URLs and moved them to their own module. Particularly for published datasets. -- New PR: https://github.com/genenetwork/genenetwork2/pull/614 \ No newline at end of file +- New PR: https://github.com/genenetwork/genenetwork2/pull/614 diff --git a/issues/genenetwork1/gn1-symbols.gmi b/issues/genenetwork1/gn1-symbols.gmi new file mode 100644 index 0000000..ac4cf71 --- /dev/null +++ b/issues/genenetwork1/gn1-symbols.gmi @@ -0,0 +1,23 @@ +# GN1 symbols + +# Tags + +* assigned: pjotrp, arthurc + +# Tasks + +* [ ] fix font issue +* [ ] fix linked webqtl images + +# Info + +The fonts differ on + +=> http://gn1-lily.genenetwork.org/webqtl/main.py?FormID=sharinginfo&GN_AccessionId=2 +=> http://gn1.genenetwork.org/webqtl/main.py?FormID=sharinginfo&GN_AccessionId=2 + +There is a deeper problem. When you click on one of the symbols it wants to show, for example: + +=> http://www.webqtl.org/array_images/S092UM-1F2.png + +This fails both for GN1 and Lily. Do we still want to show those images, or simply disable? If we want to show them, where are they? diff --git a/tasks/pjotrp.gmi b/tasks/pjotrp.gmi index 5b5178e..f6554fd 100644 --- a/tasks/pjotrp.gmi +++ b/tasks/pjotrp.gmi @@ -7,7 +7,8 @@ ## Tasks * [ ] Rabbit https and GN2 https -* [X] Allow @bonfacem to run redis on P2 +* [ ] fix GN1 images linking to http://www.webqtl.org/array_images/S238-1F1-U74Av2.png +=> http://gn1.genenetwork.org/webqtl/main.py?FormID=sharinginfo&GN_AccessionId=2 => ../topics/systems/tux02-production.gmi Set up stable GN service * [ ] Pluto tool with Zach => ../issues/genenetwork1/gn1-production-system-issues.gmi diff --git a/topics/systems/machine-room.gmi b/topics/systems/machine-room.gmi index bece333..d75883a 100644 --- a/topics/systems/machine-room.gmi +++ b/topics/systems/machine-room.gmi @@ -6,9 +6,10 @@ ## Tasks -* [ ] Make tux02e visible from outside -* [ ] Network switch 10Gbs -* [ ] Add disks to tux01 - Sep 28 To Brock +* [X] Make tux02e visible from outside +* [ ] Network switch 10Gbs - add hosts +* [ ] Add disks to tux01 and tux02 - need to reboot +* [ ] Set up E-mail relay for tux01 and tux02 smtp.uthsc.edu, port 25 => tux02-production.gmi setup new production machine => decommission-machines.gmi Decommission machines -- cgit v1.2.3 From 9c6b56fafd8647a6ea4385e24b24d47ae2b975c9 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Tue, 23 Nov 2021 17:42:44 +0300 Subject: editing-data: Update notes --- topics/data-uploads/editing-data.gmi | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/topics/data-uploads/editing-data.gmi b/topics/data-uploads/editing-data.gmi index 1bc1bf8..a6029fa 100644 --- a/topics/data-uploads/editing-data.gmi +++ b/topics/data-uploads/editing-data.gmi @@ -60,3 +60,9 @@ After editing traits, you could see changes in a diff format at the very top of That needs to be improved. - For Probeset data, atm some tables aren't yet updated(unfortunately those tables aren't used anywhere). I need to figure that out. + +#### Tue 23 Nov 2021 + +- Fixed excel issue described here: + +=> https://github.com/genenetwork/genenetwork2/compare/testing...BonfaceKilz:bug/fix-excel-adding-new-line?expand=1 \ No newline at end of file -- cgit v1.2.3 From f46ed96a01617514547da4636b6121fadc5f2326 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 24 Nov 2021 00:05:03 -0500 Subject: Add Shapiro-Wilk Test QC Check scipy has an implementation --- topics/quality-control/qc-checks.gmi | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index f2e47fd..30236e1 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -39,3 +39,22 @@ is allowed into the GeneNetwork will be 22,000 entries. 9. There should be no trailing spaces in data cells. 10. Entries should have the same capitalization style. + +11. Assesing Phenotypes for normality with Shapiro-Wilk Test. + +https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.shapiro.html +https://vedexcel.com/how-to-perform-a-shapiro-wilk-test-in-python/ + +``` +Here is a simple QC procedure we may want to consider that was used by +Megan and Camron in a recent paper that I have attached. + + +Phenotypes were assessed for normality using the Shapiro–Wilk Test. Because +some of the data residuals devi- ated significantly from normality, we used +the orderNorm function to perform Ordered Quantile normalization43 on all +phenotypes. + +-- +Rob +``` -- cgit v1.2.3 From 1259ed80a7f60a9f7ba82f3751990ac24a8e9436 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 24 Nov 2021 00:39:40 -0500 Subject: Add citation for Shapiro-Wilk Test --- topics/quality-control/qc-checks.gmi | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index 30236e1..713fa0b 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -58,3 +58,16 @@ phenotypes. -- Rob ``` + +``` +QTL analysis was performed in F2 mice using the R package R/qtl +(RRID:SCR_009085) as previously described.11,29,42 Quality checking +of genotypes and QTL analysis were performed in R (https://www.r- +project.org/) using R/bestNormalize (https://github.com/petersonR/ +bestNormalize) and R/qtl.42 Phenotypes were assessed for normality +using the Shapiro–Wilk Test. Because some of the data residuals devi- +ated significantly from normality, we used the orderNorm function to +perform Ordered Quantile normalization43 on all phenotypes. QTL +-- +A quantitative trait variant in Gabra2 underlies increased methamphetamine stimulant sensitivity +``` -- cgit v1.2.3 From e867ad5edbebfab57e2efd09cde7246f3cb79cc7 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 24 Nov 2021 01:26:09 -0500 Subject: Check for annotations file --- topics/quality-control/qc-checks.gmi | 30 +----------------------------- 1 file changed, 1 insertion(+), 29 deletions(-) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index 713fa0b..149557c 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -42,32 +42,4 @@ is allowed into the GeneNetwork will be 22,000 entries. 11. Assesing Phenotypes for normality with Shapiro-Wilk Test. -https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.shapiro.html -https://vedexcel.com/how-to-perform-a-shapiro-wilk-test-in-python/ - -``` -Here is a simple QC procedure we may want to consider that was used by -Megan and Camron in a recent paper that I have attached. - - -Phenotypes were assessed for normality using the Shapiro–Wilk Test. Because -some of the data residuals devi- ated significantly from normality, we used -the orderNorm function to perform Ordered Quantile normalization43 on all -phenotypes. - --- -Rob -``` - -``` -QTL analysis was performed in F2 mice using the R package R/qtl -(RRID:SCR_009085) as previously described.11,29,42 Quality checking -of genotypes and QTL analysis were performed in R (https://www.r- -project.org/) using R/bestNormalize (https://github.com/petersonR/ -bestNormalize) and R/qtl.42 Phenotypes were assessed for normality -using the Shapiro–Wilk Test. Because some of the data residuals devi- -ated significantly from normality, we used the orderNorm function to -perform Ordered Quantile normalization43 on all phenotypes. QTL --- -A quantitative trait variant in Gabra2 underlies increased methamphetamine stimulant sensitivity -``` +12. Check for annotations file. -- cgit v1.2.3 From ba56073820818a8edc198c823b016248fde2c244 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 24 Nov 2021 01:26:26 -0500 Subject: Move Shapiro-Wilk test to separate file --- topics/quality-control/shapiro-wilk-test.gmi | 33 ++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 topics/quality-control/shapiro-wilk-test.gmi diff --git a/topics/quality-control/shapiro-wilk-test.gmi b/topics/quality-control/shapiro-wilk-test.gmi new file mode 100644 index 0000000..b1bdbfd --- /dev/null +++ b/topics/quality-control/shapiro-wilk-test.gmi @@ -0,0 +1,33 @@ +# Shapiro Wilk Test + +This document contains more info about QC step 11. + +https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.shapiro.html +https://vedexcel.com/how-to-perform-a-shapiro-wilk-test-in-python/ + +``` +Here is a simple QC procedure we may want to consider that was used by +Megan and Camron in a recent paper that I have attached. + + +Phenotypes were assessed for normality using the Shapiro–Wilk Test. Because +some of the data residuals devi- ated significantly from normality, we used +the orderNorm function to perform Ordered Quantile normalization43 on all +phenotypes. + +-- +Rob +``` + +``` +QTL analysis was performed in F2 mice using the R package R/qtl +(RRID:SCR_009085) as previously described.11,29,42 Quality checking +of genotypes and QTL analysis were performed in R (https://www.r- +project.org/) using R/bestNormalize (https://github.com/petersonR/ +bestNormalize) and R/qtl.42 Phenotypes were assessed for normality +using the Shapiro–Wilk Test. Because some of the data residuals devi- +ated significantly from normality, we used the orderNorm function to +perform Ordered Quantile normalization43 on all phenotypes. QTL +-- +A quantitative trait variant in Gabra2 underlies increased methamphetamine stimulant sensitivity +``` -- cgit v1.2.3 From c97aa198bf95cb59ece70f3d8460d4537800e92c Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Thu, 25 Nov 2021 08:29:50 +0100 Subject: Using locking code in sheepdog too --- issues/gemma/gemma-wrapper-has-incomplete-files.gmi | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi index 6441376..6cb92f6 100644 --- a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi +++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi @@ -19,9 +19,9 @@ The 'obvious' fix would be to create an error handler in GEMMA itself that would => https://www.cplusplus.com/reference/exception/set_terminate/ -The problem is that it is NOT a catch all. If there is a hardware fault - a hanging CPU core, for example, which we see - or a problem in a library, such as openblas, there is no guarantee that the terminate handler will be called. Another complication is that a terminate handler needs to be aware of the files being output - i.e., we need to carry the state down somehow. I think we can probably address these issues as much is handled in the GEMMA PARAM class, but it is not worth the effort. +The problem is that it is NOT a catch all. If there is a hardware fault or a problem in a library, such as openblas, there is no guarantee that the terminate handler will be called. Another complication is that a terminate handler needs to be aware of the files being output - i.e., we need to carry the state down somehow. I think we can probably address these issues as much is handled in the GEMMA PARAM class, but it is not worth the effort (I'll take care of it in a GEMMA rewrite). -It turns out that GNU parallel can keep track of jobs in a job log - and even rerun the ones missing. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! To me this is the obvious solution because 'parallel' is monitoring outside the GEMMA process and is a hardened piece of software. On failure it simply designates runs that way and we can clean up any (partly) produced files followed by a safe rerun. The lock routine below ascertains no processes are creating the same output at the same time. +It turns out that GNU parallel can keep track of jobs in a job log - and even rerun the ones missing using the `--joblog` and `--resume` switches. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! To me this is the obvious solution because 'parallel' is monitoring outside the GEMMA process and is a hardened piece of software. On failure it simply designates runs that way and we can clean up any (partly) produced files followed by a safe rerun. The lock routine below ascertains no processes are creating the same output at the same time. ## Dealing with locks @@ -30,3 +30,7 @@ There is another parallel issue (pun intended) where gemma-wrapper is invoked tw One solution is to write a lock file using the inputs as a hash. The lock file can contain a PID and we can check if that is still alive. I should do the same for sheepdog locks(!) => https://github.com/genetics-statistics/gemma-wrapper/commit/e7e516ec5a6ffc5b398302fa204685a40e76e171 Added locking support + +I added the same code to replace sheepdog locks. Sheepdog is running every minute on our machines so it is a great test case. + +=> https://github.com/pjotrp/deploy/commit/4790b81ee897c8244280169edb3cac751eb0a9b3 sheepdog locking -- cgit v1.2.3 From d326fd0036a43b4e2b1edfe8137ff5c36570d221 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Thu, 25 Nov 2021 09:25:55 +0100 Subject: Fixed incomplete output for parallel gemma --- issues/gemma/gemma-wrapper-has-incomplete-files.gmi | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi index 6cb92f6..d530fb4 100644 --- a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi +++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi @@ -8,7 +8,7 @@ Gemma wrapper caches files - but it can happen a cached file is incomplete and n ## Tasks -* [ ] parse parallel job log for failed tasks and remove the output files. +* [X] parse parallel job log for failed tasks and remove the output files. * [X] create a (global) lock file for gemma-wrapper ## Info @@ -23,6 +23,12 @@ The problem is that it is NOT a catch all. If there is a hardware fault or a pro It turns out that GNU parallel can keep track of jobs in a job log - and even rerun the ones missing using the `--joblog` and `--resume` switches. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files! To me this is the obvious solution because 'parallel' is monitoring outside the GEMMA process and is a hardened piece of software. On failure it simply designates runs that way and we can clean up any (partly) produced files followed by a safe rerun. The lock routine below ascertains no processes are creating the same output at the same time. +## Delete files on failure + +Implemented in + +=> https://github.com/genetics-statistics/gemma-wrapper/commit/624ed0d805f29ab682cffbe46bc104dffd0d713c + ## Dealing with locks There is another parallel issue (pun intended) where gemma-wrapper is invoked twice for the same job. This is quite possible when people get impatient waiting for a first job to finish. -- cgit v1.2.3 From ff4ce4c1a520d4f7cd539b7366f9d090bae4e205 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Sat, 27 Nov 2021 09:50:54 +0300 Subject: issue: Add need for redis documentation --- issues/redis-data.gmi | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 issues/redis-data.gmi diff --git a/issues/redis-data.gmi b/issues/redis-data.gmi new file mode 100644 index 0000000..2246c96 --- /dev/null +++ b/issues/redis-data.gmi @@ -0,0 +1,6 @@ +# Document redis data + +ATM how we store data in redis is undocumented. Without this, it's +hard for contributors to work out how to modify some things. For +example, PJ would want to add himself to the editors group. Doing +this was non-trivial for him. \ No newline at end of file -- cgit v1.2.3 From c10687b09b144b0fd38b4eeeea4b92f6dc968aa1 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 1 Dec 2021 01:05:09 -0500 Subject: remove mention of conversion of gene symbols since we are not converting --- topics/quality-control/qc-checks.gmi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index 149557c..756d2ee 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -1,6 +1,6 @@ # Quality Control Checks -1. Gene Symbols to ProbeSetId (Affymetrix format): +1. ProbeSetId (Affymetrix format): We favour using Illumina, Affimetrix, and other platform formats. -- cgit v1.2.3 From 615a6aca8a1db5401870cc133baa33f0bbc06b0d Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 1 Dec 2021 01:05:52 -0500 Subject: add notes on ui design --- topics/quality-control/ui-design.gmi | 39 ++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 topics/quality-control/ui-design.gmi diff --git a/topics/quality-control/ui-design.gmi b/topics/quality-control/ui-design.gmi new file mode 100644 index 0000000..1ea6f21 --- /dev/null +++ b/topics/quality-control/ui-design.gmi @@ -0,0 +1,39 @@ +# UI Design + +1. Input/Receive Data in UI (drag and drop/upload submit form) + +2. Select Mouse + +"What type of Group are you using?" + +> (AKXD, BXH, Mouse Diversity Panel, BXD) + +3. "What is your platform?" + +> (Aff, Ilumina, ...) + +If Affymetrix (Aff) is selected then there should be various options +like Clarion S. + +If the platform you chose is not available: + + Tell PI that they should solicit for their platform to be added to the list. + + They can contact us via email. + +## More Example UI Interactions and Checks + +"If your dataset does not comply with GN then you can try uploading your +dataset so that we can inspect it." + +"Your dataset has two erroneous entries: Gene Accession Gene." + +"The last two columns have the wrong format for the strain name." + +"Here's our format of how your dataset should look like." + +> ProbeSetID Strains ... + +"Inbred Set ID 1 is the same as BXD" + +> These are the strains: ... -- cgit v1.2.3 From 89c182b5418852fbe8ef1efeb66866f7bef550f9 Mon Sep 17 00:00:00 2001 From: jgart Date: Wed, 1 Dec 2021 01:18:50 -0500 Subject: qc-checks: check for crlf --- topics/quality-control/qc-checks.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index 756d2ee..cbf7e5b 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -43,3 +43,5 @@ is allowed into the GeneNetwork will be 22,000 entries. 11. Assesing Phenotypes for normality with Shapiro-Wilk Test. 12. Check for annotations file. + +13. Check for CRLF. -- cgit v1.2.3 From 655b3a4d69f7c79d87796fc50886d62c7ba3860e Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Fri, 3 Dec 2021 15:41:23 +0300 Subject: authorisation: Check off completed tasks --- issues/authorisation.gmi | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/issues/authorisation.gmi b/issues/authorisation.gmi index 751ec86..0adac56 100644 --- a/issues/authorisation.gmi +++ b/issues/authorisation.gmi @@ -15,9 +15,9 @@ clean up authorisation. # TODO -- [ ] Make user able to delete/ add extra value +- [x] Make user able to delete/ add extra value -- [ ] Fetch complete list of samples from database and genotype file +- [x] Fetch complete list of samples from database and genotype file instead of only fetching that list from the database. Look at trait page for reference. -- cgit v1.2.3 From 96b9a25c0911b6e8411ee12cc3eac048729e217b Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Fri, 3 Dec 2021 15:42:02 +0300 Subject: editing-data: Add bonfacem to assignees --- topics/data-uploads/editing-data.gmi | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/topics/data-uploads/editing-data.gmi b/topics/data-uploads/editing-data.gmi index a6029fa..01144a0 100644 --- a/topics/data-uploads/editing-data.gmi +++ b/topics/data-uploads/editing-data.gmi @@ -1,3 +1,7 @@ +## Tags + +* assigned: bonfacem + ### Introduction At the moment, you can edit metadata related to a published phenotype and a probeset. When an edit is done, the diff data is stored in a table, `metadata_audit` in json format that looks something like: -- cgit v1.2.3 From cbe94eb40a84a6a071f671d685d83284030d4cd8 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Fri, 3 Dec 2021 15:42:19 +0300 Subject: deploy.gmi: New issue to track configuration settings --- issues/deploy.gmi | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 issues/deploy.gmi diff --git a/issues/deploy.gmi b/issues/deploy.gmi new file mode 100644 index 0000000..f701805 --- /dev/null +++ b/issues/deploy.gmi @@ -0,0 +1,26 @@ +# Handle Configuration Files + +How we handle our configuration is a mess; and this makes deployment a +hassle. In genenetwork2, our configs live in different places, and it +takes some fiddling tinged with a bit of domain (gn2) knowledge to +work out what setting live where. With the introduction of blueprints +in our code-base, this duplication became a necessary evil because +most of the settings QC checks happen in a "utility/settings.py" which +unfortunately imports the "app" object. This import leads to circular +imports whenever we use blueprints. Unfortunately, a huge chunk of +the code base uses "utility/settings.py" to get configuration +settings; and removing this would make gn2 as is disfunctional. + +# Tags + +* assigned: bonfacem + +# Notes + +Here's an example of how to use json as our settings: + +=> https://stackoverflow.com/questions/15122312/how-to-import-from-config-file-in-flask + +Here's Flask's official documentation on handling configuration: + +=> https://flask.palletsprojects.com/en/1.1.x/config/ -- cgit v1.2.3 From c630ecb79cec6d22562f33566191034f3eaab8c3 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Fri, 3 Dec 2021 15:48:12 +0300 Subject: Reference correct module --- issues/deploy.gmi | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/issues/deploy.gmi b/issues/deploy.gmi index f701805..94d73ec 100644 --- a/issues/deploy.gmi +++ b/issues/deploy.gmi @@ -5,10 +5,10 @@ hassle. In genenetwork2, our configs live in different places, and it takes some fiddling tinged with a bit of domain (gn2) knowledge to work out what setting live where. With the introduction of blueprints in our code-base, this duplication became a necessary evil because -most of the settings QC checks happen in a "utility/settings.py" which +most of the settings QC checks happen in a "utility/tools.py" which unfortunately imports the "app" object. This import leads to circular imports whenever we use blueprints. Unfortunately, a huge chunk of -the code base uses "utility/settings.py" to get configuration +the code base uses "utility/tools.py" to get configuration settings; and removing this would make gn2 as is disfunctional. # Tags -- cgit v1.2.3 From bb67515f8b8c7b1098836e615b970ba1415ae113 Mon Sep 17 00:00:00 2001 From: jgart Date: Sun, 5 Dec 2021 21:05:48 -0500 Subject: issue: genenetwork: http->https --- issues/genenetwork/http-https.gmi | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 issues/genenetwork/http-https.gmi diff --git a/issues/genenetwork/http-https.gmi b/issues/genenetwork/http-https.gmi new file mode 100644 index 0000000..80306bf --- /dev/null +++ b/issues/genenetwork/http-https.gmi @@ -0,0 +1,15 @@ +# Arthur mentioned last meet that GeneNetwork2 does not redirect http urls to https. + +He'd like http to redirect to https. + +http://genenetwork.org/ -> https://genenetwork.org/ + +# nginx snippet + +``` +if ($scheme = http) { + return 301 https://$server_name$request_uri; +} +``` + +* assigned: pjotr, arthur, jgart -- cgit v1.2.3 From 57dd5d991631b25790a10798c5ba17c8a4ab15f0 Mon Sep 17 00:00:00 2001 From: jgart Date: Mon, 6 Dec 2021 06:10:10 -0500 Subject: allow excel file upload? --- topics/quality-control/ui-design.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/topics/quality-control/ui-design.gmi b/topics/quality-control/ui-design.gmi index 1ea6f21..84f7748 100644 --- a/topics/quality-control/ui-design.gmi +++ b/topics/quality-control/ui-design.gmi @@ -21,6 +21,8 @@ If the platform you chose is not available: They can contact us via email. +4. Allow excel file upload? + ## More Example UI Interactions and Checks "If your dataset does not comply with GN then you can try uploading your -- cgit v1.2.3 From fe53e6c30090cdea1cd947de876d6ee2f7b28983 Mon Sep 17 00:00:00 2001 From: jgart Date: Tue, 14 Dec 2021 04:04:19 -0500 Subject: add qc check for utf-8 encoding --- topics/quality-control/qc-checks.gmi | 2 ++ 1 file changed, 2 insertions(+) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index cbf7e5b..9c8fcfd 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -45,3 +45,5 @@ is allowed into the GeneNetwork will be 22,000 entries. 12. Check for annotations file. 13. Check for CRLF. + +14. Check for UTF-8 encoding. -- cgit v1.2.3 From 7c7df39586d850fb560987081984069c66d70b33 Mon Sep 17 00:00:00 2001 From: jgart Date: Sat, 18 Dec 2021 09:33:59 -0500 Subject: qc-checks: Update SE values decimal size --- topics/quality-control/qc-checks.gmi | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/topics/quality-control/qc-checks.gmi b/topics/quality-control/qc-checks.gmi index 9c8fcfd..0d04625 100644 --- a/topics/quality-control/qc-checks.gmi +++ b/topics/quality-control/qc-checks.gmi @@ -28,22 +28,20 @@ is allowed into the GeneNetwork will be 22,000 entries. 4. The standard error between male and female mice has to be computed. -5. SE values have to be computed to 8 decimal places. +5. SE values have to be computed to 6 or greater decimal places. -6. The average between male and female mice has to be computed. +6. The average between male and female mice has to be computed to 3 decimal places. -7. AVG values have to be computed to only 3 decimal places. +7. Datasets/studies having the same ProbeSetID should be grouped together. -8. Datasets/studies having the same ProbeSetID should be grouped together. +8. There should be no trailing spaces in data cells. -9. There should be no trailing spaces in data cells. +9. Entries should have the same capitalization style. -10. Entries should have the same capitalization style. +10. Assesing Phenotypes for normality with Shapiro-Wilk Test. -11. Assesing Phenotypes for normality with Shapiro-Wilk Test. +11. Check for annotations file. -12. Check for annotations file. +12. Check for CRLF. -13. Check for CRLF. - -14. Check for UTF-8 encoding. +13. Check for UTF-8 encoding. -- cgit v1.2.3 From 9a9f8ac1f6789b0bd3305dd91e30f5b4a9736140 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Tue, 21 Dec 2021 08:25:31 +0100 Subject: Update database not responding issue --- issues/database-not-responding.gmi | 134 ++++++++++++++++++++++++++++++++++++- 1 file changed, 133 insertions(+), 1 deletion(-) diff --git a/issues/database-not-responding.gmi b/issues/database-not-responding.gmi index c8e752b..e66c939 100644 --- a/issues/database-not-responding.gmi +++ b/issues/database-not-responding.gmi @@ -1,4 +1,136 @@ -# Mariadb table locked +# Hanging database + +Mariadb occassionally stops responding. + +## Tasks + +* assigned: pjotrp, zsloan +* bug + +# Info + +## Mariadb is 'hanging' + +In the last 12 hours GN2 monitoring shows the website is responding intermittendly. A quick check shows the database is blocking. Rather than simply restarting the database - which is known to sort the issue - the timing is that the US is sleeping so I can do some checking. Let's take a look. + +Mariadb is at 4x CPU + +``` +PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND + 40514 mysql 20 0 37.3g 1.8g 18268 S 394.4 0.7 4855:33 mysqld +``` + +The ps table shows a backup is ongoing + +``` +root 57559 0.0 0.0 2388 756 ? Ss 03:00 0:00 /bin/sh -c /bin/su mysql -c /export/backup/scripts/tux01/backup_mariadb.sh >> ~/cron.log 2>&1 +mysql 57588 0.2 0.0 200112 27292 ? Sl 03:00 0:25 mariabackup --backup --target-dir=/home/backup/tux01_mariadb_new/latest/ --user=webqtlout --password=x xxxxxxx +``` + +Tales in use: + +``` +MariaDB [db_webqtl]> show open tables where in_use > 1; ++-----------+----------------+--------+-------------+ +| Database | Table | In_use | Name_locked | ++-----------+----------------+--------+-------------+ +| db_webqtl | InfoFiles | 5 | 0 | +| db_webqtl | GeneRIF_BASIC | 3 | 0 | +| db_webqtl | ProbeFreeze | 4 | 0 | +| db_webqtl | ProbeSet | 4 | 0 | +| db_webqtl | ProbeSetXRef | 4 | 0 | +| db_webqtl | Species | 4 | 0 | +| db_webqtl | Geno | 4 | 0 | +| db_webqtl | Chr_Length | 2 | 0 | +| db_webqtl | Tissue | 4 | 0 | +| db_webqtl | ProbeSetFreeze | 4 | 0 | +| db_webqtl | InbredSet | 4 | 0 | ++-----------+----------------+--------+-------------+ +11 rows in set (0.001 sec) +``` + +In the error log we seeing a lot of + +``` +2021-12-21 6:17:57 256179 [Warning] Aborted connection 256179 to db: 'db_webqtl' user: 'webqtlout' host: '128.169.5.59' (Got timeout reading communication packets) +``` + +``` +SHOW FULL PROCESSLIST; +458 rows in set (0.003 sec) +``` + +with entries + +``` +| 256363 | webqtlout | 128.169.5.59:59120 | db_webqtl | Query | 15 | Waiting for table flush | SELECT Id, Name, FullName, ShortName, DataScale FROM ProbeSetFreeze WHERE public > 0 AND (Name = "CB_M_0305_R" OR FullName = "CB_M_0305_R" OR ShortName = "CB_M_0305_R") +``` + +waiting for tables to flush! + +at the top of the process list we find + +``` +Id User Host db Command Time State Info Progress 1 system user NULL Daemon NULL InnoDB purge coordinator NULL 0.000 2 system user NULL Daemon NULL InnoDB purge worker NULL 0.000 3 system user NULL Daemon NULL InnoDB purge worker NULL 0.000 4 system user NULL Daemon NULL InnoDB purge worker NULL 0.000 5 system user NULL Daemon NULL InnoDB shutdown handler NULL 0.000 227365 webqtlout 127.0.0.1:33950 db_webqtl Sleep 13015 NULL 0.000 245634 webqtlout 127.0.0.1:38098 db_webqtl Sleep 23180 NULL 0.000 +``` + +This is quite informative: + +=> https://programmer.group/analysis-of-mysql-process-in-waiting-for-table-flush.html + +it suggests that the backup can be the root of the problem. + +And then + +=> https://www.thegeekdiary.com/troubleshooting-mysql-query-hung-waiting-for-table-flush/ + +suggests + +* Wait for the long-running queries which are blocking the FLUSH TABLE to complete; +* Identify the long-running queries and kill them; +* Restart the server + +and that is somewhat amusing. + +Stripping out all reqular queries we get: + +``` +grep localhost test.out |grep -vi probesetfreeze|grep -vi species +255559 webqtlout localhost NULL Query 13092 Waiting for table flush FLUSH NO_WRITE_TO_BINLOG TABLES0.000 +256351 webqtlout localhost db_webqtl Field List 1588 Waiting for table flush NULL 0.000 +256383 webqtlout localhost db_webqtl Query 0 Init SHOW FULL PROCESSLIST 0.000 +``` + +and it appears everyone is waiting for id 255559. Let's kill that. + +``` +kill 255559; +``` + +and inspect + +``` +mysql -u webqtlout -pwebqtlout db_webqtl -A -e "show processlist;"|less +``` + +it started processing again. To speed up recovery back to: + +``` +systemctl restart mysql +``` + +of course that stopped the running backup. But processing is back in business. + +My first conclusion is that this problem was triggered by the backup procedure. Interestingly, it happens irregularly. We also have seen this issue before this backup procedure was instated, so I figure it has to do with Mariadb. + +The version on production is from 2017 - we should update that soon: + +Server version: 10.3.27-MariaDB-0+deb10u1-log Debian 10 + +we have been running a more recent version of mariadb on luna. Still, that is unlikely to fix this issue because I think it really has to do with myisam and locking of large tables. Switching to innodb does away with global locks and is the default on mariadb (there are less and less people using myisam). + + +## Mariadb table locked Arthur reports: MariaDB is not responding Saturday, July 24 2021 at 10:48 pm. I tried to enter data to the table ProbeSetXRef.pValue and when normally takes few seconds, now is more than 10 minutes without completion/responding. -- cgit v1.2.3 From 125a4fb150e1209dc89ed1ea4f4026965da7524f Mon Sep 17 00:00:00 2001 From: jgart Date: Tue, 21 Dec 2021 03:15:11 -0500 Subject: add topic on using pudb with pytest --- topics/pudb/pytest-pudb.gmi | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 topics/pudb/pytest-pudb.gmi diff --git a/topics/pudb/pytest-pudb.gmi b/topics/pudb/pytest-pudb.gmi new file mode 100644 index 0000000..5a979c0 --- /dev/null +++ b/topics/pudb/pytest-pudb.gmi @@ -0,0 +1,7 @@ +# Using pudb with pytest + +``` +pytest --pdbcls pudb.debugger:Debugger --pdb --capture=no +``` + +=> https://documen.tician.de/pudb/starting.html?highlight=pytest#usage-with-pytest -- cgit v1.2.3 From 8c14607dbd48af1beaed057c165a2c21230d923e Mon Sep 17 00:00:00 2001 From: jgart Date: Tue, 21 Dec 2021 03:22:21 -0500 Subject: add topic on using pudb --- topics/pudb/using-pudb.gmi | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 topics/pudb/using-pudb.gmi diff --git a/topics/pudb/using-pudb.gmi b/topics/pudb/using-pudb.gmi new file mode 100644 index 0000000..877a700 --- /dev/null +++ b/topics/pudb/using-pudb.gmi @@ -0,0 +1,21 @@ +# Using pudb + +## Setting Breakpoints + +You can add the following line anywhere in your code to start a pudb breakpoint: + +``` +import pudb; pudb.set_trace() +``` + +You can also press `b` in pudb to add a breakpoint with a keyboard shortcut. + +## pdb and pudb API similarity + +The standard pdb docs should work for pudb with one small exception. + +> At the programming language level, PuDB displays the same interface +> as Python’s built-in pdb module. Just replace pdb with pudb. (One +> exception: run is called runstatement.) + +=> https://docs.python.org/3/library/pdb.html?highlight=pdb#module-pdb -- cgit v1.2.3 From a1ab0aabb27a43b8111fcb29040b31b241bcae28 Mon Sep 17 00:00:00 2001 From: jgart Date: Tue, 21 Dec 2021 03:23:47 -0500 Subject: gemini: ideas_and_references: formatting --- topics/gemini/ideas_and_references.gmi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/gemini/ideas_and_references.gmi b/topics/gemini/ideas_and_references.gmi index 2544594..dfb026d 100644 --- a/topics/gemini/ideas_and_references.gmi +++ b/topics/gemini/ideas_and_references.gmi @@ -1,3 +1,3 @@ Ideas and References -=> gemini://drewdevault.com/2021/11/15/Reframing-gemini.gmi Reframing the philosophy of Gemini \ No newline at end of file +=> gemini://drewdevault.com/2021/11/15/Reframing-gemini.gmi Reframing the philosophy of Gemini -- cgit v1.2.3 From 6f2174fe1b3b945b00dd0de6918fce3fbe64afe4 Mon Sep 17 00:00:00 2001 From: jgart Date: Tue, 21 Dec 2021 03:27:59 -0500 Subject: add gemini cheatsheet --- topics/gemini/cheatsheet.gmi | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 topics/gemini/cheatsheet.gmi diff --git a/topics/gemini/cheatsheet.gmi b/topics/gemini/cheatsheet.gmi new file mode 100644 index 0000000..b2158ea --- /dev/null +++ b/topics/gemini/cheatsheet.gmi @@ -0,0 +1,5 @@ +# gemtext cheatsheet + +=> gemini://gemini.circumlunar.space/docs/cheatsheet.gmi + +=> https://gemini.circumlunar.space/docs/cheatsheet.gmi proxy -- cgit v1.2.3 From 741a3a9b7b66d121021127eabfd0e4b617c46094 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Tue, 21 Dec 2021 13:51:30 +0300 Subject: Add task on broken builds --- issues/fix-broken-builds-in-gn2-and-gn3.gmi | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/issues/fix-broken-builds-in-gn2-and-gn3.gmi b/issues/fix-broken-builds-in-gn2-and-gn3.gmi index cf9506f..85cc2dd 100644 --- a/issues/fix-broken-builds-in-gn2-and-gn3.gmi +++ b/issues/fix-broken-builds-in-gn2-and-gn3.gmi @@ -7,5 +7,24 @@ gn3 and gn2. * assigned: bonfacekilz -* [ ] Fix pylint/ mypy issues in gn3 -* [ ] Update docker image in gn2 +### Notes + +At the moment, we are getting the following build failures in gn2, + +``` +Compiling '/gnu/store/bm0909ha5nv5chgp3bk5fw0my6mrsabr-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/wqflask/wgcna/wgcna_analysis.py'... +Compiling '/gnu/store/bm0909ha5nv5chgp3bk5fw0my6mrsabr-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/wsgi.py'... +error: in phase 'install': uncaught exception: +%exception #<&invoke-error program: "python" arguments: ("-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/bm0909ha5nv5chgp3bk5fw0my6mrsabr-genenetwork2-3.11-guix-fe345c9") exit-status: 1 term-signal: #f stop-signal: #f> +phase `install' failed after 0.9 seconds +command "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/bm0909ha5nv5chgp3bk5fw0my6mrsabr-genenetwork2-3.11-guix-fe345c9" failed with status 1 +note: keeping build directory `/tmp/guix-build-genenetwork2-3.11-guix-fe345c9.drv-0' +builder for `/gnu/store/l5hb8bfz3di3zga93963arlka03lijn8-genenetwork2-3.11-guix-fe345c9.drv' failed with exit code 1 +build of /gnu/store/l5hb8bfz3di3zga93963arlka03lijn8-genenetwork2-3.11-guix-fe345c9.drv failed +View build log at '/var/log/guix/drvs/l5/hb8bfz3di3zga93963arlka03lijn8-genenetwork2-3.11-guix-fe345c9.drv.bz2'. +guix build: error: build of `/gnu/store/l5hb8bfz3di3zga93963arlka03lijn8-genenetwork2-3.11-guix-fe345c9.drv' failed + +``` + +This starts happening (not yet comfirmed) when python is bumped up +from 3.8 to 3.9 -- cgit v1.2.3 From 580f1b72704b0df0f7fac80afe02553ec2a3ddfa Mon Sep 17 00:00:00 2001 From: Efraim Flashner Date: Tue, 21 Dec 2021 14:32:10 +0200 Subject: update to genenetwork2 building issues --- issues/fix-broken-builds-in-gn2-and-gn3.gmi | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/issues/fix-broken-builds-in-gn2-and-gn3.gmi b/issues/fix-broken-builds-in-gn2-and-gn3.gmi index 85cc2dd..d52d121 100644 --- a/issues/fix-broken-builds-in-gn2-and-gn3.gmi +++ b/issues/fix-broken-builds-in-gn2-and-gn3.gmi @@ -5,7 +5,7 @@ gn3 and gn2. ## Tasks -* assigned: bonfacekilz +* assigned: bonfacekilz, efraim ### Notes @@ -28,3 +28,15 @@ guix build: error: build of `/gnu/store/l5hb8bfz3di3zga93963arlka03lijn8-genenet This starts happening (not yet comfirmed) when python is bumped up from 3.8 to 3.9 + +In addition, there are some dependencies of genenetwork2 which FTBFS, due to incorrect versions of inputs. + +``` + starting phase `sanity-check'¬ + validating 'pingouin' /gnu/store/dpy5ln4zjgv53l8jpm1qalwn45cagxfm-python-pingouin-0.5.0/lib/python3.9/site-packages¬ + ...checking requirements: ERROR: pingouin==0.5.0 (statsmodels 0.11.1 (/gnu/store/ab8k9bwmaml2bplq4bd9xkg08z7bn8fr-python-statsmodels-0.11.1/lib/python3.9/site-packages), Requirement.parse('statsmodels>=0.12.0'), {'pingouin'})¬ + error: in phase 'sanity-check': uncaught exception:¬ + %exception #<&invoke-error program: "python" arguments: ("/gnu/store/nwwr89v2vyg1hs48i49m083vhczsgh3m-sanity-check.py" "/gnu/store/dpy5ln4zjgv53l8jpm1qalwn45cagxfm-python-pingouin-0.5.0/lib/python3.9/site-packages") exit-status: 1 term-signal: #f stop-signal: #f>▸¬ + phase `sanity-check' failed after 0.1 seconds¬ +``` + -- cgit v1.2.3 From be16a6a7f1a7e2dfa074e858c26ff6a9b6aa86de Mon Sep 17 00:00:00 2001 From: Efraim Flashner Date: Tue, 21 Dec 2021 15:42:47 +0200 Subject: update on genenetwork2 not building --- issues/fix-broken-builds-in-gn2-and-gn3.gmi | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/issues/fix-broken-builds-in-gn2-and-gn3.gmi b/issues/fix-broken-builds-in-gn2-and-gn3.gmi index d52d121..3c0e6aa 100644 --- a/issues/fix-broken-builds-in-gn2-and-gn3.gmi +++ b/issues/fix-broken-builds-in-gn2-and-gn3.gmi @@ -26,6 +26,22 @@ guix build: error: build of `/gnu/store/l5hb8bfz3di3zga93963arlka03lijn8-genenet ``` +Some missing bits from the build log: + +``` +Compiling '/gnu/store/n2w3bdka4zfl3jw0iqk49km1hgl8i7g5-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/utility/redis_tools.py'... +Compiling '/gnu/store/n2w3bdka4zfl3jw0iqk49km1hgl8i7g5-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/utility/startup_config.py'... +Compiling '/gnu/store/n2w3bdka4zfl3jw0iqk49km1hgl8i7g5-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/utility/svg.py'... +*** File "/gnu/store/n2w3bdka4zfl3jw0iqk49km1hgl8i7g5-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/utility/svg.py", line 113 + True = 1 + ^ + SyntaxError: cannot assign to True + +Compiling '/gnu/store/n2w3bdka4zfl3jw0iqk49km1hgl8i7g5-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/utility/temp_data.py'... +Compiling '/gnu/store/n2w3bdka4zfl3jw0iqk49km1hgl8i7g5-genenetwork2-3.11-guix-fe345c9/lib/python3.9/site-packages/wqflask/utility/tools.py'... + +``` + This starts happening (not yet comfirmed) when python is bumped up from 3.8 to 3.9 @@ -40,3 +56,4 @@ In addition, there are some dependencies of genenetwork2 which FTBFS, due to inc phase `sanity-check' failed after 0.1 seconds¬ ``` +This was fixed by reverting the update to python-pingouin, dropping it back to 0.3.12. -- cgit v1.2.3