diff options
Diffstat (limited to 'issues/genenetwork2')
6 files changed, 242 insertions, 3 deletions
diff --git a/issues/genenetwork2/genenetwork2_configurations.gmi b/issues/genenetwork2/genenetwork2_configurations.gmi new file mode 100644 index 0000000..4ba0a89 --- /dev/null +++ b/issues/genenetwork2/genenetwork2_configurations.gmi @@ -0,0 +1,81 @@ +# Genenetwork2 Configurations + +## Tags + +* assigned: fredm +* priority: normal +* status: closed, obsoleted +* keywords: configuration, config, gn2, genenetwork, genenetwork2 +* type: bug + +## Description + +The current way the configuration in Genenetwork2 works is as follows: + +* The configuration setup begins in `bin/genenetwork` script in the genenetwork2 repository. This script sets up the `GN2_PROFILE` environment variable, and all other environment variables depending on it: see the "`GN2_PROFILE`-Dependent Environment Variables" section below. +* In `wqflask/wqflask/__init__.py` the settings in the file pointed to by `GN2_SETTINGS` environment variable are then loaded. If there is no such file, then it defaults to loading the settings in the `etc/default_settings.py` within the genenetwork2 repository. +* In `wqflask/utility/tools.py` in the genenetwork2 repository, the settings loaded thus far, can then have their values overriden by values set in the environment variables. There is an "OVERRIDES" value that is currently not in use and can be ignored. +* Some other minor configurations are updated/overriden in `wqflask/utility/startup_config.py` + +From these, `GN2_PROFILE` is the only environment variable that needs to be set up. All the other variables can be set in the settings file. + +The file `wqflask/base/webqtlConfig.py` also has some configurations that need to be handled. + +The sub-sections that follow will contain more information on variables and what needs to be done + +### `GN2_PROFILE`-Dependent Environment Variables + +* `PATH`: Exposes the binaries within the profile +* `PYTHONPATH`: Exposes the Python modules within the profile +* `R_LIBS_SITE`: Exposes the R module within the profile +* `JS_GUIX_PATH`: Exposes the Javascript modules within the profile +* `GUIX_GTK3_PATH`: Exposes the GTK modules within the profile +* `GI_TYPELIB_PATH="$GN2_PROFILE/lib/girepository-1.0" +* `XDG_DATA_DIRS`: Exposes the "share" directories within the profile +* `GIO_EXTRA_MODULES`: Expose the GIO modules +* `GUIX_GENENETWORK_FILES`: Expose the Genenetwork shared files +* `PLINK_COMMAND`: Point to the "PLINK" binary +* `GEMMA_COMMAND`: Point to the "GEMMA" binary +* `GEMMA_WRAPPER_COMMAND`: Point to the `GEMMA` wrapper binary + +### `wqflask/utility/tools.py` + +This is our biggest pain-point. The following are some actions that can be taken to reduce the pain in this module: + +* Refactor functions in to take the "application object" as one of their arguments, to decouple the module from the module where the application is declared +* Remove all settings from this module, instead, override any of the variables that need it at the application creation step, possibly within the `wqflask/wqflask/__init__.py` module. +* Replace all uses of variables from this module with configurations from `flask.current_app.config` as appropriate +* Get rid of this module entirely if possible, moving the functions elsewhere. + +### Modules with Routes but no Blueprints + +These modules are: + +* `wqflask/run_gunicorn.py` +* `wqflask/wqflask/user_session.py` +* `wqflask/wqflask/collect.py` +* `wqflask/wqflask/views.py` +* `wqflask/wqflask/partial_correlations_views.py` +* `wqflask/wqflask/user_login.py` +* `wqflask/wqflask/api/router.py` +* `wqflask/base/trait.py` + +Except for `wqflask/run_gunicorn.py`, each module will have its own Blueprint and then we'll have some top-level Blueprint that import all the others and sets them up. The top-level blueprint will then be registered with the app at the application-creation time. + +For `wqflask/run_gunicorn.py`, the route can remain as is, since this is an entry-point module. + +### `wqflask/wqflask/__init__.py` + +* Move application initialisation into an application factory function +* Replace all uses of `wqflask.app` variable with `flask.current_app` or `create_app()` as necessary + +### Non-Executable Configuration Files + +Eschew executable formats (*.py) for configuration files and prefer non-executable formats e.g. *.cfg, *.json, *.conf etc + + +## Closed as obsoleted + +I am closing this issue as obsoleted, since a lot of things have changed since this issue was set up. The `bin/genenetwork2` script no longer exists and most of the paths mentioned have changed. + +The configuration issue(s) mentioned above still abound, but the changes will have to be incremental to avoid breaking the system. diff --git a/issues/genenetwork2/handle-oauth-errors-better.gmi b/issues/genenetwork2/handle-oauth-errors-better.gmi index 462ded5..77ad7ad 100644 --- a/issues/genenetwork2/handle-oauth-errors-better.gmi +++ b/issues/genenetwork2/handle-oauth-errors-better.gmi @@ -3,7 +3,7 @@ ## Tags * type: bug -* status: open +* status: closed, completed * priority: high * assigned: fredm * interested: zachs, robw @@ -15,3 +15,7 @@ When a session expires, for whatever reason, a notification is displayed to the => ./session_expiry_oauth_error.png The message is a little jarring to the end user. Make it gentler, and probably more informative, so the user is not as surprised. + +## Close as complete + +This should be fixed at this point. Closing this as complete. diff --git a/issues/genenetwork2/mapping-error.gmi b/issues/genenetwork2/mapping-error.gmi index 2e28491..7e7d0a7 100644 --- a/issues/genenetwork2/mapping-error.gmi +++ b/issues/genenetwork2/mapping-error.gmi @@ -3,7 +3,7 @@ ## Tags * type: bug -* status: open +* status: closed * priority: medium * assigned: zachs, fredm, flisso * keywords: gn2, genenetwork2, genenetwork 2, mapping @@ -49,3 +49,18 @@ TypeError: 'NoneType' object is not iterable ### Updates This is likely just because the genotype file doesn't exist in the necessary format (BIMBAM). We probably need to convert the R/qtl2 genotypes to BIMBAM. + +## Stalled + +This is currently stalled, until we can upload genotypes via the uploader. + + +## Notes + +### 2025-12-31 + +I am closing this issue as WONTFIX because of the following reasons: + +- Better fix is to prevent mapping in the first place, if no genotypes exist for the given trait(s) +- Issue relies on non-implemented feature (Genotypes upload) to fix it +- Issue does not exist on production diff --git a/issues/genenetwork2/mechanical-rob-add-partial-correlations-tests.gmi b/issues/genenetwork2/mechanical-rob-add-partial-correlations-tests.gmi new file mode 100644 index 0000000..e38f653 --- /dev/null +++ b/issues/genenetwork2/mechanical-rob-add-partial-correlations-tests.gmi @@ -0,0 +1,22 @@ +# mechanical-rob: Add Partial Correlations Tests + +## Tags + +* assigned: fredm +* priority: medium +* status: open +* keywords: genenetwork2, gn2, mechanical-rob, partial correlations, tests, regression +* type: enhancement + +## Description + +Add regression tests to verify that the partial correlations feature still works +as expected. + +### TODOS + +- [-] Tests for "entry-point" page +- [x] Tests for partial correlation using Pearson's R against select traits +- [ ] Tests for partial correlation using Spearman's Rho against select traits +- [ ] Tests for partial correlation using Pearson's R against an entire dataset +- [ ] Tests for partial correlation using Spearman's Rho against an entire dataset diff --git a/issues/genenetwork2/refresh-token-failure.gmi b/issues/genenetwork2/refresh-token-failure.gmi index dd33341..c488820 100644 --- a/issues/genenetwork2/refresh-token-failure.gmi +++ b/issues/genenetwork2/refresh-token-failure.gmi @@ -2,7 +2,7 @@ ## Tags -* status: open +* status: closed, obsoleted * priority: high * type: bug * assigned: fredm, zsloan, zachs @@ -106,3 +106,6 @@ The following commits were done as part of the troubleshooting: => https://git.genenetwork.org/guix-bioinformatics/commit/?id=955e4ce9370be9811262d7c73fa5398385cc04d8 +# Closed as Obsolete + +We no longer rely on refresh tokens. This issue is no longer present. diff --git a/issues/genenetwork2/remove-bin-genenetwork2-script.gmi b/issues/genenetwork2/remove-bin-genenetwork2-script.gmi new file mode 100644 index 0000000..da11be7 --- /dev/null +++ b/issues/genenetwork2/remove-bin-genenetwork2-script.gmi @@ -0,0 +1,114 @@ +# Remove `bin/genenetwork2` Script + +## Tags + +* type: improvement +* status: closed, completed +* priority: medium +* assigned: fredm, bonfacem, alexm, zachs +* interested: pjotrp, aruni +* keywords: gn2, bin/genenetwork2, startup script + +## Description + +The `bin/genenetwork2` script was used for a really long time to launch Genenetwork2, and has served that purpose with honour and dedication. We applaud that. + +It is, however, time to retire the script, since at this point in time, it serves more to obfuscate the startup that as a helpful tool. + +On production, we have all but abandoned the use of the script, and we need to do the same for CI/CD, and eventually, development. + +This issue tracks the process, and problems that come up during the move to retire the script. + +### Process + +* [x] Identify how to run unit tests without the script +* [x] Document how to run unit tests without the script +* [x] Identify how to run mechanical-rob tests without the script +* [x] Document how to run mechanical-rob tests without the script +* [x] Update CI/CD definitions to get rid of the references to the script +* [x] Delete the script from the repository + +### Setup + +First, we need to setup the following mandatory environment variables: + +* GN2_PROFILE +* GN2_SETTINGS +* JS_GUIX_PATH +* GEMMA_COMMAND +* PLINK_COMMAND +* GEMMA_WRAPPER_COMMAND +* REQUESTS_CA_BUNDLE + +Within a guix shell, you could do that with something like: + +``` +export GN2_PROFILE="${GUIX_ENVIRONMENT}" +export GN2_SETTINGS="/home/frederick/genenetwork/gn2_settings.conf" +export JS_GUIX_PATH="${GN2_PROFILE}/share/genenetwork2/javascript" +export GEMMA_COMMAND="${GN2_PROFILE}/bin/gemma" +export PLINK_COMMAND="${GN2_PROFILE}/bin/plink2" +export GEMMA_WRAPPER_COMMAND="${GN2_PROFILE}/bin/gemma-wrapper" +export REQUESTS_CA_BUNDLE="${GUIX_ENVIRONMENT}/etc/ssl/certs/ca-certificates.crt" +``` + +Note that, you can define all the variables derived from "GN2_PROFILE" in your settings file, if such a settings file is computed. + +### Running Unit Tests + +To run unit tests, run pytest at the root of the repository. + +``` +$ cd /path/to/genenetwork2 +$ pytest +``` + +### Running "mechanical-rob" Tests + +At the root of the repository, run something like: + +``` +python test/requests/test-website.py --all http://localhost:5033 +``` + +Change the port, as appropriate. + + +### Launching Application + +In addition to the minimum set of envvars defined in the "Setup" section above, we need the following variables defined to get the application to launch: + +* FLASK_APP + +In a guix shell, you could do: + +``` +export FLASK_APP="gn2.wsgi" +``` + +Now you can launch the application with flask with something like: + +``` +flask run --port=5033 --with-threads +``` + +or with green unicorn with something like: + +``` +gunicorn --reload \ + --workers 3 \ + --timeout 1200 \ + --log-level="debug" \ + --keep-alive 6000 \ + --max-requests 10 \ + --bind="127.0.0.1:5033" \ + --max-requests-jitter 5 \ + gn2.wsgi:application +``` + +You can change the gunicorn setting to fit your scenario. + + +## Close as completed + +The script has been deleted. |