From 2a33ac494aecbe43b54f79afefae7b7d7c362d6c Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sun, 17 Feb 2019 10:47:36 +0000 Subject: RELEASE-NOTES --- RELEASE-NOTES.md | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md index d15cad1a..9ad75aa0 100644 --- a/RELEASE-NOTES.md +++ b/RELEASE-NOTES.md @@ -1,4 +1,4 @@ -## ChangeLog v2.11 (date unknown) +## ChangeLog v2.11 pre-release (date unknown) This is a massive bug fix release with many improvements. For contributions see @@ -6,12 +6,37 @@ see and [commits](https://github.com/genenetwork/genenetwork2/commits/master). +The current GNU Guix checkout is at + +https://gitlab.com/genenetwork/guix-bioinformatics/tree/gn-latest-20190213 + +and + +https://gitlab.com/genenetwork/guix/tree/gn-latest-20190213 + +With these branches a binary install can be fetched with + + env GUIX_PACKAGE_PATH=../guix-bioinformatics ./pre-inst-env guix package --substitute-urls="https://berlin.guixsd.org http://guix.genenetwork.org https://mirror.hydra.gnu.org" -i genenetwork2 -p ~/opt/gn-latest-20190213 + +The key for guix.genenetwork.org is + +```scheme +(public-key + (ecc + (curve Ed25519) + (q #D54AA5C8CBE268CBC82418AB83709611BAC88B39D44E68C18F6E16197B5B5CA0#) + ) + ) +``` + +### Migrated PIL to pillow (preparing for Python3 migration) + +### Dropped pylmm support + ### Added GEMMA support -* Front-end support +* Front-end support ### Added test framework and unit tests * Added python integration and unit tests - - -- cgit v1.2.3 From 496759ad08efb02b4268ea7f3bbb7905974237e9 Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Mon, 18 Feb 2019 09:39:41 +0000 Subject: Updated installation instructions and SERVER_PORT for single flask server --- doc/README.org | 71 ++++++++++++++++++++++++++++++++++-------------- wqflask/runserver.py | 8 ++++-- wqflask/utility/tools.py | 1 + 3 files changed, 57 insertions(+), 23 deletions(-) diff --git a/doc/README.org b/doc/README.org index 5dc9e994..620c946c 100644 --- a/doc/README.org +++ b/doc/README.org @@ -12,13 +12,15 @@ - [[#load-the-small-database-in-mysql][Load the small database in MySQL]] - [[#gn2-dependency-graph][GN2 Dependency Graph]] - [[#working-with-the-gn2-source-code][Working with the GN2 source code]] + - [[#running-elasticsearch][Running ElasticSearch]] + - [[#systemd][SystemD]] + - [[#read-more][Read more]] - [[#trouble-shooting][Trouble shooting]] - [[#importerror-no-module-named-jinja2][ImportError: No module named jinja2]] - - [[#error-can-not-find-directory-homegn2_data][ERROR: can not find directory $HOME/gn2_data]] + - [[#error-can-not-find-directory-homegn2_data-or-can-not-find-directory-homegenotype_filesgenotype][ERROR: 'can not find directory $HOME/gn2_data' or 'can not find directory $HOME/genotype_files/genotype']] - [[#cant-run-a-module][Can't run a module]] - [[#rpy2-error-show-now-found][Rpy2 error 'show' now found]] - [[#mysql-cant-connect-server-through-socket-error][Mysql can't connect server through socket ERROR]] - - [[#read-more][Read more]] - [[#irc-session][IRC session]] * Introduction @@ -100,11 +102,13 @@ mysql (which comes as part of the GNU Guix genenetwork2 install). As root configure and run -: adduser mysql && addgroup mysql -: mysqld --datadir=/var/mysql --initialize-insecure -: mkdir -p /var/run/mysqld -: chown mysql.mysql ~/mysql /var/run/mysqld -: mysqld -u mysql --datadir=/var/mysql --explicit_defaults_for_timestamp -P 12048" +#+BEGIN_SRC bash +adduser mysql && addgroup mysql +mysqld --datadir=/var/mysql --initialize-insecure +mkdir -p /var/run/mysqld +chown mysql.mysql ~/mysql /var/run/mysqld +mysqld -u mysql --datadir=/var/mysql --explicit_defaults_for_timestamp -P 12048" +#+END_SRC If you want to run as root you may have to set @@ -192,6 +196,41 @@ http://biogems.info/contrib/genenetwork/gn2.svg See [[development.org]]. +* Running ElasticSearch + +In order to start up elasticsearch: +Penguin - change user to "elasticsearch" and use the following command: "env JAVA_HOME=/opt/jdk-9.0.4 /opt/elasticsearch-6.2.1/bin/elasticsearch" + + +** SystemD + +New server - as root run "systemctl restart elasticsearch" + +#+BEGIN_SRC +tux01:/etc/systemd/system# cat elasticsearch.service +[Unit] +Description=Run Elasticsearch + +[Service] +ExecStart=/opt/elasticsearch-6.2.1/bin/elasticsearch +Environment=JAVA_HOME=/opt/jdk-9.0.4 +Environment="ES_JAVA_OPTS=-Xms1g -Xmx8g" +Environment="PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/jdk-9.0.4/bin" +LimitNOFILE=65536 +StandardOutput=syslog +StandardError=syslog +User=elasticsearch + +[Install] +WantedBy=multi-user.target +#+END_SRC + +* Read more + +If you want to understand the architecture of GN2 read +[[Architecture.org]]. The rest of this document is mostly on deployment +of GN2. + * Trouble shooting ** ImportError: No module named jinja2 @@ -210,13 +249,17 @@ On one system: : export GEM_PATH="$HOME/.guix-profile/lib/ruby/gems/2.2.0" and perhaps a few more. -** ERROR: can not find directory $HOME/gn2_data +** ERROR: 'can not find directory $HOME/gn2_data' or 'can not find directory $HOME/genotype_files/genotype' The default settings file looks in your $HOME/gn2_data. Since these files come with a Guix installation you should take a hint from the values in the installed version of default_settings.py (see above in this document). +You can use the GENENETWORK_FILES switch to set the datadir, for example + +: env GN2_PROFILE=~/opt/gn-latest GENENETWORK_FILES=/gnu/data/gn2_data ./bin/genenetwork2 + ** Can't run a module In rare cases, development modules are not brought in with Guix @@ -257,18 +300,6 @@ if that works run genenetwork after setting SQL_URI to something like : export SQL_URI=mysql://gn2:mysql_password@127.0.0.1/db_webqtl_s -* Running ElasticSearch - -In order to start up elasticsearch: -Penguin - change user to "elasticsearch" and use the following command: "env JAVA_HOME=/opt/jdk-9.0.4 /opt/elasticsearch-6.2.1/bin/elasticsearch" - -New server - as root run "systemctl restart elasticsearch" - -* Read more - -If you want to understand the architecture of GN2 read -[[Architecture.org]]. The rest of this document is mostly on deployment -of GN2. * IRC session diff --git a/wqflask/runserver.py b/wqflask/runserver.py index 5f41d04d..7c06356b 100644 --- a/wqflask/runserver.py +++ b/wqflask/runserver.py @@ -27,9 +27,11 @@ app_config() werkzeug_logger = logging.getLogger('werkzeug') +from utility.tools import WEBSERVER_MODE, SERVER_PORT + if WEBSERVER_MODE == 'DEBUG': app.run(host='0.0.0.0', - port=port, + port=SERVER_PORT, debug=True, use_debugger=False, threaded=False, @@ -38,7 +40,7 @@ if WEBSERVER_MODE == 'DEBUG': elif WEBSERVER_MODE == 'DEV': werkzeug_logger.setLevel(logging.WARNING) app.run(host='0.0.0.0', - port=port, + port=SERVER_PORT, debug=False, use_debugger=False, threaded=False, @@ -46,7 +48,7 @@ elif WEBSERVER_MODE == 'DEV': use_reloader=True) else: # staging/production modes app.run(host='0.0.0.0', - port=port, + port=SERVER_PORT, debug=False, use_debugger=False, threaded=True, diff --git a/wqflask/utility/tools.py b/wqflask/utility/tools.py index 86ef2e1e..8b2260f5 100644 --- a/wqflask/utility/tools.py +++ b/wqflask/utility/tools.py @@ -232,6 +232,7 @@ GN_VERSION = get_setting('GN_VERSION') HOME = get_setting('HOME') WEBSERVER_MODE = get_setting('WEBSERVER_MODE') GN_SERVER_URL = get_setting('GN_SERVER_URL') +SERVER_PORT = get_setting_int('SERVER_PORT') SQL_URI = get_setting('SQL_URI') LOG_LEVEL = get_setting('LOG_LEVEL') LOG_LEVEL_DEBUG = get_setting_int('LOG_LEVEL_DEBUG') -- cgit v1.2.3 From 96e4efe928773b292f3d3b9b7ef3c23e5ec003fc Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Mon, 18 Feb 2019 11:28:41 +0000 Subject: Note on running tests --- README.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 59645994..c9582d0f 100644 --- a/README.md +++ b/README.md @@ -27,9 +27,18 @@ genenetwork2 (default is http://localhost:5003/). For full examples (you'll need to set a number of environment variables), including running scripts and -a Python REPL, see the startup script +a Python REPL, for example + +```sh +env GN2_PROFILE=~/opt/gn-latest GENENETWORK_FILES=~/data/gn2_data ./bin/genenetrk2 +``` + +see the startup script [./bin/genenetwork2](https://github.com/genenetwork/genenetwork2/blob/testing/bin/genenetwork2). +Also Mysql and Elasticsearch need to be running, see +[INSTALL](./doc/README.org). + ## Testing We are building 'Mechanical Rob' automated testing using Python -- cgit v1.2.3 From 6167e78f7c37b26f5226936e8eff0fe31be8691e Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Mon, 18 Feb 2019 11:32:51 +0000 Subject: typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c9582d0f..523eeb76 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ set a number of environment variables), including running scripts and a Python REPL, for example ```sh -env GN2_PROFILE=~/opt/gn-latest GENENETWORK_FILES=~/data/gn2_data ./bin/genenetrk2 +env GN2_PROFILE=~/opt/gn-latest GENENETWORK_FILES=~/data/gn2_data ./bin/genenetwork2 ``` see the startup script -- cgit v1.2.3