From 918dcce5f42fcbe9327eb30bac06a69000a1fc8b Mon Sep 17 00:00:00 2001 From: Pjotr Prins Date: Sat, 25 Jul 2020 16:47:24 +0100 Subject: Updating docs and show how to include a local Python module --- README.md | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 47519118..a2a4625a 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,8 @@ developers). See the [installation docs](doc/README.org). ## Run -Once installed GN2 can be run online through a browser interface +Once having installed GN2 it can be run through a browser +interface ```sh genenetwork2 @@ -35,18 +36,12 @@ For full examples (you may need to set a number of environment variables), including running scripts and a Python REPL, also see the startup script [./bin/genenetwork2](https://github.com/genenetwork/genenetwork2/blob/testing/bin/genenetwork2). -Also Mysql and Elasticsearch need to be running, see +Also mariadb and redis need to be running, see [INSTALL](./doc/README.org). ## Testing -We are building 'Mechanical Rob' automated testing using Python -[requests](https://github.com/genenetwork/genenetwork2/tree/master/test/lib) -which can be run with something like - -```sh -env GN2_PROFILE=~/opt/gn-latest ./bin/genenetwork2 ./etc/default_settings.py -c ../test/requests/test-website.py -a http://localhost:5003 -``` +We are building 'Mechanical Rob' automated testing using Python. The GN2_PROFILE is the Guix profile that contains all dependencies. The ./bin/genenetwork2 script sets up the environment @@ -73,10 +68,10 @@ Contribute to GN2 source code by forking the [github repository](https://github.com/genenetwork/genenetwork2/) with git and sending us pull requests. -For development GN2 has a -[mailing list](http://listserv.uthsc.edu/mailman/listinfo/genenetwork-dev) -and an active IRC channel #genenetwork on freenode.net with a -[web interface](http://webchat.freenode.net/). +For development GN2 has a [mailing +list](http://listserv.uthsc.edu/mailman/listinfo/genenetwork-dev) and +an active IRC channel #genenetwork on freenode.net with a [web +interface](http://webchat.freenode.net/). ## License -- cgit v1.2.3 From c1143ffc544abf9c4f12d6676d2584ef1d1cb95e Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Sun, 26 Jul 2020 23:18:54 +0300 Subject: Document how to run tests * README.md: - Add details on how to run unit tests - Update link that points to Mechanical Rob --- README.md | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 47519118..45aca1b2 100644 --- a/README.md +++ b/README.md @@ -40,9 +40,13 @@ Also Mysql and Elasticsearch need to be running, see ## Testing +To have tests pass, the redis and mariadb instance should be running, because of +asserts sprinkled in the code base(these will be removed in due time). + +#### Mechanical Rob We are building 'Mechanical Rob' automated testing using Python -[requests](https://github.com/genenetwork/genenetwork2/tree/master/test/lib) -which can be run with something like +[requests](https://github.com/genenetwork/genenetwork2/tree/testing/test/requests) +which can be run with: ```sh env GN2_PROFILE=~/opt/gn-latest ./bin/genenetwork2 ./etc/default_settings.py -c ../test/requests/test-website.py -a http://localhost:5003 @@ -54,6 +58,32 @@ and executes test-website.py in a Python interpreter. The -a switch says to run all tests and the URL points to the running GN2 http server. +#### Unit tests + +To run unittests, first `cd` into the genenetwork2 directory: + +```sh +# You can use the coverage tool to run the tests +# You could omit the -v which makes the output verbose +runcmd coverage run -m unittest discover -v + +# Alternatively, you could run the unittests using: +runpython -m unittest discover -v + +# To generate a report in wqflask/coverage_html_report/: +runcmd coverage html +``` + +The `runcmd` and `runpython` are shell aliases defined in the following way: + +```sh +alias runpython="env GN2_PROFILE=~/opt/gn-latest TMPDIR=/tmp SERVER_PORT=5004 GENENETWORK_FILES=/gnu/data/gn2_data/ ./bin/genenetwork2 + +alias runcmd="time env GN2_PROFILE=~/opt/gn-latest TMPDIR=//tmp SERVER_PORT=5004 GENENETWORK_FILES=/gnu/data/gn2_data/ ./bin/genenetwork2 ./etc/default_settings.py -cli" +``` + +Replace some of the env variables as per your use case. + ## Documentation User documentation can be found -- cgit v1.2.3 From 9f49e2dbdf4e06fcb39ed5146560937f228682a7 Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Tue, 22 Sep 2020 04:31:05 +0300 Subject: Update documentation on testing and CI * README.md: Update section on testing. * doc/docker-container.org: Add it. --- README.md | 10 +++++- doc/docker-container.org | 86 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 1 deletion(-) create mode 100644 doc/docker-container.org (limited to 'README.md') diff --git a/README.md b/README.md index 46264252..f1decb3c 100644 --- a/README.md +++ b/README.md @@ -42,9 +42,17 @@ Also mariadb and redis need to be running, see ## Testing To have tests pass, the redis and mariadb instance should be running, because of -asserts sprinkled in the code base(these will be removed in due time). +asserts sprinkled in the code base. + +Right now, the only tests running in CI are unittests. Please make +sure the existing unittests are green when submitting a PR. + +See +[./bin/genenetwork2](https://github.com/genenetwork/genenetwork2/blob/testing/doc/docker-container.org) +for more details. #### Mechanical Rob + We are building 'Mechanical Rob' automated testing using Python [requests](https://github.com/genenetwork/genenetwork2/tree/testing/test/requests) which can be run with: diff --git a/doc/docker-container.org b/doc/docker-container.org new file mode 100644 index 00000000..3c9864c5 --- /dev/null +++ b/doc/docker-container.org @@ -0,0 +1,86 @@ +#+TITLE: Genenetwork2 Dockerized + +* Table of Contents :TOC: +- [[#introduction][Introduction]] +- [[#creating-the-docker-images][Creating the Docker Images]] +- [[#pushing-to-dockerhub][Pushing to DockerHub]] + +* Introduction + +The CI(Continuous Integration) system for Genenetwork2 uses [[https://github.com/features/actions][Github +Actions]]. As such, it's important to have a way to run tests using +facilities provided by GUIX in a reproducible way. This project +leverages GUIX to generate a docker container from which the unittests +are ran from. + +Find instructions on how to set docker up inside GUIX [[https://github.com/pjotrp/guix-notes/blob/master/CONTAINERS.org#run-docker][here]]. This +document will not get into that. It's assumed that you have a working +dockec setup. + +The rest of this document outlines how the docker container used in +the CI builds were created. + +* Creating the Docker Images + +First create the image by running: + +#+begin_src sh +# For the Python 2 version: +env GUIX_PACKAGE_PATH="/home/bonface/projects/guix-bioinformatics::/home/bonface/projects/guix-past/modules" \ + ./pre-inst-env guix pack -f docker --no-grafts \ + -S /usr/bin=/bin -S /etc/profile=/etc/profile \ + -S /share/genenetwork2=/share/genenetwork2 \ + -S /share/javascript=/share/javascript \ + -S /lib=/lib \ + -S /usr/gn2-profile=/ \ + coreutils bash genenetwork2 + +# For the Python 3 version: +env GUIX_PACKAGE_PATH="/home/bonface/projects/guix-bioinformatics::/home/bonface/projects/guix-past/modules" \ + ./pre-inst-env guix pack -f docker --no-grafts \ + -S /usr/bin=/bin -S /etc/profile=/etc/profile \ + -S /share/genenetwork2=/share/genenetwork2 \ + -S /share/javascript=/share/javascript \ + -S /lib=/lib \ + -S /usr/gn2-profile=/ \ + coreutils bash python3-genenetwork2 + #+end_src + +The output will look something similar to: + +: /gnu/store/dj1xh19jq1l9vwq24w3nay2954x0wabb-docker-pack.tar.gz + +Load the docker image by running: + +: docker load --input /gnu/store/dj1xh19jq1l9vwq24w3nay2954x0wabb-docker-pack.tar.gz + +Results look something similar to: + +#+begin_export ascii +a93f52b7f565: Loading layer 3.174GB/3.174GB +Loaded image: coreutils-genenetwork2:latest +#+end_export + +Assuming you have a docker instance running, you could always run +commands in it e.g: + +: docker run "coreutils-genenetwork2:latest" python --version + + +* Pushing to DockerHub + +We use DockerHub to store the docker images from which we use on our +CI environment using Github Actions. + +To push to dockerhub, first get the image name by running =docker +images=. Push to dockerhub using a command similar to: + +: docker push bonfacekilz/python2-genenetwork2:latest + +Right now, we have 2 images on DockerHub: + +- https://hub.docker.com/repository/docker/bonfacekilz/python2-genenetwork2: + Contains the python2 version of gn2. Don't use this. Please use the + python3 image! +- https://hub.docker.com/repository/docker/bonfacekilz/python3-genenetwork2: + Contains the python3 version of gn2. -- cgit v1.2.3 From 564b235bdc518a3ed91ca506ad2ac93d5bc91eef Mon Sep 17 00:00:00 2001 From: BonfaceKilz Date: Tue, 22 Sep 2020 04:32:00 +0300 Subject: Add testing badge on README * README.md: Badge becomes green on successful runs. --- README.md | 1 + 1 file changed, 1 insertion(+) (limited to 'README.md') diff --git a/README.md b/README.md index f1decb3c..cd35defb 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ [![DOI](https://zenodo.org/badge/5591/genenetwork/genenetwork2.svg)](https://zenodo.org/badge/latestdoi/5591/genenetwork/genenetwork2) [![JOSS](http://joss.theoj.org/papers/10.21105/joss.00025/status.svg)](http://joss.theoj.org/papers/10.21105/joss.00025) +[![Actions Status](https://github.com/genenetwork/genenetwork2/workflows/tests/badge.svg)](https://github.com/genenetwork/genenetwork2/actions) # GeneNetwork -- cgit v1.2.3 From 4e8a9dfabf3580f1dd2cfeaa407bae726e7659a0 Mon Sep 17 00:00:00 2001 From: robwwilliams Date: Wed, 13 Jan 2021 11:56:32 -0600 Subject: Just some tweaks to first paragrfaph --- README.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index cd35defb..bfb16ddb 100644 --- a/README.md +++ b/README.md @@ -3,13 +3,12 @@ # GeneNetwork -This repository contains the source code for the GeneNetwork (GN) -server http://gn2.genenetwork.org/ (version 2 aka GN2). GN is a Web -2.0 style framework with included tools for doing genetics online -using high-throughput data. GN is used for a wide range of studies. An -exhaustive list of publications mentioning GN and its previous -incarnation WebQTL can be found -[here](http://www.genenetwork.org/reference.html). +This repository contains the current source code for GeneNetwork (GN) +(https://www.genenetwork.org/ (version 2). GN2 is a Web +2.0-style framework that includes data and computational tools for online genetics and genomic analysis of +many different populations and many types of molecular, cellular, and physiological data. +The system is used by scientists and clinians in the field of precision health care and systems genetics. +GN and its predecessors have been in operation since Jan 1994, making it one of the longest-lived web services in biomedical research (https://en.wikipedia.org/wiki/GeneNetwork, and see a partial list of publications using GN and its predecessor, WebQTL (https://genenetwork.org/references/). ## Install -- cgit v1.2.3