Merge branch 'main' of https://github.com/genenetwork/genenetwork3 into bug/fix_rqtl_covariates

1 files changed, 81 insertions, 8 deletions

diff --git a/README.md b/README.md
index c3a9848..84a7a54 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,17 @@ GeneNetwork3 REST API for data science and machine  learning
 
 ## Installation
 
-#### Using guix
+#### GNU Guix packages
+
+Install GNU Guix - this can be done on every running Linux system.
+
+There are at least three ways to start GeneNetwork3 with GNU Guix:
+
+1. Create an environment with `guix environment`
+2. Create a container with `guix environment -C`
+3. Use a profile and shell settings with `source ~/opt/genenetwork3/etc/profile`
+
+#### Create an environment:
 
 Simply load up the environment (for development purposes):
 
@@ -14,17 +24,44 @@ guix environment --load=guix.scm
 Also, make sure you have the [guix-bioinformatics](https://git.genenetwork.org/guix-bioinformatics/guix-bioinformatics) channel set up.
 
 ```bash
-env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/.config/guix/current/bin/guix environment --load=guix.scm
+env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/.config/guix/current/bin/guix environment --expose=$HOME/genotype_files/ --load=guix.scm
 python3
   import redis
 ```
 
-Better run a proper container
+#### Run a Guix container
+
+```
+env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/.config/guix/current/bin/guix environment -C --network --expose=$HOME/genotype_files/ --load=guix.scm
+```
+
+
+#### Using a Guix profile (or rolling back)
+
+Create a new profile with
+
+```
+env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/.config/guix/current/bin/guix package -i genenetwork3 -p ~/opt/genenetwork3
+```
+
+and load the profile settings with
+
+```
+source ~/opt/genenetwork3/etc/profile
+start server...
+```
+
+Note that GN2 profiles include the GN3 profile (!). To roll genenetwork3 back you can use either in the same fashion (probably best to start a new shell first)
 
 ```
-env GUIX_PACKAGE_PATH=~/guix-bioinformatics/ ~/.config/guix/current/bin/guix environment -C --network --load=guix.scm
+bash
+source ~/opt/genenetwork2-older-version/etc/profile
+set|grep store
+run tests, server etc...
 ```
 
+#### Troubleshooting Guix packages
+
 If you get a Guix error, such as `ice-9/boot-9.scm:1669:16: In procedure raise-exception:
 error: python-sqlalchemy-stubs: unbound variable` it typically means an update to guix latest is required (i.e., guix pull):
 
@@ -33,11 +70,11 @@ guix pull
 source ~/.config/guix/current/etc/profile
 ```
 
-and try again.
+and try again. Also make sure your ~/guix-bioinformatics is up to date.
 
 See also instructions in [.guix.scm](.guix.scm).
 
-#### Running Tests
+## Running Tests
 
 (assuming you are in a guix container; otherwise use venv!)
 
@@ -59,7 +96,7 @@ Running mypy(type-checker):
 mypy .
 ```
 
-#### Running the flask app
+## Running the GN3 web service
 
 To spin up the server on its own (for development):
 
@@ -88,7 +125,7 @@ And for the scalable production version run
 gunicorn --bind 0.0.0.0:8080 --workers 8 --keep-alive 6000 --max-requests 10 --max-requests-jitter 5 --timeout 1200 wsgi:app
 ```
 
-##### Using python-pip
+## Using python-pip
 
 IMPORTANT NOTE: we do not recommend using pip tools, use Guix instead
 
@@ -120,3 +157,39 @@ guix. To freeze dependencies:
 pip freeze --path venv/lib/python3.8/site-packages > requirements.txt
 
 ```
+
+## Genotype Files
+
+You can get the genotype files from http://ipfs.genenetwork.org/ipfs/QmXQy3DAUWJuYxubLHLkPMNCEVq1oV7844xWG2d1GSPFPL and save them on your host machine at, say `$HOME/genotype_files` with something like:
+
+```bash
+$ mkdir -p $HOME/genotype_files
+$ cd $HOME/genotype_files
+$ yes | 7z x genotype_files.tar.7z
+$ tar xf genotype_files.tar
+```
+
+The `genotype_files.tar.7z` file seems to only contain the **BXD.geno** genotype file.
+
+## QTLReaper (rust-qtlreaper) and Trait Files
+
+To run QTL computations, this system makes use of the [rust-qtlreaper](https://github.com/chfi/rust-qtlreaper.git) utility.
+
+To do this, the system needs to export the trait data into a tab-separated file, that can then be passed to the utility using the `--traits` option. For more information about the available options, please [see the rust-qtlreaper](https://github.com/chfi/rust-qtlreaper.git) repository.
+
+### Traits File Format
+
+The traits file begins with a header row/line with the column headers. The first column in the file has the header **"Trait"**. Every other column has a header for one of the strains in consideration.
+
+Under the **"Trait"** column, the traits are numbered from **T1** to **T<n>** where **<n>** is the count of the total number of traits in consideration.
+
+As an example, you could end up with a trait file like the following:
+
+```txt
+Trait	BXD27	BXD32	DBA/2J	BXD21	...
+T1	10.5735	9.27408	9.48255	9.18253	...
+T2	6.4471	6.7191	5.98015	6.68051	...
+...
+```
+
+It is very important that the column header names for the strains correspond to the genotype file used.