* GeneNetwork Architecture

#+TITLE: Installing GeneNetwork services

* Table of Contents                                                     :TOC:
 - [[#genenetwork-architecture][GeneNetwork Architecture]]
   - [[#introduction][Introduction]]
   - [[#webserver][Webserver]]
   - [[#gnserver-rest][GnServer (REST)]]

** Introduction

This document describes the architecture of GN2. Because GN2 is
evolving, only a high-level overview is given here.

** Webserver

The main [[https://github.com/genenetwork/genenetwork2][GN2 webserver]] is built on [[http://flask.pocoo.org/][Python flask]] and this GN2 source
code can be found on [[https://github.com/genenetwork/genenetwork2/tree/master/wqflask/wqflask][github]] in the wqflask directory. The routing
tables are defined in [[https://github.com/genenetwork/genenetwork2/blob/master/wqflask/wqflask/views.py][views.py]]. For example the main page is loaded
from a template named [[https://github.com/genenetwork/genenetwork2/blob/master/wqflask/wqflask/templates/index_page.htm][index_page.html]] in the [[https://github.com/genenetwork/genenetwork2/tree/master/wqflask/wqflask/templates][templates]] directory. In
the template you can find get the form gets filled by a Javascript
routine defined in [[https://github.com/genenetwork/genenetwork2/blob/master/wqflask/wqflask/static/new/javascript/dataset_select_menu.js][data_select_menu.js]] which picks up a static JSON
file for the menu. This static file is generated with
[[https://github.com/genenetwork/genenetwork2/blob/master/wqflask/maintenance/gen_select_dataset.py][gen_select_dataset.py]].  Note that this JSON data is served by
gn_server in the latest version, see [[#gnserver-rest][GnServer (REST)]].

When you hit a search with, for example,
'http://localhost:5003/search?species=mouse&group=BXD&type=Hippocampus+mRNA&dataset=HC_M2_0606_P&search_terms_or=&search_terms_and=MEAN%3D%2815+16%29+LRS%3D%2823+46%29+&FormID=searchResult'
it has the menu items as parameters. According to the routing table,
the search is executed and Redis caching is used (we'll probably
change that to the level of the gn_server). The logic is in
search_result.py which invokes database functions in
wqflask/dbFunction/webqtlDatabaseFunction.py, for example. The
receiving template lives at [[https://github.com/genenetwork/genenetwork2/blob/master/wqflask/wqflask/templates/search_result_page.html][search_result_page.html]].

For what happens at the database level see [[database.org]].

A view consists of an HTML template with JS libraries for managing
menus, tables etc. For example, for the search results see the
[[https://github.com/genenetwork/genenetwork2/blob/master/wqflask/wqflask/templates/search_result_page.html][search_result_page.html]] which is a Flask template. The first section
puts the search in plain English, e.g. 'We searched Hippocampus
Consortium M430v2 (Jun06) PDNN to find all records with MEAN between
15 and 16 and with LRS between 23 and 46.'. Then the results are added
to a table which is displayed using a JS [[https://datatables.net/][DataTable container]].

** GnServer (REST)

The [[https://github.com/genenetwork/gn_server][GnServer REST API]] is built on high performance [[http://elixir-lang.org/][Elixir]] with [[https://github.com/falood/maru][Maru]].
Mainly the GnServer serves JSON requests, for example to fetch data
from the database. To get the menu data in YAML you can do something like

: curl localhost:8880/int/menu/main.json|ruby extra/json2yaml.rb

(json2yaml.rb is in the gn_server repo).