aboutsummaryrefslogtreecommitdiff
path: root/gn3
AgeCommit message (Expand)Author
2021-09-20Enable Cross-Origin Resource Sharing...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/api/heatmaps.py: Fix bugs in data parsing * gn3/app.py: enable CORS * gn3/settings.py: add flask-cors configurations * guix.scm: Add flask-cors dependency For easier testing of the heatmaps generation feature, this commit activates the cross-origin resource sharing for all "localhost" origins. Frederick Muriuki Muriithi
2021-09-20Return only the data...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/api/heatmaps.py: Parse incoming data to build up correct trait names and respond with only the computed heatmap data. * gn3/heatmaps.py: Return only the computed data for heatmaps and clustering. Since GN3 is supposed to handle only the data, and db-access, this commit ensures that GN3 responds to the client with only the computed heatmap data, and does not try to generate the heatmaps themselves. The generation of the heatmaps will be delegated to the UI clients, such as GeneNetwork2. Frederick Muriuki Muriithi
2021-09-17Fix a number of linting issues...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi Frederick Muriuki Muriithi
2021-09-17Return path to generated filename for now...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * To help with demonstrating that the code is producing the expected output, for now, we return the path to the generated html file that displays the interactive heatmap. At this point, it is mostly useful in the development environment. Moving forward, we might have to actually stream the raw html, or if we can get the Kaleido library packaged for GNU Guix, stream the images binary data instead. Frederick Muriuki Muriithi
2021-09-17Fix some layout issues and update colorscale...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Update the plot layouts and size to display the dendrogram and individual chromosome heatmaps side by side * Update the colour scale to begin with the grays rather than absolute black Frederick Muriuki Muriithi
2021-09-17Create dendrogram to show clustering tree...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Provide the clustering data to be used for the creation of the clustering dendrogram in the final clustered heatmap plot. Frederick Muriuki Muriithi
2021-09-16Add WGCNA_SCRIT env to settingsAlexander Kabui
2021-09-16pass user input to call scriptAlexander Kabui
2021-09-16Intergrate the heatmap generation with the API...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Intergrate the heatmap generation code on the /api/heatmaps/clustered endpoint. The endpoint should take a json query of the form: {"traits_names": [ ... ] } where the "traits_name" value is a list of the full names of traits. A sample query to the endpoint could be something like the following: curl -i -X POST "http://localhost:8080/api/heatmaps/clustered" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -d '{ "traits_names": [ "UCLA_BXDBXH_CARTILAGE_V2::ILM103710672", "UCLA_BXDBXH_CARTILAGE_V2::ILM2260338", "UCLA_BXDBXH_CARTILAGE_V2::ILM3140576", "UCLA_BXDBXH_CARTILAGE_V2::ILM5670577", "UCLA_BXDBXH_CARTILAGE_V2::ILM2070121", "UCLA_BXDBXH_CARTILAGE_V2::ILM103990541", "UCLA_BXDBXH_CARTILAGE_V2::ILM1190722", "UCLA_BXDBXH_CARTILAGE_V2::ILM6590722", "UCLA_BXDBXH_CARTILAGE_V2::ILM4200064", "UCLA_BXDBXH_CARTILAGE_V2::ILM3140463" ] }' which should respond with a json response containing the raw binary string for the png format and possibly another for the svg format. Frederick Muriuki Muriithi
2021-09-16Fix minor bugs...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Fix a few minor bugs left over from the integration of code from the proof-of-concept code. Frederick Muriuki Muriithi
2021-09-16Add missing imports...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add missing imports that are needed in the code. Frederick Muriuki Muriithi
2021-09-16run cmd and add exception handlerAlexander Kabui
2021-09-16register wgcna blueprintAlexander Kabui
2021-09-16add initial endpoint for wgcnaAlexander Kabui
2021-09-16add function to compose and run wgcna scriptAlexander Kabui
2021-09-16function to parse form data and write to json fileAlexander Kabui
2021-09-15init wgcna file to run r script and preprocess dataAlexander Kabui
2021-09-15Update entry-point function for heatmap generation...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Copy over code from the proof-of-concept implementation and clean it up a little for the entry-point function for heatmap generation via the API Frederick Muriuki Muriithi
2021-09-15Generate heatmaps in a single plot...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Add a function to generate the heatmaps for each chromosome into a single plot. Frederick Muriuki Muriithi
2021-09-15Process data into format usable by heatmaps...* gn3/heatmaps.py: implement `process_traits_data_for_heatmap` function, that will process the data into a form usable by heatmaps. * tests/unit/test_heatmaps.py: check that the function processes the data into the correct form. Frederick Muriuki Muriithi
2021-09-15Integrate get_lsr_from_chr function...* gn3/heatmaps.py: copy over function * tests/unit/test_heatmaps.py: add tests Copy function over from proof of concept and add some tests to ensure it works as expected. Frederick Muriuki Muriithi
2021-09-15Reorganise modules...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * The heatmap generation does not fall cleanly within the computations or db modules. This commit moves it to the higher level gn3 module. Frederick Muriuki Muriithi
2021-09-08Ease search for traits and chromosomes...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Return a dict of values rather than list for the traits and chromosomes to ease searching through the data. Muriithi Frederick Muriuki
2021-09-08Parse Chr value as int where possible...* To ease sorting of data by numerical order down the line, sort the "Chr" values by numerical order. Muriithi Frederick Muriuki
2021-09-08Remove extraneous text to ease sorting...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Change the id from 'T<n>' to simply '<n>' to ease sorting of the trait results by numerical order rather than string order. Muriithi Frederick Muriuki
2021-09-08Fix the traits order computations for clustering...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/heatmap.py: Fix ordering function * tests/unit/computations/test_heatmap.py: update test The order of the traits is important for the clustering algorithm, since the clustering seems to use the distance of one trait from another to determine how to order them. This commit also gets rid of the xoffset argument that is not important to the ordering, and was used in the older GN1 to determine how to draw the clustering lines. Muriithi Frederick Muriuki
2021-09-06Provide function to organise parsed QTLReaper results...* gn3/computations/qtlreaper.py: Provide a function to organise the results by trait for easier use down the line. * tests/unit/computations/test_qtlreaper.py: provide a test to ensure that the organising function works as expected. Muriithi Frederick Muriuki
2021-09-06Leave "Chr" value as string when parsing...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * The "Chr" value seems to be mostly a name of some sort, despite it being, seemingly an number. This commit parses the "Chr" value as a string. It also updates the tests to expec a string, rather than a number for "Chr" values. Muriithi Frederick Muriuki
2021-09-06Handle type-coercion exceptions...* gn3/computations/qtlreaper.py: handle exceptions Sometimes, the values being parsed are plain strings and cannot be cast to the float types. This commit handles that by casting only those values that can be cast to float, and returning the others as strings. Muriithi Frederick Muriuki
2021-09-06Find nearest marker...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Migrate the `web.webqtl.heatmap.Heatmap.getNearestMarker` function in GN1 to GN3. Muriithi Frederick Muriuki
2021-09-03Add covarstruct as a possible keyword for the rqtl_wrapper.R script (not inte...zsloan
2021-09-01Fix linting and typing issues...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi Muriithi Frederick Muriuki
2021-09-01Built top-level genotype file parsing function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: parse genotype files * tests/unit/db/test_genotypes.py: test parsing is correct Add the overall genotype files parsing function and tests to check that the parsing works as expected. Muriithi Frederick Muriuki
2021-09-01Parse data lines into markers...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: parse data lines in file to genetic markers. * tests/unit/db/test_genotypes.py: test that parsing works. Add some tests to check that the parsing of the markers works as expected, and add the code to actually parse the markers. Muriithi Frederick Muriuki
2021-09-01Parse the genotype file's data header...* gn3/db/genotypes.py: parse data header * tests/unit/db/test_genotypes.py: check that header's parse works correctly. Add tests to check that the parser works as expected. Add code to implement the parsing and pass the tests. Muriithi Frederick Muriuki
2021-09-01Implement parsing of genotype labels...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: parse genotype labels * tests/unit/db/test_genotypes.py: test that genotype labels are parsed correctly As part of parsing the genotype files into usable python data structures, this commit adds a function to parse the label lines (beginning with "@") into the appropriate values. Muriithi Frederick Muriuki
2021-08-31Fix linting errors, minor bugs and reorganise code...* Fix some linting errors and some minor bugs caught by the linter. Move the `random_string` function to separate module for use in multiple places in the code. Muriithi Frederick Muriuki
2021-08-31Update `heatmap_data` function: remove extraneous data...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/heatmap.py: update function * gn3/db/traits.py: new function Remove extraneous data and arguments from the function. - Load the genotype file - Generate traits file - Provide both raw traits data, and exported traits data in return Muriithi Frederick Muriuki
2021-08-31Parse QTLReaper outputs...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/qtlreaper.py: pass output files * tests/unit/computations/data/qtlreaper/main_output_sample.txt: sample test data * tests/unit/computations/data/qtlreaper/permu_output_sample.txt: sample test data * tests/unit/computations/test_qtlreaper.py: add tests Add code to parse the QTLReaper output data files. Muriithi Frederick Muriuki
2021-08-31Fix bugs with `run_reaper` function...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/qtlreaper.py: Fix some bugs * qtlfilesexport.py: Test out running rust-qtlreaper Test out the qtlreaper interface code and fix some bugs caught in the process. Muriithi Frederick Muriuki
2021-08-31Provide utilities for genotype files...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/db/genotypes.py: New module * gn3/settings.py: Add new configuration variable * qtlfilesexport.py: Test out new code Add a module containing functions dealing with the genotype files. Add a configuration variable to point to the location of the genotype files. Muriithi Frederick Muriuki
2021-08-30Fix issues with traits file format...* README.md: update header: Traits ==> Trait * gn3/computations/qtlreaper.py: update header: Traits ==> Trait * qtlfilesexport.py: Choose only BXD strains Rename the first column header from "Traits" to "Trait" to correspond with what `rust-qtlreaper` expects. Choose only the BXD strains for the proof-of-concept example - this helped bring out the fact that the traits file SHOULD NOT contain a strain column for a strain that does not exist in the genotype file in consideration. If the traits file has a strain column which does not exist in the genotype file, then `rust-qtlreaper` fails with a panic, since, from what I can tell, it tries to get a value from the genotype file for the non-existent strain, which results to a `None` type. Subsequent attempts at running an operation on the `None` type lead to the panic. Muriithi Frederick Muriuki
2021-08-30Remove empty line...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Remove empty line at the end of the traits file Muriithi Frederick Muriuki
2021-08-30Fix some linting errors and minor bugs.Muriithi Frederick Muriuki
2021-08-30Implement module for interfacing with rust-qtlreaper...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/heatmap.py: move `generate_traits_file` function to new module * gn3/computations/qtlreaper.py: new module to interface with the `rust-qtlreaper` utility. * gn3/settings.py: Provide setting for the path to the `rust-qtlreaper` utility * qtlfilesexport.py: Move `random_string` function to new module. Update to use functions in new module. Provide a module with functions to be used to interface with `rust-qtlreaper`. This module essentially contains all the functions that are needed to build the files needed for, and to run the qtlreaper utility. Muriithi Frederick Muriuki
2021-08-27Provide intermediate data in final results...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Seeing as not every requirement/feature has been migrated over at this time, this commit just provides all the intermediate data representations in the final return of the function for later use down the line. Muriithi Frederick Muriuki
2021-08-27Export trait data to file...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Provide a function to export the given strains and traits data into a traits file for use with `rust-qtlreaper`. Muriithi Frederick Muriuki
2021-08-27Rework strains and trait values retrieval...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * Rework the strains and values retrieval function to more closely correspond to the working of the original code in GN1 Muriithi Frederick Muriuki
2021-08-20Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_gener...Muriithi Frederick Muriuki
2021-08-20Fix typing issue(s) caught by mypy...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi * gn3/computations/heatmap.py: Use `Sequence` type not `Iterator` type Muriithi Frederick Muriuki