Age | Commit message (Collapse) | Author |
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/api/heatmaps.py: Fix bugs in data parsing
* gn3/app.py: enable CORS
* gn3/settings.py: add flask-cors configurations
* guix.scm: Add flask-cors dependency
For easier testing of the heatmaps generation feature, this commit activates
the cross-origin resource sharing for all "localhost" origins.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Remove the proof-of-concept CLI-only module that was used to learn how the
heatmaps work and identify the appropriate data for use with them.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/api/heatmaps.py: Parse incoming data to build up correct trait names and
respond with only the computed heatmap data.
* gn3/heatmaps.py: Return only the computed data for heatmaps and clustering.
Since GN3 is supposed to handle only the data, and db-access, this commit
ensures that GN3 responds to the client with only the computed heatmap data,
and does not try to generate the heatmaps themselves.
The generation of the heatmaps will be delegated to the UI clients, such as
GeneNetwork2.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* To help with demonstrating that the code is producing the expected output,
for now, we return the path to the generated html file that displays the
interactive heatmap.
At this point, it is mostly useful in the development environment. Moving
forward, we might have to actually stream the raw html, or if we can get the
Kaleido library packaged for GNU Guix, stream the images binary data instead.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Update the plot layouts and size to display the dendrogram and individual
chromosome heatmaps side by side
* Update the colour scale to begin with the grays rather than absolute black
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Provide the clustering data to be used for the creation of the clustering
dendrogram in the final clustered heatmap plot.
|
|
|
|
|
|
|
|
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Intergrate the heatmap generation code on the /api/heatmaps/clustered
endpoint.
The endpoint should take a json query of the form:
{"traits_names": [ ... ] }
where the "traits_name" value is a list of the full names of traits.
A sample query to the endpoint could be something like the following:
curl -i -X POST "http://localhost:8080/api/heatmaps/clustered" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{
"traits_names": [
"UCLA_BXDBXH_CARTILAGE_V2::ILM103710672",
"UCLA_BXDBXH_CARTILAGE_V2::ILM2260338",
"UCLA_BXDBXH_CARTILAGE_V2::ILM3140576",
"UCLA_BXDBXH_CARTILAGE_V2::ILM5670577",
"UCLA_BXDBXH_CARTILAGE_V2::ILM2070121",
"UCLA_BXDBXH_CARTILAGE_V2::ILM103990541",
"UCLA_BXDBXH_CARTILAGE_V2::ILM1190722",
"UCLA_BXDBXH_CARTILAGE_V2::ILM6590722",
"UCLA_BXDBXH_CARTILAGE_V2::ILM4200064",
"UCLA_BXDBXH_CARTILAGE_V2::ILM3140463"
]
}'
which should respond with a json response containing the raw binary string
for the png format and possibly another for the svg format.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Fix a few minor bugs left over from the integration of code from the
proof-of-concept code.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Add missing imports that are needed in the code.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Copy over code from the proof-of-concept implementation and clean it up a
little for the entry-point function for heatmap generation via the API
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Add a function to generate the heatmaps for each chromosome into a single
plot.
|
|
* gn3/heatmaps.py: implement `process_traits_data_for_heatmap` function, that
will process the data into a form usable by heatmaps.
* tests/unit/test_heatmaps.py: check that the function processes the data into
the correct form.
|
|
* gn3/heatmaps.py: copy over function
* tests/unit/test_heatmaps.py: add tests
Copy function over from proof of concept and add some tests to ensure it
works as expected.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The heatmap generation does not fall cleanly within the computations or db
modules. This commit moves it to the higher level gn3 module.
|
|
* tests/unit/computations/test_heatmap.py: ordering is not longer provided as
a list of tuples; the ordering values are just a list of numbers now. This
commit updates the test to take this into consideration.
* tests/unit/computations/test_qtlreaper.py: the 'Chr' value if numeric, is
represented by an actual number, not a string. This commit updates the code
to take this into consideration.
|
|
* tests/unit/db/data/genotypes/genotype_sample1.geno: new file
Add a missing sample data file needed for unit tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Add individual heatmaps
* Add dendograms
* Merge multiple heatmaps to single plot
Updated the proof of concept code to provide a sample of what is needed to
generate the appropriate heatmaps.
To generate the sample heatmaps, one can run something like:
env SQL_URI="mysql://webqtlout:webqtlout@127.0.0.1:3306/db_webqtl" \
python3 qtlfilesexport.py
assuming that the database can be accessed at 127.0.0.1:3306
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Return a dict of values rather than list for the traits and chromosomes to
ease searching through the data.
|
|
* To ease sorting of data by numerical order down the line, sort the "Chr"
values by numerical order.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Change the id from 'T<n>' to simply '<n>' to ease sorting of the trait
results by numerical order rather than string order.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix ordering function
* tests/unit/computations/test_heatmap.py: update test
The order of the traits is important for the clustering algorithm, since the
clustering seems to use the distance of one trait from another to determine
how to order them.
This commit also gets rid of the xoffset argument that is not important to
the ordering, and was used in the older GN1 to determine how to draw the
clustering lines.
|
|
* gn3/computations/qtlreaper.py: Provide a function to organise the results by
trait for easier use down the line.
* tests/unit/computations/test_qtlreaper.py: provide a test to ensure that the
organising function works as expected.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The "Chr" value seems to be mostly a name of some sort, despite it being,
seemingly an number. This commit parses the "Chr" value as a string.
It also updates the tests to expec a string, rather than a number for "Chr"
values.
|
|
* gn3/computations/qtlreaper.py: handle exceptions
Sometimes, the values being parsed are plain strings and cannot be cast to
the float types. This commit handles that by casting only those values that
can be cast to float, and returning the others as strings.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Migrate the `web.webqtl.heatmap.Heatmap.getNearestMarker` function in GN1 to
GN3.
|
|
integrated into the script yet though)
|