1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
# Migrate GN1 Clustering
GeneNetwork1 has clustering output that we want to migrate to GN2. For example, go to
=> http://gn1-lily.genenetwork.org/ GN1 on Lily
Select Type -> CRTD mRNA data and do a search for 'synap*'. Click on the top 10 results checkboxes and Add to the basket (add is an icon). Now click on 'Select all' and the 'Heat map' icons. Computation may take a while.
Next click on 'Cluster traits' and 'Redraw' map. This is the one we want in GN3/GN2!
=> ./heatmap.png
## Members
* fredm
* pjotr
## Useful links
=> https://github.com/genenetwork/genenetwork1/tree/master/web/webqtl/heatmap Implementation
## Implementation
As a first step the computations should move to GN3 with proper regression/unit testing scripts. Next wire them up as REST API endpoints and add it to the GN2 web interface:
- [ ] Move computation to GN3
- [ ] Create REST API endpoints
- [ ] Add regression/unit tests in GN3
- [ ] Add to GN2 web interface
@pjotr I couldn't quite figure out what "CRTD mRNA", so I selected "Cartilage mRNA" , to try and produce a heat map, and I did get some results.
The selected data was:
* Species: Mouse (mm10)
* Group: BXD Family
* Type: Cartilage mRNA
I got
=> http://gn1-lily.genenetwork.org/image/Heatmap_8CJbN7Iy.png the initial heatmap
and
=> http://gn1-lily.genenetwork.org/image/Heatmap_syjdiq5z.png the 'Cluster Traits' heatmap
is that anything like what I should expect?
I (@fredmanglis) have noticed that the computation of the heatmap data is intertwined with the drawing of the heatmap.
I think this will be among the first things to separate, as part of moving the computation over to GN3. My initial impression of this, is that the GN3 should handle the computation and GN2 the display.
Please correct me if I am wrong.
## 2021-08-11
I have had a look at the Python implementation of Plotly, and tried out a few of the examples to get a feel for the library and its capabilities. I think it is possible to use the library to draw the heatmaps, though I'll need more time to fully grasp how it works, and how I could use it to actually provide some of the features in the clustering heatmaps that do not seem to be out-of-the-box with plotly, e.g. the lines with clustering distance.
It does also seem like the library might provide more complex heatmaps, such as
=> https://plotly.com/python/imshow/#display-an-xarray-image-with-pximshow heatmap of xarray data
which sort of approximates the heatmap example linked in
=> https://github.com/genenetwork/genenetwork3/pull/31#issuecomment-891539359 zsloan's comment
I wrote a simple script to test out the library as shown:
```
import random
import plotly.express as px
def generate_random_data(width=10, height=30):
return [[random.uniform(0,2) for i in range(0, width)]
for j in range(0, height)]
def main():
fig = px.imshow(generate_random_data())
fig.show()
if __name__ == "__main__":
main()
```
Thankfully, the following python packages seem to already be packaged with guix-bioinformatics:
* python-plotly
* python-pandas
* python-xarray
* python-scipy
The only (seemingly) missing package is `python-pooch`
## python-pooch
python-pooch is in guix as of commit 211c933 in master from this March:
=> https://guix.gnu.org/packages/python-pooch-1.3.0/ python-pooch package listing
=> https://issues.guix.gnu.org/47022 python-pooch patch thread
## 2021-08-12
When the "Single Spectrum" colour scheme is selected, the heatmap's "colour-scale" in genenetwork1 is a single spectrum that flows from Blue, through green, to red. This one is easy to reproduce somewhat by setting the colour scale with something like:
```
fig.update_coloraxes(colorscale=[
[0.0, '#0000FF'],
[0.5, '#00FF00'],
[1.0, '#FF0000']])
```
When the "Blue + Red" colour scheme is selected, the heatmap's "colour-scale" in genenetwork1 is split into 2 separate colour scales depending on whether the data value corresponds to one of:
* C57BL/6J +
* DBA/2J +
When the "Grey + Blue + Red" colour scheme is selected, the heatmap's "colour-scale" in genenetwork1 goes from a dark-grey at 0 to a light-grey at 0.5, before splitting into 2 separate colour scales for values greater than 0.5, depending on whether the data value corresponds to one of:
* C57BL/6J +
* DBA/2J +
I (@fredmanglis) have not yet figured out how to represent these more complex splits on the Plotly heatmaps, but I have a suspicion this might be achieved if there is a way to label the data as it goes into the `px.imshow(...)` call.
Maybe look into using the
=> https://plotly.com/python/creating-and-updating-figures/#conditionally-updating-traces conditional trace update
feature to set up the colours as appropriate, when different colour-schemes are selected. Failing that, have a look at the
=> https://plotly.com/python/colorscales/ colour scales documentation
=> https://plotly.com/python/plotly-fundamentals/ plotly fundamentals page
=> https://plotly.com/python/categorical-axes/ categorical axes
|
