1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
|
#+STARTUP: contents inlineimages shrink
#+OPTIONS: ^:{}
#+TITLE: Technical Specification for GeneNetwork
#+AUTHOR: Frederick Muriuki Muriithi
* Introduction
This document will detail the internal implementation for the software.
- Data structures
- Programming languages
- Database schema(s)
- Programming Language(s)
* Data
Below is a flowchart of the data and the dependencies between the data:
** TODO Complete this flowchart
#+BEGIN_SRC dot :cmd dot :cmdline -Tpng :file images/data_flow.png :exports results
digraph Data {
Node[shape=none,margin=0];
species [label=<
<b>Species</b><br />
<i>e.g. mouse, rat, human, etc.</i>
>];
group [label=<<b>Group</b>>];
datasets [label=<<b>Datasets</b>>];
traits [label=<<b>Traits</b>>];
sample_data [label=<
<b>Sample Data</b><br />
<i>The actual data for each sample (individual/strain mean).</i>
>];
species -> group [style=dotted];
group -> datasets [style=dotted];
species -> datasets;
datasets -> traits;
traits -> sample_data;
}
#+END_SRC
#+RESULTS:
[[file:images/data_flow.png]]
** Species
This is a string that denotes which biological species the data in consideration belongs to.
Examples of these are:
- mouse
- rat
- human
** Group
A "category" for datasets with unique samplelists.
A group can contain data from a single study or from many studies.
This is a crutch due to the database design, and is not a true representation of
the data. We should strive to get rid of it.
Group information is found in the *InbredSet* table in the database
** TODO Datasets
These are groupings for the traits. The main differences between the datasets
has to do with the relevant metadata for each grouping.
There are three main groupings, which are:
- Phenotype
- Genotype
- mRNA Expression (ProbeSet)
*** Phenotype
This grouping is for traits that are *NOT* expressions of genotype data (e.g.
sex, weight, etc).
Phenotype traits are authenticated at the trait level, since there are no
sub-groupings of them within a group.
*** Genotype
The traits in this category are markers. Their sample data are the genotypes
(encoded 1/0/1, though genotype probabilities could also be stored).
This data also exists in genotype files (in both .geno and BIMBAM formats). This
data is stored in the database to allow for its use as cofactors when mapping.
*** mRNA Expression (ProbeSet)
The traits in this category are mRNA transcripts from locations in the genome.
The sample data for these traits are expression values for the traits.
Individual datasets are from individual analyses, split by tissue.
*** Temp
This is a pseudo-grouping which costitutes of User-generated data. These have no
real metadata. They are just lists of samples/strains (from a specific group)
and values
The data for these is entered directly via the "Submit Trait" page or it is derived from the PCA traits generated from the "Correlation Matrix" page
**** TODO Samplelists: What are they? Are they related to sample data in traits?
** TODO Traits
A trait is a set of values for each sample/strain.
A trait can have metadata. The metadata varies with the dataset in which the trait belongs.
The metadata is not useful, for most part, in any computations, but it is useful
for the display of the traits to the user, and in search.
*** Sample Data
This is the actual data for each sample.
A sample can be an individual, or the strain mean (average of multiple
individuals)
The following are the expected "fields" for each instance of sample data:
- *value*: a numerical value. For "strain mean" items, it is the mean value.
- *variance*: /variance/ for an individual and /standard-error/ for "strain mean"
- *ndata*/*num_cases*: Number of samples for "strain mean" values. NIL for
individuals (should probably be set to 1)
Sample data can have metadata, referred to as "*Case Attributes*". This is
mostly non-numeric data e.g. sex, treatment type, etc.
**** TODO Can there be a trait with no sample data? What are the consequences of such a trait existing?
*** Common Metadata
- *num_overlap*: When doing correlations, it sets the number of overlapping
samples for all target traits (number of samples where both the primary and
target traits have values). This is not a true attribute, rather, it is a
temporary attribute assigned to a trait "object".
- *haveinfo*: Get rid of this. An implementation detail. Indicates whether there
was information found in the database for the given trait.
- *mean*: Average of all sample data values
- *description_display*/*abbreviation*: Phenotype description and abbreviated
version (for display in figures or as a "trait symbol"). Depends on the
authentication.
*** Phenotype Metadata
- *name*: PublishXRef ID. In GeneNetwork, it is a combination of a three-letter group code (InbredSet.InbredSetCode) and this ID makes up the displayed name
- *lrs*: Likelihood Ratio Statistic. In the code, this is the maximim QTL[fn:1]
association value when the trait in question is mapped using qtlreaper[fn:2]
- *LRS_location_repr*: The location of the top *QTL* (above) as a printable
string (chromosome + megabase location) e.g. "Chr2:20.019888"
- *pvalue*: P-value for the maximum *QTL*. There's a chance that the
*lrs score* is derived from this. *VERIFY!!!!!*
- *additive*: additive effect[fn:3] for the maximum *QTL*
- *pre_publication_description*: Description of the phenotype before publication.
Depending on the authentication, the *pre_publication_description* (which is
not really human readable) is shown instead of the
*post_publication_description*.
- *post_publication_description*: For published phenotypes, this is the
description of the phenotype after publication.
- *pubmed_id*/*pubmed_link*/*pubmed_text*: For published phenotypes, these are
the identifier, link to, and text of the publication on the PubMed[fn:4] site
*** mRNA Expression (ProbeSet) Metadata
- *name*: Transcript name
- *location*: Location of the mRNA transcript in the genome (chromosome +
megabase location)
- *lrs*: Likelihood Ratio Statistic. In the code, this is the maximim QTL[fn:1]
association value when the trait in question is mapped using qtlreaper[fn:2]
- *LRS_location_repr*: The location of the top *QTL* (above) as a printable
string (chromosome + megabase location) e.g. "Chr2:20.019888"
- *pvalue*: P-value for the maximum *QTL*. There's a chance that the
*lrs score* is derived from this. *VERIFY!!!!!*
- *additive*: additive effect[fn:3] for the maximum *QTL*
- *symbol*: gene symbol
- *sequence*: The DNA sequence
- *probe_target_description*/*strand_probe*/*cell_id*: *VERIFY!!!!*
*** Genotype
- *location*: Location of the genetic marker
* Operations on Traits
**** TODO What are the valid operations on traits?
**** TODO What data is expected for each operation?
* Operations on Datasets
**** TODO Are there operations on whole datasets?
**** TODO If yes, what are they and what data does each operation expect?
* Database Design and Schema
....
* Other headlines go here ...
* Footnotes
[fn:1] Quantitative Trait Locus (QTL): A genetic region that most strongly modulates a particular trait. https://en.wikipedia.org/wiki/Genome-wide_association_study (-log10 p-value is basically an alternative measure of association to LRS). The mapping results that lrs/LRS_location_repr/additive are derived from are usually represented by a Manhattan Plot (as shown in the first figure of the wiki page). The max QTL would be the peak value in that figure (and its associated chromosome location)
[fn:2] qtlreaper: <provide link to qtlreaper>
[fn:3] https://genenetwork.org/glossary/#a (Additive Allele Effect) – the GN glossary entry for this can explain it better than I can.
It’s basically another value associated with the max QTL.
[fn:4] PubMed https://pubmed.ncbi.nlm.nih.gov/
|
