1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
|
{%extends "phenotypes/base.html"%}
{%from "flash_messages.html" import flash_all_messages%}
{%from "macro-table-pagination.html" import table_pagination%}
{%from "phenotypes/macro-display-pheno-dataset-card.html" import display_pheno_dataset_card%}
{%block title%}Phenotypes{%endblock%}
{%block pagetitle%}Phenotypes{%endblock%}
{%block lvl4_breadcrumbs%}
<li {%if activelink=="add-phenotypes"%}
class="breadcrumb-item active"
{%else%}
class="breadcrumb-item"
{%endif%}>
<a href="{{url_for('species.populations.phenotypes.add_phenotypes',
species_id=species.SpeciesId,
population_id=population.Id,
dataset_id=dataset.Id)}}">View Datasets</a>
</li>
{%endblock%}
{%block contents%}
{{flash_all_messages()}}
<div class="row">
<form id="frm-add-phenotypes"
method="POST"
enctype="multipart/form-data"
action="{{url_for('species.populations.phenotypes.add_phenotypes',
species_id=species.SpeciesId,
population_id=population.Id,
dataset_id=dataset.Id)}}">
<legend>Add New Phenotypes</legend>
<div class="form-text help-block">
<p>Select the zip file bundle containing information on the phenotypes you
wish to upload, then click the "Upload Phenotypes" button below to
upload the data.</p>
<p>See the <a href="#section-file-formats">File Formats</a> section below
to get an understanding of what is expected of the bundle files you
upload.</p>
<p><strong>This will not update any existing phenotypes!</strong></p>
</div>
<div class="form-group">
<label for="finput-phenotypes-bundle" class="form-label">
Phenotypes Bundle</label>
<input type="file"
id="finput-phenotypes-bundle"
name="phenotypes-bundle"
accept="application/zip, .zip"
required="required"
class="form-control" />
</div>
<div class="form-group">
<input type="submit"
value="upload phenotypes"
class="not-implemented btn btn-primary" />
</div>
</form>
</div>
<div class="row">
<h2 class="heading" id="section-file-formats">File Formats</h2>
<p>We accept an extended form of the
<a href="https://kbroman.org/qtl2/assets/vignettes/input_files.html#format-of-the-data-files"
title="R/qtl2 software input file format documentation">
input files' format used with the R/qtl2 software</a> as a single ZIP
file</p>
<p>The following section will detail the expectations for each of the
different file types within the uploaded ZIP file bundle for phenotypes:</p>
<h3 class="subheading">Control File</h3>
<p>There <strong>MUST be <em>one, and only one</em></strong> file that acts
as the control file. This file can be:
<ul>
<li>a <em>JSON</em> file, or</li>
<li>a <em>YAML</em> file.</li>
</ul>
</p>
<p>The control file is useful for defining things about the bundle such as:</p>
<ul>
<li>The field separator value (default: <code>sep: ','</code>). There can
only ever be one field separator and it <strong>MUST</strong> be the same
one for <strong>ALL</strong> files in the bundle.</li>
<li>The comment character (default: <code>comment.char: '#'</code>). Any
line that starts with this character will be considered a comment line and
be ignored in its entirety.</li>
<li>Code for missing values (default: <code>na.strings: 'NA'</code>). You
can specify more than one code to indicate missing values, e.g.
<code>{…, "na.strings": ["NA", "N/A", "-"], …}</code></li>
</ul>
<h3 class="subheading"><em>pheno</em> File(s)</h3>
<p>These files are the main data files. You must have at least one of these
files in your bundle for it to be valid for this step.</p>
<p>The data is a matrix of <em>individuals × phenotypes</em> by default, as
below:<br />
<code>
id,10001,10002,10003,10004,…<br />
BXD1,61.400002,54.099998,483,49.799999,…<br />
BXD2,49,50.099998,403,45.5,…<br />
BXD5,62.5,53.299999,501,62.900002,…<br />
BXD6,53.099998,55.099998,403,NA,…<br />
⋮<br /></code>
</p>
<p>If the <code>pheno_transposed</code> value is set to <code>True</code>,
then the data will be a <em>phenotypes × individuals</em> matrix as in the
example below:<br />
<code>
id,BXD1,BXD2,BXD5,BXD6,…<br />
10001,61.400002,49,62.5,53.099998,…<br />
10002,54.099998,50.099998,53.299999,55.099998,…<br />
10003,483,403,501,403,…<br />
10004,49.799999,45.5,62.900002,NA,…<br />
⋮
</code>
</p>
<h3 class="subheading"><em>phenocovar</em> File(s)</h3>
<p>At least one phenotypes metadata file with the metadata values such as
descriptions, PubMed Identifier, publication titles (if present), etc.</p>
<p>The data in this/these file(s) is a matrix of
<em>phenotypes × phenotypes-covariates</em>. The first column is always the
phenotype names/identifiers — same as in the R/qtl2 format.</p>
<p><em>phenocovar</em> files <strong>should never be transposed</strong>!</p>
<p>This file <strong>MUST</strong> be present in the bundle, and have data for
the bundle to be considered valid by our system for this step.<br />
In addition to that, the following are the fields that <strong>must be
present</strong>, and
have values, in the file before the file is considered valid:
<ul>
<li><em>description</em>: A description for each phenotype. Useful
for users to know what the phenotype is about.</li>
<li><em>units</em>: The units of measurement for the phenotype,
e.g. milligrams for brain weight, centimetres/millimetres for
tail-length, etc.</li>
</ul></p>
<p>The following <em>optional</em> fields can also be provided:
<ul>
<li><em>pubmedid</em>: A PubMed Identifier for the publication where
the phenotype is published. If this field is not provided, the system will
assume your phenotype is not published.</li>
</ul>
</p>
<p>These files will be marked up in the control file with the
<code>phenocovar</code> key, as in the examples below:
<ol>
<li>JSON: single file<br />
<code>{<br />
⋮,<br />
"phenocovar": "your_covariates_file.csv",<br />
⋮<br />
}
</code>
</li>
<li>JSON: multiple files<br />
<code>{<br />
⋮,<br />
"phenocovar": [<br />
"covariates_file_01.csv",<br />
"covariates_file_01.csv",<br />
⋮<br />
],<br />
⋮<br />
}
</code>
</li>
<li>YAML: single file or<br />
<code>
⋮<br />
phenocovar: your_covariates_file.csv<br />
⋮
</code>
</li>
<li>YAML: multiple files<br />
<code>
⋮<br />
phenocovar:<br />
- covariates_file_01.csv<br />
- covariates_file_02.csv<br />
- covariates_file_03.csv<br />
…<br />
⋮
</code>
</li>
</ol>
</p>
<h3 class="subheading"><em>phenose</em> and <em>phenonum</em> File(s)</h3>
<p>These are extensions to the R/qtl2 standard, i.e. these types ofs file are
not supported by the original R/qtl2 file format</p>
<p>We use these files to upload the standard errors (<em>phenose</em>) when
the data file (<em>pheno</em>) is average data. In that case, the
<em>phenonum</em> file(s) contains the number of individuals that were
involved when computing the averages.</p>
<p>Both types of files are matrices of <em>individuals × phenotypes</em> by
default. Like the related <em>pheno</em> files, if
<code>pheno_transposed: True</code>, then the file will be a matrix of
<em>phenotypes × individuals</em>.</p>
</div>
<div class="row text-warning">
<h3 class="subheading">Notes for Devs (well… Fred, really.)</h3>
<p>Use the following resources for automated retrieval of certain data</p>
<ul>
<li><a href="https://www.ncbi.nlm.nih.gov/pmc/tools/developers/"
title="NCBI APIs: Retrieve articles' metadata etc.">
NCBI APIS</a></li>
</ul>
</div>
{%endblock%}
{%block sidebarcontents%}
{{display_pheno_dataset_card(species, population, dataset)}}
{%endblock%}
|
