1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
|
This file will provide an overview of data flow within BNW and describe the output files that are created for the network.
Files created by BNW are now written to /var/lib/genenet/bnw.
This discusses what happens after clicking on "Learn network model from data" on homepage (home.php) and focuses on the core functions of BNW.
Step 1: Upload data
Clicking "Learn network model from data" on homepage (home.php) goes to:
bn_file_load_gom.php which is responsible for uploading the input file
and preparing it for structure/parameter learning.
Specifically, uploading a file runs the 'run_prep_input' script in /sourcecodes.
This runs the prepareInput.m MATLAB/Octave function.
One additional note:
BNW assigns each network a randomly generated 3 letter prefix when the file is uploaded.
I will use ??? to signify this prefix.
This writes many files to /var/lib/genenet/bnw directory during this step:
1) ???continuous_input_orig.txt: This is an exact copy of the input file uploaded by the user.
2) ???continuous_input.txt: Input file after processing by prepareInput.m.
-A couple of notable changes are made that differentiate input.txt from input_orig.txt.
a) An additional header type row containing the variable/node type is added as the second row of the file.
See ???type.txt below.
b) Discrete variable values are recoded to be integers.
3) ???input_desc.txt: Variable descriptions that can be viewed by user clicking on "View uploaded variables and data"
4) ???nnode.txt: The number of variables/nodes in the input file (# of columns in input file).
5) ???nrows.txt: The number of cases/samples in the input file (# of rows in the input file, excluding header lines).
The following 3 files are used to keep track of variable names, variable types, and discrete variable states.
6) ???name.txt: The names of the variables in the input file in the same order as they are presented in the input file.
7) ???nlevels.txt: The original possible states of discrete variables in the input file. The order of these states
corresponds to the integers in ???continuous_input.txt (i.e., the first listed state for the variable has a value
of 1 in ???continuous_input.txt, the second listed state has a value of 2, . . .)
8) ???type.txt: The type of each variable/node. Continuous nodes have a value of 1. Discrete nodes have a value equal to the
number of possible states.
The following 5 files contain default values of structure learning options.
They will be modified if the structural constraint interface is used.
9) ???ban.txt: List of banned edges (will contain only a header line at this point).
10) ???white.txt: List of required edges (will contain only a header line at this point).
11) ???k.txt: Number of top-scoring models to include in model averaging (default value is 1, indicating no model averaging).
12) ???parent.txt: Maximum number of parents for each node during structure learning (default value is 4).
13) ???thr.txt: Threshold value to include edges after model averaging (default value is 0.5).
Step 2a: Structure learning with default parameters
Click "Perform Bayesian network modeling using default settings"
This is a link to executionprogress.php which just displays a "progress bar" and starts structure learning.
To start structure learning, it runs execute_bn_gom.php.
execute_bn_gom.php starts structure learning using run.sh and, then, run_octave in the run_scripts directory discussed in Step 3.
Output files from structure learning that are written to /var/lib/genenet/bnw:
1) ???structure_input.txt: The structure matrix, containing the edges included in the network model.
The format of this file is a header line containing the variable names.
The rest is the strucure matrix where 1 indicates an edge is included in the network and a 2 is not.
The matrix had dimensions n x n where n is the number of variables.
A '1' in the ith column in the jth row indicates that there is an edge from variable i to variable j.
2) ???structure_input_temp.txt: Structure matrix with model averaging data.
This file has the model averaging probability of each edge.
The ???structure_input.txt file is generated from this file. If the value of an edge is this file
is greater than the value in ???thr.txt, the edge is included in the network (a '1' in ???structure_input.txt).
Step 2b: Structure learning using the structural constraint interface
Before the process described in Step 2a, additional constraints or changes to default parameters
can be added using the structural constraint interface.
This can be accessed by clicking on "Go to structure learning settings and the BNW
structural constraint interface" which links to create_tiers_gom_part1.php
This page allows for modifying "Global structure learning settings"
This will change the values in ???parent.txt, ???k.txt, and ???thr.txt.
Clicking "Continue assign variables to tiers" will go to create_tiers_gom_part2.php.
This page will modify the ???ban.txt and ???white.txt files.
Clicking on "Click here to perform Bayesian network modeling after creating tiers" goes to tier_description_processing_gom.php and Step 2a is followed.
Step 3: Parameter learning of the original network.
The step begins near the end of execute_bn_gom.php file where the run_octave script is run.
run_octave starts up the runBN_initial.m Matlab/Octave script.
This prepares parameter learning for the data file in ???continuous_input.txt and
network structure defined in ???structure_input.txt.
runBN_initial makes 2 main output files and 2 files that may be used by other Matlab/Octave scripts.
1) ???net_figure.txt: This file provides the data that is used to draw the network in BNW. I will describe
the format of this file elsewhere.
2) ???parameters.txt: The parameters of the network using the original data set.
This file is broken into different sections that correspond to each variable/node.
Each section has the name of the variable and whether it is discrete or continuout.
If it is discrete, the number of possible states is also provided.
For discrete nodes, the percentage of each state is listed.
For continuous nodes, the mean and standard deviation of the Gaussian fit to that variable is provided.
3) ???map.txt: The standard deviations and means of the original data set.
The parameter learning tools used in BNW work best if continuous data is normalized.
Each line of this file provides the variable name, the standard deviation, and the mean of the original data
for each variable for use in standardizing data in the Matlab/Octave scripts.
4) ???mapdata.txt: The topological order of the nodes used in parameter learning.
Sometimes before parameter learning, the nodes are resorted so they are in topological order (i.e.,
parents are always listed before parents).
A couple other Octave functions (createSVG and createJSON) are run to make files related to visualizing the network:
5) ???grviz_name_file.txt: A list of the names of the variables that is used by graphviz/dot when laying out the network.
6) ???graphviz.txt: Input file for dot that creates the edges included in the network that is used by layout.php.
0 is the first node listed in ???grviz_name_file.txt, 1 is the second node, . . .
7) ???graphviz_svg_no_edge.txt: Input file for dot that is used to create svg file to make SVG file without edge weights.
8) ???graphviz_svg.txt: Input file for dot that is used to create svg file to make SVG file with edge weights.
9) ???network_no_edge.svg: SVG file without edge weight
10) ???network.svg: SVG file with edge weights
11) ???network.json: JSON file used
Step 4: Display of the network.
The network can be displayed in three ways:
1) The network is initially displayed using the SVG files and Graphviz with layout_svg_wt.php/layout_svg_no.php.
This provides a simple display of the network that is suitable for dowloading.
The user then has the option to either use the modify the network or use the network to make predictions.
2) The network structure can be modified using the JSON file and Cytoscape with layout_cyto.php.
3) The network can be used to make predictions using layout.php.
The network is drawn in a division using either
network_layout_evd.php if Evidence mode is being used or
network_layout_inv.php if Intervention mode is being used.
These php files basically just read and ???net_figure.txt and run dot using the graphviz file
described above to draw the network.
Step 5a: Adding evidence or intervention to the network.
Clicking on a node and entering evidence/intervention runs add_evd.php/add_inv.php.
This creates 3 files:
(I am going to describe these as evidence here, intervention mode is the same.)
1) ???vardata.txt: The value of the evidence that was entered.
2) ???varname.txt: The name of the node for which evidence was entered.
3) ???var.txt: The index of the node for which evidence was entered in the ???mapdata.txt file.
If evidence is entered for more than one node, these files will contain tab-separated lists.
add_evd.php/add_inv.php then runs a script called either run_octave_evd or run_octave_inv.
This script runs the Predictmultiple or Predictmultipleintervention Matlab/Octave functions.
The output files from this script are:
4) ???net_figure_new.txt: The net_figure file after entering evidence/intervention.
5) ???parameters_ev.txt: The parameters of the network considering the evidence/intervention.
Then, network_layout_evd_2.php or network_layout_inv_2.php is used to draw the network with the evidence/intervention.
Step 5b: Additional files created in Intervention mode.
In Intervention mode, the network structure can change. The node with intervention no longer depends on its parents.
Several files must be created to account for this. They are the same as described above, but with the new network structure.
1) ???graphviz_new.txt
2) ???grviz_name_file_new.txt
3) ???structure_input_new.txt
Step 6: Cross-validations and predictions
I will add discussion of this step later.
|
