diff options
| author | ziejd2 | 2017-09-28 15:04:40 -0500 |
|---|---|---|
| committer | ziejd2 | 2017-09-28 15:04:40 -0500 |
| commit | 8070dc963753142bb86c4ed698d91fd623ed28e7 (patch) | |
| tree | d0f6dd8fc46a49b819aa55c1a90faa14d8448883 /sourcecodes/bnt-master/docs | |
| parent | 7cc31810d53176e805532b2789955f4eedbce6bb (diff) | |
| download | BNW-8070dc963753142bb86c4ed698d91fd623ed28e7.tar.gz | |
BNW using Octave instead of Matlab.
This version of BNW should perform the same as the original version. The only difference is that it uses Octave instead of Matlab when running BayesNet Toolbox during parameter learning. I am calling this BNW_1.02. It can be accessed at: compbio.uthsc.edu/BNW_1.02
Diffstat (limited to 'sourcecodes/bnt-master/docs')
155 files changed, 40442 insertions, 0 deletions
diff --git a/sourcecodes/bnt-master/docs/ChangeLog.Sourceforge.txt b/sourcecodes/bnt-master/docs/ChangeLog.Sourceforge.txt new file mode 100644 index 00000000..17ec3df1 --- /dev/null +++ b/sourcecodes/bnt-master/docs/ChangeLog.Sourceforge.txt @@ -0,0 +1,4436 @@ + + +2007-02-11 17:12 nsaunier + + * BNT/learning/learn_struct_pdag_pc.m: Bug submitted by Imme Ebert-Uphoff (ebert@tree.com) (see Thu Feb 8, 2007 email on the BNT mailing list). + +2005-11-26 12:12 yozhik + + * BNT/inference/dynamic/@hmm_inf_engine/fwdback_twoslice.m: merged + fwdback_twoslice.m to release branch + +2005-11-25 17:24 nsaunier + + * BNT/inference/dynamic/@hmm_inf_engine/fwdback_twoslice.m: adding + old missing fwdback_twoslice.m + +2005-11-25 17:24 yozhik + + * BNT/inference/dynamic/@hmm_inf_engine/fwdback_twoslice.m: file + fwdback_twoslice.m was added on branch release-1_0 on 2005-11-26 + 20:12:05 +0000 + +2005-09-25 15:54 yozhik + + * BNT/add_BNT_to_path.m: fix paths + +2005-09-25 15:30 yozhik + + * BNT/add_BNT_to_path.m: Restored directories to path. + +2005-09-25 15:29 yozhik + + * HMM/fwdback_twoslice.m: added missing fwdback_twoslice + +2005-09-17 11:14 yozhik + + * ChangeLog, BNT/add_BNT_to_path.m, BNT/test_BNT.m, + BNT/examples/static/cmp_inference_static.m, + BNT/inference/dynamic/@hmm_inf_engine/enter_evidence.m: Merged + bug fixes from HEAD. + +2005-09-17 11:11 yozhik + + * ChangeLog: added change log + +2005-09-17 11:11 yozhik + + * ChangeLog: file ChangeLog was added on branch release-1_0 on + 2005-09-17 18:14:47 +0000 + +2005-09-17 10:00 yozhik + + * BNT/inference/dynamic/@hmm_inf_engine/enter_evidence.m: Temporary + rollback to fix error, per Kevin. + +2005-09-17 09:59 yozhik + + * BNT/examples/static/cmp_inference_static.m: Commented out + erroneous line, per Kevin. + +2005-09-17 09:58 yozhik + + * BNT/add_BNT_to_path.m: Changed to require BNT_HOME to be + predefined. + +2005-09-17 09:56 yozhik + + * BNT/test_BNT.m: Commented out problematic tests. + +2005-09-17 09:38 yozhik + + * BNT/test_BNT.m: renable tests + +2005-09-12 22:18 yozhik + + * KPMtools/pca_kpm.m: Initial import of code base from Kevin + Murphy. + +2005-09-12 22:18 yozhik + + * KPMtools/pca_kpm.m: Initial revision + +2005-08-29 10:44 yozhik + + * graph/: README.txt, Old/best_first_elim_order.m, + Old/dag_to_jtree.m, Old/dfs.m, Old/dsep_test.m, + Old/mk_2D_lattice_slow.m, acyclic.m, assignEdgeNums.m, + best_first_elim_order.m, check_jtree_property.m, + check_triangulated.m, children.m, cliques_to_jtree.m, + cliques_to_strong_jtree.m, connected_graph.m, + dag_to_essential_graph.m, dfs.m, dfs_test.m, dijkstra.m, + family.m, graph_separated.m, graph_to_jtree.m, + min_subtree_con_nodes.m, minimum_spanning_tree.m, minspan.m, + mk_2D_lattice.m, mk_2D_lattice_slow.m, mk_adj_mat.m, + mk_adjmat_chain.m, mk_all_dags.m, mk_nbrs_of_dag.m, + mk_nbrs_of_digraph.m, mk_nbrs_of_digraph_broken.m, + mk_nbrs_of_digraph_not_vectorized.m, mk_rnd_dag.m, + mk_rnd_dag_given_edge_prob.m, mk_rooted_tree.m, mk_undirected.m, + moralize.m, neighbors.m, parents.m, pred2path.m, + reachability_graph.m, scc.m, strong_elim_order.m, test.m, + test_strong_root.m, topological_sort.m, trees.txt, triangulate.c, + triangulate.m, triangulate_2Dlattice_demo.m, triangulate_test.m: + Initial import of code base from Kevin Murphy. + +2005-08-29 10:44 yozhik + + * graph/: README.txt, Old/best_first_elim_order.m, + Old/dag_to_jtree.m, Old/dfs.m, Old/dsep_test.m, + Old/mk_2D_lattice_slow.m, acyclic.m, assignEdgeNums.m, + best_first_elim_order.m, check_jtree_property.m, + check_triangulated.m, children.m, cliques_to_jtree.m, + cliques_to_strong_jtree.m, connected_graph.m, + dag_to_essential_graph.m, dfs.m, dfs_test.m, dijkstra.m, + family.m, graph_separated.m, graph_to_jtree.m, + min_subtree_con_nodes.m, minimum_spanning_tree.m, minspan.m, + mk_2D_lattice.m, mk_2D_lattice_slow.m, mk_adj_mat.m, + mk_adjmat_chain.m, mk_all_dags.m, mk_nbrs_of_dag.m, + mk_nbrs_of_digraph.m, mk_nbrs_of_digraph_broken.m, + mk_nbrs_of_digraph_not_vectorized.m, mk_rnd_dag.m, + mk_rnd_dag_given_edge_prob.m, mk_rooted_tree.m, mk_undirected.m, + moralize.m, neighbors.m, parents.m, pred2path.m, + reachability_graph.m, scc.m, strong_elim_order.m, test.m, + test_strong_root.m, topological_sort.m, trees.txt, triangulate.c, + triangulate.m, triangulate_2Dlattice_demo.m, triangulate_test.m: + Initial revision + +2005-08-26 18:08 yozhik + + * KPMtools/fullfileKPM.m: Initial import of code base from Kevin + Murphy. + +2005-08-26 18:08 yozhik + + * KPMtools/fullfileKPM.m: Initial revision + +2005-08-21 13:00 yozhik + + * + BNT/inference/static/@pearl_inf_engine/private/parallel_protocol.m: + Initial import of code base from Kevin Murphy. + +2005-08-21 13:00 yozhik + + * + BNT/inference/static/@pearl_inf_engine/private/parallel_protocol.m: + Initial revision + +2005-07-11 12:07 yozhik + + * KPMtools/plotcov2New.m: Initial import of code base from Kevin + Murphy. + +2005-07-11 12:07 yozhik + + * KPMtools/plotcov2New.m: Initial revision + +2005-07-06 12:32 yozhik + + * KPMtools/montageKPM2.m: Initial import of code base from Kevin + Murphy. + +2005-07-06 12:32 yozhik + + * KPMtools/montageKPM2.m: Initial revision + +2005-06-27 18:35 yozhik + + * KPMtools/montageKPM3.m: Initial import of code base from Kevin + Murphy. + +2005-06-27 18:35 yozhik + + * KPMtools/montageKPM3.m: Initial revision + +2005-06-27 18:30 yozhik + + * KPMtools/cell2matPad.m: Initial import of code base from Kevin + Murphy. + +2005-06-27 18:30 yozhik + + * KPMtools/cell2matPad.m: Initial revision + +2005-06-15 14:13 yozhik + + * BNT/CPDs/@gaussian_CPD/gaussian_CPD.m: Initial import of code + base from Kevin Murphy. + +2005-06-15 14:13 yozhik + + * BNT/CPDs/@gaussian_CPD/gaussian_CPD.m: Initial revision + +2005-06-08 18:56 yozhik + + * Kalman/testKalman.m: Initial import of code base from Kevin + Murphy. + +2005-06-08 18:56 yozhik + + * Kalman/testKalman.m: Initial revision + +2005-06-08 18:25 yozhik + + * HMM/: testHMM.m, fixed_lag_smoother_demo.m: Initial import of + code base from Kevin Murphy. + +2005-06-08 18:25 yozhik + + * HMM/: testHMM.m, fixed_lag_smoother_demo.m: Initial revision + +2005-06-08 18:22 yozhik + + * HMM/: README.txt, dhmm_em.m: Initial import of code base from + Kevin Murphy. + +2005-06-08 18:22 yozhik + + * HMM/: README.txt, dhmm_em.m: Initial revision + +2005-06-08 18:17 yozhik + + * HMM/fwdback.m: Initial import of code base from Kevin Murphy. + +2005-06-08 18:17 yozhik + + * HMM/fwdback.m: Initial revision + +2005-06-05 11:46 yozhik + + * KPMtools/: rectintLoopC.c, rectintLoopC.dll: Initial import of + code base from Kevin Murphy. + +2005-06-05 11:46 yozhik + + * KPMtools/: rectintLoopC.c, rectintLoopC.dll: Initial revision + +2005-06-01 12:39 yozhik + + * KPMtools/montageKPM.m: Initial import of code base from Kevin + Murphy. + +2005-06-01 12:39 yozhik + + * KPMtools/montageKPM.m: Initial revision + +2005-05-31 21:49 yozhik + + * KPMtools/initFigures.m: Initial import of code base from Kevin + Murphy. + +2005-05-31 21:49 yozhik + + * KPMtools/initFigures.m: Initial revision + +2005-05-31 11:19 yozhik + + * KPMstats/unidrndKPM.m: Initial import of code base from Kevin + Murphy. + +2005-05-31 11:19 yozhik + + * KPMstats/unidrndKPM.m: Initial revision + +2005-05-30 15:08 yozhik + + * KPMtools/filepartsLast.m: Initial import of code base from Kevin + Murphy. + +2005-05-30 15:08 yozhik + + * KPMtools/filepartsLast.m: Initial revision + +2005-05-29 23:01 yozhik + + * KPMtools/plotBox.m: Initial import of code base from Kevin + Murphy. + +2005-05-29 23:01 yozhik + + * KPMtools/plotBox.m: Initial revision + +2005-05-25 18:31 yozhik + + * KPMtools/plotColors.m: Initial import of code base from Kevin + Murphy. + +2005-05-25 18:31 yozhik + + * KPMtools/plotColors.m: Initial revision + +2005-05-25 12:11 yozhik + + * KPMtools/genpathKPM.m: Initial import of code base from Kevin + Murphy. + +2005-05-25 12:11 yozhik + + * KPMtools/genpathKPM.m: Initial revision + +2005-05-23 17:03 yozhik + + * netlab3.3/demhmc1.m: Initial import of code base from Kevin + Murphy. + +2005-05-23 17:03 yozhik + + * netlab3.3/demhmc1.m: Initial revision + +2005-05-23 16:44 yozhik + + * netlab3.3/gmminit.m: Initial import of code base from Kevin + Murphy. + +2005-05-23 16:44 yozhik + + * netlab3.3/gmminit.m: Initial revision + +2005-05-23 16:07 yozhik + + * netlab3.3/metrop.m: Initial import of code base from Kevin + Murphy. + +2005-05-23 16:07 yozhik + + * netlab3.3/metrop.m: Initial revision + +2005-05-22 23:23 yozhik + + * netlab3.3/demmet1.m: Initial import of code base from Kevin + Murphy. + +2005-05-22 23:23 yozhik + + * netlab3.3/demmet1.m: Initial revision + +2005-05-22 16:32 yozhik + + * KPMstats/: dirichletrnd.m, dirichletpdf.m, test_dir.m, + multirnd.m, multipdf.m: Initial import of code base from Kevin + Murphy. + +2005-05-22 16:32 yozhik + + * KPMstats/: dirichletrnd.m, dirichletpdf.m, test_dir.m, + multirnd.m, multipdf.m: Initial revision + +2005-05-13 13:52 yozhik + + * KPMtools/: asort.m, dirKPM.m: Initial import of code base from + Kevin Murphy. + +2005-05-13 13:52 yozhik + + * KPMtools/: asort.m, dirKPM.m: Initial revision + +2005-05-09 18:32 yozhik + + * netlab3.3/dem2ddat.m: Initial import of code base from Kevin + Murphy. + +2005-05-09 18:32 yozhik + + * netlab3.3/dem2ddat.m: Initial revision + +2005-05-09 15:20 yozhik + + * KPMtools/: mkdirKPM.m, optimalMatching.m, optimalMatchingTest.m, + subsets1.m: Initial import of code base from Kevin Murphy. + +2005-05-09 15:20 yozhik + + * KPMtools/: mkdirKPM.m, optimalMatching.m, optimalMatchingTest.m, + subsets1.m: Initial revision + +2005-05-09 09:47 yozhik + + * KPMtools/bipartiteMatchingDemo.m: Initial import of code base + from Kevin Murphy. + +2005-05-09 09:47 yozhik + + * KPMtools/bipartiteMatchingDemo.m: Initial revision + +2005-05-08 22:25 yozhik + + * KPMtools/bipartiteMatchingIntProg.m: Initial import of code base + from Kevin Murphy. + +2005-05-08 22:25 yozhik + + * KPMtools/bipartiteMatchingIntProg.m: Initial revision + +2005-05-08 21:45 yozhik + + * KPMtools/bipartiteMatchingDemoPlot.m: Initial import of code base + from Kevin Murphy. + +2005-05-08 21:45 yozhik + + * KPMtools/bipartiteMatchingDemoPlot.m: Initial revision + +2005-05-08 19:55 yozhik + + * KPMtools/subsetsFixedSize.m: Initial import of code base from + Kevin Murphy. + +2005-05-08 19:55 yozhik + + * KPMtools/subsetsFixedSize.m: Initial revision + +2005-05-08 15:48 yozhik + + * KPMtools/centeringMatrix.m: Initial import of code base from + Kevin Murphy. + +2005-05-08 15:48 yozhik + + * KPMtools/centeringMatrix.m: Initial revision + +2005-05-08 10:51 yozhik + + * netlab3.3/demgmm1.m: Initial import of code base from Kevin + Murphy. + +2005-05-08 10:51 yozhik + + * netlab3.3/demgmm1.m: Initial revision + +2005-05-06 18:09 yozhik + + * BNT/add_BNT_to_path.m: Initial import of code base from Kevin + Murphy. + +2005-05-06 18:09 yozhik + + * BNT/add_BNT_to_path.m: Initial revision + +2005-05-03 21:35 yozhik + + * KPMstats/standardize.m: Initial import of code base from Kevin + Murphy. + +2005-05-03 21:35 yozhik + + * KPMstats/standardize.m: Initial revision + +2005-05-03 13:18 yozhik + + * KPMstats/histCmpChi2.m: Initial import of code base from Kevin + Murphy. + +2005-05-03 13:18 yozhik + + * KPMstats/histCmpChi2.m: Initial revision + +2005-05-03 12:01 yozhik + + * KPMtools/strsplit.m: Initial import of code base from Kevin + Murphy. + +2005-05-03 12:01 yozhik + + * KPMtools/strsplit.m: Initial revision + +2005-05-02 13:19 yozhik + + * KPMtools/hsvKPM.m: Initial import of code base from Kevin Murphy. + +2005-05-02 13:19 yozhik + + * KPMtools/hsvKPM.m: Initial revision + +2005-04-27 11:34 yozhik + + * BNT/potentials/@dpot/: subsasgn.m, subsref.m: Initial import of + code base from Kevin Murphy. + +2005-04-27 11:34 yozhik + + * BNT/potentials/@dpot/: subsasgn.m, subsref.m: Initial revision + +2005-04-27 10:58 yozhik + + * KPMtools/mahal2conf.m, nethelp3.3/conffig.htm, + nethelp3.3/confmat.htm, nethelp3.3/conjgrad.htm, + nethelp3.3/consist.htm, nethelp3.3/convertoldnet.htm, + nethelp3.3/datread.htm, nethelp3.3/datwrite.htm, + nethelp3.3/dem2ddat.htm, nethelp3.3/demard.htm, + nethelp3.3/demev1.htm, nethelp3.3/demev2.htm, + nethelp3.3/demev3.htm, nethelp3.3/demgauss.htm, + nethelp3.3/demglm1.htm, nethelp3.3/demglm2.htm, + nethelp3.3/demgmm1.htm, nethelp3.3/demgmm2.htm, + nethelp3.3/demgmm3.htm, nethelp3.3/demgmm4.htm, + nethelp3.3/demgmm5.htm, nethelp3.3/demgp.htm, + nethelp3.3/demgpard.htm, nethelp3.3/demgpot.htm, + nethelp3.3/demgtm1.htm, nethelp3.3/demgtm2.htm, + nethelp3.3/demhint.htm, nethelp3.3/demhmc1.htm, + nethelp3.3/demhmc2.htm, nethelp3.3/demhmc3.htm, + nethelp3.3/demkmn1.htm, nethelp3.3/demknn1.htm, + nethelp3.3/demmdn1.htm, nethelp3.3/demmet1.htm, + nethelp3.3/demmlp1.htm, nethelp3.3/demmlp2.htm, + nethelp3.3/demnlab.htm, nethelp3.3/demns1.htm, + nethelp3.3/demolgd1.htm, nethelp3.3/demopt1.htm, + nethelp3.3/dempot.htm, nethelp3.3/demprgp.htm, + nethelp3.3/demprior.htm, nethelp3.3/demrbf1.htm, + nethelp3.3/demsom1.htm, nethelp3.3/demtrain.htm, + nethelp3.3/dist2.htm, nethelp3.3/eigdec.htm, + nethelp3.3/errbayes.htm, nethelp3.3/evidence.htm, + nethelp3.3/fevbayes.htm, nethelp3.3/gauss.htm, + nethelp3.3/gbayes.htm, nethelp3.3/glm.htm, + nethelp3.3/glmderiv.htm, nethelp3.3/glmerr.htm, + nethelp3.3/glmevfwd.htm, nethelp3.3/glmfwd.htm, + nethelp3.3/glmgrad.htm, nethelp3.3/glmhess.htm, + nethelp3.3/glminit.htm, nethelp3.3/glmpak.htm, + nethelp3.3/glmtrain.htm, nethelp3.3/glmunpak.htm, + nethelp3.3/gmm.htm, nethelp3.3/gmmactiv.htm, + nethelp3.3/gmmem.htm, nethelp3.3/gmminit.htm, + nethelp3.3/gmmpak.htm, nethelp3.3/gmmpost.htm, + nethelp3.3/gmmprob.htm, nethelp3.3/gmmsamp.htm, + nethelp3.3/gmmunpak.htm, nethelp3.3/gp.htm, + nethelp3.3/gpcovar.htm, nethelp3.3/gpcovarf.htm, + nethelp3.3/gpcovarp.htm, nethelp3.3/gperr.htm, + nethelp3.3/gpfwd.htm, nethelp3.3/gpgrad.htm, + nethelp3.3/gpinit.htm, nethelp3.3/gppak.htm, + nethelp3.3/gpunpak.htm, nethelp3.3/gradchek.htm, + nethelp3.3/graddesc.htm, nethelp3.3/gsamp.htm, + nethelp3.3/gtm.htm, nethelp3.3/gtmem.htm, nethelp3.3/gtmfwd.htm, + nethelp3.3/gtminit.htm, nethelp3.3/gtmlmean.htm, + nethelp3.3/gtmlmode.htm, nethelp3.3/gtmmag.htm, + nethelp3.3/gtmpost.htm, nethelp3.3/gtmprob.htm, + nethelp3.3/hbayes.htm, nethelp3.3/hesschek.htm, + nethelp3.3/hintmat.htm, nethelp3.3/hinton.htm, + nethelp3.3/histp.htm, nethelp3.3/hmc.htm, nethelp3.3/index.htm, + nethelp3.3/kmeans.htm, nethelp3.3/knn.htm, nethelp3.3/knnfwd.htm, + nethelp3.3/linef.htm, nethelp3.3/linemin.htm, + nethelp3.3/maxitmess.htm, nethelp3.3/mdn.htm, + nethelp3.3/mdn2gmm.htm, nethelp3.3/mdndist2.htm, + nethelp3.3/mdnerr.htm, nethelp3.3/mdnfwd.htm, + nethelp3.3/mdngrad.htm, nethelp3.3/mdninit.htm, + nethelp3.3/mdnpak.htm, nethelp3.3/mdnpost.htm, + nethelp3.3/mdnprob.htm, nethelp3.3/mdnunpak.htm, + nethelp3.3/metrop.htm, nethelp3.3/minbrack.htm, + nethelp3.3/mlp.htm, nethelp3.3/mlpbkp.htm, + nethelp3.3/mlpderiv.htm, nethelp3.3/mlperr.htm, + nethelp3.3/mlpevfwd.htm, nethelp3.3/mlpfwd.htm, + nethelp3.3/mlpgrad.htm, nethelp3.3/mlphdotv.htm, + nethelp3.3/mlphess.htm, nethelp3.3/mlphint.htm, + nethelp3.3/mlpinit.htm, nethelp3.3/mlppak.htm, + nethelp3.3/mlpprior.htm, nethelp3.3/mlptrain.htm, + nethelp3.3/mlpunpak.htm, nethelp3.3/netderiv.htm, + nethelp3.3/neterr.htm, nethelp3.3/netevfwd.htm, + nethelp3.3/netgrad.htm, nethelp3.3/nethelp3.3.zip, + nethelp3.3/nethess.htm, nethelp3.3/netinit.htm, + nethelp3.3/netopt.htm, nethelp3.3/netpak.htm, + nethelp3.3/netunpak.htm, nethelp3.3/olgd.htm, nethelp3.3/pca.htm, + nethelp3.3/plotmat.htm, nethelp3.3/ppca.htm, + nethelp3.3/quasinew.htm, nethelp3.3/rbf.htm, + nethelp3.3/rbfbkp.htm, nethelp3.3/rbfderiv.htm, + nethelp3.3/rbferr.htm, nethelp3.3/rbfevfwd.htm, + nethelp3.3/rbffwd.htm, nethelp3.3/rbfgrad.htm, + nethelp3.3/rbfhess.htm, nethelp3.3/rbfjacob.htm, + nethelp3.3/rbfpak.htm, nethelp3.3/rbfprior.htm, + nethelp3.3/rbfsetbf.htm, nethelp3.3/rbfsetfw.htm, + nethelp3.3/rbftrain.htm, nethelp3.3/rbfunpak.htm, + nethelp3.3/rosegrad.htm, nethelp3.3/rosen.htm, + nethelp3.3/scg.htm, nethelp3.3/som.htm, nethelp3.3/somfwd.htm, + nethelp3.3/sompak.htm, nethelp3.3/somtrain.htm, + nethelp3.3/somunpak.htm, netlab3.3/Contents.m, netlab3.3/LICENSE, + netlab3.3/conffig.m, netlab3.3/confmat.m, netlab3.3/conjgrad.m, + netlab3.3/consist.m, netlab3.3/convertoldnet.m, + netlab3.3/datread.m, netlab3.3/datwrite.m, netlab3.3/demard.m, + netlab3.3/demev1.m, netlab3.3/demev2.m, netlab3.3/demev3.m, + netlab3.3/demgauss.m, netlab3.3/demglm1.m, netlab3.3/demglm2.m, + netlab3.3/demgmm2.m, netlab3.3/demgmm3.m, netlab3.3/demgmm4.m, + netlab3.3/demgmm5.m, netlab3.3/demgp.m, netlab3.3/demgpard.m, + netlab3.3/demgpot.m, netlab3.3/demgtm1.m, netlab3.3/demgtm2.m, + netlab3.3/demhint.m, netlab3.3/demhmc2.m, netlab3.3/demhmc3.m, + netlab3.3/demkmn1.m, netlab3.3/demknn1.m, netlab3.3/demmdn1.m, + netlab3.3/demmlp1.m, netlab3.3/demmlp2.m, netlab3.3/demnlab.m, + netlab3.3/demns1.m, netlab3.3/demolgd1.m, netlab3.3/demopt1.m, + netlab3.3/dempot.m, netlab3.3/demprgp.m, netlab3.3/demprior.m, + netlab3.3/demrbf1.m, netlab3.3/demsom1.m, netlab3.3/demtrain.m, + netlab3.3/dist2.m, netlab3.3/eigdec.m, netlab3.3/errbayes.m, + netlab3.3/evidence.m, netlab3.3/fevbayes.m, netlab3.3/gauss.m, + netlab3.3/gbayes.m, netlab3.3/glm.m, netlab3.3/glmderiv.m, + netlab3.3/glmerr.m, netlab3.3/glmevfwd.m, netlab3.3/glmfwd.m, + netlab3.3/glmgrad.m, netlab3.3/glmhess.m, netlab3.3/glminit.m, + netlab3.3/glmpak.m, netlab3.3/glmtrain.m, netlab3.3/glmunpak.m, + netlab3.3/gmm.m, netlab3.3/gmmactiv.m, netlab3.3/gmmem.m, + netlab3.3/gmmpak.m, netlab3.3/gmmpost.m, netlab3.3/gmmprob.m, + netlab3.3/gmmsamp.m, netlab3.3/gmmunpak.m, netlab3.3/gp.m, + netlab3.3/gpcovar.m, netlab3.3/gpcovarf.m, netlab3.3/gpcovarp.m, + netlab3.3/gperr.m, netlab3.3/gpfwd.m, netlab3.3/gpgrad.m, + netlab3.3/gpinit.m, netlab3.3/gppak.m, netlab3.3/gpunpak.m, + netlab3.3/gradchek.m, netlab3.3/graddesc.m, netlab3.3/gsamp.m, + netlab3.3/gtm.m, netlab3.3/gtmem.m, netlab3.3/gtmfwd.m, + netlab3.3/gtminit.m, netlab3.3/gtmlmean.m, netlab3.3/gtmlmode.m, + netlab3.3/gtmmag.m, netlab3.3/gtmpost.m, netlab3.3/gtmprob.m, + netlab3.3/hbayes.m, netlab3.3/hesschek.m, netlab3.3/hintmat.m, + netlab3.3/hinton.m, netlab3.3/histp.m, netlab3.3/hmc.m, + netlab3.3/kmeansNetlab.m, netlab3.3/knn.m, netlab3.3/knnfwd.m, + netlab3.3/linef.m, netlab3.3/linemin.m, netlab3.3/maxitmess.m, + netlab3.3/mdn.m, netlab3.3/mdn2gmm.m, netlab3.3/mdndist2.m, + netlab3.3/mdnerr.m, netlab3.3/mdnfwd.m, netlab3.3/mdngrad.m, + netlab3.3/mdninit.m, netlab3.3/mdnnet.mat, netlab3.3/mdnpak.m, + netlab3.3/mdnpost.m, netlab3.3/mdnprob.m, netlab3.3/mdnunpak.m, + netlab3.3/minbrack.m, netlab3.3/mlp.m, netlab3.3/mlpbkp.m, + netlab3.3/mlpderiv.m, netlab3.3/mlperr.m, netlab3.3/mlpevfwd.m, + netlab3.3/mlpfwd.m, netlab3.3/mlpgrad.m, netlab3.3/mlphdotv.m, + netlab3.3/mlphess.m, netlab3.3/mlphint.m, netlab3.3/mlpinit.m, + netlab3.3/mlppak.m, netlab3.3/mlpprior.m, netlab3.3/mlptrain.m, + netlab3.3/mlpunpak.m, netlab3.3/netderiv.m, netlab3.3/neterr.m, + netlab3.3/netevfwd.m, netlab3.3/netgrad.m, netlab3.3/nethess.m, + netlab3.3/netinit.m, netlab3.3/netlab3.3.zip, + netlab3.3/netlogo.mat, netlab3.3/netopt.m, netlab3.3/netpak.m, + netlab3.3/netunpak.m, netlab3.3/oilTrn.dat, netlab3.3/oilTst.dat, + netlab3.3/olgd.m, netlab3.3/pca.m, netlab3.3/plotmat.m, + netlab3.3/ppca.m, netlab3.3/quasinew.m, netlab3.3/rbf.m, + netlab3.3/rbfbkp.m, netlab3.3/rbfderiv.m, netlab3.3/rbferr.m, + netlab3.3/rbfevfwd.m, netlab3.3/rbffwd.m, netlab3.3/rbfgrad.m, + netlab3.3/rbfhess.m, netlab3.3/rbfjacob.m, netlab3.3/rbfpak.m, + netlab3.3/rbfprior.m, netlab3.3/rbfsetbf.m, netlab3.3/rbfsetfw.m, + netlab3.3/rbftrain.m, netlab3.3/rbfunpak.m, netlab3.3/rosegrad.m, + netlab3.3/rosen.m, netlab3.3/scg.m, netlab3.3/som.m, + netlab3.3/somfwd.m, netlab3.3/sompak.m, netlab3.3/somtrain.m, + netlab3.3/somunpak.m, netlab3.3/xor.dat, netlabKPM/README.txt, + netlabKPM/demgmm1_movie.m, netlabKPM/evidence_weighted.m, + netlabKPM/glmerr_weighted.m, netlabKPM/glmgrad_weighted.m, + netlabKPM/glmhess_weighted.m, netlabKPM/glmtrain_weighted.m, + netlabKPM/gmm1.avi, netlabKPM/gmmem2.m, + netlabKPM/gmmem_multi_restart.m, netlabKPM/kmeans_demo.m, + netlabKPM/mlperr_weighted.m, netlabKPM/mlpgrad_weighted.m, + netlabKPM/mlphdotv_weighted.m, netlabKPM/mlphess_weighted.m, + netlabKPM/neterr_weighted.m, netlabKPM/netgrad_weighted.m, + netlabKPM/nethess_weighted.m, netlabKPM/netopt_weighted.m, + netlabKPM/process_options.m: Initial import of code base from + Kevin Murphy. + +2005-04-27 10:58 yozhik + + * KPMtools/mahal2conf.m, nethelp3.3/conffig.htm, + nethelp3.3/confmat.htm, nethelp3.3/conjgrad.htm, + nethelp3.3/consist.htm, nethelp3.3/convertoldnet.htm, + nethelp3.3/datread.htm, nethelp3.3/datwrite.htm, + nethelp3.3/dem2ddat.htm, nethelp3.3/demard.htm, + nethelp3.3/demev1.htm, nethelp3.3/demev2.htm, + nethelp3.3/demev3.htm, nethelp3.3/demgauss.htm, + nethelp3.3/demglm1.htm, nethelp3.3/demglm2.htm, + nethelp3.3/demgmm1.htm, nethelp3.3/demgmm2.htm, + nethelp3.3/demgmm3.htm, nethelp3.3/demgmm4.htm, + nethelp3.3/demgmm5.htm, nethelp3.3/demgp.htm, + nethelp3.3/demgpard.htm, nethelp3.3/demgpot.htm, + nethelp3.3/demgtm1.htm, nethelp3.3/demgtm2.htm, + nethelp3.3/demhint.htm, nethelp3.3/demhmc1.htm, + nethelp3.3/demhmc2.htm, nethelp3.3/demhmc3.htm, + nethelp3.3/demkmn1.htm, nethelp3.3/demknn1.htm, + nethelp3.3/demmdn1.htm, nethelp3.3/demmet1.htm, + nethelp3.3/demmlp1.htm, nethelp3.3/demmlp2.htm, + nethelp3.3/demnlab.htm, nethelp3.3/demns1.htm, + nethelp3.3/demolgd1.htm, nethelp3.3/demopt1.htm, + nethelp3.3/dempot.htm, nethelp3.3/demprgp.htm, + nethelp3.3/demprior.htm, nethelp3.3/demrbf1.htm, + nethelp3.3/demsom1.htm, nethelp3.3/demtrain.htm, + nethelp3.3/dist2.htm, nethelp3.3/eigdec.htm, + nethelp3.3/errbayes.htm, nethelp3.3/evidence.htm, + nethelp3.3/fevbayes.htm, nethelp3.3/gauss.htm, + nethelp3.3/gbayes.htm, nethelp3.3/glm.htm, + nethelp3.3/glmderiv.htm, nethelp3.3/glmerr.htm, + nethelp3.3/glmevfwd.htm, nethelp3.3/glmfwd.htm, + nethelp3.3/glmgrad.htm, nethelp3.3/glmhess.htm, + nethelp3.3/glminit.htm, nethelp3.3/glmpak.htm, + nethelp3.3/glmtrain.htm, nethelp3.3/glmunpak.htm, + nethelp3.3/gmm.htm, nethelp3.3/gmmactiv.htm, + nethelp3.3/gmmem.htm, nethelp3.3/gmminit.htm, + nethelp3.3/gmmpak.htm, nethelp3.3/gmmpost.htm, + nethelp3.3/gmmprob.htm, nethelp3.3/gmmsamp.htm, + nethelp3.3/gmmunpak.htm, nethelp3.3/gp.htm, + nethelp3.3/gpcovar.htm, nethelp3.3/gpcovarf.htm, + nethelp3.3/gpcovarp.htm, nethelp3.3/gperr.htm, + nethelp3.3/gpfwd.htm, nethelp3.3/gpgrad.htm, + nethelp3.3/gpinit.htm, nethelp3.3/gppak.htm, + nethelp3.3/gpunpak.htm, nethelp3.3/gradchek.htm, + nethelp3.3/graddesc.htm, nethelp3.3/gsamp.htm, + nethelp3.3/gtm.htm, nethelp3.3/gtmem.htm, nethelp3.3/gtmfwd.htm, + nethelp3.3/gtminit.htm, nethelp3.3/gtmlmean.htm, + nethelp3.3/gtmlmode.htm, nethelp3.3/gtmmag.htm, + nethelp3.3/gtmpost.htm, nethelp3.3/gtmprob.htm, + nethelp3.3/hbayes.htm, nethelp3.3/hesschek.htm, + nethelp3.3/hintmat.htm, nethelp3.3/hinton.htm, + nethelp3.3/histp.htm, nethelp3.3/hmc.htm, nethelp3.3/index.htm, + nethelp3.3/kmeans.htm, nethelp3.3/knn.htm, nethelp3.3/knnfwd.htm, + nethelp3.3/linef.htm, nethelp3.3/linemin.htm, + nethelp3.3/maxitmess.htm, nethelp3.3/mdn.htm, + nethelp3.3/mdn2gmm.htm, nethelp3.3/mdndist2.htm, + nethelp3.3/mdnerr.htm, nethelp3.3/mdnfwd.htm, + nethelp3.3/mdngrad.htm, nethelp3.3/mdninit.htm, + nethelp3.3/mdnpak.htm, nethelp3.3/mdnpost.htm, + nethelp3.3/mdnprob.htm, nethelp3.3/mdnunpak.htm, + nethelp3.3/metrop.htm, nethelp3.3/minbrack.htm, + nethelp3.3/mlp.htm, nethelp3.3/mlpbkp.htm, + nethelp3.3/mlpderiv.htm, nethelp3.3/mlperr.htm, + nethelp3.3/mlpevfwd.htm, nethelp3.3/mlpfwd.htm, + nethelp3.3/mlpgrad.htm, nethelp3.3/mlphdotv.htm, + nethelp3.3/mlphess.htm, nethelp3.3/mlphint.htm, + nethelp3.3/mlpinit.htm, nethelp3.3/mlppak.htm, + nethelp3.3/mlpprior.htm, nethelp3.3/mlptrain.htm, + nethelp3.3/mlpunpak.htm, nethelp3.3/netderiv.htm, + nethelp3.3/neterr.htm, nethelp3.3/netevfwd.htm, + nethelp3.3/netgrad.htm, nethelp3.3/nethelp3.3.zip, + nethelp3.3/nethess.htm, nethelp3.3/netinit.htm, + nethelp3.3/netopt.htm, nethelp3.3/netpak.htm, + nethelp3.3/netunpak.htm, nethelp3.3/olgd.htm, nethelp3.3/pca.htm, + nethelp3.3/plotmat.htm, nethelp3.3/ppca.htm, + nethelp3.3/quasinew.htm, nethelp3.3/rbf.htm, + nethelp3.3/rbfbkp.htm, nethelp3.3/rbfderiv.htm, + nethelp3.3/rbferr.htm, nethelp3.3/rbfevfwd.htm, + nethelp3.3/rbffwd.htm, nethelp3.3/rbfgrad.htm, + nethelp3.3/rbfhess.htm, nethelp3.3/rbfjacob.htm, + nethelp3.3/rbfpak.htm, nethelp3.3/rbfprior.htm, + nethelp3.3/rbfsetbf.htm, nethelp3.3/rbfsetfw.htm, + nethelp3.3/rbftrain.htm, nethelp3.3/rbfunpak.htm, + nethelp3.3/rosegrad.htm, nethelp3.3/rosen.htm, + nethelp3.3/scg.htm, nethelp3.3/som.htm, nethelp3.3/somfwd.htm, + nethelp3.3/sompak.htm, nethelp3.3/somtrain.htm, + nethelp3.3/somunpak.htm, netlab3.3/Contents.m, netlab3.3/LICENSE, + netlab3.3/conffig.m, netlab3.3/confmat.m, netlab3.3/conjgrad.m, + netlab3.3/consist.m, netlab3.3/convertoldnet.m, + netlab3.3/datread.m, netlab3.3/datwrite.m, netlab3.3/demard.m, + netlab3.3/demev1.m, netlab3.3/demev2.m, netlab3.3/demev3.m, + netlab3.3/demgauss.m, netlab3.3/demglm1.m, netlab3.3/demglm2.m, + netlab3.3/demgmm2.m, netlab3.3/demgmm3.m, netlab3.3/demgmm4.m, + netlab3.3/demgmm5.m, netlab3.3/demgp.m, netlab3.3/demgpard.m, + netlab3.3/demgpot.m, netlab3.3/demgtm1.m, netlab3.3/demgtm2.m, + netlab3.3/demhint.m, netlab3.3/demhmc2.m, netlab3.3/demhmc3.m, + netlab3.3/demkmn1.m, netlab3.3/demknn1.m, netlab3.3/demmdn1.m, + netlab3.3/demmlp1.m, netlab3.3/demmlp2.m, netlab3.3/demnlab.m, + netlab3.3/demns1.m, netlab3.3/demolgd1.m, netlab3.3/demopt1.m, + netlab3.3/dempot.m, netlab3.3/demprgp.m, netlab3.3/demprior.m, + netlab3.3/demrbf1.m, netlab3.3/demsom1.m, netlab3.3/demtrain.m, + netlab3.3/dist2.m, netlab3.3/eigdec.m, netlab3.3/errbayes.m, + netlab3.3/evidence.m, netlab3.3/fevbayes.m, netlab3.3/gauss.m, + netlab3.3/gbayes.m, netlab3.3/glm.m, netlab3.3/glmderiv.m, + netlab3.3/glmerr.m, netlab3.3/glmevfwd.m, netlab3.3/glmfwd.m, + netlab3.3/glmgrad.m, netlab3.3/glmhess.m, netlab3.3/glminit.m, + netlab3.3/glmpak.m, netlab3.3/glmtrain.m, netlab3.3/glmunpak.m, + netlab3.3/gmm.m, netlab3.3/gmmactiv.m, netlab3.3/gmmem.m, + netlab3.3/gmmpak.m, netlab3.3/gmmpost.m, netlab3.3/gmmprob.m, + netlab3.3/gmmsamp.m, netlab3.3/gmmunpak.m, netlab3.3/gp.m, + netlab3.3/gpcovar.m, netlab3.3/gpcovarf.m, netlab3.3/gpcovarp.m, + netlab3.3/gperr.m, netlab3.3/gpfwd.m, netlab3.3/gpgrad.m, + netlab3.3/gpinit.m, netlab3.3/gppak.m, netlab3.3/gpunpak.m, + netlab3.3/gradchek.m, netlab3.3/graddesc.m, netlab3.3/gsamp.m, + netlab3.3/gtm.m, netlab3.3/gtmem.m, netlab3.3/gtmfwd.m, + netlab3.3/gtminit.m, netlab3.3/gtmlmean.m, netlab3.3/gtmlmode.m, + netlab3.3/gtmmag.m, netlab3.3/gtmpost.m, netlab3.3/gtmprob.m, + netlab3.3/hbayes.m, netlab3.3/hesschek.m, netlab3.3/hintmat.m, + netlab3.3/hinton.m, netlab3.3/histp.m, netlab3.3/hmc.m, + netlab3.3/kmeansNetlab.m, netlab3.3/knn.m, netlab3.3/knnfwd.m, + netlab3.3/linef.m, netlab3.3/linemin.m, netlab3.3/maxitmess.m, + netlab3.3/mdn.m, netlab3.3/mdn2gmm.m, netlab3.3/mdndist2.m, + netlab3.3/mdnerr.m, netlab3.3/mdnfwd.m, netlab3.3/mdngrad.m, + netlab3.3/mdninit.m, netlab3.3/mdnnet.mat, netlab3.3/mdnpak.m, + netlab3.3/mdnpost.m, netlab3.3/mdnprob.m, netlab3.3/mdnunpak.m, + netlab3.3/minbrack.m, netlab3.3/mlp.m, netlab3.3/mlpbkp.m, + netlab3.3/mlpderiv.m, netlab3.3/mlperr.m, netlab3.3/mlpevfwd.m, + netlab3.3/mlpfwd.m, netlab3.3/mlpgrad.m, netlab3.3/mlphdotv.m, + netlab3.3/mlphess.m, netlab3.3/mlphint.m, netlab3.3/mlpinit.m, + netlab3.3/mlppak.m, netlab3.3/mlpprior.m, netlab3.3/mlptrain.m, + netlab3.3/mlpunpak.m, netlab3.3/netderiv.m, netlab3.3/neterr.m, + netlab3.3/netevfwd.m, netlab3.3/netgrad.m, netlab3.3/nethess.m, + netlab3.3/netinit.m, netlab3.3/netlab3.3.zip, + netlab3.3/netlogo.mat, netlab3.3/netopt.m, netlab3.3/netpak.m, + netlab3.3/netunpak.m, netlab3.3/oilTrn.dat, netlab3.3/oilTst.dat, + netlab3.3/olgd.m, netlab3.3/pca.m, netlab3.3/plotmat.m, + netlab3.3/ppca.m, netlab3.3/quasinew.m, netlab3.3/rbf.m, + netlab3.3/rbfbkp.m, netlab3.3/rbfderiv.m, netlab3.3/rbferr.m, + netlab3.3/rbfevfwd.m, netlab3.3/rbffwd.m, netlab3.3/rbfgrad.m, + netlab3.3/rbfhess.m, netlab3.3/rbfjacob.m, netlab3.3/rbfpak.m, + netlab3.3/rbfprior.m, netlab3.3/rbfsetbf.m, netlab3.3/rbfsetfw.m, + netlab3.3/rbftrain.m, netlab3.3/rbfunpak.m, netlab3.3/rosegrad.m, + netlab3.3/rosen.m, netlab3.3/scg.m, netlab3.3/som.m, + netlab3.3/somfwd.m, netlab3.3/sompak.m, netlab3.3/somtrain.m, + netlab3.3/somunpak.m, netlab3.3/xor.dat, netlabKPM/README.txt, + netlabKPM/demgmm1_movie.m, netlabKPM/evidence_weighted.m, + netlabKPM/glmerr_weighted.m, netlabKPM/glmgrad_weighted.m, + netlabKPM/glmhess_weighted.m, netlabKPM/glmtrain_weighted.m, + netlabKPM/gmm1.avi, netlabKPM/gmmem2.m, + netlabKPM/gmmem_multi_restart.m, netlabKPM/kmeans_demo.m, + netlabKPM/mlperr_weighted.m, netlabKPM/mlpgrad_weighted.m, + netlabKPM/mlphdotv_weighted.m, netlabKPM/mlphess_weighted.m, + netlabKPM/neterr_weighted.m, netlabKPM/netgrad_weighted.m, + netlabKPM/nethess_weighted.m, netlabKPM/netopt_weighted.m, + netlabKPM/process_options.m: Initial revision + +2005-04-25 19:29 yozhik + + * KPMstats/KLgauss.m, KPMstats/README.txt, KPMstats/beta_sample.m, + KPMstats/chisquared_histo.m, KPMstats/chisquared_prob.m, + KPMstats/chisquared_readme.txt, KPMstats/chisquared_table.m, + KPMstats/clg_Mstep.m, KPMstats/clg_Mstep_simple.m, + KPMstats/clg_prob.m, KPMstats/condGaussToJoint.m, + KPMstats/cond_indep_fisher_z.m, + KPMstats/condgaussTrainObserved.m, KPMstats/condgauss_sample.m, + KPMstats/convertBinaryLabels.m, KPMstats/cwr_demo.m, + KPMstats/cwr_em.m, KPMstats/cwr_predict.m, KPMstats/cwr_prob.m, + KPMstats/cwr_readme.txt, KPMstats/cwr_test.m, + KPMstats/dirichlet_sample.m, KPMstats/distchck.m, + KPMstats/eigdec.m, KPMstats/est_transmat.m, + KPMstats/fit_paritioned_model_testfn.m, + KPMstats/fit_partitioned_model.m, KPMstats/gamma_sample.m, + KPMstats/gaussian_prob.m, KPMstats/gaussian_sample.m, + KPMstats/linear_regression.m, KPMstats/logist2.m, + KPMstats/logist2Apply.m, KPMstats/logist2ApplyRegularized.m, + KPMstats/logist2Fit.m, KPMstats/logist2FitRegularized.m, + KPMstats/logistK.m, KPMstats/logistK_eval.m, + KPMstats/marginalize_gaussian.m, KPMstats/matrix_T_pdf.m, + KPMstats/matrix_normal_pdf.m, KPMstats/mc_stat_distrib.m, + KPMstats/mixgauss_Mstep.m, KPMstats/mixgauss_classifier_apply.m, + KPMstats/mixgauss_classifier_train.m, KPMstats/mixgauss_em.m, + KPMstats/mixgauss_init.m, KPMstats/mixgauss_prob.m, + KPMstats/mixgauss_prob_test.m, KPMstats/mixgauss_sample.m, + KPMstats/mkPolyFvec.m, KPMstats/mk_unit_norm.m, + KPMstats/multinomial_prob.m, KPMstats/multinomial_sample.m, + KPMstats/normal_coef.m, KPMstats/partial_corr_coef.m, + KPMstats/parzen.m, KPMstats/parzenC.c, KPMstats/parzenC.dll, + KPMstats/parzenC.mexglx, KPMstats/parzenC_test.m, + KPMstats/parzen_fit_select_unif.m, KPMstats/pca.m, + KPMstats/rndcheck.m, KPMstats/sample.m, + KPMstats/sample_discrete.m, KPMstats/sample_gaussian.m, + KPMstats/student_t_logprob.m, KPMstats/student_t_prob.m, + KPMstats/unif_discrete_sample.m, KPMstats/weightedRegression.m, + KPMtools/README.txt, KPMtools/approx_unique.m, + KPMtools/approxeq.m, KPMtools/argmax.m, KPMtools/argmin.m, + KPMtools/assert.m, KPMtools/assignEdgeNums.m, + KPMtools/assign_cols.m, KPMtools/axis_pct.m, KPMtools/block.m, + KPMtools/cell2num.m, KPMtools/chi2inv.m, KPMtools/choose.m, + KPMtools/collapse_mog.m, KPMtools/colmult.c, + KPMtools/colmult.mexglx, KPMtools/computeROC.m, + KPMtools/compute_counts.m, KPMtools/conf2mahal.m, + KPMtools/cross_entropy.m, KPMtools/div.m, KPMtools/draw_circle.m, + KPMtools/draw_ellipse.m, KPMtools/draw_ellipse_axes.m, + KPMtools/em_converged.m, KPMtools/entropy.m, + KPMtools/exportfig.m, KPMtools/extend_domain_table.m, + KPMtools/factorial.m, KPMtools/find_equiv_posns.m, + KPMtools/hash_add.m, KPMtools/hash_del.m, KPMtools/hash_lookup.m, + KPMtools/hungarian.m, KPMtools/image_rgb.m, + KPMtools/imresizeAspect.m, KPMtools/ind2subv.c, + KPMtools/ind2subv.m, KPMtools/installC_KPMtools.m, + KPMtools/is_psd.m, KPMtools/is_stochastic.m, + KPMtools/isemptycell.m, KPMtools/isposdef.m, KPMtools/isscalar.m, + KPMtools/isvector.m, KPMtools/junk.c, KPMtools/loadcell.m, + KPMtools/logb.m, KPMtools/logdet.m, KPMtools/logsum.m, + KPMtools/logsum_simple.m, KPMtools/logsum_test.m, + KPMtools/logsumexp.m, KPMtools/logsumexpv.m, + KPMtools/marg_table.m, KPMtools/marginalize_table.m, + KPMtools/matprint.m, KPMtools/max_mult.c, KPMtools/max_mult.m, + KPMtools/mexutil.c, KPMtools/mexutil.h, + KPMtools/mk_multi_index.m, KPMtools/mk_stochastic.m, + KPMtools/mult_by_table.m, KPMtools/myintersect.m, + KPMtools/myismember.m, KPMtools/myones.m, KPMtools/myplot.m, + KPMtools/myrand.m, KPMtools/myrepmat.m, KPMtools/myreshape.m, + KPMtools/mysetdiff.m, KPMtools/mysize.m, KPMtools/mysubset.m, + KPMtools/mysymsetdiff.m, KPMtools/bipartiteMatchingHungarian.m, + KPMtools/myunion.m, KPMtools/nchoose2.m, KPMtools/ncols.m, + KPMtools/nonmaxsup.m, KPMtools/normalise.m, + KPMtools/normaliseC.c, KPMtools/normaliseC.dll, + KPMtools/normalize.m, KPMtools/nrows.m, KPMtools/num2strcell.m, + KPMtools/partitionData.m, KPMtools/partition_matrix_vec.m, + KPMtools/pca_netlab.m, KPMtools/pick.m, KPMtools/plotROC.m, + KPMtools/plotROCkpm.m, KPMtools/plot_axis_thru_origin.m, + KPMtools/plot_ellipse.m, KPMtools/plot_matrix.m, + KPMtools/plot_polygon.m, KPMtools/plotcov2.m, + KPMtools/plotcov3.m, KPMtools/plotgauss1d.m, + KPMtools/plotgauss2d.m, KPMtools/plotgauss2d_old.m, + KPMtools/polygon_area.m, KPMtools/polygon_centroid.m, + KPMtools/polygon_intersect.m, KPMtools/previewfig.m, + KPMtools/process_options.m, KPMtools/rand_psd.m, + KPMtools/rectintC.m, KPMtools/rectintLoopC.mexglx, + KPMtools/rectintSparse.m, KPMtools/rectintSparseC.m, + KPMtools/rectintSparseLoopC.c, KPMtools/rectintSparseLoopC.dll, + KPMtools/repmatC.c, KPMtools/repmatC.dll, + KPMtools/repmatC.mexglx, KPMtools/rgb2grayKPM.m, + KPMtools/rnd_partition.m, KPMtools/rotate_xlabel.m, + KPMtools/safeStr.m, KPMtools/sampleUniformInts.m, + KPMtools/sample_discrete.m, KPMtools/set_xtick_label.m, + KPMtools/set_xtick_label_demo.m, KPMtools/setdiag.m, + KPMtools/softeye.m, KPMtools/sort_evec.m, + KPMtools/splitLongSeqIntoManyShort.m, KPMtools/sprintf_intvec.m, + KPMtools/sqdist.m, KPMtools/strmatch_multi.m, + KPMtools/strmatch_substr.m, KPMtools/subplot2.m, + KPMtools/subplot3.m, KPMtools/subsets.m, KPMtools/subv2ind.c, + KPMtools/subv2ind.m, KPMtools/sumv.m, KPMtools/suptitle.m, + KPMtools/unaryEncoding.m, KPMtools/wrap.m, + KPMtools/xticklabel_rotate90.m, KPMtools/zipload.m, + KPMtools/zipsave.m: Initial import of code base from Kevin + Murphy. + +2005-04-25 19:29 yozhik + + * KPMstats/KLgauss.m, KPMstats/README.txt, KPMstats/beta_sample.m, + KPMstats/chisquared_histo.m, KPMstats/chisquared_prob.m, + KPMstats/chisquared_readme.txt, KPMstats/chisquared_table.m, + KPMstats/clg_Mstep.m, KPMstats/clg_Mstep_simple.m, + KPMstats/clg_prob.m, KPMstats/condGaussToJoint.m, + KPMstats/cond_indep_fisher_z.m, + KPMstats/condgaussTrainObserved.m, KPMstats/condgauss_sample.m, + KPMstats/convertBinaryLabels.m, KPMstats/cwr_demo.m, + KPMstats/cwr_em.m, KPMstats/cwr_predict.m, KPMstats/cwr_prob.m, + KPMstats/cwr_readme.txt, KPMstats/cwr_test.m, + KPMstats/dirichlet_sample.m, KPMstats/distchck.m, + KPMstats/eigdec.m, KPMstats/est_transmat.m, + KPMstats/fit_paritioned_model_testfn.m, + KPMstats/fit_partitioned_model.m, KPMstats/gamma_sample.m, + KPMstats/gaussian_prob.m, KPMstats/gaussian_sample.m, + KPMstats/linear_regression.m, KPMstats/logist2.m, + KPMstats/logist2Apply.m, KPMstats/logist2ApplyRegularized.m, + KPMstats/logist2Fit.m, KPMstats/logist2FitRegularized.m, + KPMstats/logistK.m, KPMstats/logistK_eval.m, + KPMstats/marginalize_gaussian.m, KPMstats/matrix_T_pdf.m, + KPMstats/matrix_normal_pdf.m, KPMstats/mc_stat_distrib.m, + KPMstats/mixgauss_Mstep.m, KPMstats/mixgauss_classifier_apply.m, + KPMstats/mixgauss_classifier_train.m, KPMstats/mixgauss_em.m, + KPMstats/mixgauss_init.m, KPMstats/mixgauss_prob.m, + KPMstats/mixgauss_prob_test.m, KPMstats/mixgauss_sample.m, + KPMstats/mkPolyFvec.m, KPMstats/mk_unit_norm.m, + KPMstats/multinomial_prob.m, KPMstats/multinomial_sample.m, + KPMstats/normal_coef.m, KPMstats/partial_corr_coef.m, + KPMstats/parzen.m, KPMstats/parzenC.c, KPMstats/parzenC.dll, + KPMstats/parzenC.mexglx, KPMstats/parzenC_test.m, + KPMstats/parzen_fit_select_unif.m, KPMstats/pca.m, + KPMstats/rndcheck.m, KPMstats/sample.m, + KPMstats/sample_discrete.m, KPMstats/sample_gaussian.m, + KPMstats/student_t_logprob.m, KPMstats/student_t_prob.m, + KPMstats/unif_discrete_sample.m, KPMstats/weightedRegression.m, + KPMtools/README.txt, KPMtools/approx_unique.m, + KPMtools/approxeq.m, KPMtools/argmax.m, KPMtools/argmin.m, + KPMtools/assert.m, KPMtools/assignEdgeNums.m, + KPMtools/assign_cols.m, KPMtools/axis_pct.m, KPMtools/block.m, + KPMtools/cell2num.m, KPMtools/chi2inv.m, KPMtools/choose.m, + KPMtools/collapse_mog.m, KPMtools/colmult.c, + KPMtools/colmult.mexglx, KPMtools/computeROC.m, + KPMtools/compute_counts.m, KPMtools/conf2mahal.m, + KPMtools/cross_entropy.m, KPMtools/div.m, KPMtools/draw_circle.m, + KPMtools/draw_ellipse.m, KPMtools/draw_ellipse_axes.m, + KPMtools/em_converged.m, KPMtools/entropy.m, + KPMtools/exportfig.m, KPMtools/extend_domain_table.m, + KPMtools/factorial.m, KPMtools/find_equiv_posns.m, + KPMtools/hash_add.m, KPMtools/hash_del.m, KPMtools/hash_lookup.m, + KPMtools/hungarian.m, KPMtools/image_rgb.m, + KPMtools/imresizeAspect.m, KPMtools/ind2subv.c, + KPMtools/ind2subv.m, KPMtools/installC_KPMtools.m, + KPMtools/is_psd.m, KPMtools/is_stochastic.m, + KPMtools/isemptycell.m, KPMtools/isposdef.m, KPMtools/isscalar.m, + KPMtools/isvector.m, KPMtools/junk.c, KPMtools/loadcell.m, + KPMtools/logb.m, KPMtools/logdet.m, KPMtools/logsum.m, + KPMtools/logsum_simple.m, KPMtools/logsum_test.m, + KPMtools/logsumexp.m, KPMtools/logsumexpv.m, + KPMtools/marg_table.m, KPMtools/marginalize_table.m, + KPMtools/matprint.m, KPMtools/max_mult.c, KPMtools/max_mult.m, + KPMtools/mexutil.c, KPMtools/mexutil.h, + KPMtools/mk_multi_index.m, KPMtools/mk_stochastic.m, + KPMtools/mult_by_table.m, KPMtools/myintersect.m, + KPMtools/myismember.m, KPMtools/myones.m, KPMtools/myplot.m, + KPMtools/myrand.m, KPMtools/myrepmat.m, KPMtools/myreshape.m, + KPMtools/mysetdiff.m, KPMtools/mysize.m, KPMtools/mysubset.m, + KPMtools/mysymsetdiff.m, KPMtools/bipartiteMatchingHungarian.m, + KPMtools/myunion.m, KPMtools/nchoose2.m, KPMtools/ncols.m, + KPMtools/nonmaxsup.m, KPMtools/normalise.m, + KPMtools/normaliseC.c, KPMtools/normaliseC.dll, + KPMtools/normalize.m, KPMtools/nrows.m, KPMtools/num2strcell.m, + KPMtools/partitionData.m, KPMtools/partition_matrix_vec.m, + KPMtools/pca_netlab.m, KPMtools/pick.m, KPMtools/plotROC.m, + KPMtools/plotROCkpm.m, KPMtools/plot_axis_thru_origin.m, + KPMtools/plot_ellipse.m, KPMtools/plot_matrix.m, + KPMtools/plot_polygon.m, KPMtools/plotcov2.m, + KPMtools/plotcov3.m, KPMtools/plotgauss1d.m, + KPMtools/plotgauss2d.m, KPMtools/plotgauss2d_old.m, + KPMtools/polygon_area.m, KPMtools/polygon_centroid.m, + KPMtools/polygon_intersect.m, KPMtools/previewfig.m, + KPMtools/process_options.m, KPMtools/rand_psd.m, + KPMtools/rectintC.m, KPMtools/rectintLoopC.mexglx, + KPMtools/rectintSparse.m, KPMtools/rectintSparseC.m, + KPMtools/rectintSparseLoopC.c, KPMtools/rectintSparseLoopC.dll, + KPMtools/repmatC.c, KPMtools/repmatC.dll, + KPMtools/repmatC.mexglx, KPMtools/rgb2grayKPM.m, + KPMtools/rnd_partition.m, KPMtools/rotate_xlabel.m, + KPMtools/safeStr.m, KPMtools/sampleUniformInts.m, + KPMtools/sample_discrete.m, KPMtools/set_xtick_label.m, + KPMtools/set_xtick_label_demo.m, KPMtools/setdiag.m, + KPMtools/softeye.m, KPMtools/sort_evec.m, + KPMtools/splitLongSeqIntoManyShort.m, KPMtools/sprintf_intvec.m, + KPMtools/sqdist.m, KPMtools/strmatch_multi.m, + KPMtools/strmatch_substr.m, KPMtools/subplot2.m, + KPMtools/subplot3.m, KPMtools/subsets.m, KPMtools/subv2ind.c, + KPMtools/subv2ind.m, KPMtools/sumv.m, KPMtools/suptitle.m, + KPMtools/unaryEncoding.m, KPMtools/wrap.m, + KPMtools/xticklabel_rotate90.m, KPMtools/zipload.m, + KPMtools/zipsave.m: Initial revision + +2005-04-03 18:39 yozhik + + * BNT/learning/score_dags.m: Initial import of code base from Kevin + Murphy. + +2005-04-03 18:39 yozhik + + * BNT/learning/score_dags.m: Initial revision + +2005-03-31 11:20 yozhik + + * BNT/installC_BNT.m: Initial import of code base from Kevin + Murphy. + +2005-03-31 11:20 yozhik + + * BNT/installC_BNT.m: Initial revision + +2005-03-30 11:59 yozhik + + * KPMtools/asdemo.html: Initial import of code base from Kevin + Murphy. + +2005-03-30 11:59 yozhik + + * KPMtools/asdemo.html: Initial revision + +2005-03-26 18:51 yozhik + + * KPMtools/asdemo.m: Initial import of code base from Kevin Murphy. + +2005-03-26 18:51 yozhik + + * KPMtools/asdemo.m: Initial revision + +2005-01-15 18:27 yozhik + + * BNT/CPDs/@tabular_CPD/: get_field.m, set_fields.m, tabular_CPD.m: + Initial import of code base from Kevin Murphy. + +2005-01-15 18:27 yozhik + + * BNT/CPDs/@tabular_CPD/: get_field.m, set_fields.m, tabular_CPD.m: + Initial revision + +2004-11-24 12:12 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/junk: Initial import of + code base from Kevin Murphy. + +2004-11-24 12:12 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/junk: Initial revision + +2004-11-22 14:41 yozhik + + * BNT/examples/dynamic/orig_water1.m: Initial import of code base + from Kevin Murphy. + +2004-11-22 14:41 yozhik + + * BNT/examples/dynamic/orig_water1.m: Initial revision + +2004-11-22 14:15 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/cbk_inf_engine.m: Initial + import of code base from Kevin Murphy. + +2004-11-22 14:15 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/cbk_inf_engine.m: Initial + revision + +2004-11-06 13:52 yozhik + + * BNT/examples/static/StructLearn/model_select2.m: Initial import + of code base from Kevin Murphy. + +2004-11-06 13:52 yozhik + + * BNT/examples/static/StructLearn/model_select2.m: Initial revision + +2004-11-06 12:55 yozhik + + * BNT/examples/static/StructLearn/model_select1.m: Initial import + of code base from Kevin Murphy. + +2004-11-06 12:55 yozhik + + * BNT/examples/static/StructLearn/model_select1.m: Initial revision + +2004-10-22 18:18 yozhik + + * HMM/viterbi_path.m: Initial import of code base from Kevin + Murphy. + +2004-10-22 18:18 yozhik + + * HMM/viterbi_path.m: Initial revision + +2004-09-29 20:09 yozhik + + * BNT/inference/static/@var_elim_inf_engine/marginal_nodes.m: + Initial import of code base from Kevin Murphy. + +2004-09-29 20:09 yozhik + + * BNT/inference/static/@var_elim_inf_engine/marginal_nodes.m: + Initial revision + +2004-09-12 20:21 yozhik + + * BNT/examples/limids/amnio.m: Initial import of code base from + Kevin Murphy. + +2004-09-12 20:21 yozhik + + * BNT/examples/limids/amnio.m: Initial revision + +2004-09-12 19:27 yozhik + + * BNT/examples/limids/oil1.m: Initial import of code base from + Kevin Murphy. + +2004-09-12 19:27 yozhik + + * BNT/examples/limids/oil1.m: Initial revision + +2004-09-12 14:01 yozhik + + * BNT/examples/static/sprinkler1.m: Initial import of code base + from Kevin Murphy. + +2004-09-12 14:01 yozhik + + * BNT/examples/static/sprinkler1.m: Initial revision + +2004-08-29 05:41 yozhik + + * HMM/transmat_train_observed.m: Initial import of code base from + Kevin Murphy. + +2004-08-29 05:41 yozhik + + * HMM/transmat_train_observed.m: Initial revision + +2004-08-05 08:25 yozhik + + * BNT/potentials/: @dpot/divide_by_pot.m, Tables/divide_by_table.m: + Initial import of code base from Kevin Murphy. + +2004-08-05 08:25 yozhik + + * BNT/potentials/: @dpot/divide_by_pot.m, Tables/divide_by_table.m: + Initial revision + +2004-08-04 12:59 yozhik + + * BNT/potentials/@dpot/: marginalize_pot.m, multiply_by_pot.m: + Initial import of code base from Kevin Murphy. + +2004-08-04 12:59 yozhik + + * BNT/potentials/@dpot/: marginalize_pot.m, multiply_by_pot.m: + Initial revision + +2004-08-04 12:36 yozhik + + * BNT/@assocarray/subsref.m: Initial import of code base from Kevin + Murphy. + +2004-08-04 12:36 yozhik + + * BNT/@assocarray/subsref.m: Initial revision + +2004-08-04 08:54 yozhik + + * BNT/potentials/@dpot/normalize_pot.m: Initial import of code base + from Kevin Murphy. + +2004-08-04 08:54 yozhik + + * BNT/potentials/@dpot/normalize_pot.m: Initial revision + +2004-08-04 08:51 yozhik + + * BNT/potentials/Tables/: marg_table.m, mult_by_table.m, + extend_domain_table.m: Initial import of code base from Kevin + Murphy. + +2004-08-04 08:51 yozhik + + * BNT/potentials/Tables/: marg_table.m, mult_by_table.m, + extend_domain_table.m: Initial revision + +2004-08-02 15:23 yozhik + + * BNT/CPDs/@noisyor_CPD/CPD_to_CPT.m: Initial import of code base + from Kevin Murphy. + +2004-08-02 15:23 yozhik + + * BNT/CPDs/@noisyor_CPD/CPD_to_CPT.m: Initial revision + +2004-08-02 15:05 yozhik + + * BNT/general/noisyORtoTable.m: Initial import of code base from + Kevin Murphy. + +2004-08-02 15:05 yozhik + + * BNT/general/noisyORtoTable.m: Initial revision + +2004-06-29 10:46 yozhik + + * BNT/learning/learn_struct_pdag_pc.m: Initial import of code base + from Kevin Murphy. + +2004-06-29 10:46 yozhik + + * BNT/learning/learn_struct_pdag_pc.m: Initial revision + +2004-06-15 10:50 yozhik + + * GraphViz/graph_to_dot.m: Initial import of code base from Kevin + Murphy. + +2004-06-15 10:50 yozhik + + * GraphViz/graph_to_dot.m: Initial revision + +2004-06-11 14:16 yozhik + + * BNT/CPDs/@tabular_CPD/log_marg_prob_node.m: Initial import of + code base from Kevin Murphy. + +2004-06-11 14:16 yozhik + + * BNT/CPDs/@tabular_CPD/log_marg_prob_node.m: Initial revision + +2004-06-09 18:56 yozhik + + * BNT/README.txt: Initial import of code base from Kevin Murphy. + +2004-06-09 18:56 yozhik + + * BNT/README.txt: Initial revision + +2004-06-09 18:53 yozhik + + * BNT/CPDs/@generic_CPD/learn_params.m: Initial import of code base + from Kevin Murphy. + +2004-06-09 18:53 yozhik + + * BNT/CPDs/@generic_CPD/learn_params.m: Initial revision + +2004-06-09 18:42 yozhik + + * BNT/examples/static/nodeorderExample.m: Initial import of code + base from Kevin Murphy. + +2004-06-09 18:42 yozhik + + * BNT/examples/static/nodeorderExample.m: Initial revision + +2004-06-09 18:33 yozhik + + * BNT/: learning/score_family.m, test_BNT.m: Initial import of code + base from Kevin Murphy. + +2004-06-09 18:33 yozhik + + * BNT/: learning/score_family.m, test_BNT.m: Initial revision + +2004-06-09 18:28 yozhik + + * BNT/: learning/learn_params.m, CPDs/@gaussian_CPD/learn_params.m, + examples/static/gaussian2.m: Initial import of code base from + Kevin Murphy. + +2004-06-09 18:28 yozhik + + * BNT/: learning/learn_params.m, CPDs/@gaussian_CPD/learn_params.m, + examples/static/gaussian2.m: Initial revision + +2004-06-09 18:25 yozhik + + * BNT/CPDs/@tabular_CPD/learn_params.m: Initial import of code base + from Kevin Murphy. + +2004-06-09 18:25 yozhik + + * BNT/CPDs/@tabular_CPD/learn_params.m: Initial revision + +2004-06-09 18:17 yozhik + + * BNT/general/sample_bnet.m: Initial import of code base from Kevin + Murphy. + +2004-06-09 18:17 yozhik + + * BNT/general/sample_bnet.m: Initial revision + +2004-06-07 12:45 yozhik + + * BNT/examples/static/discrete1.m: Initial import of code base from + Kevin Murphy. + +2004-06-07 12:45 yozhik + + * BNT/examples/static/discrete1.m: Initial revision + +2004-06-07 12:04 yozhik + + * BNT/: inference/static/@global_joint_inf_engine/marginal_nodes.m, + inference/static/@global_joint_inf_engine/enter_evidence.m, + examples/dynamic/mk_bat_dbn.m: Initial import of code base from + Kevin Murphy. + +2004-06-07 12:04 yozhik + + * BNT/: inference/static/@global_joint_inf_engine/marginal_nodes.m, + inference/static/@global_joint_inf_engine/enter_evidence.m, + examples/dynamic/mk_bat_dbn.m: Initial revision + +2004-06-07 08:53 yozhik + + * BNT/examples/limids/asia_dt1.m: Initial import of code base from + Kevin Murphy. + +2004-06-07 08:53 yozhik + + * BNT/examples/limids/asia_dt1.m: Initial revision + +2004-06-07 08:48 yozhik + + * BNT/general/: solve_limid.m, compute_joint_pot.m: Initial import + of code base from Kevin Murphy. + +2004-06-07 08:48 yozhik + + * BNT/general/: solve_limid.m, compute_joint_pot.m: Initial + revision + +2004-06-07 07:39 yozhik + + * Kalman/README.txt: Initial import of code base from Kevin Murphy. + +2004-06-07 07:39 yozhik + + * Kalman/README.txt: Initial revision + +2004-06-07 07:33 yozhik + + * GraphViz/README.txt: Initial import of code base from Kevin + Murphy. + +2004-06-07 07:33 yozhik + + * GraphViz/README.txt: Initial revision + +2004-05-31 15:19 yozhik + + * HMM/dhmm_sample.m: Initial import of code base from Kevin Murphy. + +2004-05-31 15:19 yozhik + + * HMM/dhmm_sample.m: Initial revision + +2004-05-25 17:32 yozhik + + * HMM/mhmm_sample.m: Initial import of code base from Kevin Murphy. + +2004-05-25 17:32 yozhik + + * HMM/mhmm_sample.m: Initial revision + +2004-05-24 15:26 yozhik + + * HMM/mc_sample.m: Initial import of code base from Kevin Murphy. + +2004-05-24 15:26 yozhik + + * HMM/mc_sample.m: Initial revision + +2004-05-18 07:50 yozhik + + * BNT/installC_graph.m: Initial import of code base from Kevin + Murphy. + +2004-05-18 07:50 yozhik + + * BNT/installC_graph.m: Initial revision + +2004-05-13 18:13 yozhik + + * BNT/inference/static/@gaussian_inf_engine/gaussian_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2004-05-13 18:13 yozhik + + * BNT/inference/static/@gaussian_inf_engine/gaussian_inf_engine.m: + Initial revision + +2004-05-11 12:23 yozhik + + * BNT/examples/dynamic/mk_chmm.m: Initial import of code base from + Kevin Murphy. + +2004-05-11 12:23 yozhik + + * BNT/examples/dynamic/mk_chmm.m: Initial revision + +2004-05-11 11:45 yozhik + + * BNT/examples/dynamic/mk_water_dbn.m: Initial import of code base + from Kevin Murphy. + +2004-05-11 11:45 yozhik + + * BNT/examples/dynamic/mk_water_dbn.m: Initial revision + +2004-05-05 06:32 yozhik + + * GraphViz/draw_dot.m: Initial import of code base from Kevin + Murphy. + +2004-05-05 06:32 yozhik + + * GraphViz/draw_dot.m: Initial revision + +2004-03-30 09:18 yozhik + + * BNT/: general/mk_named_CPT.m, + CPDs/@softmax_CPD/convert_to_table.m: Initial import of code base + from Kevin Murphy. + +2004-03-30 09:18 yozhik + + * BNT/: general/mk_named_CPT.m, + CPDs/@softmax_CPD/convert_to_table.m: Initial revision + +2004-03-22 14:32 yozhik + + * GraphViz/draw_graph.m: Initial import of code base from Kevin + Murphy. + +2004-03-22 14:32 yozhik + + * GraphViz/draw_graph.m: Initial revision + +2004-03-12 15:21 yozhik + + * GraphViz/dot_to_graph.m: Initial import of code base from Kevin + Murphy. + +2004-03-12 15:21 yozhik + + * GraphViz/dot_to_graph.m: Initial revision + +2004-03-04 14:34 yozhik + + * BNT/examples/static/burglary.m: Initial import of code base from + Kevin Murphy. + +2004-03-04 14:34 yozhik + + * BNT/examples/static/burglary.m: Initial revision + +2004-03-04 14:27 yozhik + + * BNT/examples/static/burglar-alarm-net.lisp.txt: Initial import of + code base from Kevin Murphy. + +2004-03-04 14:27 yozhik + + * BNT/examples/static/burglar-alarm-net.lisp.txt: Initial revision + +2004-02-28 09:25 yozhik + + * BNT/examples/static/learn1.m: Initial import of code base from + Kevin Murphy. + +2004-02-28 09:25 yozhik + + * BNT/examples/static/learn1.m: Initial revision + +2004-02-22 11:43 yozhik + + * BNT/examples/static/brainy.m: Initial import of code base from + Kevin Murphy. + +2004-02-22 11:43 yozhik + + * BNT/examples/static/brainy.m: Initial revision + +2004-02-20 14:00 yozhik + + * BNT/CPDs/@discrete_CPD/convert_to_pot.m: Initial import of code + base from Kevin Murphy. + +2004-02-20 14:00 yozhik + + * BNT/CPDs/@discrete_CPD/convert_to_pot.m: Initial revision + +2004-02-18 17:12 yozhik + + * + BNT/inference/dynamic/@jtree_dbn_inf_engine/enter_soft_evidence.m: + Initial import of code base from Kevin Murphy. + +2004-02-18 17:12 yozhik + + * + BNT/inference/dynamic/@jtree_dbn_inf_engine/enter_soft_evidence.m: + Initial revision + +2004-02-13 18:06 yozhik + + * HMM/mhmmParzen_train_observed.m: Initial import of code base from + Kevin Murphy. + +2004-02-13 18:06 yozhik + + * HMM/mhmmParzen_train_observed.m: Initial revision + +2004-02-12 15:08 yozhik + + * HMM/gausshmm_train_observed.m: Initial import of code base from + Kevin Murphy. + +2004-02-12 15:08 yozhik + + * HMM/gausshmm_train_observed.m: Initial revision + +2004-02-12 04:57 yozhik + + * BNT/examples/static/HME/hmemenu.m: Initial import of code base + from Kevin Murphy. + +2004-02-12 04:57 yozhik + + * BNT/examples/static/HME/hmemenu.m: Initial revision + +2004-02-07 20:52 yozhik + + * HMM/mhmm_em.m: Initial import of code base from Kevin Murphy. + +2004-02-07 20:52 yozhik + + * HMM/mhmm_em.m: Initial revision + +2004-02-04 15:53 yozhik + + * BNT/examples/dynamic/mk_orig_bat_dbn.m: Initial import of code + base from Kevin Murphy. + +2004-02-04 15:53 yozhik + + * BNT/examples/dynamic/mk_orig_bat_dbn.m: Initial revision + +2004-02-03 23:42 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/enter_soft_evidence.m: + Initial import of code base from Kevin Murphy. + +2004-02-03 23:42 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/enter_soft_evidence.m: + Initial revision + +2004-02-03 09:15 yozhik + + * GraphViz/Old/graphToDot.m: Initial import of code base from Kevin + Murphy. + +2004-02-03 09:15 yozhik + + * GraphViz/Old/graphToDot.m: Initial revision + +2004-01-30 18:57 yozhik + + * BNT/examples/dynamic/mk_orig_water_dbn.m: Initial import of code + base from Kevin Murphy. + +2004-01-30 18:57 yozhik + + * BNT/examples/dynamic/mk_orig_water_dbn.m: Initial revision + +2004-01-27 13:08 yozhik + + * GraphViz/: my_call.m, editGraphGUI.m: Initial import of code base + from Kevin Murphy. + +2004-01-27 13:08 yozhik + + * GraphViz/: my_call.m, editGraphGUI.m: Initial revision + +2004-01-27 13:01 yozhik + + * GraphViz/Old/: dot_to_graph.m, draw_graph.m: Initial import of + code base from Kevin Murphy. + +2004-01-27 13:01 yozhik + + * GraphViz/Old/: dot_to_graph.m, draw_graph.m: Initial revision + +2004-01-27 12:47 yozhik + + * GraphViz/Old/pre_pesha_graph_to_dot.m: Initial import of code + base from Kevin Murphy. + +2004-01-27 12:47 yozhik + + * GraphViz/Old/pre_pesha_graph_to_dot.m: Initial revision + +2004-01-27 12:42 yozhik + + * GraphViz/Old/draw_dot.m: Initial import of code base from Kevin + Murphy. + +2004-01-27 12:42 yozhik + + * GraphViz/Old/draw_dot.m: Initial revision + +2004-01-14 17:06 yozhik + + * BNT/examples/static/Models/mk_hmm_bnet.m: Initial import of code + base from Kevin Murphy. + +2004-01-14 17:06 yozhik + + * BNT/examples/static/Models/mk_hmm_bnet.m: Initial revision + +2004-01-12 12:53 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/enter_evidence.m: Initial + import of code base from Kevin Murphy. + +2004-01-12 12:53 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/enter_evidence.m: Initial + revision + +2004-01-04 17:23 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/bp_mrf2.m: Initial + import of code base from Kevin Murphy. + +2004-01-04 17:23 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/bp_mrf2.m: Initial + revision + +2003-12-15 22:17 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/marginal_nodes.m: Initial + import of code base from Kevin Murphy. + +2003-12-15 22:17 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/marginal_nodes.m: Initial + revision + +2003-10-31 14:37 yozhik + + * BNT/inference/static/@jtree_inf_engine/jtree_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2003-10-31 14:37 yozhik + + * BNT/inference/static/@jtree_inf_engine/jtree_inf_engine.m: + Initial revision + +2003-09-05 07:06 yozhik + + * BNT/learning/learn_struct_mcmc.m: Initial import of code base + from Kevin Murphy. + +2003-09-05 07:06 yozhik + + * BNT/learning/learn_struct_mcmc.m: Initial revision + +2003-08-18 14:50 yozhik + + * BNT/learning/learn_params_dbn_em.m: Initial import of code base + from Kevin Murphy. + +2003-08-18 14:50 yozhik + + * BNT/learning/learn_params_dbn_em.m: Initial revision + +2003-07-30 06:37 yozhik + + * BNT/potentials/: @mpot/set_domain_pot.m, + @cgpot/Old/set_domain_pot.m, @cgpot/set_domain_pot.m: Initial + import of code base from Kevin Murphy. + +2003-07-30 06:37 yozhik + + * BNT/potentials/: @mpot/set_domain_pot.m, + @cgpot/Old/set_domain_pot.m, @cgpot/set_domain_pot.m: Initial + revision + +2003-07-28 19:44 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/: dbn_init_bel.m, + dbn_marginal_from_bel.m, dbn_update_bel.m, dbn_update_bel1.m, + marginal_family.m, update_engine.m: Initial import of code base + from Kevin Murphy. + +2003-07-28 19:44 yozhik + + * BNT/inference/dynamic/@cbk_inf_engine/: dbn_init_bel.m, + dbn_marginal_from_bel.m, dbn_update_bel.m, dbn_update_bel1.m, + marginal_family.m, update_engine.m: Initial revision + +2003-07-28 15:44 yozhik + + * GraphViz/: approxeq.m, process_options.m: Initial import of code + base from Kevin Murphy. + +2003-07-28 15:44 yozhik + + * GraphViz/: approxeq.m, process_options.m: Initial revision + +2003-07-24 06:41 yozhik + + * BNT/CPDs/@hhmmQ_CPD/update_ess.m: Initial import of code base + from Kevin Murphy. + +2003-07-24 06:41 yozhik + + * BNT/CPDs/@hhmmQ_CPD/update_ess.m: Initial revision + +2003-07-22 15:55 yozhik + + * BNT/CPDs/@gaussian_CPD/update_ess.m: Initial import of code base + from Kevin Murphy. + +2003-07-22 15:55 yozhik + + * BNT/CPDs/@gaussian_CPD/update_ess.m: Initial revision + +2003-07-06 13:57 yozhik + + * BNT/inference/static/@pearl_inf_engine/bethe_free_energy.m: + Initial import of code base from Kevin Murphy. + +2003-07-06 13:57 yozhik + + * BNT/inference/static/@pearl_inf_engine/bethe_free_energy.m: + Initial revision + +2003-05-21 06:49 yozhik + + * BNT/potentials/@scgpot/: complement_pot.m, normalize_pot.m, + recursive_combine_pots.m: Initial import of code base from Kevin + Murphy. + +2003-05-21 06:49 yozhik + + * BNT/potentials/@scgpot/: complement_pot.m, normalize_pot.m, + recursive_combine_pots.m: Initial revision + +2003-05-20 07:10 yozhik + + * BNT/CPDs/@gaussian_CPD/maximize_params.m: Initial import of code + base from Kevin Murphy. + +2003-05-20 07:10 yozhik + + * BNT/CPDs/@gaussian_CPD/maximize_params.m: Initial revision + +2003-05-13 09:11 yozhik + + * HMM/mhmm_em_demo.m: Initial import of code base from Kevin + Murphy. + +2003-05-13 09:11 yozhik + + * HMM/mhmm_em_demo.m: Initial revision + +2003-05-13 07:35 yozhik + + * BNT/examples/dynamic/viterbi1.m: Initial import of code base from + Kevin Murphy. + +2003-05-13 07:35 yozhik + + * BNT/examples/dynamic/viterbi1.m: Initial revision + +2003-05-11 16:31 yozhik + + * BNT/CPDs/@gaussian_CPD/convert_to_table.m: Initial import of code + base from Kevin Murphy. + +2003-05-11 16:31 yozhik + + * BNT/CPDs/@gaussian_CPD/convert_to_table.m: Initial revision + +2003-05-11 16:13 yozhik + + * BNT/CPDs/@gaussian_CPD/gaussian_CPD_params_given_dps.m: Initial + import of code base from Kevin Murphy. + +2003-05-11 16:13 yozhik + + * BNT/CPDs/@gaussian_CPD/gaussian_CPD_params_given_dps.m: Initial + revision + +2003-05-11 08:39 yozhik + + * BNT/inference/static/@stab_cond_gauss_inf_engine/README: Initial + import of code base from Kevin Murphy. + +2003-05-11 08:39 yozhik + + * BNT/inference/static/@stab_cond_gauss_inf_engine/README: Initial + revision + +2003-05-04 15:31 yozhik + + * BNT/uninstallC_BNT.m: Initial import of code base from Kevin + Murphy. + +2003-05-04 15:31 yozhik + + * BNT/uninstallC_BNT.m: Initial revision + +2003-05-04 15:23 yozhik + + * BNT/examples/dynamic/: dhmm1.m, ghmm1.m, mhmm1.m: Initial import + of code base from Kevin Murphy. + +2003-05-04 15:23 yozhik + + * BNT/examples/dynamic/: dhmm1.m, ghmm1.m, mhmm1.m: Initial + revision + +2003-05-04 15:11 yozhik + + * HMM/mhmm_logprob.m: Initial import of code base from Kevin + Murphy. + +2003-05-04 15:11 yozhik + + * HMM/mhmm_logprob.m: Initial revision + +2003-05-04 15:01 yozhik + + * HMM/: dhmm_logprob.m, dhmm_em_online.m, dhmm_em_online_demo.m: + Initial import of code base from Kevin Murphy. + +2003-05-04 15:01 yozhik + + * HMM/: dhmm_logprob.m, dhmm_em_online.m, dhmm_em_online_demo.m: + Initial revision + +2003-05-04 14:58 yozhik + + * HMM/: pomdp_sample.m, dhmm_sample_endstate.m, dhmm_em_demo.m: + Initial import of code base from Kevin Murphy. + +2003-05-04 14:58 yozhik + + * HMM/: pomdp_sample.m, dhmm_sample_endstate.m, dhmm_em_demo.m: + Initial revision + +2003-05-04 14:47 yozhik + + * + BNT/inference/online/@hmm_2TBN_inf_engine/private/mk_hmm_obs_lik_vec.m: + Initial import of code base from Kevin Murphy. + +2003-05-04 14:47 yozhik + + * + BNT/inference/online/@hmm_2TBN_inf_engine/private/mk_hmm_obs_lik_vec.m: + Initial revision + +2003-05-04 14:42 yozhik + + * + BNT/inference/dynamic/@hmm_inf_engine/private/mk_hmm_obs_lik_matrix.m: + Initial import of code base from Kevin Murphy. + +2003-05-04 14:42 yozhik + + * + BNT/inference/dynamic/@hmm_inf_engine/private/mk_hmm_obs_lik_matrix.m: + Initial revision + +2003-04-22 14:00 yozhik + + * BNT/CPDs/@tabular_CPD/display.m: Initial import of code base from + Kevin Murphy. + +2003-04-22 14:00 yozhik + + * BNT/CPDs/@tabular_CPD/display.m: Initial revision + +2003-03-28 09:22 yozhik + + * BNT/examples/dynamic/ho1.m: Initial import of code base from + Kevin Murphy. + +2003-03-28 09:22 yozhik + + * BNT/examples/dynamic/ho1.m: Initial revision + +2003-03-28 09:12 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/stab_cond_gauss_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2003-03-28 09:12 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/stab_cond_gauss_inf_engine.m: + Initial revision + +2003-03-28 08:35 yozhik + + * GraphViz/arrow.m: Initial import of code base from Kevin Murphy. + +2003-03-28 08:35 yozhik + + * GraphViz/arrow.m: Initial revision + +2003-03-25 16:06 yozhik + + * BNT/examples/static/Models/mk_asia_bnet.m: Initial import of code + base from Kevin Murphy. + +2003-03-25 16:06 yozhik + + * BNT/examples/static/Models/mk_asia_bnet.m: Initial revision + +2003-03-20 07:07 yozhik + + * BNT/potentials/@scgpot/README: Initial import of code base from + Kevin Murphy. + +2003-03-20 07:07 yozhik + + * BNT/potentials/@scgpot/README: Initial revision + +2003-03-14 01:45 yozhik + + * + BNT/inference/dynamic/@stable_ho_inf_engine/stable_ho_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2003-03-14 01:45 yozhik + + * + BNT/inference/dynamic/@stable_ho_inf_engine/stable_ho_inf_engine.m: + Initial revision + +2003-03-12 02:38 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/enter_evidence.m: + Initial import of code base from Kevin Murphy. + +2003-03-12 02:38 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/enter_evidence.m: + Initial revision + +2003-03-11 10:07 yozhik + + * BNT/potentials/@scgpot/reduce_pot.m: Initial import of code base + from Kevin Murphy. + +2003-03-11 10:07 yozhik + + * BNT/potentials/@scgpot/reduce_pot.m: Initial revision + +2003-03-11 09:49 yozhik + + * BNT/potentials/@scgpot/combine_pots.m: Initial import of code + base from Kevin Murphy. + +2003-03-11 09:49 yozhik + + * BNT/potentials/@scgpot/combine_pots.m: Initial revision + +2003-03-11 09:37 yozhik + + * BNT/potentials/@scgcpot/reduce_pot.m: Initial import of code base + from Kevin Murphy. + +2003-03-11 09:37 yozhik + + * BNT/potentials/@scgcpot/reduce_pot.m: Initial revision + +2003-03-11 09:06 yozhik + + * BNT/potentials/@scgpot/marginalize_pot.m: Initial import of code + base from Kevin Murphy. + +2003-03-11 09:06 yozhik + + * BNT/potentials/@scgpot/marginalize_pot.m: Initial revision + +2003-03-11 06:04 yozhik + + * BNT/potentials/@scgpot/scgpot.m: Initial import of code base from + Kevin Murphy. + +2003-03-11 06:04 yozhik + + * BNT/potentials/@scgpot/scgpot.m: Initial revision + +2003-03-09 15:03 yozhik + + * BNT/CPDs/@gaussian_CPD/convert_to_pot.m: Initial import of code + base from Kevin Murphy. + +2003-03-09 15:03 yozhik + + * BNT/CPDs/@gaussian_CPD/convert_to_pot.m: Initial revision + +2003-03-09 14:44 yozhik + + * BNT/CPDs/@tabular_CPD/maximize_params.m: Initial import of code + base from Kevin Murphy. + +2003-03-09 14:44 yozhik + + * BNT/CPDs/@tabular_CPD/maximize_params.m: Initial revision + +2003-02-21 03:20 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/marginal_difclq_nodes.m: + Initial import of code base from Kevin Murphy. + +2003-02-21 03:20 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/marginal_difclq_nodes.m: + Initial revision + +2003-02-21 03:13 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/marginal_nodes.m: + Initial import of code base from Kevin Murphy. + +2003-02-21 03:13 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/marginal_nodes.m: + Initial revision + +2003-02-19 01:52 yozhik + + * BNT/inference/dynamic/@stable_ho_inf_engine/: enter_evidence.m, + marginal_family.m, marginal_nodes.m, test_ho_inf_enginge.m, + update_engine.m: Initial import of code base from Kevin Murphy. + +2003-02-19 01:52 yozhik + + * BNT/inference/dynamic/@stable_ho_inf_engine/: enter_evidence.m, + marginal_family.m, marginal_nodes.m, test_ho_inf_enginge.m, + update_engine.m: Initial revision + +2003-02-10 07:38 yozhik + + * BNT/inference/static/@stab_cond_gauss_inf_engine/push.m: Initial + import of code base from Kevin Murphy. + +2003-02-10 07:38 yozhik + + * BNT/inference/static/@stab_cond_gauss_inf_engine/push.m: Initial + revision + +2003-02-06 18:25 yozhik + + * KPMtools/checkpsd.m: Initial import of code base from Kevin + Murphy. + +2003-02-06 18:25 yozhik + + * KPMtools/checkpsd.m: Initial revision + +2003-02-05 19:16 yozhik + + * GraphViz/draw_hmm.m: Initial import of code base from Kevin + Murphy. + +2003-02-05 19:16 yozhik + + * GraphViz/draw_hmm.m: Initial revision + +2003-02-01 16:23 yozhik + + * BNT/: general/dbn_to_hmm.m, learning/learn_params_dbn.m: Initial + import of code base from Kevin Murphy. + +2003-02-01 16:23 yozhik + + * BNT/: general/dbn_to_hmm.m, learning/learn_params_dbn.m: Initial + revision + +2003-02-01 11:42 yozhik + + * BNT/general/mk_dbn.m: Initial import of code base from Kevin + Murphy. + +2003-02-01 11:42 yozhik + + * BNT/general/mk_dbn.m: Initial revision + +2003-01-30 16:13 yozhik + + * BNT/CPDs/@gaussian_CPD/maximize_params_debug.m: Initial import of + code base from Kevin Murphy. + +2003-01-30 16:13 yozhik + + * BNT/CPDs/@gaussian_CPD/maximize_params_debug.m: Initial revision + +2003-01-30 14:38 yozhik + + * BNT/CPDs/@gaussian_CPD/Old/maximize_params.m: Initial import of + code base from Kevin Murphy. + +2003-01-30 14:38 yozhik + + * BNT/CPDs/@gaussian_CPD/Old/maximize_params.m: Initial revision + +2003-01-29 03:23 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/marginal_singleclq_nodes.m: + Initial import of code base from Kevin Murphy. + +2003-01-29 03:23 yozhik + + * + BNT/inference/static/@stab_cond_gauss_inf_engine/marginal_singleclq_nodes.m: + Initial revision + +2003-01-24 11:36 yozhik + + * Kalman/sample_lds.m: Initial import of code base from Kevin + Murphy. + +2003-01-24 11:36 yozhik + + * Kalman/sample_lds.m: Initial revision + +2003-01-24 04:52 yozhik + + * BNT/potentials/@scgpot/extension_pot.m: Initial import of code + base from Kevin Murphy. + +2003-01-24 04:52 yozhik + + * BNT/potentials/@scgpot/extension_pot.m: Initial revision + +2003-01-23 10:49 yozhik + + * BNT/: general/convert_dbn_CPDs_to_tables1.m, + inference/dynamic/@hmm_inf_engine/private/mk_hmm_obs_lik_vec.m: + Initial import of code base from Kevin Murphy. + +2003-01-23 10:49 yozhik + + * BNT/: general/convert_dbn_CPDs_to_tables1.m, + inference/dynamic/@hmm_inf_engine/private/mk_hmm_obs_lik_vec.m: + Initial revision + +2003-01-23 10:44 yozhik + + * BNT/general/convert_dbn_CPDs_to_tables.m: Initial import of code + base from Kevin Murphy. + +2003-01-23 10:44 yozhik + + * BNT/general/convert_dbn_CPDs_to_tables.m: Initial revision + +2003-01-22 13:38 yozhik + + * BNT/inference/dynamic/@hmm_inf_engine/enter_evidence.m: Initial + import of code base from Kevin Murphy. + +2003-01-22 13:38 yozhik + + * BNT/inference/dynamic/@hmm_inf_engine/enter_evidence.m: Initial + revision + +2003-01-22 12:32 yozhik + + * HMM/mc_sample_endstate.m: Initial import of code base from Kevin + Murphy. + +2003-01-22 12:32 yozhik + + * HMM/mc_sample_endstate.m: Initial revision + +2003-01-22 09:56 yozhik + + * HMM/fixed_lag_smoother.m: Initial import of code base from Kevin + Murphy. + +2003-01-22 09:56 yozhik + + * HMM/fixed_lag_smoother.m: Initial revision + +2003-01-20 08:56 yozhik + + * GraphViz/draw_graph_test.m: Initial import of code base from + Kevin Murphy. + +2003-01-20 08:56 yozhik + + * GraphViz/draw_graph_test.m: Initial revision + +2003-01-18 15:10 yozhik + + * BNT/general/dsep_test.m: Initial import of code base from Kevin + Murphy. + +2003-01-18 15:10 yozhik + + * BNT/general/dsep_test.m: Initial revision + +2003-01-18 15:00 yozhik + + * BNT/copyright.txt: Initial import of code base from Kevin Murphy. + +2003-01-18 15:00 yozhik + + * BNT/copyright.txt: Initial revision + +2003-01-18 14:49 yozhik + + * Kalman/tracking_demo.m: Initial import of code base from Kevin + Murphy. + +2003-01-18 14:49 yozhik + + * Kalman/tracking_demo.m: Initial revision + +2003-01-18 14:22 yozhik + + * BNT/: examples/dummy, inference/dummy, inference/dynamic/dummy, + inference/online/dummy, inference/static/dummy: Initial import of + code base from Kevin Murphy. + +2003-01-18 14:22 yozhik + + * BNT/: examples/dummy, inference/dummy, inference/dynamic/dummy, + inference/online/dummy, inference/static/dummy: Initial revision + +2003-01-18 14:16 yozhik + + * BNT/examples/dynamic/: ehmm1.m, jtree_clq_test.m: Initial import + of code base from Kevin Murphy. + +2003-01-18 14:16 yozhik + + * BNT/examples/dynamic/: ehmm1.m, jtree_clq_test.m: Initial + revision + +2003-01-18 14:11 yozhik + + * BNT/inference/static/: + @jtree_sparse_inf_engine/jtree_sparse_inf_engine.m, + @jtree_mnet_inf_engine/jtree_mnet_inf_engine.m: Initial import of + code base from Kevin Murphy. + +2003-01-18 14:11 yozhik + + * BNT/inference/static/: + @jtree_sparse_inf_engine/jtree_sparse_inf_engine.m, + @jtree_mnet_inf_engine/jtree_mnet_inf_engine.m: Initial revision + +2003-01-18 13:17 yozhik + + * GraphViz/draw_dbn_test.m: Initial import of code base from Kevin + Murphy. + +2003-01-18 13:17 yozhik + + * GraphViz/draw_dbn_test.m: Initial revision + +2003-01-11 10:53 yozhik + + * BNT/inference/static/@pearl_inf_engine/pearl_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2003-01-11 10:53 yozhik + + * BNT/inference/static/@pearl_inf_engine/pearl_inf_engine.m: + Initial revision + +2003-01-11 10:48 yozhik + + * BNT/examples/dynamic/HHMM/Map/learn_map.m: Initial import of code + base from Kevin Murphy. + +2003-01-11 10:48 yozhik + + * BNT/examples/dynamic/HHMM/Map/learn_map.m: Initial revision + +2003-01-11 10:41 yozhik + + * BNT/inference/dynamic/@jtree_dbn_inf_engine/enter_evidence.m: + Initial import of code base from Kevin Murphy. + +2003-01-11 10:41 yozhik + + * BNT/inference/dynamic/@jtree_dbn_inf_engine/enter_evidence.m: + Initial revision + +2003-01-11 10:13 yozhik + + * BNT/inference/dynamic/@bk_inf_engine/enter_soft_evidence.m: + Initial import of code base from Kevin Murphy. + +2003-01-11 10:13 yozhik + + * BNT/inference/dynamic/@bk_inf_engine/enter_soft_evidence.m: + Initial revision + +2003-01-07 08:25 yozhik + + * BNT/CPDs/@softmax_CPD/softmax_CPD.m: Initial import of code base + from Kevin Murphy. + +2003-01-07 08:25 yozhik + + * BNT/CPDs/@softmax_CPD/softmax_CPD.m: Initial revision + +2003-01-03 14:01 yozhik + + * + BNT/inference/static/@belprop_mrf2_inf_engine/belprop_mrf2_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2003-01-03 14:01 yozhik + + * + BNT/inference/static/@belprop_mrf2_inf_engine/belprop_mrf2_inf_engine.m: + Initial revision + +2003-01-02 09:49 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/find_mpe.m: Initial + import of code base from Kevin Murphy. + +2003-01-02 09:49 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/find_mpe.m: Initial + revision + +2003-01-02 09:28 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/: set_params.m, + enter_soft_evidence.m: Initial import of code base from Kevin + Murphy. + +2003-01-02 09:28 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/: set_params.m, + enter_soft_evidence.m: Initial revision + +2002-12-31 14:06 yozhik + + * BNT/general/mk_mrf2.m: Initial import of code base from Kevin + Murphy. + +2002-12-31 14:06 yozhik + + * BNT/general/mk_mrf2.m: Initial revision + +2002-12-31 13:24 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/marginal_nodes.m: + Initial import of code base from Kevin Murphy. + +2002-12-31 13:24 yozhik + + * BNT/inference/static/@belprop_mrf2_inf_engine/marginal_nodes.m: + Initial revision + +2002-12-31 11:00 yozhik + + * BNT/inference/static/@belprop_inf_engine/belprop_inf_engine.m: + Initial import of code base from Kevin Murphy. + +2002-12-31 11:00 yozhik + + * BNT/inference/static/@belprop_inf_engine/belprop_inf_engine.m: + Initial revision + +2002-12-16 11:16 yozhik + + * BNT/examples/dynamic/HHMM/remove_hhmm_end_state.m: Initial import + of code base from Kevin Murphy. + +2002-12-16 11:16 yozhik + + * BNT/examples/dynamic/HHMM/remove_hhmm_end_state.m: Initial + revision + +2002-12-16 09:57 yozhik + + * BNT/general/unroll_set.m: Initial import of code base from Kevin + Murphy. + +2002-12-16 09:57 yozhik + + * BNT/general/unroll_set.m: Initial revision + +2002-11-26 14:14 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/mgram3.m: Initial import of code + base from Kevin Murphy. + +2002-11-26 14:14 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/mgram3.m: Initial revision + +2002-11-26 14:04 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/mgram2.m: Initial import of code + base from Kevin Murphy. + +2002-11-26 14:04 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/mgram2.m: Initial revision + +2002-11-22 16:44 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/Old/mgram2.m: Initial import of + code base from Kevin Murphy. + +2002-11-22 16:44 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/Old/mgram2.m: Initial revision + +2002-11-22 15:59 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/mgram1.m: Initial import of code + base from Kevin Murphy. + +2002-11-22 15:59 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/mgram1.m: Initial revision + +2002-11-22 15:51 yozhik + + * BNT/inference/dynamic/@jtree_dbn_inf_engine/marginal_nodes.m: + Initial import of code base from Kevin Murphy. + +2002-11-22 15:51 yozhik + + * BNT/inference/dynamic/@jtree_dbn_inf_engine/marginal_nodes.m: + Initial revision + +2002-11-22 15:07 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/: num2letter.m, letter2num.m: + Initial import of code base from Kevin Murphy. + +2002-11-22 15:07 yozhik + + * BNT/examples/dynamic/HHMM/Mgram/: num2letter.m, letter2num.m: + Initial revision + +2002-11-22 14:35 yozhik + + * BNT/general/convert_dbn_CPDs_to_pots.m: Initial import of code + base from Kevin Murphy. + +2002-11-22 14:35 yozhik + + * BNT/general/convert_dbn_CPDs_to_pots.m: Initial revision + +2002-11-22 13:45 yozhik + + * HMM/mk_rightleft_transmat.m: Initial import of code base from + Kevin Murphy. + +2002-11-22 13:45 yozhik + + * HMM/mk_rightleft_transmat.m: Initial revision + +2002-11-14 12:33 yozhik + + * BNT/examples/dynamic/water2.m: Initial import of code base from + Kevin Murphy. + +2002-11-14 12:33 yozhik + + * BNT/examples/dynamic/water2.m: Initial revision + +2002-11-14 12:07 yozhik + + * BNT/examples/dynamic/water1.m: Initial import of code base from + Kevin Murphy. + +2002-11-14 12:07 yozhik + + * BNT/examples/dynamic/water1.m: Initial revision + +2002-11-14 12:02 yozhik + + * BNT/inference/: online/@hmm_2TBN_inf_engine/marginal_nodes.m, + dynamic/@hmm_inf_engine/marginal_nodes.m, + online/@hmm_2TBN_inf_engine/hmm_2TBN_inf_engine.m, + dynamic/@hmm_inf_engine/hmm_inf_engine.m, + dynamic/@hmm_inf_engine/marginal_family.m, + online/@hmm_2TBN_inf_engine/marginal_family.m: Initial import of + code base from Kevin Murphy. + +2002-11-14 12:02 yozhik + + * BNT/inference/: online/@hmm_2TBN_inf_engine/marginal_nodes.m, + dynamic/@hmm_inf_engine/marginal_nodes.m, + online/@hmm_2TBN_inf_engine/hmm_2TBN_inf_engine.m, + dynamic/@hmm_inf_engine/hmm_inf_engine.m, + dynamic/@hmm_inf_engine/marginal_family.m, + online/@hmm_2TBN_inf_engine/marginal_family.m: Initial revision + +2002-11-14 08:31 yozhik + + * BNT/inference/: + online/@jtree_2TBN_inf_engine/jtree_2TBN_inf_engine.m, + dynamic/@jtree_dbn_inf_engine/jtree_dbn_inf_engine.m: Initial + import of code base from Kevin Murphy. + +2002-11-14 08:31 yozhik + + * BNT/inference/: + online/@jtree_2TBN_inf_engine/jtree_2TBN_inf_engine.m, + dynamic/@jtree_dbn_inf_engine/jtree_dbn_inf_engine.m: Initial + revision + +2002-11-13 17:01 yozhik + + * BNT/examples/: static/qmr2.m, dynamic/arhmm1.m: Initial import of + code base from Kevin Murphy. + +2002-11-13 17:01 yozhik + + * BNT/examples/: static/qmr2.m, dynamic/arhmm1.m: Initial revision + +2002-11-03 08:44 yozhik + + * BNT/examples/static/Models/mk_alarm_bnet.m: Initial import of + code base from Kevin Murphy. + +2002-11-03 08:44 yozhik + + * BNT/examples/static/Models/mk_alarm_bnet.m: Initial revision + +2002-11-01 16:32 yozhik + + * Kalman/kalman_forward_backward.m: Initial import of code base + from Kevin Murphy. + +2002-11-01 16:32 yozhik + + * Kalman/kalman_forward_backward.m: Initial revision + +2002-10-23 08:17 yozhik + + * Kalman/learning_demo.m: Initial import of code base from Kevin + Murphy. + +2002-10-23 08:17 yozhik + + * Kalman/learning_demo.m: Initial revision + +2002-10-18 13:05 yozhik + + * BNT/inference/static/@pearl_inf_engine/marginal_family.m: Initial + import of code base from Kevin Murphy. + +2002-10-18 13:05 yozhik + + * BNT/inference/static/@pearl_inf_engine/marginal_family.m: Initial + revision + +2002-10-10 16:45 yozhik + + * BNT/examples/dynamic/jtree_clq_test2.m: Initial import of code + base from Kevin Murphy. + +2002-10-10 16:45 yozhik + + * BNT/examples/dynamic/jtree_clq_test2.m: Initial revision + +2002-10-10 16:14 yozhik + + * BNT/examples/dynamic/: mk_mildew_dbn.m, mk_uffe_dbn.m: Initial + import of code base from Kevin Murphy. + +2002-10-10 16:14 yozhik + + * BNT/examples/dynamic/: mk_mildew_dbn.m, mk_uffe_dbn.m: Initial + revision + +2002-10-09 13:36 yozhik + + * BNT/examples/dynamic/mk_ps_from_clqs.m: Initial import of code + base from Kevin Murphy. + +2002-10-09 13:36 yozhik + + * BNT/examples/dynamic/mk_ps_from_clqs.m: Initial revision + +2002-10-07 06:26 yozhik + + * BNT/CPDs/@deterministic_CPD/deterministic_CPD.m: Initial import + of code base from Kevin Murphy. + +2002-10-07 06:26 yozhik + + * BNT/CPDs/@deterministic_CPD/deterministic_CPD.m: Initial revision + +2002-10-02 08:39 yozhik + + * BNT/potentials/Tables/marg_tableC.c: Initial import of code base + from Kevin Murphy. + +2002-10-02 08:39 yozhik + + * BNT/potentials/Tables/marg_tableC.c: Initial revision + +2002-10-02 08:28 yozhik + + * BNT/potentials/Tables/: mult_by_tableM.m, mult_by_table2.m: + Initial import of code base from Kevin Murphy. + +2002-10-02 08:28 yozhik + + * BNT/potentials/Tables/: mult_by_tableM.m, mult_by_table2.m: + Initial revision + +2002-10-01 14:33 yozhik + + * BNT/potentials/Tables/mult_by_tableC.c: Initial import of code + base from Kevin Murphy. + +2002-10-01 14:33 yozhik + + * BNT/potentials/Tables/mult_by_tableC.c: Initial revision + +2002-10-01 14:23 yozhik + + * BNT/potentials/Tables/mult_by_table.c: Initial import of code + base from Kevin Murphy. + +2002-10-01 14:23 yozhik + + * BNT/potentials/Tables/mult_by_table.c: Initial revision + +2002-10-01 14:20 yozhik + + * BNT/potentials/Tables/repmat_and_mult.c: Initial import of code + base from Kevin Murphy. + +2002-10-01 14:20 yozhik + + * BNT/potentials/Tables/repmat_and_mult.c: Initial revision + +2002-10-01 12:04 yozhik + + * BNT/potentials/@dpot/dpot.m: Initial import of code base from + Kevin Murphy. + +2002-10-01 12:04 yozhik + + * BNT/potentials/@dpot/dpot.m: Initial revision + +2002-10-01 11:21 yozhik + + * BNT/examples/static/Belprop/belprop_polytree_discrete.m: Initial + import of code base from Kevin Murphy. + +2002-10-01 11:21 yozhik + + * BNT/examples/static/Belprop/belprop_polytree_discrete.m: Initial + revision + +2002-10-01 11:16 yozhik + + * BNT/examples/static/cmp_inference_static.m: Initial import of + code base from Kevin Murphy. + +2002-10-01 11:16 yozhik + + * BNT/examples/static/cmp_inference_static.m: Initial revision + +2002-10-01 10:39 yozhik + + * BNT/potentials/Tables/marg_tableM.m: Initial import of code base + from Kevin Murphy. + +2002-10-01 10:39 yozhik + + * BNT/potentials/Tables/marg_tableM.m: Initial revision + +2002-09-29 03:21 yozhik + + * BNT/potentials/Tables/mult_by_table_global.m: Initial import of + code base from Kevin Murphy. + +2002-09-29 03:21 yozhik + + * BNT/potentials/Tables/mult_by_table_global.m: Initial revision + +2002-09-26 01:39 yozhik + + * BNT/learning/learn_struct_K2.m: Initial import of code base from + Kevin Murphy. + +2002-09-26 01:39 yozhik + + * BNT/learning/learn_struct_K2.m: Initial revision + +2002-09-24 15:43 yozhik + + * BNT/: CPDs/@hhmm2Q_CPD/update_ess.m, + CPDs/@hhmm2Q_CPD/maximize_params.m, + examples/dynamic/HHMM/Map/disp_map_hhmm.m: Initial import of code + base from Kevin Murphy. + +2002-09-24 15:43 yozhik + + * BNT/: CPDs/@hhmm2Q_CPD/update_ess.m, + CPDs/@hhmm2Q_CPD/maximize_params.m, + examples/dynamic/HHMM/Map/disp_map_hhmm.m: Initial revision + +2002-09-24 15:34 yozhik + + * BNT/CPDs/@hhmm2Q_CPD/: hhmm2Q_CPD.m, reset_ess.m: Initial import + of code base from Kevin Murphy. + +2002-09-24 15:34 yozhik + + * BNT/CPDs/@hhmm2Q_CPD/: hhmm2Q_CPD.m, reset_ess.m: Initial + revision + +2002-09-24 15:13 yozhik + + * BNT/examples/dynamic/HHMM/Map/mk_rnd_map_hhmm.m: Initial import + of code base from Kevin Murphy. + +2002-09-24 15:13 yozhik + + * BNT/examples/dynamic/HHMM/Map/mk_rnd_map_hhmm.m: Initial revision + +2002-09-24 06:10 yozhik + + * BNT/CPDs/@hhmmQ_CPD/maximize_params.m: Initial import of code + base from Kevin Murphy. + +2002-09-24 06:10 yozhik + + * BNT/CPDs/@hhmmQ_CPD/maximize_params.m: Initial revision + +2002-09-24 06:02 yozhik + + * BNT/examples/dynamic/HHMM/Map/sample_from_map.m: Initial import + of code base from Kevin Murphy. + +2002-09-24 06:02 yozhik + + * BNT/examples/dynamic/HHMM/Map/sample_from_map.m: Initial revision + +2002-09-24 05:46 yozhik + + * BNT/CPDs/@hhmm2Q_CPD/CPD_to_CPT.m: Initial import of code base + from Kevin Murphy. + +2002-09-24 05:46 yozhik + + * BNT/CPDs/@hhmm2Q_CPD/CPD_to_CPT.m: Initial revision + +2002-09-24 03:49 yozhik + + * BNT/examples/dynamic/HHMM/Map/mk_map_hhmm.m: Initial import of + code base from Kevin Murphy. + +2002-09-24 03:49 yozhik + + * BNT/examples/dynamic/HHMM/Map/mk_map_hhmm.m: Initial revision + +2002-09-24 00:02 yozhik + + * BNT/examples/dynamic/HHMM/Map/Old/mk_map_hhmm.m: Initial import + of code base from Kevin Murphy. + +2002-09-24 00:02 yozhik + + * BNT/examples/dynamic/HHMM/Map/Old/mk_map_hhmm.m: Initial revision + +2002-09-23 21:19 yozhik + + * BNT/CPDs/@hhmmQ_CPD/hhmmQ_CPD.m: Initial import of code base from + Kevin Murphy. + +2002-09-23 21:19 yozhik + + * BNT/CPDs/@hhmmQ_CPD/hhmmQ_CPD.m: Initial revision + +2002-09-23 19:58 yozhik + + * BNT/CPDs/@hhmmQ_CPD/update_CPT.m: Initial import of code base + from Kevin Murphy. + +2002-09-23 19:58 yozhik + + * BNT/CPDs/@hhmmQ_CPD/update_CPT.m: Initial revision + +2002-09-23 19:30 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_CPT.m: Initial import of code base + from Kevin Murphy. + +2002-09-23 19:30 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_CPT.m: Initial revision + +2002-09-21 14:37 yozhik + + * BNT/examples/dynamic/HHMM/abcd_hhmm.m: Initial import of code + base from Kevin Murphy. + +2002-09-21 14:37 yozhik + + * BNT/examples/dynamic/HHMM/abcd_hhmm.m: Initial revision + +2002-09-21 13:58 yozhik + + * BNT/examples/dynamic/HHMM/mk_hhmm.m: Initial import of code base + from Kevin Murphy. + +2002-09-21 13:58 yozhik + + * BNT/examples/dynamic/HHMM/mk_hhmm.m: Initial revision + +2002-09-10 10:44 yozhik + + * BNT/CPDs/@gaussian_CPD/log_prob_node.m: Initial import of code + base from Kevin Murphy. + +2002-09-10 10:44 yozhik + + * BNT/CPDs/@gaussian_CPD/log_prob_node.m: Initial revision + +2002-07-28 16:09 yozhik + + * BNT/learning/: learn_struct_pdag_pc_constrain.m, CovMat.m: + Initial import of code base from Kevin Murphy. + +2002-07-28 16:09 yozhik + + * BNT/learning/: learn_struct_pdag_pc_constrain.m, CovMat.m: + Initial revision + +2002-07-24 07:48 yozhik + + * BNT/general/hodbn_to_bnet.m: Initial import of code base from + Kevin Murphy. + +2002-07-24 07:48 yozhik + + * BNT/general/hodbn_to_bnet.m: Initial revision + +2002-07-23 06:17 yozhik + + * BNT/general/mk_higher_order_dbn.m: Initial import of code base + from Kevin Murphy. + +2002-07-23 06:17 yozhik + + * BNT/general/mk_higher_order_dbn.m: Initial revision + +2002-07-20 18:25 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/set_fields.m: Initial + import of code base from Kevin Murphy. + +2002-07-20 18:25 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/set_fields.m: Initial + revision + +2002-07-20 17:32 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/back_mpe.m: Initial + import of code base from Kevin Murphy. + +2002-07-20 17:32 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/back_mpe.m: Initial + revision + +2002-07-02 15:56 yozhik + + * BNT/examples/dynamic/HHMM/Motif/learn_motif_hhmm.m: Initial + import of code base from Kevin Murphy. + +2002-07-02 15:56 yozhik + + * BNT/examples/dynamic/HHMM/Motif/learn_motif_hhmm.m: Initial + revision + +2002-06-27 13:34 yozhik + + * BNT/general/add_ev_to_dmarginal.m: Initial import of code base + from Kevin Murphy. + +2002-06-27 13:34 yozhik + + * BNT/general/add_ev_to_dmarginal.m: Initial revision + +2002-06-24 16:54 yozhik + + * BNT/CPDs/@hhmmF_CPD/update_ess.m: Initial import of code base + from Kevin Murphy. + +2002-06-24 16:54 yozhik + + * BNT/CPDs/@hhmmF_CPD/update_ess.m: Initial revision + +2002-06-24 16:38 yozhik + + * BNT/CPDs/@hhmmF_CPD/hhmmF_CPD.m: Initial import of code base from + Kevin Murphy. + +2002-06-24 16:38 yozhik + + * BNT/CPDs/@hhmmF_CPD/hhmmF_CPD.m: Initial revision + +2002-06-24 15:45 yozhik + + * BNT/CPDs/@hhmmF_CPD/update_CPT.m: Initial import of code base + from Kevin Murphy. + +2002-06-24 15:45 yozhik + + * BNT/CPDs/@hhmmF_CPD/update_CPT.m: Initial revision + +2002-06-24 15:35 yozhik + + * BNT/CPDs/@hhmmF_CPD/Old/: hhmmF_CPD.m, log_prior.m, + maximize_params.m, reset_ess.m, update_CPT.m, update_ess.m: + Initial import of code base from Kevin Murphy. + +2002-06-24 15:35 yozhik + + * BNT/CPDs/@hhmmF_CPD/Old/: hhmmF_CPD.m, log_prior.m, + maximize_params.m, reset_ess.m, update_CPT.m, update_ess.m: + Initial revision + +2002-06-24 15:23 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_ess4.m: Initial import of code + base from Kevin Murphy. + +2002-06-24 15:23 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_ess4.m: Initial revision + +2002-06-24 15:08 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_ess3.m: Initial import of code + base from Kevin Murphy. + +2002-06-24 15:08 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_ess3.m: Initial revision + +2002-06-24 14:20 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_ess2.m: Initial import of code + base from Kevin Murphy. + +2002-06-24 14:20 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/update_ess2.m: Initial revision + +2002-06-24 11:56 yozhik + + * BNT/: general/mk_fgraph_given_ev.m, + CPDs/mk_isolated_tabular_CPD.m: Initial import of code base from + Kevin Murphy. + +2002-06-24 11:56 yozhik + + * BNT/: general/mk_fgraph_given_ev.m, + CPDs/mk_isolated_tabular_CPD.m: Initial revision + +2002-06-24 11:19 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/: hhmmQ_CPD.m, log_prior.m, + maximize_params.m, reset_ess.m, update_ess.m: Initial import of + code base from Kevin Murphy. + +2002-06-24 11:19 yozhik + + * BNT/CPDs/@hhmmQ_CPD/Old/: hhmmQ_CPD.m, log_prior.m, + maximize_params.m, reset_ess.m, update_ess.m: Initial revision + +2002-06-20 13:30 yozhik + + * BNT/examples/dynamic/mildew1.m: Initial import of code base from + Kevin Murphy. + +2002-06-20 13:30 yozhik + + * BNT/examples/dynamic/mildew1.m: Initial revision + +2002-06-19 17:18 yozhik + + * BNT/: inference/dynamic/@hmm_inf_engine/find_mpe.m, + examples/dynamic/HHMM/Square/learn_square_hhmm_cts.m: Initial + import of code base from Kevin Murphy. + +2002-06-19 17:18 yozhik + + * BNT/: inference/dynamic/@hmm_inf_engine/find_mpe.m, + examples/dynamic/HHMM/Square/learn_square_hhmm_cts.m: Initial + revision + +2002-06-19 17:03 yozhik + + * BNT/examples/static/fgraph/fg1.m: Initial import of code base + from Kevin Murphy. + +2002-06-19 17:03 yozhik + + * BNT/examples/static/fgraph/fg1.m: Initial revision + +2002-06-19 16:59 yozhik + + * BNT/: examples/static/softev1.m, + inference/static/@belprop_fg_inf_engine/find_mpe.m: Initial + import of code base from Kevin Murphy. + +2002-06-19 16:59 yozhik + + * BNT/: examples/static/softev1.m, + inference/static/@belprop_fg_inf_engine/find_mpe.m: Initial + revision + +2002-06-19 15:11 yozhik + + * BNT/inference/static/@var_elim_inf_engine/find_mpe.m: Initial + import of code base from Kevin Murphy. + +2002-06-19 15:11 yozhik + + * BNT/inference/static/@var_elim_inf_engine/find_mpe.m: Initial + revision + +2002-06-19 15:08 yozhik + + * BNT/: inference/static/@belprop_inf_engine/find_mpe.m, + examples/static/mpe1.m, examples/static/mpe2.m: Initial import of + code base from Kevin Murphy. + +2002-06-19 15:08 yozhik + + * BNT/: inference/static/@belprop_inf_engine/find_mpe.m, + examples/static/mpe1.m, examples/static/mpe2.m: Initial revision + +2002-06-19 15:04 yozhik + + * BNT/inference/static/@var_elim_inf_engine/: + var_elim_inf_engine.m, enter_evidence.m: Initial import of code + base from Kevin Murphy. + +2002-06-19 15:04 yozhik + + * BNT/inference/static/@var_elim_inf_engine/: + var_elim_inf_engine.m, enter_evidence.m: Initial revision + +2002-06-19 14:56 yozhik + + * BNT/inference/static/@global_joint_inf_engine/find_mpe.m: Initial + import of code base from Kevin Murphy. + +2002-06-19 14:56 yozhik + + * BNT/inference/static/@global_joint_inf_engine/find_mpe.m: Initial + revision + +2002-06-17 16:49 yozhik + + * BNT/inference/online/: @jtree_2TBN_inf_engine/back1_mpe.m, + @smoother_engine/find_mpe.m: Initial import of code base from + Kevin Murphy. + +2002-06-17 16:49 yozhik + + * BNT/inference/online/: @jtree_2TBN_inf_engine/back1_mpe.m, + @smoother_engine/find_mpe.m: Initial revision + +2002-06-17 16:46 yozhik + + * BNT/inference/online/: @jtree_2TBN_inf_engine/fwd.m, + @jtree_2TBN_inf_engine/fwd1.m, @smoother_engine/enter_evidence.m: + Initial import of code base from Kevin Murphy. + +2002-06-17 16:46 yozhik + + * BNT/inference/online/: @jtree_2TBN_inf_engine/fwd.m, + @jtree_2TBN_inf_engine/fwd1.m, @smoother_engine/enter_evidence.m: + Initial revision + +2002-06-17 16:38 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/backT_mpe.m: Initial + import of code base from Kevin Murphy. + +2002-06-17 16:38 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/backT_mpe.m: Initial + revision + +2002-06-17 16:34 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/: back.m, backT.m, + back1.m: Initial import of code base from Kevin Murphy. + +2002-06-17 16:34 yozhik + + * BNT/inference/online/@jtree_2TBN_inf_engine/: back.m, backT.m, + back1.m: Initial revision + +2002-06-17 16:14 yozhik + + * BNT/inference/static/@jtree_inf_engine/: find_mpe.m, + find_max_config.m: Initial import of code base from Kevin Murphy. + +2002-06-17 16:14 yozhik + + * BNT/inference/static/@jtree_inf_engine/: find_mpe.m, + find_max_config.m: Initial revision + +2002-06-17 14:58 yozhik + + * BNT/general/Old/calc_mpe.m: Initial import of code base from + Kevin Murphy. + +2002-06-17 14:58 yozhik + + * BNT/general/Old/calc_mpe.m: Initial revision + +2002-06-17 13:59 yozhik + + * BNT/inference/static/@jtree_inf_engine/: enter_evidence.m, + distribute_evidence.m: Initial import of code base from Kevin + Murphy. + +2002-06-17 13:59 yozhik + + * BNT/inference/static/@jtree_inf_engine/: enter_evidence.m, + distribute_evidence.m: Initial revision + +2002-06-17 13:29 yozhik + + * BNT/inference/static/@jtree_mnet_inf_engine/: find_mpe.m, + enter_evidence.m: Initial import of code base from Kevin Murphy. + +2002-06-17 13:29 yozhik + + * BNT/inference/static/@jtree_mnet_inf_engine/: find_mpe.m, + enter_evidence.m: Initial revision + +2002-06-16 13:01 yozhik + + * BNT/general/is_mnet.m: Initial import of code base from Kevin + Murphy. + +2002-06-16 13:01 yozhik + + * BNT/general/is_mnet.m: Initial revision + +2002-06-16 12:52 yozhik + + * BNT/general/mk_mnet.m: Initial import of code base from Kevin + Murphy. + +2002-06-16 12:52 yozhik + + * BNT/general/mk_mnet.m: Initial revision + +2002-06-16 12:34 yozhik + + * BNT/inference/static/@jtree_inf_engine/init_pot.m: Initial import + of code base from Kevin Murphy. + +2002-06-16 12:34 yozhik + + * BNT/inference/static/@jtree_inf_engine/init_pot.m: Initial + revision + +2002-06-16 12:06 yozhik + + * BNT/potentials/@dpot/find_most_prob_entry.m: Initial import of + code base from Kevin Murphy. + +2002-06-16 12:06 yozhik + + * BNT/potentials/@dpot/find_most_prob_entry.m: Initial revision + +2002-05-31 03:25 yozhik + + * BNT/general/unroll_higher_order_topology.m: Initial import of + code base from Kevin Murphy. + +2002-05-31 03:25 yozhik + + * BNT/general/unroll_higher_order_topology.m: Initial revision + +2002-05-29 08:59 yozhik + + * BNT/@assocarray/assocarray.m, + BNT/CPDs/@boolean_CPD/boolean_CPD.m, + BNT/CPDs/@discrete_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@discrete_CPD/CPD_to_pi.m, + BNT/CPDs/@discrete_CPD/CPD_to_scgpot.m, + BNT/CPDs/@discrete_CPD/README, + BNT/CPDs/@discrete_CPD/convert_CPD_to_table_hidden_ps.m, + BNT/CPDs/@discrete_CPD/convert_obs_CPD_to_table.m, + BNT/CPDs/@discrete_CPD/convert_to_sparse_table.c, + BNT/CPDs/@discrete_CPD/convert_to_table.m, + BNT/CPDs/@discrete_CPD/discrete_CPD.m, + BNT/CPDs/@discrete_CPD/dom_sizes.m, + BNT/CPDs/@discrete_CPD/log_prob_node.m, + BNT/CPDs/@discrete_CPD/prob_node.m, + BNT/CPDs/@discrete_CPD/sample_node.m, + BNT/CPDs/@discrete_CPD/Old/convert_to_pot.m, + BNT/CPDs/@discrete_CPD/Old/convert_to_table.m, + BNT/CPDs/@discrete_CPD/Old/prob_CPD.m, + BNT/CPDs/@discrete_CPD/Old/prob_node.m, + BNT/CPDs/@discrete_CPD/private/prod_CPT_and_pi_msgs.m, + BNT/CPDs/@gaussian_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@gaussian_CPD/CPD_to_pi.m, + BNT/CPDs/@gaussian_CPD/CPD_to_scgpot.m, + BNT/CPDs/@gaussian_CPD/adjustable_CPD.m, + BNT/CPDs/@gaussian_CPD/convert_CPD_to_table_hidden_ps.m, + BNT/CPDs/@gaussian_CPD/display.m, + BNT/CPDs/@gaussian_CPD/get_field.m, + BNT/CPDs/@gaussian_CPD/reset_ess.m, + BNT/CPDs/@gaussian_CPD/sample_node.m, + BNT/CPDs/@gaussian_CPD/set_fields.m, + BNT/CPDs/@gaussian_CPD/Old/CPD_to_lambda_msg.m, + BNT/CPDs/@gaussian_CPD/Old/gaussian_CPD.m, + BNT/CPDs/@gaussian_CPD/Old/log_prob_node.m, + BNT/CPDs/@gaussian_CPD/Old/update_ess.m, + BNT/CPDs/@gaussian_CPD/Old/update_tied_ess.m, + BNT/CPDs/@gaussian_CPD/private/CPD_to_linear_gaussian.m, + BNT/CPDs/@generic_CPD/README, + BNT/CPDs/@generic_CPD/adjustable_CPD.m, + BNT/CPDs/@generic_CPD/display.m, + BNT/CPDs/@generic_CPD/generic_CPD.m, + BNT/CPDs/@generic_CPD/log_prior.m, + BNT/CPDs/@generic_CPD/set_clamped.m, + BNT/CPDs/@generic_CPD/Old/BIC_score_CPD.m, + BNT/CPDs/@generic_CPD/Old/CPD_to_dpots.m, + BNT/CPDs/@gmux_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@gmux_CPD/convert_to_pot.m, + BNT/CPDs/@gmux_CPD/CPD_to_pi.m, BNT/CPDs/@gmux_CPD/display.m, + BNT/CPDs/@gmux_CPD/gmux_CPD.m, BNT/CPDs/@gmux_CPD/sample_node.m, + BNT/CPDs/@gmux_CPD/Old/gmux_CPD.m, + BNT/CPDs/@hhmmF_CPD/log_prior.m, + BNT/CPDs/@hhmmF_CPD/maximize_params.m, + BNT/CPDs/@hhmmF_CPD/reset_ess.m, BNT/CPDs/@hhmmQ_CPD/log_prior.m, + BNT/CPDs/@hhmmQ_CPD/reset_ess.m, + BNT/CPDs/@mlp_CPD/convert_to_table.m, + BNT/CPDs/@mlp_CPD/maximize_params.m, BNT/CPDs/@mlp_CPD/mlp_CPD.m, + BNT/CPDs/@mlp_CPD/reset_ess.m, BNT/CPDs/@mlp_CPD/update_ess.m, + BNT/CPDs/@noisyor_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@noisyor_CPD/CPD_to_pi.m, + BNT/CPDs/@noisyor_CPD/noisyor_CPD.m, + BNT/CPDs/@noisyor_CPD/private/sum_prod_CPD_and_pi_msgs.m, + BNT/CPDs/@root_CPD/CPD_to_pi.m, + BNT/CPDs/@root_CPD/convert_to_pot.m, + BNT/CPDs/@root_CPD/log_marg_prob_node.m, + BNT/CPDs/@root_CPD/log_prob_node.m, + BNT/CPDs/@root_CPD/root_CPD.m, BNT/CPDs/@root_CPD/sample_node.m, + BNT/CPDs/@root_CPD/Old/CPD_to_CPT.m, + BNT/CPDs/@softmax_CPD/convert_to_pot.m, + BNT/CPDs/@softmax_CPD/display.m, + BNT/CPDs/@softmax_CPD/get_field.m, + BNT/CPDs/@softmax_CPD/maximize_params.m, + BNT/CPDs/@softmax_CPD/reset_ess.m, + BNT/CPDs/@softmax_CPD/sample_node.m, + BNT/CPDs/@softmax_CPD/set_fields.m, + BNT/CPDs/@softmax_CPD/update_ess.m, + BNT/CPDs/@softmax_CPD/private/extract_params.m, + BNT/CPDs/@tabular_CPD/CPD_to_CPT.m, + BNT/CPDs/@tabular_CPD/bayes_update_params.m, + BNT/CPDs/@tabular_CPD/log_nextcase_prob_node.m, + BNT/CPDs/@tabular_CPD/log_prior.m, + BNT/CPDs/@tabular_CPD/reset_ess.m, + BNT/CPDs/@tabular_CPD/update_ess.m, + BNT/CPDs/@tabular_CPD/update_ess_simple.m, + BNT/CPDs/@tabular_CPD/Old/BIC_score_CPD.m, + BNT/CPDs/@tabular_CPD/Old/bayesian_score_CPD.m, + BNT/CPDs/@tabular_CPD/Old/log_marg_prob_node_case.m, + BNT/CPDs/@tabular_CPD/Old/mult_CPD_and_pi_msgs.m, + BNT/CPDs/@tabular_CPD/Old/prob_CPT.m, + BNT/CPDs/@tabular_CPD/Old/prob_node.m, + BNT/CPDs/@tabular_CPD/Old/sample_node.m, + BNT/CPDs/@tabular_CPD/Old/sample_node_single_case.m, + BNT/CPDs/@tabular_CPD/Old/tabular_CPD.m, + BNT/CPDs/@tabular_CPD/Old/update_params.m, + BNT/CPDs/@tabular_decision_node/CPD_to_CPT.m, + BNT/CPDs/@tabular_decision_node/display.m, + BNT/CPDs/@tabular_decision_node/get_field.m, + BNT/CPDs/@tabular_decision_node/set_fields.m, + BNT/CPDs/@tabular_decision_node/tabular_decision_node.m, + BNT/CPDs/@tabular_decision_node/Old/tabular_decision_node.m, + BNT/CPDs/@tabular_kernel/convert_to_pot.m, + BNT/CPDs/@tabular_kernel/convert_to_table.m, + BNT/CPDs/@tabular_kernel/get_field.m, + BNT/CPDs/@tabular_kernel/set_fields.m, + BNT/CPDs/@tabular_kernel/tabular_kernel.m, + BNT/CPDs/@tabular_kernel/Old/tabular_kernel.m, + BNT/CPDs/@tabular_utility_node/convert_to_pot.m, + BNT/CPDs/@tabular_utility_node/display.m, + BNT/CPDs/@tabular_utility_node/tabular_utility_node.m, + BNT/CPDs/@tree_CPD/display.m, + BNT/CPDs/@tree_CPD/evaluate_tree_performance.m, + BNT/CPDs/@tree_CPD/get_field.m, + BNT/CPDs/@tree_CPD/learn_params.m, BNT/CPDs/@tree_CPD/readme.txt, + BNT/CPDs/@tree_CPD/set_fields.m, BNT/CPDs/@tree_CPD/tree_CPD.m, + BNT/CPDs/Old/@linear_gaussian_CPD/linear_gaussian_CPD.m, + BNT/CPDs/Old/@linear_gaussian_CPD/log_marg_prob_node.m, + BNT/CPDs/Old/@linear_gaussian_CPD/update_params_complete.m, + BNT/CPDs/Old/@root_gaussian_CPD/log_marg_prob_node.m, + BNT/CPDs/Old/@root_gaussian_CPD/root_gaussian_CPD.m, + BNT/CPDs/Old/@root_gaussian_CPD/update_params_complete.m, + BNT/CPDs/Old/@tabular_chance_node/CPD_to_upot.m, + BNT/CPDs/Old/@tabular_chance_node/tabular_chance_node.m, + BNT/examples/dynamic/bat1.m, BNT/examples/dynamic/bkff1.m, + BNT/examples/dynamic/chmm1.m, + BNT/examples/dynamic/cmp_inference_dbn.m, + BNT/examples/dynamic/cmp_learning_dbn.m, + BNT/examples/dynamic/cmp_online_inference.m, + BNT/examples/dynamic/fhmm_infer.m, + BNT/examples/dynamic/filter_test1.m, + BNT/examples/dynamic/kalman1.m, + BNT/examples/dynamic/kjaerulff1.m, + BNT/examples/dynamic/loopy_dbn1.m, + BNT/examples/dynamic/mk_collage_from_clqs.m, + BNT/examples/dynamic/mk_fhmm.m, BNT/examples/dynamic/reveal1.m, + BNT/examples/dynamic/scg_dbn.m, + BNT/examples/dynamic/skf_data_assoc_gmux.m, + BNT/examples/dynamic/HHMM/add_hhmm_end_state.m, + BNT/examples/dynamic/HHMM/hhmm_jtree_clqs.m, + BNT/examples/dynamic/HHMM/mk_hhmm_topo.m, + BNT/examples/dynamic/HHMM/mk_hhmm_topo_F1.m, + BNT/examples/dynamic/HHMM/pretty_print_hhmm_parse.m, + BNT/examples/dynamic/HHMM/Motif/fixed_args_mk_motif_hhmm.m, + BNT/examples/dynamic/HHMM/Motif/mk_motif_hhmm.m, + BNT/examples/dynamic/HHMM/Motif/sample_motif_hhmm.m, + BNT/examples/dynamic/HHMM/Old/mk_abcd_hhmm.m, + BNT/examples/dynamic/HHMM/Old/mk_arrow_alpha_hhmm3.m, + BNT/examples/dynamic/HHMM/Old/mk_hhmm2.m, + BNT/examples/dynamic/HHMM/Old/mk_hhmm3.m, + BNT/examples/dynamic/HHMM/Old/mk_hhmm3_args.m, + BNT/examples/dynamic/HHMM/Old/motif_hhmm.m, + BNT/examples/dynamic/HHMM/Old/remove_hhmm_end_state.m, + BNT/examples/dynamic/HHMM/Square/get_square_data.m, + BNT/examples/dynamic/HHMM/Square/hhmm_inference.m, + BNT/examples/dynamic/HHMM/Square/is_F2_true_D3.m, + BNT/examples/dynamic/HHMM/Square/learn_square_hhmm_discrete.m, + BNT/examples/dynamic/HHMM/Square/mk_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/plot_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/sample_square_hhmm_cts.m, + BNT/examples/dynamic/HHMM/Square/sample_square_hhmm_discrete.m, + BNT/examples/dynamic/HHMM/Square/square4.mat, + BNT/examples/dynamic/HHMM/Square/square4_cases.mat, + BNT/examples/dynamic/HHMM/Square/test_square_fig.m, + BNT/examples/dynamic/HHMM/Square/test_square_fig.mat, + BNT/examples/dynamic/HHMM/Square/Old/learn_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/Old/mk_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/Old/plot_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/Old/sample_square_hhmm.m, + BNT/examples/dynamic/Old/chmm1.m, + BNT/examples/dynamic/Old/cmp_inference.m, + BNT/examples/dynamic/Old/kalman1.m, + BNT/examples/dynamic/Old/old.water1.m, + BNT/examples/dynamic/Old/online1.m, + BNT/examples/dynamic/Old/online2.m, + BNT/examples/dynamic/Old/scg_dbn.m, + BNT/examples/dynamic/SLAM/mk_gmux_robot_dbn.m, + BNT/examples/dynamic/SLAM/mk_linear_slam.m, + BNT/examples/dynamic/SLAM/slam_kf.m, + BNT/examples/dynamic/SLAM/slam_offline_loopy.m, + BNT/examples/dynamic/SLAM/slam_partial_kf.m, + BNT/examples/dynamic/SLAM/slam_stationary_loopy.m, + BNT/examples/dynamic/SLAM/Old/offline_loopy_slam.m, + BNT/examples/dynamic/SLAM/Old/paskin1.m, + BNT/examples/dynamic/SLAM/Old/skf_data_assoc_gmux2.m, + BNT/examples/dynamic/SLAM/Old/slam_kf.m, + BNT/examples/limids/id1.m, BNT/examples/limids/pigs1.m, + BNT/examples/static/cg1.m, BNT/examples/static/cg2.m, + BNT/examples/static/discrete2.m, BNT/examples/static/discrete3.m, + BNT/examples/static/fa1.m, BNT/examples/static/gaussian1.m, + BNT/examples/static/gibbs_test1.m, BNT/examples/static/lw1.m, + BNT/examples/static/mfa1.m, BNT/examples/static/mixexp1.m, + BNT/examples/static/mixexp2.m, BNT/examples/static/mixexp3.m, + BNT/examples/static/mog1.m, BNT/examples/static/qmr1.m, + BNT/examples/static/sample1.m, BNT/examples/static/softmax1.m, + BNT/examples/static/Belprop/belprop_loop1_discrete.m, + BNT/examples/static/Belprop/belprop_loop1_gauss.m, + BNT/examples/static/Belprop/belprop_loopy_cg.m, + BNT/examples/static/Belprop/belprop_loopy_discrete.m, + BNT/examples/static/Belprop/belprop_loopy_gauss.m, + BNT/examples/static/Belprop/belprop_polytree_cg.m, + BNT/examples/static/Belprop/belprop_polytree_gauss.m, + BNT/examples/static/Belprop/bp1.m, + BNT/examples/static/Belprop/gmux1.m, + BNT/examples/static/Brutti/Belief_IOhmm.m, + BNT/examples/static/Brutti/Belief_hmdt.m, + BNT/examples/static/Brutti/Belief_hme.m, + BNT/examples/static/Brutti/Sigmoid_Belief.m, + BNT/examples/static/HME/HMEforMatlab.jpg, + BNT/examples/static/HME/README, BNT/examples/static/HME/fhme.m, + BNT/examples/static/HME/gen_data.m, + BNT/examples/static/HME/hme_class_plot.m, + BNT/examples/static/HME/hme_reg_plot.m, + BNT/examples/static/HME/hme_topobuilder.m, + BNT/examples/static/HME/test_data_class.mat, + BNT/examples/static/HME/test_data_class2.mat, + BNT/examples/static/HME/test_data_reg.mat, + BNT/examples/static/HME/train_data_class.mat, + BNT/examples/static/HME/train_data_reg.mat, + BNT/examples/static/Misc/mixexp_data.txt, + BNT/examples/static/Misc/mixexp_graddesc.m, + BNT/examples/static/Misc/mixexp_plot.m, + BNT/examples/static/Misc/sprinkler.bif, + BNT/examples/static/Models/mk_cancer_bnet.m, + BNT/examples/static/Models/mk_car_bnet.m, + BNT/examples/static/Models/mk_ideker_bnet.m, + BNT/examples/static/Models/mk_incinerator_bnet.m, + BNT/examples/static/Models/mk_markov_chain_bnet.m, + BNT/examples/static/Models/mk_minimal_qmr_bnet.m, + BNT/examples/static/Models/mk_qmr_bnet.m, + BNT/examples/static/Models/mk_vstruct_bnet.m, + BNT/examples/static/Models/Old/mk_hmm_bnet.m, + BNT/examples/static/SCG/scg1.m, BNT/examples/static/SCG/scg2.m, + BNT/examples/static/SCG/scg3.m, + BNT/examples/static/SCG/scg_3node.m, + BNT/examples/static/SCG/scg_unstable.m, + BNT/examples/static/StructLearn/bic1.m, + BNT/examples/static/StructLearn/cooper_yoo.m, + BNT/examples/static/StructLearn/k2demo1.m, + BNT/examples/static/StructLearn/mcmc1.m, + BNT/examples/static/StructLearn/pc1.m, + BNT/examples/static/StructLearn/pc2.m, + BNT/examples/static/Zoubin/README, + BNT/examples/static/Zoubin/csum.m, + BNT/examples/static/Zoubin/ffa.m, + BNT/examples/static/Zoubin/mfa.m, + BNT/examples/static/Zoubin/mfa_cl.m, + BNT/examples/static/Zoubin/mfademo.m, + BNT/examples/static/Zoubin/rdiv.m, + BNT/examples/static/Zoubin/rprod.m, + BNT/examples/static/Zoubin/rsum.m, + BNT/examples/static/dtree/test_housing.m, + BNT/examples/static/dtree/test_restaurants.m, + BNT/examples/static/dtree/test_zoo1.m, + BNT/examples/static/dtree/tmp.dot, + BNT/examples/static/dtree/transform_data_into_bnt_format.m, + BNT/examples/static/fgraph/fg2.m, + BNT/examples/static/fgraph/fg3.m, + BNT/examples/static/fgraph/fg_mrf1.m, + BNT/examples/static/fgraph/fg_mrf2.m, + BNT/general/bnet_to_fgraph.m, + BNT/general/compute_fwd_interface.m, + BNT/general/compute_interface_nodes.m, + BNT/general/compute_minimal_interface.m, + BNT/general/dbn_to_bnet.m, + BNT/general/determine_elim_constraints.m, + BNT/general/do_intervention.m, BNT/general/dsep.m, + BNT/general/enumerate_scenarios.m, BNT/general/fgraph_to_bnet.m, + BNT/general/log_lik_complete.m, + BNT/general/log_marg_lik_complete.m, BNT/general/mk_bnet.m, + BNT/general/mk_fgraph.m, BNT/general/mk_limid.m, + BNT/general/mk_mutilated_samples.m, + BNT/general/mk_slice_and_half_dbn.m, + BNT/general/partition_dbn_nodes.m, + BNT/general/sample_bnet_nocell.m, BNT/general/sample_dbn.m, + BNT/general/score_bnet_complete.m, + BNT/general/unroll_dbn_topology.m, + BNT/general/Old/bnet_to_gdl_graph.m, + BNT/general/Old/calc_mpe_bucket.m, + BNT/general/Old/calc_mpe_dbn.m, + BNT/general/Old/calc_mpe_given_inf_engine.m, + BNT/general/Old/calc_mpe_global.m, + BNT/general/Old/compute_interface_nodes.m, + BNT/general/Old/mk_gdl_graph.m, GraphViz/draw_dbn.m, + GraphViz/make_layout.m, BNT/license.gpl.txt, + BNT/general/add_evidence_to_gmarginal.m, + BNT/inference/@inf_engine/bnet_from_engine.m, + BNT/inference/@inf_engine/get_field.m, + BNT/inference/@inf_engine/inf_engine.m, + BNT/inference/@inf_engine/marginal_family.m, + BNT/inference/@inf_engine/set_fields.m, + BNT/inference/@inf_engine/update_engine.m, + BNT/inference/@inf_engine/Old/marginal_family_pot.m, + BNT/inference/@inf_engine/Old/observed_nodes.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/bk_ff_hmm_inf_engine.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_init_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_marginal_from_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_predict_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_update_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_update_bel1.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/marginal_family.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/bk_ff_fb.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/combine_marginals_into_joint.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/dbn_to_hmm.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/mk_hmm_obs_lik_mat.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/mk_hmm_obs_lik_vec.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/mk_hmm_obs_lik_vec1.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/project_joint_onto_marginals.m, + BNT/inference/dynamic/@bk_inf_engine/bk_inf_engine.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_init_bel.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_marginal_from_bel.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_update_bel.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_update_bel1.m, + BNT/inference/dynamic/@bk_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@bk_inf_engine/marginal_family.m, + BNT/inference/dynamic/@bk_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@bk_inf_engine/update_engine.m, + BNT/inference/dynamic/@ff_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/enter_soft_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/ff_inf_engine.m, + BNT/inference/dynamic/@ff_inf_engine/filter_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/marginal_family.m, + BNT/inference/dynamic/@ff_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@ff_inf_engine/smooth_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/Old/enter_soft_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/Old/enter_soft_evidence1.m, + BNT/inference/dynamic/@ff_inf_engine/Old/marginal_family.m, + BNT/inference/dynamic/@frontier_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@frontier_inf_engine/enter_soft_evidence.m, + BNT/inference/dynamic/@frontier_inf_engine/frontier_inf_engine.m, + BNT/inference/dynamic/@frontier_inf_engine/marginal_family.m, + BNT/inference/dynamic/@frontier_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@frontier_inf_engine/set_fwdback.m, + BNT/inference/dynamic/@hmm_inf_engine/update_engine.m, + BNT/inference/dynamic/@hmm_inf_engine/Old/dhmm_inf_engine.m, + BNT/inference/dynamic/@hmm_inf_engine/Old/marginal_family.m, + BNT/inference/dynamic/@hmm_inf_engine/Old/marginal_nodes.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/marginal_family.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence1.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence2.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence3.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence4.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/marginal_nodes.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/enter_soft_evidence_nonint.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/enter_soft_evidence_trans.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/jtree_dbn_inf_engine.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/jtree_dbn_inf_engine1.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/jtree_dbn_inf_engine2.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/jtree_unrolled_dbn_inf_engine.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/marginal_family.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/update_engine.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/Old/marginal_family.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/Old/marginal_nodes.m, + BNT/inference/dynamic/@kalman_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@kalman_inf_engine/kalman_inf_engine.m, + BNT/inference/dynamic/@kalman_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@kalman_inf_engine/update_engine.m, + BNT/inference/dynamic/@kalman_inf_engine/private/dbn_to_lds.m, + BNT/inference/dynamic/@kalman_inf_engine/private/extract_params_from_gbn.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/enter_soft_ev.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/pearl_dbn_inf_engine.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/correct_smooth.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/enter_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/filter_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/filter_evidence_obj_oriented.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/smooth_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/smooth_evidence_fast.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/wrong_smooth.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/private/init_pearl_dbn_ev_msgs.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/marginal_family.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/pearl_unrolled_dbn_inf_engine.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/update_engine.m, + BNT/inference/online/@filter_engine/bnet_from_engine.m, + BNT/inference/online/@filter_engine/enter_evidence.m, + BNT/inference/online/@filter_engine/filter_engine.m, + BNT/inference/online/@filter_engine/marginal_family.m, + BNT/inference/online/@filter_engine/marginal_nodes.m, + BNT/inference/online/@hmm_2TBN_inf_engine/back.m, + BNT/inference/online/@hmm_2TBN_inf_engine/backT.m, + BNT/inference/online/@hmm_2TBN_inf_engine/fwd.m, + BNT/inference/online/@hmm_2TBN_inf_engine/fwd1.m, + BNT/inference/online/@hmm_2TBN_inf_engine/update_engine.m, + BNT/inference/online/@jtree_2TBN_inf_engine/marginal_family.m, + BNT/inference/online/@jtree_2TBN_inf_engine/marginal_nodes.m, + BNT/inference/online/@jtree_2TBN_inf_engine/Old/jtree_2TBN_inf_engine.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/back.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/back1.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/backT.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/enter_evidence.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/fwd.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/fwd1.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/jtree_sparse_2TBN_inf_engine.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/marginal_family.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/marginal_nodes.m, + BNT/inference/online/@smoother_engine/bnet_from_engine.m, + BNT/inference/online/@smoother_engine/marginal_family.m, + BNT/inference/online/@smoother_engine/marginal_nodes.m, + BNT/inference/online/@smoother_engine/smoother_engine.m, + BNT/inference/online/@smoother_engine/update_engine.m, + BNT/inference/static/@belprop_fg_inf_engine/belprop_fg_inf_engine.m, + BNT/inference/static/@belprop_fg_inf_engine/enter_evidence.m, + BNT/inference/static/@belprop_fg_inf_engine/loopy_converged.m, + BNT/inference/static/@belprop_fg_inf_engine/marginal_nodes.m, + BNT/inference/static/@belprop_fg_inf_engine/set_params.m, + BNT/inference/static/@belprop_inf_engine/enter_evidence.m, + BNT/inference/static/@belprop_inf_engine/loopy_converged.m, + BNT/inference/static/@belprop_inf_engine/marginal_family.m, + BNT/inference/static/@belprop_inf_engine/marginal_nodes.m, + BNT/inference/static/@belprop_inf_engine/Old/belprop_gdl_inf_engine.m, + BNT/inference/static/@belprop_inf_engine/Old/belprop_inf_engine_nostr.m, + BNT/inference/static/@belprop_inf_engine/Old/enter_evidence.m, + BNT/inference/static/@belprop_inf_engine/Old/enter_evidence1.m, + BNT/inference/static/@belprop_inf_engine/Old/marginal_domain.m, + BNT/inference/static/@belprop_inf_engine/private/junk, + BNT/inference/static/@belprop_inf_engine/private/parallel_protocol.m, + BNT/inference/static/@belprop_inf_engine/private/tree_protocol.m, + BNT/inference/static/@cond_gauss_inf_engine/cond_gauss_inf_engine.m, + BNT/inference/static/@cond_gauss_inf_engine/enter_evidence.m, + BNT/inference/static/@cond_gauss_inf_engine/marginal_nodes.m, + BNT/inference/static/@enumerative_inf_engine/enter_evidence.m, + BNT/inference/static/@enumerative_inf_engine/enumerative_inf_engine.m, + BNT/inference/static/@enumerative_inf_engine/marginal_nodes.m, + BNT/inference/static/@gaussian_inf_engine/enter_evidence.m, + BNT/inference/static/@gaussian_inf_engine/marginal_nodes.m, + BNT/inference/static/@gaussian_inf_engine/private/extract_params_from_gbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/enter_evidence.m, + BNT/inference/static/@gibbs_sampling_inf_engine/gibbs_sampling_inf_engine.m, + BNT/inference/static/@gibbs_sampling_inf_engine/marginal_nodes.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/CPT.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_children.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_families.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_families_dbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_posterior.c, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_posterior_dbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_strides.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/get_cpts.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/get_slice_dbn.c, + BNT/inference/static/@gibbs_sampling_inf_engine/private/get_slice_dbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/my_sample_discrete.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/sample_single_discrete.c, + BNT/inference/static/@global_joint_inf_engine/global_joint_inf_engine.m, + BNT/inference/static/@global_joint_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_inf_engine/cliques_from_engine.m, + BNT/inference/static/@jtree_inf_engine/clq_containing_nodes.m, + BNT/inference/static/@jtree_inf_engine/collect_evidence.m, + BNT/inference/static/@jtree_inf_engine/enter_soft_evidence.m, + BNT/inference/static/@jtree_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_inf_engine/marginal_nodes.m, + BNT/inference/static/@jtree_inf_engine/set_fields.m, + BNT/inference/static/@jtree_inf_engine/Old/collect_evidence.m, + BNT/inference/static/@jtree_inf_engine/Old/distribute_evidence.m, + BNT/inference/static/@jtree_inf_engine/Old/enter_evidence.m, + BNT/inference/static/@jtree_inf_engine/Old/enter_soft_evidence.m, + BNT/inference/static/@jtree_limid_inf_engine/enter_evidence.m, + BNT/inference/static/@jtree_limid_inf_engine/jtree_limid_inf_engine.m, + BNT/inference/static/@jtree_limid_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_limid_inf_engine/marginal_nodes.m, + BNT/inference/static/@jtree_limid_inf_engine/Old/marginal_family.m, + BNT/inference/static/@jtree_limid_inf_engine/Old/marginal_nodes_SS.m, + BNT/inference/static/@jtree_sparse_inf_engine/cliques_from_engine.m, + BNT/inference/static/@jtree_sparse_inf_engine/clq_containing_nodes.m, + BNT/inference/static/@jtree_sparse_inf_engine/collect_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/distribute_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/enter_evidence.m, + BNT/inference/static/@jtree_sparse_inf_engine/enter_soft_evidence.m, + BNT/inference/static/@jtree_sparse_inf_engine/init_pot.c, + BNT/inference/static/@jtree_sparse_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_sparse_inf_engine/marginal_nodes.m, + BNT/inference/static/@jtree_sparse_inf_engine/set_fields.m, + BNT/inference/static/@jtree_sparse_inf_engine/old/collect_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/distribute_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/init_pot.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/init_pot1.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/init_pot1.m, + BNT/inference/static/@likelihood_weighting_inf_engine/enter_evidence.m, + BNT/inference/static/@likelihood_weighting_inf_engine/likelihood_weighting_inf_engine.m, + BNT/inference/static/@likelihood_weighting_inf_engine/marginal_nodes.m, + BNT/inference/static/@pearl_inf_engine/enter_evidence.m, + BNT/inference/static/@pearl_inf_engine/loopy_converged.m, + BNT/inference/static/@pearl_inf_engine/marginal_nodes.m, + BNT/inference/static/@pearl_inf_engine/private/compute_bel.m, + BNT/inference/static/@pearl_inf_engine/private/prod_lambda_msgs.m, + BNT/inference/static/@pearl_inf_engine/private/tree_protocol.m, + BNT/inference/static/@quickscore_inf_engine/enter_evidence.m, + BNT/inference/static/@quickscore_inf_engine/marginal_nodes.m, + BNT/inference/static/@quickscore_inf_engine/quickscore_inf_engine.m, + BNT/inference/static/@quickscore_inf_engine/private/C_quickscore.c, + BNT/inference/static/@quickscore_inf_engine/private/nr.h, + BNT/inference/static/@quickscore_inf_engine/private/nrutil.c, + BNT/inference/static/@quickscore_inf_engine/private/nrutil.h, + BNT/inference/static/@quickscore_inf_engine/private/quickscore.m, + BNT/learning/bayes_update_params.m, + BNT/learning/bic_score_family.m, + BNT/learning/compute_cooling_schedule.m, + BNT/learning/dirichlet_score_family.m, + BNT/learning/kpm_learn_struct_mcmc.m, + BNT/learning/learn_params_em.m, + BNT/learning/learn_struct_dbn_reveal.m, + BNT/learning/learn_struct_pdag_ic_star.m, + BNT/learning/mcmc_sample_to_hist.m, BNT/learning/mk_schedule.m, + BNT/learning/mk_tetrad_data_file.m, + BNT/learning/score_dags_old.m, HMM/dhmm_logprob_brute_force.m, + HMM/dhmm_logprob_path.m, HMM/mdp_sample.m, Kalman/AR_to_SS.m, + Kalman/SS_to_AR.m, Kalman/convert_to_lagged_form.m, + Kalman/ensure_AR.m, Kalman/eval_AR_perf.m, + Kalman/kalman_filter.m, Kalman/kalman_smoother.m, + Kalman/kalman_update.m, Kalman/learn_AR.m, + Kalman/learn_AR_diagonal.m, Kalman/learn_kalman.m, + Kalman/smooth_update.m, + BNT/general/convert_dbn_CPDs_to_tables_slow.m, + BNT/general/dispcpt.m, BNT/general/linear_gaussian_to_cpot.m, + BNT/general/partition_matrix_vec_3.m, + BNT/general/shrink_obs_dims_in_gaussian.m, + BNT/general/shrink_obs_dims_in_table.m, + BNT/potentials/CPD_to_pot.m, BNT/potentials/README, + BNT/potentials/check_for_cd_arcs.m, + BNT/potentials/determine_pot_type.m, + BNT/potentials/mk_initial_pot.m, + BNT/potentials/@cgpot/cg_can_to_mom.m, + BNT/potentials/@cgpot/cg_mom_to_can.m, + BNT/potentials/@cgpot/cgpot.m, BNT/potentials/@cgpot/display.m, + BNT/potentials/@cgpot/divide_by_pot.m, + BNT/potentials/@cgpot/domain_pot.m, + BNT/potentials/@cgpot/enter_cts_evidence_pot.m, + BNT/potentials/@cgpot/enter_discrete_evidence_pot.m, + BNT/potentials/@cgpot/marginalize_pot.m, + BNT/potentials/@cgpot/multiply_by_pot.m, + BNT/potentials/@cgpot/multiply_pots.m, + BNT/potentials/@cgpot/normalize_pot.m, + BNT/potentials/@cgpot/pot_to_marginal.m, + BNT/potentials/@cgpot/Old/normalize_pot.m, + BNT/potentials/@cgpot/Old/simple_marginalize_pot.m, + BNT/potentials/@cpot/cpot.m, BNT/potentials/@cpot/cpot_to_mpot.m, + BNT/potentials/@cpot/display.m, + BNT/potentials/@cpot/divide_by_pot.m, + BNT/potentials/@cpot/domain_pot.m, + BNT/potentials/@cpot/enter_cts_evidence_pot.m, + BNT/potentials/@cpot/marginalize_pot.m, + BNT/potentials/@cpot/multiply_by_pot.m, + BNT/potentials/@cpot/multiply_pots.m, + BNT/potentials/@cpot/normalize_pot.m, + BNT/potentials/@cpot/pot_to_marginal.m, + BNT/potentials/@cpot/rescale_pot.m, + BNT/potentials/@cpot/set_domain_pot.m, + BNT/potentials/@cpot/Old/cpot_to_mpot.m, + BNT/potentials/@cpot/Old/normalize_pot.convert.m, + BNT/potentials/@dpot/approxeq_pot.m, + BNT/potentials/@dpot/display.m, + BNT/potentials/@dpot/domain_pot.m, + BNT/potentials/@dpot/dpot_to_table.m, + BNT/potentials/@dpot/get_fields.m, + BNT/potentials/@dpot/multiply_pots.m, + BNT/potentials/@dpot/pot_to_marginal.m, + BNT/potentials/@dpot/set_domain_pot.m, + BNT/potentials/@mpot/display.m, + BNT/potentials/@mpot/marginalize_pot.m, + BNT/potentials/@mpot/mpot.m, BNT/potentials/@mpot/mpot_to_cpot.m, + BNT/potentials/@mpot/normalize_pot.m, + BNT/potentials/@mpot/pot_to_marginal.m, + BNT/potentials/@mpot/rescale_pot.m, + BNT/potentials/@upot/approxeq_pot.m, + BNT/potentials/@upot/display.m, + BNT/potentials/@upot/divide_by_pot.m, + BNT/potentials/@upot/marginalize_pot.m, + BNT/potentials/@upot/multiply_by_pot.m, + BNT/potentials/@upot/normalize_pot.m, + BNT/potentials/@upot/pot_to_marginal.m, + BNT/potentials/@upot/upot.m, + BNT/potentials/@upot/upot_to_opt_policy.m, + BNT/potentials/Old/comp_eff_node_sizes.m, + BNT/potentials/Tables/divide_by_sparse_table.c, + BNT/potentials/Tables/divide_by_table.c, + BNT/potentials/Tables/marg_sparse_table.c, + BNT/potentials/Tables/marg_table.c, + BNT/potentials/Tables/mult_by_sparse_table.c, + BNT/potentials/Tables/rep_mult.c, HMM/mk_leftright_transmat.m: + Initial import of code base from Kevin Murphy. + +2002-05-29 08:59 yozhik + + * BNT/@assocarray/assocarray.m, + BNT/CPDs/@boolean_CPD/boolean_CPD.m, + BNT/CPDs/@discrete_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@discrete_CPD/CPD_to_pi.m, + BNT/CPDs/@discrete_CPD/CPD_to_scgpot.m, + BNT/CPDs/@discrete_CPD/README, + BNT/CPDs/@discrete_CPD/convert_CPD_to_table_hidden_ps.m, + BNT/CPDs/@discrete_CPD/convert_obs_CPD_to_table.m, + BNT/CPDs/@discrete_CPD/convert_to_sparse_table.c, + BNT/CPDs/@discrete_CPD/convert_to_table.m, + BNT/CPDs/@discrete_CPD/discrete_CPD.m, + BNT/CPDs/@discrete_CPD/dom_sizes.m, + BNT/CPDs/@discrete_CPD/log_prob_node.m, + BNT/CPDs/@discrete_CPD/prob_node.m, + BNT/CPDs/@discrete_CPD/sample_node.m, + BNT/CPDs/@discrete_CPD/Old/convert_to_pot.m, + BNT/CPDs/@discrete_CPD/Old/convert_to_table.m, + BNT/CPDs/@discrete_CPD/Old/prob_CPD.m, + BNT/CPDs/@discrete_CPD/Old/prob_node.m, + BNT/CPDs/@discrete_CPD/private/prod_CPT_and_pi_msgs.m, + BNT/CPDs/@gaussian_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@gaussian_CPD/CPD_to_pi.m, + BNT/CPDs/@gaussian_CPD/CPD_to_scgpot.m, + BNT/CPDs/@gaussian_CPD/adjustable_CPD.m, + BNT/CPDs/@gaussian_CPD/convert_CPD_to_table_hidden_ps.m, + BNT/CPDs/@gaussian_CPD/display.m, + BNT/CPDs/@gaussian_CPD/get_field.m, + BNT/CPDs/@gaussian_CPD/reset_ess.m, + BNT/CPDs/@gaussian_CPD/sample_node.m, + BNT/CPDs/@gaussian_CPD/set_fields.m, + BNT/CPDs/@gaussian_CPD/Old/CPD_to_lambda_msg.m, + BNT/CPDs/@gaussian_CPD/Old/gaussian_CPD.m, + BNT/CPDs/@gaussian_CPD/Old/log_prob_node.m, + BNT/CPDs/@gaussian_CPD/Old/update_ess.m, + BNT/CPDs/@gaussian_CPD/Old/update_tied_ess.m, + BNT/CPDs/@gaussian_CPD/private/CPD_to_linear_gaussian.m, + BNT/CPDs/@generic_CPD/README, + BNT/CPDs/@generic_CPD/adjustable_CPD.m, + BNT/CPDs/@generic_CPD/display.m, + BNT/CPDs/@generic_CPD/generic_CPD.m, + BNT/CPDs/@generic_CPD/log_prior.m, + BNT/CPDs/@generic_CPD/set_clamped.m, + BNT/CPDs/@generic_CPD/Old/BIC_score_CPD.m, + BNT/CPDs/@generic_CPD/Old/CPD_to_dpots.m, + BNT/CPDs/@gmux_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@gmux_CPD/convert_to_pot.m, + BNT/CPDs/@gmux_CPD/CPD_to_pi.m, BNT/CPDs/@gmux_CPD/display.m, + BNT/CPDs/@gmux_CPD/gmux_CPD.m, BNT/CPDs/@gmux_CPD/sample_node.m, + BNT/CPDs/@gmux_CPD/Old/gmux_CPD.m, + BNT/CPDs/@hhmmF_CPD/log_prior.m, + BNT/CPDs/@hhmmF_CPD/maximize_params.m, + BNT/CPDs/@hhmmF_CPD/reset_ess.m, BNT/CPDs/@hhmmQ_CPD/log_prior.m, + BNT/CPDs/@hhmmQ_CPD/reset_ess.m, + BNT/CPDs/@mlp_CPD/convert_to_table.m, + BNT/CPDs/@mlp_CPD/maximize_params.m, BNT/CPDs/@mlp_CPD/mlp_CPD.m, + BNT/CPDs/@mlp_CPD/reset_ess.m, BNT/CPDs/@mlp_CPD/update_ess.m, + BNT/CPDs/@noisyor_CPD/CPD_to_lambda_msg.m, + BNT/CPDs/@noisyor_CPD/CPD_to_pi.m, + BNT/CPDs/@noisyor_CPD/noisyor_CPD.m, + BNT/CPDs/@noisyor_CPD/private/sum_prod_CPD_and_pi_msgs.m, + BNT/CPDs/@root_CPD/CPD_to_pi.m, + BNT/CPDs/@root_CPD/convert_to_pot.m, + BNT/CPDs/@root_CPD/log_marg_prob_node.m, + BNT/CPDs/@root_CPD/log_prob_node.m, + BNT/CPDs/@root_CPD/root_CPD.m, BNT/CPDs/@root_CPD/sample_node.m, + BNT/CPDs/@root_CPD/Old/CPD_to_CPT.m, + BNT/CPDs/@softmax_CPD/convert_to_pot.m, + BNT/CPDs/@softmax_CPD/display.m, + BNT/CPDs/@softmax_CPD/get_field.m, + BNT/CPDs/@softmax_CPD/maximize_params.m, + BNT/CPDs/@softmax_CPD/reset_ess.m, + BNT/CPDs/@softmax_CPD/sample_node.m, + BNT/CPDs/@softmax_CPD/set_fields.m, + BNT/CPDs/@softmax_CPD/update_ess.m, + BNT/CPDs/@softmax_CPD/private/extract_params.m, + BNT/CPDs/@tabular_CPD/CPD_to_CPT.m, + BNT/CPDs/@tabular_CPD/bayes_update_params.m, + BNT/CPDs/@tabular_CPD/log_nextcase_prob_node.m, + BNT/CPDs/@tabular_CPD/log_prior.m, + BNT/CPDs/@tabular_CPD/reset_ess.m, + BNT/CPDs/@tabular_CPD/update_ess.m, + BNT/CPDs/@tabular_CPD/update_ess_simple.m, + BNT/CPDs/@tabular_CPD/Old/BIC_score_CPD.m, + BNT/CPDs/@tabular_CPD/Old/bayesian_score_CPD.m, + BNT/CPDs/@tabular_CPD/Old/log_marg_prob_node_case.m, + BNT/CPDs/@tabular_CPD/Old/mult_CPD_and_pi_msgs.m, + BNT/CPDs/@tabular_CPD/Old/prob_CPT.m, + BNT/CPDs/@tabular_CPD/Old/prob_node.m, + BNT/CPDs/@tabular_CPD/Old/sample_node.m, + BNT/CPDs/@tabular_CPD/Old/sample_node_single_case.m, + BNT/CPDs/@tabular_CPD/Old/tabular_CPD.m, + BNT/CPDs/@tabular_CPD/Old/update_params.m, + BNT/CPDs/@tabular_decision_node/CPD_to_CPT.m, + BNT/CPDs/@tabular_decision_node/display.m, + BNT/CPDs/@tabular_decision_node/get_field.m, + BNT/CPDs/@tabular_decision_node/set_fields.m, + BNT/CPDs/@tabular_decision_node/tabular_decision_node.m, + BNT/CPDs/@tabular_decision_node/Old/tabular_decision_node.m, + BNT/CPDs/@tabular_kernel/convert_to_pot.m, + BNT/CPDs/@tabular_kernel/convert_to_table.m, + BNT/CPDs/@tabular_kernel/get_field.m, + BNT/CPDs/@tabular_kernel/set_fields.m, + BNT/CPDs/@tabular_kernel/tabular_kernel.m, + BNT/CPDs/@tabular_kernel/Old/tabular_kernel.m, + BNT/CPDs/@tabular_utility_node/convert_to_pot.m, + BNT/CPDs/@tabular_utility_node/display.m, + BNT/CPDs/@tabular_utility_node/tabular_utility_node.m, + BNT/CPDs/@tree_CPD/display.m, + BNT/CPDs/@tree_CPD/evaluate_tree_performance.m, + BNT/CPDs/@tree_CPD/get_field.m, + BNT/CPDs/@tree_CPD/learn_params.m, BNT/CPDs/@tree_CPD/readme.txt, + BNT/CPDs/@tree_CPD/set_fields.m, BNT/CPDs/@tree_CPD/tree_CPD.m, + BNT/CPDs/Old/@linear_gaussian_CPD/linear_gaussian_CPD.m, + BNT/CPDs/Old/@linear_gaussian_CPD/log_marg_prob_node.m, + BNT/CPDs/Old/@linear_gaussian_CPD/update_params_complete.m, + BNT/CPDs/Old/@root_gaussian_CPD/log_marg_prob_node.m, + BNT/CPDs/Old/@root_gaussian_CPD/root_gaussian_CPD.m, + BNT/CPDs/Old/@root_gaussian_CPD/update_params_complete.m, + BNT/CPDs/Old/@tabular_chance_node/CPD_to_upot.m, + BNT/CPDs/Old/@tabular_chance_node/tabular_chance_node.m, + BNT/examples/dynamic/bat1.m, BNT/examples/dynamic/bkff1.m, + BNT/examples/dynamic/chmm1.m, + BNT/examples/dynamic/cmp_inference_dbn.m, + BNT/examples/dynamic/cmp_learning_dbn.m, + BNT/examples/dynamic/cmp_online_inference.m, + BNT/examples/dynamic/fhmm_infer.m, + BNT/examples/dynamic/filter_test1.m, + BNT/examples/dynamic/kalman1.m, + BNT/examples/dynamic/kjaerulff1.m, + BNT/examples/dynamic/loopy_dbn1.m, + BNT/examples/dynamic/mk_collage_from_clqs.m, + BNT/examples/dynamic/mk_fhmm.m, BNT/examples/dynamic/reveal1.m, + BNT/examples/dynamic/scg_dbn.m, + BNT/examples/dynamic/skf_data_assoc_gmux.m, + BNT/examples/dynamic/HHMM/add_hhmm_end_state.m, + BNT/examples/dynamic/HHMM/hhmm_jtree_clqs.m, + BNT/examples/dynamic/HHMM/mk_hhmm_topo.m, + BNT/examples/dynamic/HHMM/mk_hhmm_topo_F1.m, + BNT/examples/dynamic/HHMM/pretty_print_hhmm_parse.m, + BNT/examples/dynamic/HHMM/Motif/fixed_args_mk_motif_hhmm.m, + BNT/examples/dynamic/HHMM/Motif/mk_motif_hhmm.m, + BNT/examples/dynamic/HHMM/Motif/sample_motif_hhmm.m, + BNT/examples/dynamic/HHMM/Old/mk_abcd_hhmm.m, + BNT/examples/dynamic/HHMM/Old/mk_arrow_alpha_hhmm3.m, + BNT/examples/dynamic/HHMM/Old/mk_hhmm2.m, + BNT/examples/dynamic/HHMM/Old/mk_hhmm3.m, + BNT/examples/dynamic/HHMM/Old/mk_hhmm3_args.m, + BNT/examples/dynamic/HHMM/Old/motif_hhmm.m, + BNT/examples/dynamic/HHMM/Old/remove_hhmm_end_state.m, + BNT/examples/dynamic/HHMM/Square/get_square_data.m, + BNT/examples/dynamic/HHMM/Square/hhmm_inference.m, + BNT/examples/dynamic/HHMM/Square/is_F2_true_D3.m, + BNT/examples/dynamic/HHMM/Square/learn_square_hhmm_discrete.m, + BNT/examples/dynamic/HHMM/Square/mk_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/plot_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/sample_square_hhmm_cts.m, + BNT/examples/dynamic/HHMM/Square/sample_square_hhmm_discrete.m, + BNT/examples/dynamic/HHMM/Square/square4.mat, + BNT/examples/dynamic/HHMM/Square/square4_cases.mat, + BNT/examples/dynamic/HHMM/Square/test_square_fig.m, + BNT/examples/dynamic/HHMM/Square/test_square_fig.mat, + BNT/examples/dynamic/HHMM/Square/Old/learn_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/Old/mk_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/Old/plot_square_hhmm.m, + BNT/examples/dynamic/HHMM/Square/Old/sample_square_hhmm.m, + BNT/examples/dynamic/Old/chmm1.m, + BNT/examples/dynamic/Old/cmp_inference.m, + BNT/examples/dynamic/Old/kalman1.m, + BNT/examples/dynamic/Old/old.water1.m, + BNT/examples/dynamic/Old/online1.m, + BNT/examples/dynamic/Old/online2.m, + BNT/examples/dynamic/Old/scg_dbn.m, + BNT/examples/dynamic/SLAM/mk_gmux_robot_dbn.m, + BNT/examples/dynamic/SLAM/mk_linear_slam.m, + BNT/examples/dynamic/SLAM/slam_kf.m, + BNT/examples/dynamic/SLAM/slam_offline_loopy.m, + BNT/examples/dynamic/SLAM/slam_partial_kf.m, + BNT/examples/dynamic/SLAM/slam_stationary_loopy.m, + BNT/examples/dynamic/SLAM/Old/offline_loopy_slam.m, + BNT/examples/dynamic/SLAM/Old/paskin1.m, + BNT/examples/dynamic/SLAM/Old/skf_data_assoc_gmux2.m, + BNT/examples/dynamic/SLAM/Old/slam_kf.m, + BNT/examples/limids/id1.m, BNT/examples/limids/pigs1.m, + BNT/examples/static/cg1.m, BNT/examples/static/cg2.m, + BNT/examples/static/discrete2.m, BNT/examples/static/discrete3.m, + BNT/examples/static/fa1.m, BNT/examples/static/gaussian1.m, + BNT/examples/static/gibbs_test1.m, BNT/examples/static/lw1.m, + BNT/examples/static/mfa1.m, BNT/examples/static/mixexp1.m, + BNT/examples/static/mixexp2.m, BNT/examples/static/mixexp3.m, + BNT/examples/static/mog1.m, BNT/examples/static/qmr1.m, + BNT/examples/static/sample1.m, BNT/examples/static/softmax1.m, + BNT/examples/static/Belprop/belprop_loop1_discrete.m, + BNT/examples/static/Belprop/belprop_loop1_gauss.m, + BNT/examples/static/Belprop/belprop_loopy_cg.m, + BNT/examples/static/Belprop/belprop_loopy_discrete.m, + BNT/examples/static/Belprop/belprop_loopy_gauss.m, + BNT/examples/static/Belprop/belprop_polytree_cg.m, + BNT/examples/static/Belprop/belprop_polytree_gauss.m, + BNT/examples/static/Belprop/bp1.m, + BNT/examples/static/Belprop/gmux1.m, + BNT/examples/static/Brutti/Belief_IOhmm.m, + BNT/examples/static/Brutti/Belief_hmdt.m, + BNT/examples/static/Brutti/Belief_hme.m, + BNT/examples/static/Brutti/Sigmoid_Belief.m, + BNT/examples/static/HME/HMEforMatlab.jpg, + BNT/examples/static/HME/README, BNT/examples/static/HME/fhme.m, + BNT/examples/static/HME/gen_data.m, + BNT/examples/static/HME/hme_class_plot.m, + BNT/examples/static/HME/hme_reg_plot.m, + BNT/examples/static/HME/hme_topobuilder.m, + BNT/examples/static/HME/test_data_class.mat, + BNT/examples/static/HME/test_data_class2.mat, + BNT/examples/static/HME/test_data_reg.mat, + BNT/examples/static/HME/train_data_class.mat, + BNT/examples/static/HME/train_data_reg.mat, + BNT/examples/static/Misc/mixexp_data.txt, + BNT/examples/static/Misc/mixexp_graddesc.m, + BNT/examples/static/Misc/mixexp_plot.m, + BNT/examples/static/Misc/sprinkler.bif, + BNT/examples/static/Models/mk_cancer_bnet.m, + BNT/examples/static/Models/mk_car_bnet.m, + BNT/examples/static/Models/mk_ideker_bnet.m, + BNT/examples/static/Models/mk_incinerator_bnet.m, + BNT/examples/static/Models/mk_markov_chain_bnet.m, + BNT/examples/static/Models/mk_minimal_qmr_bnet.m, + BNT/examples/static/Models/mk_qmr_bnet.m, + BNT/examples/static/Models/mk_vstruct_bnet.m, + BNT/examples/static/Models/Old/mk_hmm_bnet.m, + BNT/examples/static/SCG/scg1.m, BNT/examples/static/SCG/scg2.m, + BNT/examples/static/SCG/scg3.m, + BNT/examples/static/SCG/scg_3node.m, + BNT/examples/static/SCG/scg_unstable.m, + BNT/examples/static/StructLearn/bic1.m, + BNT/examples/static/StructLearn/cooper_yoo.m, + BNT/examples/static/StructLearn/k2demo1.m, + BNT/examples/static/StructLearn/mcmc1.m, + BNT/examples/static/StructLearn/pc1.m, + BNT/examples/static/StructLearn/pc2.m, + BNT/examples/static/Zoubin/README, + BNT/examples/static/Zoubin/csum.m, + BNT/examples/static/Zoubin/ffa.m, + BNT/examples/static/Zoubin/mfa.m, + BNT/examples/static/Zoubin/mfa_cl.m, + BNT/examples/static/Zoubin/mfademo.m, + BNT/examples/static/Zoubin/rdiv.m, + BNT/examples/static/Zoubin/rprod.m, + BNT/examples/static/Zoubin/rsum.m, + BNT/examples/static/dtree/test_housing.m, + BNT/examples/static/dtree/test_restaurants.m, + BNT/examples/static/dtree/test_zoo1.m, + BNT/examples/static/dtree/tmp.dot, + BNT/examples/static/dtree/transform_data_into_bnt_format.m, + BNT/examples/static/fgraph/fg2.m, + BNT/examples/static/fgraph/fg3.m, + BNT/examples/static/fgraph/fg_mrf1.m, + BNT/examples/static/fgraph/fg_mrf2.m, + BNT/general/bnet_to_fgraph.m, + BNT/general/compute_fwd_interface.m, + BNT/general/compute_interface_nodes.m, + BNT/general/compute_minimal_interface.m, + BNT/general/dbn_to_bnet.m, + BNT/general/determine_elim_constraints.m, + BNT/general/do_intervention.m, BNT/general/dsep.m, + BNT/general/enumerate_scenarios.m, BNT/general/fgraph_to_bnet.m, + BNT/general/log_lik_complete.m, + BNT/general/log_marg_lik_complete.m, BNT/general/mk_bnet.m, + BNT/general/mk_fgraph.m, BNT/general/mk_limid.m, + BNT/general/mk_mutilated_samples.m, + BNT/general/mk_slice_and_half_dbn.m, + BNT/general/partition_dbn_nodes.m, + BNT/general/sample_bnet_nocell.m, BNT/general/sample_dbn.m, + BNT/general/score_bnet_complete.m, + BNT/general/unroll_dbn_topology.m, + BNT/general/Old/bnet_to_gdl_graph.m, + BNT/general/Old/calc_mpe_bucket.m, + BNT/general/Old/calc_mpe_dbn.m, + BNT/general/Old/calc_mpe_given_inf_engine.m, + BNT/general/Old/calc_mpe_global.m, + BNT/general/Old/compute_interface_nodes.m, + BNT/general/Old/mk_gdl_graph.m, GraphViz/draw_dbn.m, + GraphViz/make_layout.m, BNT/license.gpl.txt, + BNT/general/add_evidence_to_gmarginal.m, + BNT/inference/@inf_engine/bnet_from_engine.m, + BNT/inference/@inf_engine/get_field.m, + BNT/inference/@inf_engine/inf_engine.m, + BNT/inference/@inf_engine/marginal_family.m, + BNT/inference/@inf_engine/set_fields.m, + BNT/inference/@inf_engine/update_engine.m, + BNT/inference/@inf_engine/Old/marginal_family_pot.m, + BNT/inference/@inf_engine/Old/observed_nodes.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/bk_ff_hmm_inf_engine.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_init_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_marginal_from_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_predict_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_update_bel.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/dbn_update_bel1.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/marginal_family.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/bk_ff_fb.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/combine_marginals_into_joint.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/dbn_to_hmm.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/mk_hmm_obs_lik_mat.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/mk_hmm_obs_lik_vec.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/mk_hmm_obs_lik_vec1.m, + BNT/inference/dynamic/@bk_ff_hmm_inf_engine/private/project_joint_onto_marginals.m, + BNT/inference/dynamic/@bk_inf_engine/bk_inf_engine.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_init_bel.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_marginal_from_bel.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_update_bel.m, + BNT/inference/dynamic/@bk_inf_engine/dbn_update_bel1.m, + BNT/inference/dynamic/@bk_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@bk_inf_engine/marginal_family.m, + BNT/inference/dynamic/@bk_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@bk_inf_engine/update_engine.m, + BNT/inference/dynamic/@ff_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/enter_soft_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/ff_inf_engine.m, + BNT/inference/dynamic/@ff_inf_engine/filter_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/marginal_family.m, + BNT/inference/dynamic/@ff_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@ff_inf_engine/smooth_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/Old/enter_soft_evidence.m, + BNT/inference/dynamic/@ff_inf_engine/Old/enter_soft_evidence1.m, + BNT/inference/dynamic/@ff_inf_engine/Old/marginal_family.m, + BNT/inference/dynamic/@frontier_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@frontier_inf_engine/enter_soft_evidence.m, + BNT/inference/dynamic/@frontier_inf_engine/frontier_inf_engine.m, + BNT/inference/dynamic/@frontier_inf_engine/marginal_family.m, + BNT/inference/dynamic/@frontier_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@frontier_inf_engine/set_fwdback.m, + BNT/inference/dynamic/@hmm_inf_engine/update_engine.m, + BNT/inference/dynamic/@hmm_inf_engine/Old/dhmm_inf_engine.m, + BNT/inference/dynamic/@hmm_inf_engine/Old/marginal_family.m, + BNT/inference/dynamic/@hmm_inf_engine/Old/marginal_nodes.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/marginal_family.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence1.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence2.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence3.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/enter_soft_evidence4.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Broken/marginal_nodes.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/enter_soft_evidence_nonint.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/enter_soft_evidence_trans.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/jtree_dbn_inf_engine.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/jtree_dbn_inf_engine1.m, + BNT/inference/dynamic/@jtree_dbn_inf_engine/Old/jtree_dbn_inf_engine2.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/jtree_unrolled_dbn_inf_engine.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/marginal_family.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/update_engine.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/Old/marginal_family.m, + BNT/inference/dynamic/@jtree_unrolled_dbn_inf_engine/Old/marginal_nodes.m, + BNT/inference/dynamic/@kalman_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@kalman_inf_engine/kalman_inf_engine.m, + BNT/inference/dynamic/@kalman_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@kalman_inf_engine/update_engine.m, + BNT/inference/dynamic/@kalman_inf_engine/private/dbn_to_lds.m, + BNT/inference/dynamic/@kalman_inf_engine/private/extract_params_from_gbn.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/enter_soft_ev.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/pearl_dbn_inf_engine.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/correct_smooth.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/enter_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/filter_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/filter_evidence_obj_oriented.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/smooth_evidence.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/smooth_evidence_fast.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/Old/wrong_smooth.m, + BNT/inference/dynamic/@pearl_dbn_inf_engine/private/init_pearl_dbn_ev_msgs.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/enter_evidence.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/marginal_family.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/marginal_nodes.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/pearl_unrolled_dbn_inf_engine.m, + BNT/inference/dynamic/@pearl_unrolled_dbn_inf_engine/update_engine.m, + BNT/inference/online/@filter_engine/bnet_from_engine.m, + BNT/inference/online/@filter_engine/enter_evidence.m, + BNT/inference/online/@filter_engine/filter_engine.m, + BNT/inference/online/@filter_engine/marginal_family.m, + BNT/inference/online/@filter_engine/marginal_nodes.m, + BNT/inference/online/@hmm_2TBN_inf_engine/back.m, + BNT/inference/online/@hmm_2TBN_inf_engine/backT.m, + BNT/inference/online/@hmm_2TBN_inf_engine/fwd.m, + BNT/inference/online/@hmm_2TBN_inf_engine/fwd1.m, + BNT/inference/online/@hmm_2TBN_inf_engine/update_engine.m, + BNT/inference/online/@jtree_2TBN_inf_engine/marginal_family.m, + BNT/inference/online/@jtree_2TBN_inf_engine/marginal_nodes.m, + BNT/inference/online/@jtree_2TBN_inf_engine/Old/jtree_2TBN_inf_engine.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/back.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/back1.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/backT.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/enter_evidence.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/fwd.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/fwd1.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/jtree_sparse_2TBN_inf_engine.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/marginal_family.m, + BNT/inference/online/@jtree_sparse_2TBN_inf_engine/marginal_nodes.m, + BNT/inference/online/@smoother_engine/bnet_from_engine.m, + BNT/inference/online/@smoother_engine/marginal_family.m, + BNT/inference/online/@smoother_engine/marginal_nodes.m, + BNT/inference/online/@smoother_engine/smoother_engine.m, + BNT/inference/online/@smoother_engine/update_engine.m, + BNT/inference/static/@belprop_fg_inf_engine/belprop_fg_inf_engine.m, + BNT/inference/static/@belprop_fg_inf_engine/enter_evidence.m, + BNT/inference/static/@belprop_fg_inf_engine/loopy_converged.m, + BNT/inference/static/@belprop_fg_inf_engine/marginal_nodes.m, + BNT/inference/static/@belprop_fg_inf_engine/set_params.m, + BNT/inference/static/@belprop_inf_engine/enter_evidence.m, + BNT/inference/static/@belprop_inf_engine/loopy_converged.m, + BNT/inference/static/@belprop_inf_engine/marginal_family.m, + BNT/inference/static/@belprop_inf_engine/marginal_nodes.m, + BNT/inference/static/@belprop_inf_engine/Old/belprop_gdl_inf_engine.m, + BNT/inference/static/@belprop_inf_engine/Old/belprop_inf_engine_nostr.m, + BNT/inference/static/@belprop_inf_engine/Old/enter_evidence.m, + BNT/inference/static/@belprop_inf_engine/Old/enter_evidence1.m, + BNT/inference/static/@belprop_inf_engine/Old/marginal_domain.m, + BNT/inference/static/@belprop_inf_engine/private/junk, + BNT/inference/static/@belprop_inf_engine/private/parallel_protocol.m, + BNT/inference/static/@belprop_inf_engine/private/tree_protocol.m, + BNT/inference/static/@cond_gauss_inf_engine/cond_gauss_inf_engine.m, + BNT/inference/static/@cond_gauss_inf_engine/enter_evidence.m, + BNT/inference/static/@cond_gauss_inf_engine/marginal_nodes.m, + BNT/inference/static/@enumerative_inf_engine/enter_evidence.m, + BNT/inference/static/@enumerative_inf_engine/enumerative_inf_engine.m, + BNT/inference/static/@enumerative_inf_engine/marginal_nodes.m, + BNT/inference/static/@gaussian_inf_engine/enter_evidence.m, + BNT/inference/static/@gaussian_inf_engine/marginal_nodes.m, + BNT/inference/static/@gaussian_inf_engine/private/extract_params_from_gbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/enter_evidence.m, + BNT/inference/static/@gibbs_sampling_inf_engine/gibbs_sampling_inf_engine.m, + BNT/inference/static/@gibbs_sampling_inf_engine/marginal_nodes.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/CPT.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_children.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_families.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_families_dbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_posterior.c, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_posterior_dbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/compute_strides.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/get_cpts.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/get_slice_dbn.c, + BNT/inference/static/@gibbs_sampling_inf_engine/private/get_slice_dbn.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/my_sample_discrete.m, + BNT/inference/static/@gibbs_sampling_inf_engine/private/sample_single_discrete.c, + BNT/inference/static/@global_joint_inf_engine/global_joint_inf_engine.m, + BNT/inference/static/@global_joint_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_inf_engine/cliques_from_engine.m, + BNT/inference/static/@jtree_inf_engine/clq_containing_nodes.m, + BNT/inference/static/@jtree_inf_engine/collect_evidence.m, + BNT/inference/static/@jtree_inf_engine/enter_soft_evidence.m, + BNT/inference/static/@jtree_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_inf_engine/marginal_nodes.m, + BNT/inference/static/@jtree_inf_engine/set_fields.m, + BNT/inference/static/@jtree_inf_engine/Old/collect_evidence.m, + BNT/inference/static/@jtree_inf_engine/Old/distribute_evidence.m, + BNT/inference/static/@jtree_inf_engine/Old/enter_evidence.m, + BNT/inference/static/@jtree_inf_engine/Old/enter_soft_evidence.m, + BNT/inference/static/@jtree_limid_inf_engine/enter_evidence.m, + BNT/inference/static/@jtree_limid_inf_engine/jtree_limid_inf_engine.m, + BNT/inference/static/@jtree_limid_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_limid_inf_engine/marginal_nodes.m, + BNT/inference/static/@jtree_limid_inf_engine/Old/marginal_family.m, + BNT/inference/static/@jtree_limid_inf_engine/Old/marginal_nodes_SS.m, + BNT/inference/static/@jtree_sparse_inf_engine/cliques_from_engine.m, + BNT/inference/static/@jtree_sparse_inf_engine/clq_containing_nodes.m, + BNT/inference/static/@jtree_sparse_inf_engine/collect_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/distribute_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/enter_evidence.m, + BNT/inference/static/@jtree_sparse_inf_engine/enter_soft_evidence.m, + BNT/inference/static/@jtree_sparse_inf_engine/init_pot.c, + BNT/inference/static/@jtree_sparse_inf_engine/marginal_family.m, + BNT/inference/static/@jtree_sparse_inf_engine/marginal_nodes.m, + BNT/inference/static/@jtree_sparse_inf_engine/set_fields.m, + BNT/inference/static/@jtree_sparse_inf_engine/old/collect_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/distribute_evidence.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/init_pot.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/init_pot1.c, + BNT/inference/static/@jtree_sparse_inf_engine/old/init_pot1.m, + BNT/inference/static/@likelihood_weighting_inf_engine/enter_evidence.m, + BNT/inference/static/@likelihood_weighting_inf_engine/likelihood_weighting_inf_engine.m, + BNT/inference/static/@likelihood_weighting_inf_engine/marginal_nodes.m, + BNT/inference/static/@pearl_inf_engine/enter_evidence.m, + BNT/inference/static/@pearl_inf_engine/loopy_converged.m, + BNT/inference/static/@pearl_inf_engine/marginal_nodes.m, + BNT/inference/static/@pearl_inf_engine/private/compute_bel.m, + BNT/inference/static/@pearl_inf_engine/private/prod_lambda_msgs.m, + BNT/inference/static/@pearl_inf_engine/private/tree_protocol.m, + BNT/inference/static/@quickscore_inf_engine/enter_evidence.m, + BNT/inference/static/@quickscore_inf_engine/marginal_nodes.m, + BNT/inference/static/@quickscore_inf_engine/quickscore_inf_engine.m, + BNT/inference/static/@quickscore_inf_engine/private/C_quickscore.c, + BNT/inference/static/@quickscore_inf_engine/private/nr.h, + BNT/inference/static/@quickscore_inf_engine/private/nrutil.c, + BNT/inference/static/@quickscore_inf_engine/private/nrutil.h, + BNT/inference/static/@quickscore_inf_engine/private/quickscore.m, + BNT/learning/bayes_update_params.m, + BNT/learning/bic_score_family.m, + BNT/learning/compute_cooling_schedule.m, + BNT/learning/dirichlet_score_family.m, + BNT/learning/kpm_learn_struct_mcmc.m, + BNT/learning/learn_params_em.m, + BNT/learning/learn_struct_dbn_reveal.m, + BNT/learning/learn_struct_pdag_ic_star.m, + BNT/learning/mcmc_sample_to_hist.m, BNT/learning/mk_schedule.m, + BNT/learning/mk_tetrad_data_file.m, + BNT/learning/score_dags_old.m, HMM/dhmm_logprob_brute_force.m, + HMM/dhmm_logprob_path.m, HMM/mdp_sample.m, Kalman/AR_to_SS.m, + Kalman/SS_to_AR.m, Kalman/convert_to_lagged_form.m, + Kalman/ensure_AR.m, Kalman/eval_AR_perf.m, + Kalman/kalman_filter.m, Kalman/kalman_smoother.m, + Kalman/kalman_update.m, Kalman/learn_AR.m, + Kalman/learn_AR_diagonal.m, Kalman/learn_kalman.m, + Kalman/smooth_update.m, + BNT/general/convert_dbn_CPDs_to_tables_slow.m, + BNT/general/dispcpt.m, BNT/general/linear_gaussian_to_cpot.m, + BNT/general/partition_matrix_vec_3.m, + BNT/general/shrink_obs_dims_in_gaussian.m, + BNT/general/shrink_obs_dims_in_table.m, + BNT/potentials/CPD_to_pot.m, BNT/potentials/README, + BNT/potentials/check_for_cd_arcs.m, + BNT/potentials/determine_pot_type.m, + BNT/potentials/mk_initial_pot.m, + BNT/potentials/@cgpot/cg_can_to_mom.m, + BNT/potentials/@cgpot/cg_mom_to_can.m, + BNT/potentials/@cgpot/cgpot.m, BNT/potentials/@cgpot/display.m, + BNT/potentials/@cgpot/divide_by_pot.m, + BNT/potentials/@cgpot/domain_pot.m, + BNT/potentials/@cgpot/enter_cts_evidence_pot.m, + BNT/potentials/@cgpot/enter_discrete_evidence_pot.m, + BNT/potentials/@cgpot/marginalize_pot.m, + BNT/potentials/@cgpot/multiply_by_pot.m, + BNT/potentials/@cgpot/multiply_pots.m, + BNT/potentials/@cgpot/normalize_pot.m, + BNT/potentials/@cgpot/pot_to_marginal.m, + BNT/potentials/@cgpot/Old/normalize_pot.m, + BNT/potentials/@cgpot/Old/simple_marginalize_pot.m, + BNT/potentials/@cpot/cpot.m, BNT/potentials/@cpot/cpot_to_mpot.m, + BNT/potentials/@cpot/display.m, + BNT/potentials/@cpot/divide_by_pot.m, + BNT/potentials/@cpot/domain_pot.m, + BNT/potentials/@cpot/enter_cts_evidence_pot.m, + BNT/potentials/@cpot/marginalize_pot.m, + BNT/potentials/@cpot/multiply_by_pot.m, + BNT/potentials/@cpot/multiply_pots.m, + BNT/potentials/@cpot/normalize_pot.m, + BNT/potentials/@cpot/pot_to_marginal.m, + BNT/potentials/@cpot/rescale_pot.m, + BNT/potentials/@cpot/set_domain_pot.m, + BNT/potentials/@cpot/Old/cpot_to_mpot.m, + BNT/potentials/@cpot/Old/normalize_pot.convert.m, + BNT/potentials/@dpot/approxeq_pot.m, + BNT/potentials/@dpot/display.m, + BNT/potentials/@dpot/domain_pot.m, + BNT/potentials/@dpot/dpot_to_table.m, + BNT/potentials/@dpot/get_fields.m, + BNT/potentials/@dpot/multiply_pots.m, + BNT/potentials/@dpot/pot_to_marginal.m, + BNT/potentials/@dpot/set_domain_pot.m, + BNT/potentials/@mpot/display.m, + BNT/potentials/@mpot/marginalize_pot.m, + BNT/potentials/@mpot/mpot.m, BNT/potentials/@mpot/mpot_to_cpot.m, + BNT/potentials/@mpot/normalize_pot.m, + BNT/potentials/@mpot/pot_to_marginal.m, + BNT/potentials/@mpot/rescale_pot.m, + BNT/potentials/@upot/approxeq_pot.m, + BNT/potentials/@upot/display.m, + BNT/potentials/@upot/divide_by_pot.m, + BNT/potentials/@upot/marginalize_pot.m, + BNT/potentials/@upot/multiply_by_pot.m, + BNT/potentials/@upot/normalize_pot.m, + BNT/potentials/@upot/pot_to_marginal.m, + BNT/potentials/@upot/upot.m, + BNT/potentials/@upot/upot_to_opt_policy.m, + BNT/potentials/Old/comp_eff_node_sizes.m, + BNT/potentials/Tables/divide_by_sparse_table.c, + BNT/potentials/Tables/divide_by_table.c, + BNT/potentials/Tables/marg_sparse_table.c, + BNT/potentials/Tables/marg_table.c, + BNT/potentials/Tables/mult_by_sparse_table.c, + BNT/potentials/Tables/rep_mult.c, HMM/mk_leftright_transmat.m: + Initial revision + +2002-05-29 04:59 yozhik + + * BNT/inference/static/@stab_cond_gauss_inf_engine/: + clq_containing_nodes.m, problems.txt, push_pot_toclique.m, + Old/initialize_engine.m: Initial import of code base from Kevin + Murphy. + +2002-05-29 04:59 yozhik + + * BNT/inference/static/@stab_cond_gauss_inf_engine/: + clq_containing_nodes.m, problems.txt, push_pot_toclique.m, + Old/initialize_engine.m: Initial revision + +2002-05-19 15:11 yozhik + + * BNT/potentials/: @scgcpot/marginalize_pot.m, + @scgcpot/normalize_pot.m, @scgcpot/rescale_pot.m, + @scgcpot/scgcpot.m, @scgpot/direct_combine_pots.m, + @scgpot/pot_to_marginal.m: Initial import of code base from Kevin + Murphy. + +2002-05-19 15:11 yozhik + + * BNT/potentials/: @scgcpot/marginalize_pot.m, + @scgcpot/normalize_pot.m, @scgcpot/rescale_pot.m, + @scgcpot/scgcpot.m, @scgpot/direct_combine_pots.m, + @scgpot/pot_to_marginal.m: Initial revision + +2001-07-28 08:43 yozhik + + * BNT/potentials/genops.c: Initial import of code base from Kevin + Murphy. + +2001-07-28 08:43 yozhik + + * BNT/potentials/genops.c: Initial revision + diff --git a/sourcecodes/bnt-master/docs/Eqns/lin_reg_eqn.gif b/sourcecodes/bnt-master/docs/Eqns/lin_reg_eqn.gif new file mode 100644 index 00000000..6ba3fda5 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Eqns/lin_reg_eqn.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/CPTgrass.gif b/sourcecodes/bnt-master/docs/Figures/CPTgrass.gif new file mode 100644 index 00000000..a5ecccd1 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/CPTgrass.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/HMEforMatlab.jpg b/sourcecodes/bnt-master/docs/Figures/HMEforMatlab.jpg new file mode 100644 index 00000000..16682678 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/HMEforMatlab.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/ar1.fig b/sourcecodes/bnt-master/docs/Figures/ar1.fig new file mode 100644 index 00000000..32e11d19 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/ar1.fig @@ -0,0 +1,13 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 600 450 300 225 600 450 900 675 +1 1 0 1 -1 0 0 0 -1 0.000 1 0.0000 1725 450 300 225 1725 450 2025 675 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 450 1425 450 +4 0 -1 0 0 0 12 0.0000 4 180 2310 300 1050 Auto Regressive model AR(1)\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 450 525 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1575 525 X2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/ar1.gif b/sourcecodes/bnt-master/docs/Figures/ar1.gif new file mode 100644 index 00000000..3271dd61 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/ar1.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/batnet.fig b/sourcecodes/bnt-master/docs/Figures/batnet.fig new file mode 100644 index 00000000..a1c72a6a --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/batnet.fig @@ -0,0 +1,318 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +0 32 #dfdfdf +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 5250.000 2175.000 2250 1500 2175 2175 2250 2850 +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 6594.530 5137.500 2250 4200 2150 5137 2250 6075 +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 3562.500 3450.000 2250 3000 2175 3450 2250 3900 +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 2576.560 6412.500 2250 6225 2200 6412 2250 6600 +6 6075 3300 7125 3600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7125 3600 7125 3300 6075 3300 6075 3600 7125 3600 +4 1 -1 0 0 0 12 0.0000 4 135 1020 6600 3525 SensorValid1\001 +-6 +6 6450 3975 7500 4275 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7500 4275 7500 3975 6450 3975 6450 4275 7500 4275 +4 1 -1 0 0 0 12 0.0000 4 135 870 6975 4200 FYdotDiff1\001 +-6 +6 6600 4575 7650 4875 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7650 4875 7650 4575 6600 4575 6600 4875 7650 4875 +4 1 -1 0 0 0 12 0.0000 4 135 975 7125 4800 FcloseSlow1\001 +-6 +6 2400 3075 3000 3375 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3000 3375 3000 3075 2400 3075 2400 3375 3000 3375 +4 1 -1 0 0 0 12 0.0000 4 135 465 2700 3300 Xdot0\001 +-6 +6 5025 3075 5625 3375 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 3375 5625 3075 5025 3075 5025 3375 5625 3375 +4 1 -1 0 0 0 12 0.0000 4 135 465 5325 3300 Xdot1\001 +-6 +6 2400 3600 3150 3900 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3150 3900 3150 3600 2400 3600 2400 3900 3150 3900 +4 1 -1 0 0 0 12 0.0000 4 135 615 2775 3825 InLane0\001 +-6 +6 4875 3600 5625 3900 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 3900 5625 3600 4875 3600 4875 3900 5625 3900 +4 1 -1 0 0 0 12 0.0000 4 135 615 5250 3825 InLane1\001 +-6 +6 2400 1500 3150 1800 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3150 1800 3150 1500 2400 1500 2400 1800 3150 1800 +4 1 -1 0 0 0 12 0.0000 4 135 630 2775 1725 LeftClr0\001 +-6 +6 4875 1500 5625 1800 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 1800 5625 1500 4875 1500 4875 1800 5625 1800 +4 1 -1 0 0 0 12 0.0000 4 135 630 5250 1725 LeftClr1\001 +-6 +6 2400 2025 3300 2325 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3300 2325 3300 2025 2400 2025 2400 2325 3300 2325 +4 1 -1 0 0 0 12 0.0000 4 180 720 2850 2250 RightClr0\001 +-6 +6 4800 2025 5625 2325 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 2325 5625 2025 4800 2025 4800 2325 5625 2325 +4 1 -1 0 0 0 12 0.0000 4 180 720 5250 2250 RightClr1\001 +-6 +6 2400 2550 3300 2850 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3300 2850 3300 2550 2400 2550 2400 2850 3300 2850 +4 1 -1 0 0 0 12 0.0000 4 135 855 2850 2775 LatAction0\001 +-6 +6 4725 2550 5625 2850 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 2850 5625 2550 4725 2550 4725 2850 5625 2850 +4 1 -1 0 0 0 12 0.0000 4 135 855 5175 2775 LatAction1\001 +-6 +6 2400 4200 3450 4500 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3450 4500 3450 4200 2400 4200 2400 4500 3450 4500 +4 1 -1 0 0 0 12 0.0000 4 135 930 2925 4425 FwdAction0\001 +-6 +6 4575 4200 5625 4500 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 4500 5625 4200 4575 4200 4575 4500 5625 4500 +4 1 -1 0 0 0 12 0.0000 4 135 930 5100 4425 FwdAction1\001 +-6 +6 2400 4725 3000 5025 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3000 5025 3000 4725 2400 4725 2400 5025 3000 5025 +4 1 -1 0 0 0 12 0.0000 4 135 465 2700 4950 Ydot0\001 +-6 +6 5025 4725 5625 5025 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 5025 5625 4725 5025 4725 5025 5025 5625 5025 +4 1 -1 0 0 0 12 0.0000 4 135 465 5325 4950 Ydot1\001 +-6 +6 2400 5250 3150 5550 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3150 5550 3150 5250 2400 5250 2400 5550 3150 5550 +4 1 -1 0 0 0 12 0.0000 4 180 705 2775 5475 Stopped0\001 +-6 +6 4500 5250 5250 5550 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5250 5550 5250 5250 4500 5250 4500 5550 5250 5550 +4 1 -1 0 0 0 12 0.0000 4 180 705 4875 5475 Stopped1\001 +-6 +6 6450 5325 7200 5625 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7200 5625 7200 5325 6450 5325 6450 5625 7200 5625 +4 1 -1 0 0 0 12 0.0000 4 135 585 6825 5550 BXdot1\001 +-6 +6 2400 5775 3300 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3300 6075 3300 5775 2400 5775 2400 6075 3300 6075 +4 1 -1 0 0 0 12 0.0000 4 180 885 2850 6000 EngStatus0\001 +-6 +6 4575 5775 5475 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5475 6075 5475 5775 4575 5775 4575 6075 5475 6075 +4 1 -1 0 0 0 12 0.0000 4 180 885 5025 6000 EngStatus1\001 +-6 +6 6150 5775 7200 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7200 6075 7200 5775 6150 5775 6150 6075 7200 6075 +4 1 -1 0 0 0 12 0.0000 4 135 960 6675 6000 BcloseFast1\001 +-6 +6 2400 6300 3750 6600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3750 6600 3750 6300 2400 6300 2400 6600 3750 6600 +4 1 -1 0 0 0 12 0.0000 4 135 1380 3075 6525 FrontBackStatus0\001 +-6 +6 4125 6300 5475 6600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5475 6600 5475 6300 4125 6300 4125 6600 5475 6600 +4 1 -1 0 0 0 12 0.0000 4 135 1380 4800 6525 FrontBackStatus1\001 +-6 +6 6150 6300 7200 6600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7200 6600 7200 6300 6150 6300 6150 6600 7200 6600 +4 1 -1 0 0 0 12 0.0000 4 135 885 6675 6525 BYdotDiff1\001 +-6 +6 7650 5025 8250 5325 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 8250 5325 8250 5025 7650 5025 7650 5325 8250 5325 +4 1 -1 0 0 0 12 0.0000 4 135 390 7950 5250 Fclr1\001 +-6 +6 7650 5775 8250 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 8250 6075 8250 5775 7650 5775 7650 6075 8250 6075 +4 1 -1 0 0 0 12 0.0000 4 135 405 7950 6000 Bclr1\001 +-6 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 3750 4725 2775 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3000 3225 4875 3675 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5175 5025 5025 5250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5325 5775 5475 5025 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5400 5775 6450 4200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5175 2850 5325 3075 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5100 4500 5175 4725 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 1650 4800 2550 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3750 6375 4650 4500 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2700 4650 4200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5475 6000 6450 5475 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7725 5775 7200 5550 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3675 6300 4800 2850 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 6150 6000 5475 6525 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 3 + 3 1 1.00 60.00 120.00 + 5625 4800 6375 3750 8850 3750 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 6600 4800 5475 6375 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 6675 6300 6675 6075 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 3 + 3 1 1.00 60.00 120.00 + 5475 5850 6150 5175 7650 5175 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3000 3225 5025 3225 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 3225 8850 3225 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7125 3450 8850 3300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 3750 4875 3750 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7125 3450 8850 3675 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 1650 4875 1650 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 1650 8700 1650 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2175 4800 2175 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 2175 8550 2175 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2700 4725 2700 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2100 4725 2625 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 2700 8700 2700 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3450 4350 4575 4350 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7500 4125 8400 4350 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7125 4275 7125 4575 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3000 4875 5025 4875 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7800 5025 7650 4800 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 5400 4500 5400 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7200 5400 8700 5400 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 5925 4575 5925 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7650 5925 7200 5925 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7200 6450 8400 6450 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 8250 5175 8850 4875 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 8250 5925 8850 5925 +2 4 1 1 -1 -1 0 0 -1 4.000 0 0 20 0 0 5 + 3975 6700 1875 6700 1875 4100 3975 4100 3975 6700 +2 4 1 1 -1 -1 0 0 -1 4.000 0 0 20 0 0 5 + 3975 4000 1875 4000 1875 1400 3975 1400 3975 4000 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 6600 9750 6300 8400 6300 8400 6600 9750 6600 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 6075 9750 5775 8850 5775 8850 6075 9750 6075 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 5550 9750 5250 8700 5250 8700 5550 9750 5550 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 5025 9750 4725 8850 4725 8850 5025 9750 5025 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 4500 9750 4200 8400 4200 8400 4500 9750 4500 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 3900 9750 3600 8850 3600 8850 3900 9750 3900 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 3375 9750 3075 8850 3075 8850 3375 9750 3375 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 2850 9750 2550 8700 2550 8700 2850 9750 2850 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 2325 9750 2025 8550 2025 8550 2325 9750 2325 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 1800 9750 1500 8700 1500 8700 1800 9750 1800 +4 1 -1 0 0 0 12 0.0000 4 135 840 9300 3300 XdotSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 840 9300 3825 YdotSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 1005 9225 1725 LeftClrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 180 1095 9150 2250 RightClrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 180 900 9225 2775 TurnSignal1\001 +4 1 -1 0 0 0 12 0.0000 4 135 1245 9075 4425 FYdotDiffSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 765 9300 4950 FclrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 960 9225 5475 BXdotSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 780 9300 6000 BclrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 1260 9075 6525 BYdotDiffSens1\001 +4 1 -1 0 0 0 16 0.0000 4 165 585 2925 7050 slice t\001 +4 1 -1 0 0 0 16 0.0000 4 165 840 4800 7050 slice t+1\001 +4 1 -1 0 0 0 16 0.0000 4 165 855 9150 7050 evidence\001 + diff --git a/sourcecodes/bnt-master/docs/Figures/batnet.gif b/sourcecodes/bnt-master/docs/Figures/batnet.gif new file mode 100644 index 00000000..d7fb15b6 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/batnet.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/batnet_numbered.fig b/sourcecodes/bnt-master/docs/Figures/batnet_numbered.fig new file mode 100644 index 00000000..daa9a755 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/batnet_numbered.fig @@ -0,0 +1,345 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +0 32 #dfdfdf +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 5250.000 2175.000 2250 1500 2175 2175 2250 2850 +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 6594.530 5137.500 2250 4200 2150 5137 2250 6075 +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 3562.500 3450.000 2250 3000 2175 3450 2250 3900 +5 1 1 1 -1 -1 0 0 -1 4.000 0 1 0 0 2576.560 6412.500 2250 6225 2200 6412 2250 6600 +6 6075 3300 7125 3600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7125 3600 7125 3300 6075 3300 6075 3600 7125 3600 +4 1 -1 0 0 0 12 0.0000 4 135 1020 6600 3525 SensorValid1\001 +-6 +6 6450 3975 7500 4275 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7500 4275 7500 3975 6450 3975 6450 4275 7500 4275 +4 1 -1 0 0 0 12 0.0000 4 135 870 6975 4200 FYdotDiff1\001 +-6 +6 6600 4575 7650 4875 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7650 4875 7650 4575 6600 4575 6600 4875 7650 4875 +4 1 -1 0 0 0 12 0.0000 4 135 975 7125 4800 FcloseSlow1\001 +-6 +6 2400 3075 3000 3375 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3000 3375 3000 3075 2400 3075 2400 3375 3000 3375 +4 1 -1 0 0 0 12 0.0000 4 135 465 2700 3300 Xdot0\001 +-6 +6 5025 3075 5625 3375 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 3375 5625 3075 5025 3075 5025 3375 5625 3375 +4 1 -1 0 0 0 12 0.0000 4 135 465 5325 3300 Xdot1\001 +-6 +6 2400 3600 3150 3900 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3150 3900 3150 3600 2400 3600 2400 3900 3150 3900 +4 1 -1 0 0 0 12 0.0000 4 135 615 2775 3825 InLane0\001 +-6 +6 4875 3600 5625 3900 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 3900 5625 3600 4875 3600 4875 3900 5625 3900 +4 1 -1 0 0 0 12 0.0000 4 135 615 5250 3825 InLane1\001 +-6 +6 2400 1500 3150 1800 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3150 1800 3150 1500 2400 1500 2400 1800 3150 1800 +4 1 -1 0 0 0 12 0.0000 4 135 630 2775 1725 LeftClr0\001 +-6 +6 4875 1500 5625 1800 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 1800 5625 1500 4875 1500 4875 1800 5625 1800 +4 1 -1 0 0 0 12 0.0000 4 135 630 5250 1725 LeftClr1\001 +-6 +6 2400 2025 3300 2325 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3300 2325 3300 2025 2400 2025 2400 2325 3300 2325 +4 1 -1 0 0 0 12 0.0000 4 180 720 2850 2250 RightClr0\001 +-6 +6 4800 2025 5625 2325 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 2325 5625 2025 4800 2025 4800 2325 5625 2325 +4 1 -1 0 0 0 12 0.0000 4 180 720 5250 2250 RightClr1\001 +-6 +6 2400 2550 3300 2850 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3300 2850 3300 2550 2400 2550 2400 2850 3300 2850 +4 1 -1 0 0 0 12 0.0000 4 135 855 2850 2775 LatAction0\001 +-6 +6 4725 2550 5625 2850 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 2850 5625 2550 4725 2550 4725 2850 5625 2850 +4 1 -1 0 0 0 12 0.0000 4 135 855 5175 2775 LatAction1\001 +-6 +6 2400 4200 3450 4500 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3450 4500 3450 4200 2400 4200 2400 4500 3450 4500 +4 1 -1 0 0 0 12 0.0000 4 135 930 2925 4425 FwdAction0\001 +-6 +6 4575 4200 5625 4500 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 4500 5625 4200 4575 4200 4575 4500 5625 4500 +4 1 -1 0 0 0 12 0.0000 4 135 930 5100 4425 FwdAction1\001 +-6 +6 2400 4725 3000 5025 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3000 5025 3000 4725 2400 4725 2400 5025 3000 5025 +4 1 -1 0 0 0 12 0.0000 4 135 465 2700 4950 Ydot0\001 +-6 +6 5025 4725 5625 5025 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5625 5025 5625 4725 5025 4725 5025 5025 5625 5025 +4 1 -1 0 0 0 12 0.0000 4 135 465 5325 4950 Ydot1\001 +-6 +6 2400 5250 3150 5550 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3150 5550 3150 5250 2400 5250 2400 5550 3150 5550 +4 1 -1 0 0 0 12 0.0000 4 180 705 2775 5475 Stopped0\001 +-6 +6 4500 5250 5250 5550 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5250 5550 5250 5250 4500 5250 4500 5550 5250 5550 +4 1 -1 0 0 0 12 0.0000 4 180 705 4875 5475 Stopped1\001 +-6 +6 6450 5325 7200 5625 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7200 5625 7200 5325 6450 5325 6450 5625 7200 5625 +4 1 -1 0 0 0 12 0.0000 4 135 585 6825 5550 BXdot1\001 +-6 +6 2400 5775 3300 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3300 6075 3300 5775 2400 5775 2400 6075 3300 6075 +4 1 -1 0 0 0 12 0.0000 4 180 885 2850 6000 EngStatus0\001 +-6 +6 4575 5775 5475 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5475 6075 5475 5775 4575 5775 4575 6075 5475 6075 +4 1 -1 0 0 0 12 0.0000 4 180 885 5025 6000 EngStatus1\001 +-6 +6 6150 5775 7200 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7200 6075 7200 5775 6150 5775 6150 6075 7200 6075 +4 1 -1 0 0 0 12 0.0000 4 135 960 6675 6000 BcloseFast1\001 +-6 +6 2325 6300 3825 6600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 3750 6600 3750 6300 2400 6300 2400 6600 3750 6600 +4 1 -1 0 0 0 12 0.0000 4 135 1380 3075 6525 FrontBackStatus0\001 +-6 +6 4050 6300 5550 6600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 5475 6600 5475 6300 4125 6300 4125 6600 5475 6600 +4 1 -1 0 0 0 12 0.0000 4 135 1380 4800 6525 FrontBackStatus1\001 +-6 +6 6150 6300 7200 6600 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 7200 6600 7200 6300 6150 6300 6150 6600 7200 6600 +4 1 -1 0 0 0 12 0.0000 4 135 885 6675 6525 BYdotDiff1\001 +-6 +6 7650 5025 8250 5325 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 8250 5325 8250 5025 7650 5025 7650 5325 8250 5325 +4 1 -1 0 0 0 12 0.0000 4 135 390 7950 5250 Fclr1\001 +-6 +6 7650 5775 8250 6075 +2 4 0 1 -1 7 0 0 -1 0.000 0 0 7 0 0 5 + 8250 6075 8250 5775 7650 5775 7650 6075 8250 6075 +4 1 -1 0 0 0 12 0.0000 4 135 405 7950 6000 Bclr1\001 +-6 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 3750 4725 2775 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3000 3225 4875 3675 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5175 5025 5025 5250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5325 5775 5475 5025 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5400 5775 6450 4200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5175 2850 5325 3075 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5100 4500 5175 4725 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 1650 4800 2550 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3750 6375 4650 4500 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2700 4650 4200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5475 6000 6450 5475 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7725 5775 7200 5550 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3675 6300 4800 2850 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 6150 6000 5475 6525 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 3 + 3 1 1.00 60.00 120.00 + 5625 4800 6375 3750 8850 3750 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 6600 4800 5475 6375 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 6675 6300 6675 6075 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 3 + 3 1 1.00 60.00 120.00 + 5475 5850 6150 5175 7650 5175 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3000 3225 5025 3225 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 3225 8850 3225 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7125 3450 8850 3300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 3750 4875 3750 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7125 3450 8850 3675 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 1650 4875 1650 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 1650 8700 1650 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2175 4800 2175 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 2175 8550 2175 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2700 4725 2700 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 2100 4725 2625 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 5625 2700 8700 2700 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3450 4350 4575 4350 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7500 4125 8400 4350 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7125 4275 7125 4575 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3000 4875 5025 4875 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7800 5025 7650 4800 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3150 5400 4500 5400 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7200 5400 8700 5400 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 3300 5925 4575 5925 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7650 5925 7200 5925 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 7200 6450 8400 6450 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 8250 5175 8850 4875 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 8250 5925 8850 5925 +2 4 1 1 -1 -1 0 0 -1 4.000 0 0 20 0 0 5 + 3975 6700 1875 6700 1875 4100 3975 4100 3975 6700 +2 4 1 1 -1 -1 0 0 -1 4.000 0 0 20 0 0 5 + 3975 4000 1875 4000 1875 1400 3975 1400 3975 4000 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 6600 9750 6300 8400 6300 8400 6600 9750 6600 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 6075 9750 5775 8850 5775 8850 6075 9750 6075 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 5550 9750 5250 8700 5250 8700 5550 9750 5550 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 5025 9750 4725 8850 4725 8850 5025 9750 5025 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 4500 9750 4200 8400 4200 8400 4500 9750 4500 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 3900 9750 3600 8850 3600 8850 3900 9750 3900 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 3375 9750 3075 8850 3075 8850 3375 9750 3375 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 2850 9750 2550 8700 2550 8700 2850 9750 2850 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 2325 9750 2025 8550 2025 8550 2325 9750 2325 +2 4 0 1 -1 32 0 0 20 0.000 0 0 7 0 0 5 + 9750 1800 9750 1500 8700 1500 8700 1800 9750 1800 +4 1 -1 0 0 0 12 0.0000 4 135 840 9300 3300 XdotSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 840 9300 3825 YdotSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 1005 9225 1725 LeftClrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 180 1095 9150 2250 RightClrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 180 900 9225 2775 TurnSignal1\001 +4 1 -1 0 0 0 12 0.0000 4 135 1245 9075 4425 FYdotDiffSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 765 9300 4950 FclrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 960 9225 5475 BXdotSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 780 9300 6000 BclrSens1\001 +4 1 -1 0 0 0 12 0.0000 4 135 1260 9075 6525 BYdotDiffSens1\001 +4 1 -1 0 0 0 16 0.0000 4 165 585 2925 7050 slice t\001 +4 1 -1 0 0 0 16 0.0000 4 165 840 4800 7050 slice t+1\001 +4 1 -1 0 0 0 16 0.0000 4 165 855 9150 7050 evidence\001 +4 0 0 50 0 0 12 0.0000 4 135 180 2025 6525 14\001 +4 0 0 50 0 0 12 0.0000 4 135 90 1950 6075 7\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 5475 19\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 4950 17\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 4425 16\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 3825 20\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 3375 23\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 2775 21\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 2250 25\001 +4 0 0 50 0 0 12 0.0000 4 135 180 1950 1800 27\001 +4 0 0 50 0 0 12 0.0000 4 135 90 5925 3600 6\001 +4 0 0 50 0 0 12 0.0000 4 135 180 6225 4200 12\001 +4 0 0 50 0 0 12 0.0000 4 135 180 6375 4800 13\001 +4 0 0 50 0 0 12 0.0000 4 135 180 7875 5025 10\001 +4 0 0 50 0 0 12 0.0000 4 135 90 6300 5475 8\001 +4 0 0 50 0 0 12 0.0000 4 135 90 6000 6075 4\001 +4 0 0 50 0 0 12 0.0000 4 135 90 6000 6600 1\001 +4 0 0 50 0 0 12 0.0000 4 135 90 7575 6150 3\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 1725 28\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 2250 26\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 2775 22\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 3300 24\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 3825 18\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 4425 15\001 +4 0 0 50 0 0 12 0.0000 4 135 180 9825 4950 11\001 +4 0 0 50 0 0 12 0.0000 4 135 90 9825 5475 9\001 +4 0 0 50 0 0 12 0.0000 4 135 90 9825 6075 5\001 +4 0 0 50 0 0 12 0.0000 4 135 90 9825 6525 2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/bic.png b/sourcecodes/bnt-master/docs/Figures/bic.png new file mode 100644 index 00000000..1ad2a4e5 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/bic.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/cg1.fig b/sourcecodes/bnt-master/docs/Figures/cg1.fig new file mode 100644 index 00000000..145c8d1d --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/cg1.fig @@ -0,0 +1,60 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 1200 3000 300 300 1200 3000 1200 3300 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3900 3000 300 300 3900 3000 3900 3300 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3900 4500 300 300 3900 4500 3900 4800 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 2100 5700 300 300 2100 5700 2100 6000 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 5700 5700 300 300 5700 5700 5700 6000 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3600 1200 4200 1200 4200 1800 3600 1800 3600 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 6600 1200 7200 1200 7200 1800 6600 1800 6600 1200 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1200 1800 1200 2700 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1200 1800 3600 2850 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1200 1800 3600 4350 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1200 3300 2100 5400 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3750 4800 2100 5400 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4050 4800 5400 5550 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3900 1800 3900 2700 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3900 3300 3900 4200 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6900 1800 4200 4350 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6900 1800 6900 2700 +2 2 0 1 0 0 100 0 2 0.000 0 0 7 0 0 5 + 900 1200 1500 1200 1500 1800 900 1800 900 1200 +4 0 0 100 0 0 12 0.0000 4 135 180 1050 1650 W\001 +4 0 0 100 0 0 12 0.0000 4 135 300 1050 3150 Min\001 +4 0 0 100 0 0 12 0.0000 4 135 105 3750 1650 F\001 +4 0 0 100 0 0 12 0.0000 4 135 120 3750 3150 E\001 +4 0 0 100 0 0 12 0.0000 4 135 135 3750 4650 D\001 +4 0 0 100 0 0 12 0.0000 4 135 405 1950 5850 Mout\001 +4 0 0 100 0 0 12 0.0000 4 135 120 6750 1650 B\001 +4 0 0 100 0 0 12 0.0000 4 135 120 6750 3150 C\001 +4 0 0 100 0 0 12 0.0000 4 135 105 5550 5850 L\001 diff --git a/sourcecodes/bnt-master/docs/Figures/cg1.gif b/sourcecodes/bnt-master/docs/Figures/cg1.gif new file mode 100644 index 00000000..fe62c5c7 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/cg1.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.T5.fig b/sourcecodes/bnt-master/docs/Figures/chmm5.T5.fig new file mode 100644 index 00000000..15579df9 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.T5.fig @@ -0,0 +1,323 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 300 600 4500 9900 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 1200 900 1200 900 1800 300 1800 300 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 3300 900 3300 900 3900 300 3900 300 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 5400 900 5400 900 6000 300 6000 300 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 7500 900 7500 900 8100 300 8100 300 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 9300 900 9300 900 9900 300 9900 300 9300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 1200 3600 1200 3600 1800 3000 1800 3000 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 3300 3600 3300 3600 3900 3000 3900 3000 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 5400 3600 5400 3600 6000 3000 6000 3000 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 7500 3600 7500 3600 8100 3000 8100 3000 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 9300 3600 9300 3600 9900 3000 9900 3000 9300 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1200 3900 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3300 3900 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5400 3900 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7500 3900 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9300 3900 9000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1200 1200 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3300 1200 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5400 1200 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7500 1200 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9300 1200 9000 +-6 +6 3600 600 7200 9900 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 1200 6300 1200 6300 1800 5700 1800 5700 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 3300 6300 3300 6300 3900 5700 3900 5700 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 5400 6300 5400 6300 6000 5700 6000 5700 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 7500 6300 7500 6300 8100 5700 8100 5700 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 9300 6300 9300 6300 9900 5700 9900 5700 9300 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1200 6600 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3300 6600 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5400 6600 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7500 6600 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9300 6600 9000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 7800 +-6 +6 6300 600 9900 9900 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 9600 5100 300 300 9600 5100 9600 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 9600 7200 300 300 9600 7200 9600 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 9600 9000 300 300 9600 9000 9600 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 9600 900 300 300 9600 900 9600 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 9600 3000 300 300 9600 3000 9600 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 8400 1200 9000 1200 9000 1800 8400 1800 8400 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 8400 3300 9000 3300 9000 3900 8400 3900 8400 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 8400 5400 9000 5400 9000 6000 8400 6000 8400 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 8400 7500 9000 7500 9000 8100 8400 8100 8400 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 8400 9300 9000 9300 9000 9900 8400 9900 8400 9300 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 1200 9300 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 3300 9300 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 5400 9300 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 7500 9300 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 9300 9300 9000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1500 8400 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3600 8400 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1500 8400 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3600 8400 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5700 8400 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3600 8400 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5700 8400 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5700 8400 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7800 8400 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7800 8400 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7800 8400 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9600 8400 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9600 8400 7800 +-6 +6 9000 600 12600 9900 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 12300 5100 300 300 12300 5100 12300 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 12300 7200 300 300 12300 7200 12300 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 12300 9000 300 300 12300 9000 12300 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 12300 900 300 300 12300 900 12300 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 12300 3000 300 300 12300 3000 12300 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 11100 1200 11700 1200 11700 1800 11100 1800 11100 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 11100 3300 11700 3300 11700 3900 11100 3900 11100 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 11100 5400 11700 5400 11700 6000 11100 6000 11100 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 11100 7500 11700 7500 11700 8100 11100 8100 11100 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 11100 9300 11700 9300 11700 9900 11100 9900 11100 9300 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 11700 1200 12000 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 11700 3300 12000 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 11700 5400 12000 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 11700 7500 12000 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 11700 9300 12000 9000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 1500 11100 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 3600 11100 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 1500 11100 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 3600 11100 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 5700 11100 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 3600 11100 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 5700 11100 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 5700 11100 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 7800 11100 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 7800 11100 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 7800 11100 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 9600 11100 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9000 9600 11100 7800 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.fig b/sourcecodes/bnt-master/docs/Figures/chmm5.fig new file mode 100644 index 00000000..ae772850 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.fig @@ -0,0 +1,187 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 225 1125 975 9975 +6 225 1125 975 9975 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 1200 900 1200 900 1800 300 1800 300 1200 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 3300 900 3300 900 3900 300 3900 300 3300 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 5400 900 5400 900 6000 300 6000 300 5400 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 7500 900 7500 900 8100 300 8100 300 7500 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 9300 900 9300 900 9900 300 9900 300 9300 +-6 +-6 +6 2925 1125 3675 9975 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 1200 3600 1200 3600 1800 3000 1800 3000 1200 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 3300 3600 3300 3600 3900 3000 3900 3000 3300 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 5400 3600 5400 3600 6000 3000 6000 3000 5400 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 7500 3600 7500 3600 8100 3000 8100 3000 7500 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 9300 3600 9300 3600 9900 3000 9900 3000 9300 +-6 +6 5625 1125 6375 9975 +6 5625 1125 6375 9975 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 1200 6300 1200 6300 1800 5700 1800 5700 1200 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 3300 6300 3300 6300 3900 5700 3900 5700 3300 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 5400 6300 5400 6300 6000 5700 6000 5700 5400 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 7500 6300 7500 6300 8100 5700 8100 5700 7500 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 9300 6300 9300 6300 9900 5700 9900 5700 9300 +-6 +-6 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +1 3 0 2 7 0 50 0 20 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7500 1200 7200 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5400 1200 5100 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3300 1200 3000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 1200 1200 1050 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 1200 3900 1050 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 1500 3000 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3600 3000 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 1500 3000 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3600 3000 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3600 3000 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5700 3000 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3300 3900 3000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3600 5700 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3600 5700 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5700 3000 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7800 3000 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5700 3000 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7800 3000 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 9600 3000 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 9300 1200 9000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7800 3000 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 9600 3000 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 9600 5700 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 9300 3900 9000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 9600 5700 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7800 5700 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7800 5700 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7800 5700 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7500 3900 7200 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5700 5700 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5700 5700 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5700 5700 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3600 5700 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 1500 5700 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 1500 5700 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 1200 6600 1050 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 3300 6600 3000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 5400 6600 5100 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 9300 6600 9000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 7500 6600 7200 +2 1 0 3 0 0 100 0 5 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5400 3900 5100 diff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.gif b/sourcecodes/bnt-master/docs/Figures/chmm5.gif new file mode 100644 index 00000000..f8259c4a --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.influence.fig b/sourcecodes/bnt-master/docs/Figures/chmm5.influence.fig new file mode 100644 index 00000000..62dc8add --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.influence.fig @@ -0,0 +1,200 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 1200 900 1200 900 1800 300 1800 300 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 3300 900 3300 900 3900 300 3900 300 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 5400 900 5400 900 6000 300 6000 300 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 7500 900 7500 900 8100 300 8100 300 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 9300 900 9300 900 9900 300 9900 300 9300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 1200 3600 1200 3600 1800 3000 1800 3000 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 3300 3600 3300 3600 3900 3000 3900 3000 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 5400 3600 5400 3600 6000 3000 6000 3000 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 7500 3600 7500 3600 8100 3000 8100 3000 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 9300 3600 9300 3600 9900 3000 9900 3000 9300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 1200 6300 1200 6300 1800 5700 1800 5700 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 3300 6300 3300 6300 3900 5700 3900 5700 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 5400 6300 5400 6300 6000 5700 6000 5700 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 7500 6300 7500 6300 8100 5700 8100 5700 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 9300 6300 9300 6300 9900 5700 9900 5700 9300 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7500 1200 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9300 1200 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9300 3900 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7500 3900 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7500 6600 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9300 6600 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5400 3900 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5400 6600 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5400 1200 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3300 1200 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1200 1200 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1200 3900 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3300 3900 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3300 6600 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1200 6600 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 900 9600 3000 7800 +3 2 1 3 0 7 100 0 -1 8.000 0 1 0 4 + 2 0 6.00 180.00 360.00 + 750 1950 2775 3750 975 5475 2925 5550 + 0.000 -1.000 -1.000 0.000 +3 0 1 3 0 7 100 0 -1 8.000 0 1 0 3 + 2 0 6.00 180.00 360.00 + 900 1875 3900 3450 5700 5550 + 0.000 1.000 0.000 +4 0 0 100 0 0 25 0.0000 4 255 435 375 1575 A1\001 +4 0 0 100 0 0 25 0.0000 4 255 420 375 3750 B1\001 +4 0 0 100 0 0 25 0.0000 4 255 420 375 5775 C1\001 +4 0 0 100 0 0 25 0.0000 4 255 435 375 7950 D1\001 +4 0 0 100 0 0 25 0.0000 4 255 405 375 9750 E1\001 +4 0 0 100 0 0 25 0.0000 4 255 435 3075 1575 A2\001 +4 0 0 100 0 0 25 0.0000 4 255 420 3075 3750 B2\001 +4 0 0 100 0 0 25 0.0000 4 255 420 3075 5775 C2\001 +4 0 0 100 0 0 25 0.0000 4 255 435 3075 7875 D2\001 +4 0 0 100 0 0 25 0.0000 4 255 405 3075 9675 E2\001 +4 0 0 100 0 0 25 0.0000 4 255 435 5775 1575 A3\001 +4 0 0 100 0 0 25 0.0000 4 255 420 5700 3675 B3\001 +4 0 0 100 0 0 25 0.0000 4 255 420 5850 5775 C3\001 +4 0 0 100 0 0 25 0.0000 4 255 435 5775 7875 D3\001 +4 0 0 100 0 0 25 0.0000 4 255 405 5850 9675 E3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.jpg b/sourcecodes/bnt-master/docs/Figures/chmm5.jpg new file mode 100644 index 00000000..2aa0de57 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.named.fig b/sourcecodes/bnt-master/docs/Figures/chmm5.named.fig new file mode 100644 index 00000000..816d7429 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.named.fig @@ -0,0 +1,192 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 1200 900 1200 900 1800 300 1800 300 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 3300 900 3300 900 3900 300 3900 300 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 5400 900 5400 900 6000 300 6000 300 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 7500 900 7500 900 8100 300 8100 300 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 9300 900 9300 900 9900 300 9900 300 9300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 1200 3600 1200 3600 1800 3000 1800 3000 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 3300 3600 3300 3600 3900 3000 3900 3000 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 5400 3600 5400 3600 6000 3000 6000 3000 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 7500 3600 7500 3600 8100 3000 8100 3000 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 9300 3600 9300 3600 9900 3000 9900 3000 9300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 1200 6300 1200 6300 1800 5700 1800 5700 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 3300 6300 3300 6300 3900 5700 3900 5700 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 5400 6300 5400 6300 6000 5700 6000 5700 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 7500 6300 7500 6300 8100 5700 8100 5700 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 9300 6300 9300 6300 9900 5700 9900 5700 9300 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7500 1200 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9300 1200 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9300 3900 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7500 3900 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7500 6600 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9300 6600 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5400 3900 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5400 6600 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5400 1200 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3300 1200 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1200 1200 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1200 3900 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3300 3900 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3300 6600 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1200 6600 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 900 9600 3000 7800 +4 0 0 100 0 0 25 0.0000 4 255 435 375 1575 A1\001 +4 0 0 100 0 0 25 0.0000 4 255 420 375 3750 B1\001 +4 0 0 100 0 0 25 0.0000 4 255 420 375 5775 C1\001 +4 0 0 100 0 0 25 0.0000 4 255 435 375 7950 D1\001 +4 0 0 100 0 0 25 0.0000 4 255 405 375 9750 E1\001 +4 0 0 100 0 0 25 0.0000 4 255 435 3075 1575 A2\001 +4 0 0 100 0 0 25 0.0000 4 255 420 3075 3750 B2\001 +4 0 0 100 0 0 25 0.0000 4 255 420 3075 5775 C2\001 +4 0 0 100 0 0 25 0.0000 4 255 435 3075 7875 D2\001 +4 0 0 100 0 0 25 0.0000 4 255 405 3075 9675 E2\001 +4 0 0 100 0 0 25 0.0000 4 255 435 5775 1575 A3\001 +4 0 0 100 0 0 25 0.0000 4 255 420 5700 3675 B3\001 +4 0 0 100 0 0 25 0.0000 4 255 420 5850 5775 C3\001 +4 0 0 100 0 0 25 0.0000 4 255 435 5775 7875 D3\001 +4 0 0 100 0 0 25 0.0000 4 255 405 5850 9675 E3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/chmm5.small.fig b/sourcecodes/bnt-master/docs/Figures/chmm5.small.fig new file mode 100644 index 00000000..8d49e5cc --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5.small.fig @@ -0,0 +1,181 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 300 600 4500 9900 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 1200 900 1200 900 1800 300 1800 300 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 3300 900 3300 900 3900 300 3900 300 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 5400 900 5400 900 6000 300 6000 300 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 7500 900 7500 900 8100 300 8100 300 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 300 9300 900 9300 900 9900 300 9900 300 9300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 1200 3600 1200 3600 1800 3000 1800 3000 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 3300 3600 3300 3600 3900 3000 3900 3000 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 5400 3600 5400 3600 6000 3000 6000 3000 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 7500 3600 7500 3600 8100 3000 8100 3000 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 3000 9300 3600 9300 3600 9900 3000 9900 3000 9300 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1200 3900 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3300 3900 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5400 3900 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7500 3900 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9300 3900 9000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1200 1200 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3300 1200 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5400 1200 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7500 1200 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9300 1200 9000 +-6 +6 3600 600 7200 9900 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 1200 6300 1200 6300 1800 5700 1800 5700 1200 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 3300 6300 3300 6300 3900 5700 3900 5700 3300 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 5400 6300 5400 6300 6000 5700 6000 5700 5400 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 7500 6300 7500 6300 8100 5700 8100 5700 7500 +2 2 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 5 + 5700 9300 6300 9300 6300 9900 5700 9900 5700 9300 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1200 6600 1050 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3300 6600 3000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5400 6600 5100 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7500 6600 7200 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9300 6600 9000 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 1500 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 3600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 5700 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 7800 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 9600 +2 1 0 1 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 7800 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/chmm5_circle.fig b/sourcecodes/bnt-master/docs/Figures/chmm5_circle.fig new file mode 100644 index 00000000..bda2e382 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5_circle.fig @@ -0,0 +1,162 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 602 3534 300 300 602 3534 602 3834 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 602 5634 300 300 602 5634 602 5934 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 602 7734 300 300 602 7734 602 8034 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 602 9459 300 300 602 9459 602 9759 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 600 1425 300 300 600 1425 600 1725 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3300 1500 300 300 3300 1500 3300 1800 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3300 3600 300 300 3300 3600 3300 3900 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3300 5700 300 300 3300 5700 3300 6000 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3300 7800 300 300 3300 7800 3300 8100 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 3300 9600 300 300 3300 9600 3300 9900 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 6077 1509 300 300 6077 1509 6077 1809 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 6077 3534 300 300 6077 3534 6077 3834 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 6075 5700 300 300 6075 5700 6075 6000 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 6077 7809 300 300 6077 7809 6077 8109 +1 3 0 1 0 7 100 0 -1 0.000 1 0.0000 6002 9534 300 300 6002 9534 6002 9834 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7500 1200 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9300 1200 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 9600 3000 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 7800 3000 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9300 3900 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 9600 5700 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 9600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7800 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 7500 3900 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 7800 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 7500 6600 7200 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 9300 6600 9000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5700 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 5400 3900 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 5400 6600 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5700 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 5400 1200 5100 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 5700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3600 3000 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 3300 1200 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1200 1200 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1500 3000 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1200 3900 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 1500 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 1500 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3300 3900 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 3300 6600 3000 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 3600 3600 5700 3600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.00 60.00 120.00 + 6300 1200 6600 1050 +2 1 0 2 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 900 9600 3000 7800 diff --git a/sourcecodes/bnt-master/docs/Figures/chmm5_nobold.fig b/sourcecodes/bnt-master/docs/Figures/chmm5_nobold.fig new file mode 100644 index 00000000..d13942a8 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/chmm5_nobold.fig @@ -0,0 +1,187 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 225 1125 975 9975 +6 225 1125 975 9975 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 1200 900 1200 900 1800 300 1800 300 1200 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 3300 900 3300 900 3900 300 3900 300 3300 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 5400 900 5400 900 6000 300 6000 300 5400 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 7500 900 7500 900 8100 300 8100 300 7500 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 300 9300 900 9300 900 9900 300 9900 300 9300 +-6 +-6 +6 2925 1125 3675 9975 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 1200 3600 1200 3600 1800 3000 1800 3000 1200 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 3300 3600 3300 3600 3900 3000 3900 3000 3300 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 5400 3600 5400 3600 6000 3000 6000 3000 5400 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 7500 3600 7500 3600 8100 3000 8100 3000 7500 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 3000 9300 3600 9300 3600 9900 3000 9900 3000 9300 +-6 +6 5625 1125 6375 9975 +6 5625 1125 6375 9975 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 1200 6300 1200 6300 1800 5700 1800 5700 1200 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 3300 6300 3300 6300 3900 5700 3900 5700 3300 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 5400 6300 5400 6300 6000 5700 6000 5700 5400 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 7500 6300 7500 6300 8100 5700 8100 5700 7500 +2 2 0 4 0 0 100 0 -1 0.000 0 0 7 0 0 5 + 5700 9300 6300 9300 6300 9900 5700 9900 5700 9300 +-6 +-6 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 6900 7200 300 300 6900 7200 6900 7500 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 6900 9000 300 300 6900 9000 6900 9300 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 4200 9000 300 300 4200 9000 4200 9300 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 4200 7200 300 300 4200 7200 4200 7500 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 4200 5100 300 300 4200 5100 4200 5400 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 4200 3000 300 300 4200 3000 4200 3300 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 4200 900 300 300 4200 900 4200 1200 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 1500 900 300 300 1500 900 1500 1200 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 1500 3000 300 300 1500 3000 1500 3300 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 1500 5100 300 300 1500 5100 1500 5400 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 1500 7200 300 300 1500 7200 1500 7500 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 1500 9000 300 300 1500 9000 1500 9300 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 6900 5100 300 300 6900 5100 6900 5400 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 6900 3000 300 300 6900 3000 6900 3300 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 6900 900 300 300 6900 900 6900 1200 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7500 1200 7200 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5400 1200 5100 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3300 1200 3000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 1200 1200 1050 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 1200 3900 1050 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 1500 3000 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3600 3000 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 1500 3000 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3600 3000 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 3600 3000 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5700 3000 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3300 3900 3000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3600 5700 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3600 5700 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5700 3000 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7800 3000 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 5700 3000 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7800 3000 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 9600 3000 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 9300 1200 9000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 7800 3000 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 900 9600 3000 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 9600 5700 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 9300 3900 9000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 9600 5700 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7800 5700 9600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7800 5700 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7800 5700 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 7500 3900 7200 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5700 5700 7800 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5700 5700 5700 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5700 5700 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 3600 5700 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 1500 5700 3600 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 1500 5700 1500 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 1200 6600 1050 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 3300 6600 3000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 5400 6600 5100 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 9300 6600 9000 +2 1 0 3 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 6300 7500 6600 7200 +2 1 0 3 0 0 100 0 5 0.000 0 0 7 1 0 2 + 0 0 1.50 90.00 180.00 + 3600 5400 3900 5100 diff --git a/sourcecodes/bnt-master/docs/Figures/fa.eps b/sourcecodes/bnt-master/docs/Figures/fa.eps new file mode 100644 index 00000000..03102fbf --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa.eps @@ -0,0 +1,356 @@ +%!PS-Adobe-3.0 EPSF-3.0 +%%Creator: (ImageMagick) +%%Title: (fa.eps) +%%CreationDate: (Tue Nov 16 19:51:50 2004) +%%BoundingBox: 0 0 131 161 +%%DocumentData: Clean7Bit +%%LanguageLevel: 1 +%%Pages: 1 +%%EndComments + +%%BeginDefaults +%%EndDefaults + +%%BeginProlog +% +% Display a color image. The image is displayed in color on +% Postscript viewers or printers that support color, otherwise +% it is displayed as grayscale. +% +/DirectClassPacket +{ + % + % Get a DirectClass packet. + % + % Parameters: + % red. + % green. + % blue. + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/DirectClassImage +{ + % + % Display a DirectClass image. + % + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { DirectClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayDirectClassPacket } image + } ifelse +} bind def + +/GrayDirectClassPacket +{ + % + % Get a DirectClass packet; convert to grayscale. + % + % Parameters: + % red + % green + % blue + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/GrayPseudoClassPacket +{ + % + % Get a PseudoClass packet; convert to grayscale. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassPacket +{ + % + % Get a PseudoClass packet. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassImage +{ + % + % Display a PseudoClass image. + % + % Parameters: + % class: 0-PseudoClass or 1-Grayscale. + % + currentfile buffer readline pop + token pop /class exch def pop + class 0 gt + { + currentfile buffer readline pop + token pop /depth exch def pop + /grays columns 8 add depth sub depth mul 8 idiv string def + columns rows depth + [ + columns 0 0 + rows neg 0 rows + ] + { currentfile grays readhexstring pop } image + } + { + % + % Parameters: + % colors: number of colors in the colormap. + % colormap: red, green, blue color packets. + % + currentfile buffer readline pop + token pop /colors exch def pop + /colors colors 3 mul def + /colormap colors string def + currentfile colormap readhexstring pop pop + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { PseudoClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayPseudoClassPacket } image + } ifelse + } ifelse +} bind def + +/DisplayImage +{ + % + % Display a DirectClass or PseudoClass image. + % + % Parameters: + % x & y translation. + % x & y scale. + % label pointsize. + % image label. + % image columns & rows. + % class: 0-DirectClass or 1-PseudoClass. + % compression: 0-none or 1-RunlengthEncoded. + % hex color packets. + % + gsave + /buffer 512 string def + /byte 1 string def + /color_packet 3 string def + /pixels 768 string def + + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + x y translate + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + currentfile buffer readline pop + token pop /pointsize exch def pop + /Times-Roman findfont pointsize scalefont setfont + x y scale + currentfile buffer readline pop + token pop /columns exch def + token pop /rows exch def pop + currentfile buffer readline pop + token pop /class exch def pop + currentfile buffer readline pop + token pop /compression exch def pop + class 0 gt { PseudoClassImage } { DirectClassImage } ifelse + grestore +} bind def +%%EndProlog +%%Page: 1 1 +%%PageBoundingBox: 0 0 131 161 +userdict begin +DisplayImage +0 0 +131 161 +12.000000 +131 161 +1 +1 +1 +1 +ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffff +ffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffff +ffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffff +ffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffff +ffffffffffffffffe0fffffffffff001ffffffffffffffffffe0ffffffffff8ffe3fffff +ffffffffffffe0fffffffffc7fffc7ffffffffffffffffe0fffffffffbfffffbffffffff +ffffffffe0ffffffffe7fffffcffffffffffffffffe0ffffffffdfffffff7fffffffffff +ffffe0ffffffffbfffffffbfffffffffffffffe0ffffffff7fffffffdfffffffffffffff +e0fffffffeffffffffefffffffffffffffe0fffffffeffffffffefffffffffffffffe0ff +fffffdfffffffff7ffffffffffffffe0fffffffdfe31fffff7ffffffffffffffe0ffffff +fbff7bfffffbffffffffffffffe0fffffffbffb7fffffbffffffffffffffe0fffffffbff +8ffffffbffffffffffffffe0fffffffbffcffffffbffffffffffffffe0fffffffbffa7ff +fffbffffffffffffffe0fffffffbffb7fffffbffffffffffffffe0fffffffbff7bfffffb +ffffffffffffffe0fffffffdfe31fffff7ffffffffffffffe0fffffffdfffffffff7ffff +ffffffffffe0fffffffeffffffffefffffffffffffffe0fffffffeffffffffefffffffff +ffffffe0ffffffff7fffffffdfffffffffffffffe0ffffffffbfffffffbfffffffffffff +ffe0ffffffffdfffffff7fffffffffffffffe0ffffffffe7fffffcffffffffffffffffe0 +fffffffffbfffffbffffffffffffffffe0fffffffffc7fffc7ffffffffffffffffe0ffff +ffffff8ffe3fffffffffffffffffe0fffffffffff001ffffffffffffffffffe0ffffffff +ffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffff +ffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffbfff +ffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffff +ffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffffff +ffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffffffffff +ffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffffffffffffff +e0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ff +ffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffff +ffffffbfffffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffff +ffbfffffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbf +ffffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffff +ffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffff +ffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffffbfffffffffffff +ffffffe0ffffffffffffbfffffffffffffffffffe0fffffffffffeafffffffffffffffff +ffe0fffffffffffeafffffffffffffffffffe0fffffffffffe9fffffffffffffffffffe0 +ffffffffffff1fffffffffffffffffffe0ffffffffffff1fffffffffffffffffffe0ffff +ffffffff1fffffffffffffffffffe0ffffffffffff3fffffffffffffffffffe0ffffffff +ffffbfffffffffffffffffffe0ffffffffffffbfffffffffffffffffffe0ffffffffffff +ffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffff +ffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffff +ffffffffffffe0ffffffffffffffffffffffffffffffffe0fffffffffff001ffffffffff +ffffffffe0ffffffffff8ffe3fffffffffffffffffe0fffffffffc6eeec7ffffffffffff +ffffe0fffffffffbfffffbffffffffffffffffe0ffffffffe3bbfbb8ffffffffffffffff +e0ffffffffdfffffff7fffffffffffffffe0ffffffffaeeeeeeebfffffffffffffffe0ff +ffffff7fffffffdfffffffffffffffe0fffffffebfbfbfbfafffffffffffffffe0ffffff +feffffffffefffffffffffffffe0fffffffceeeeeeeee7ffffffffffffffe0fffffffdfe +38fffff7ffffffffffffffe0fffffffbfb79fbfbfbffffffffffffffe0fffffffbffbbff +fffbffffffffffffffe0fffffffaeecaeeeeebffffffffffffffe0fffffffbffc7fffffb +ffffffffffffffe0fffffffbbfafbfbfbbffffffffffffffe0fffffffbffeffffffbffff +ffffffffffe0fffffffaeeeeeeeeebffffffffffffffe0fffffffdffc7fffff7ffffffff +ffffffe0fffffffdfbbbfbbbf7ffffffffffffffe0fffffffeffffffffefffffffffffff +ffe0fffffffeeeeeeeeeefffffffffffffffe0ffffffff7fffffffdfffffffffffffffe0 +ffffffffbfbfbfbfbfffffffffffffffe0ffffffffdfffffff7fffffffffffffffe0ffff +ffffe6eeeeecffffffffffffffffe0fffffffffbfffffbffffffffffffffffe0ffffffff +fc7bfbc7ffffffffffffffffe0ffffffffff8ffe3fffffffffffffffffe0fffffffffff0 +01ffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffff +ffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffff +ffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffff +ffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffff +ffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffff +e0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ff +ffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffff +ffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffff +ffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffff +ffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ff80ffffffffbfffcf +ffdff01c5f7fffe0ffdeffffffffbfffeffffff5c99f7fffe0ffdffffbffff5fffefffff +f5ebdebfffe0ffdde790cd3f5d3ce9989c6dc7febfffe0ffc1db6bb67eeedb6db6db6c17 +fddfffe0ffdde37bb6fe0edc6d33d9edf7fc1fffe0ffdfdb7bb6feeedb6d7cde5dfbdddf +ffe0ffdfdb6bb6fdf6db6e76db5df99befffe0ff87e59ccc78e04c86f188d8fc31c7ffe0 +fffffffffffffffffeffffffffffffffe0fffffffffffffffffdffffffffffffffe0ffff +fffffffffffff9ffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffff +ffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffff +ffffffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffff +ffffffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffff +ffffffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffff +ffffffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffff +ffffe0ffffffffffffffffffffffffffffffffe0ffffffffffffffffffffffffffffffff +e0 +end +%%PageTrailer +%%Trailer +%%EOF diff --git a/sourcecodes/bnt-master/docs/Figures/fa.fig b/sourcecodes/bnt-master/docs/Figures/fa.fig new file mode 100644 index 00000000..db4bec4a --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa.fig @@ -0,0 +1,16 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 900 1950 300 225 900 1950 1200 2175 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 900 825 300 225 900 825 1200 1050 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1125 900 1650 +4 0 -1 0 0 0 12 0.0000 4 135 135 825 900 X\001 +4 0 -1 0 0 0 12 0.0000 4 135 135 825 2025 Y\001 diff --git a/sourcecodes/bnt-master/docs/Figures/fa.gif b/sourcecodes/bnt-master/docs/Figures/fa.gif new file mode 100644 index 00000000..420c1517 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/fa_caption.fig b/sourcecodes/bnt-master/docs/Figures/fa_caption.fig new file mode 100644 index 00000000..71cd37f2 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_caption.fig @@ -0,0 +1,13 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 900 1950 300 225 900 1950 1200 2175 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 900 825 300 225 900 825 1200 1050 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1125 900 1650 +4 0 -1 0 0 0 12 0.0000 4 135 135 750 2025 Y\001 +4 0 -1 0 0 0 12 0.0000 4 135 120 750 900 X\001 +4 0 -1 0 0 0 12 0.0000 4 180 1620 300 2625 Factor Analysis/PCA\001 diff --git a/sourcecodes/bnt-master/docs/Figures/fa_discrete.fig b/sourcecodes/bnt-master/docs/Figures/fa_discrete.fig new file mode 100644 index 00000000..b5be4ca5 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_discrete.fig @@ -0,0 +1,43 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 975 600 300 225 975 600 1275 825 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 2100 600 300 225 2100 600 2400 825 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 825 525 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 825 1425 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2025 825 600 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2025 825 1425 1425 +2 2 0 1 -1 0 0 0 2 0.000 0 0 7 0 0 5 + 300 1425 750 1425 750 1875 300 1875 300 1425 +2 2 0 1 -1 0 0 0 2 0.000 0 0 7 0 0 5 + 1275 1425 1725 1425 1725 1875 1275 1875 1275 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 825 3000 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2025 825 3000 1425 +2 2 0 1 0 0 100 0 2 3.000 0 0 7 0 0 5 + 2775 1425 3225 1425 3225 1875 2775 1875 2775 1425 +4 0 -1 0 0 0 24 0.0000 4 30 270 1350 750 ...\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1350 1725 R2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 825 675 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1950 675 Xn\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 375 1725 R1\001 +4 0 -1 0 0 0 12 0.0000 4 180 1920 600 2325 Discrete Factor Analysis\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 2100 1650 ...\001 +4 0 0 100 0 -1 12 0.0000 4 135 210 2850 1725 Rn\001 diff --git a/sourcecodes/bnt-master/docs/Figures/fa_discrete.gif b/sourcecodes/bnt-master/docs/Figures/fa_discrete.gif new file mode 100644 index 00000000..54a8bbfb --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_discrete.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/fa_discrete_single.fig b/sourcecodes/bnt-master/docs/Figures/fa_discrete_single.fig new file mode 100644 index 00000000..225a741c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_discrete_single.fig @@ -0,0 +1,18 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 1 0 1 0 7 100 0 -1 0.000 1 0.0000 825 375 300 225 825 375 1125 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 600 825 1275 +2 2 0 1 0 0 100 0 2 0.000 0 0 7 0 0 5 + 600 1275 975 1275 975 1725 600 1725 600 1275 +4 0 -1 0 0 0 12 0.0000 4 135 120 750 1575 R\001 +4 0 -1 0 0 0 12 0.0000 4 135 135 750 450 X\001 +4 0 0 100 0 0 12 0.0000 4 180 1785 300 2025 discrete factor analysis\001 diff --git a/sourcecodes/bnt-master/docs/Figures/fa_regular.fig b/sourcecodes/bnt-master/docs/Figures/fa_regular.fig new file mode 100644 index 00000000..db30e0a0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_regular.fig @@ -0,0 +1,36 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 225 300 3450 3150 +1 3 0 1 -1 7 0 0 -1 0.000 1 0.0000 999 1082 335 335 999 1082 1149 1382 +1 3 0 1 -1 7 0 0 -1 0.000 1 0.0000 2124 1082 335 335 2124 1082 2274 1382 +1 3 0 1 -1 0 0 0 2 0.000 1 0.0000 624 2807 335 335 624 2807 774 3107 +1 3 0 1 -1 0 0 0 2 0.000 1 0.0000 1674 2807 335 335 1674 2807 1824 3107 +1 3 0 1 -1 0 0 0 2 0.000 1 0.0000 2874 2807 335 335 2874 2807 3024 3107 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1425 675 2475 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 975 1425 1575 2400 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 975 1425 2775 2475 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 1425 750 2475 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 1425 1650 2325 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 1425 2775 2400 +4 0 -1 0 0 0 24 0.0000 4 330 3225 225 600 regular factor analysis\001 +-6 +4 0 -1 0 0 0 12 0.0000 4 135 210 825 1200 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1950 1200 X2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 525 2925 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1500 2850 Y2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2700 2850 Y3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/fa_regular.gif b/sourcecodes/bnt-master/docs/Figures/fa_regular.gif new file mode 100644 index 00000000..bb506a59 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_regular.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/fa_scalar.eps b/sourcecodes/bnt-master/docs/Figures/fa_scalar.eps new file mode 100644 index 00000000..ccdc01cc --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_scalar.eps @@ -0,0 +1,414 @@ +%!PS-Adobe-3.0 EPSF-3.0 +%%Creator: (ImageMagick) +%%Title: (fa_scalar.eps) +%%CreationDate: (Tue Nov 16 19:52:00 2004) +%%BoundingBox: 0 0 246 156 +%%DocumentData: Clean7Bit +%%LanguageLevel: 1 +%%Pages: 1 +%%EndComments + +%%BeginDefaults +%%EndDefaults + +%%BeginProlog +% +% Display a color image. The image is displayed in color on +% Postscript viewers or printers that support color, otherwise +% it is displayed as grayscale. +% +/DirectClassPacket +{ + % + % Get a DirectClass packet. + % + % Parameters: + % red. + % green. + % blue. + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/DirectClassImage +{ + % + % Display a DirectClass image. + % + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { DirectClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayDirectClassPacket } image + } ifelse +} bind def + +/GrayDirectClassPacket +{ + % + % Get a DirectClass packet; convert to grayscale. + % + % Parameters: + % red + % green + % blue + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/GrayPseudoClassPacket +{ + % + % Get a PseudoClass packet; convert to grayscale. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassPacket +{ + % + % Get a PseudoClass packet. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassImage +{ + % + % Display a PseudoClass image. + % + % Parameters: + % class: 0-PseudoClass or 1-Grayscale. + % + currentfile buffer readline pop + token pop /class exch def pop + class 0 gt + { + currentfile buffer readline pop + token pop /depth exch def pop + /grays columns 8 add depth sub depth mul 8 idiv string def + columns rows depth + [ + columns 0 0 + rows neg 0 rows + ] + { currentfile grays readhexstring pop } image + } + { + % + % Parameters: + % colors: number of colors in the colormap. + % colormap: red, green, blue color packets. + % + currentfile buffer readline pop + token pop /colors exch def pop + /colors colors 3 mul def + /colormap colors string def + currentfile colormap readhexstring pop pop + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { PseudoClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayPseudoClassPacket } image + } ifelse + } ifelse +} bind def + +/DisplayImage +{ + % + % Display a DirectClass or PseudoClass image. + % + % Parameters: + % x & y translation. + % x & y scale. + % label pointsize. + % image label. + % image columns & rows. + % class: 0-DirectClass or 1-PseudoClass. + % compression: 0-none or 1-RunlengthEncoded. + % hex color packets. + % + gsave + /buffer 512 string def + /byte 1 string def + /color_packet 3 string def + /pixels 768 string def + + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + x y translate + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + currentfile buffer readline pop + token pop /pointsize exch def pop + /Times-Roman findfont pointsize scalefont setfont + x y scale + currentfile buffer readline pop + token pop /columns exch def + token pop /rows exch def pop + currentfile buffer readline pop + token pop /class exch def pop + currentfile buffer readline pop + token pop /compression exch def pop + class 0 gt { PseudoClassImage } { DirectClassImage } ifelse + grestore +} bind def +%%EndProlog +%%Page: 1 1 +%%PageBoundingBox: 0 0 246 156 +userdict begin +DisplayImage +0 0 +246 156 +12.000000 +246 156 +1 +1 +1 +1 +fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffff800fff +fffffffffffffffffff800fffffffffffffffffffffcfffffffffffc7ff1ffffffffffff +ffffffffc7ff1ffffffffffffffffffffcffffffffffe3fffe3ffffffffffffffffffe3f +ffe3fffffffffffffffffffcffffffffffdfffffdffffffffffffffffffdfffffdffffff +fffffffffffffcffffffffff3fffffe7fffffffffffffffff3fffffe7fffffffffffffff +fffcfffffffffefffffffbffffffffffffffffefffffffbffffffffffffffffffcffffff +fffdfffffffdffffffffffffffffdfffffffdffffffffffffffffffcfffffffffbffffff +feffffffffffffffffbfffffffeffffffffffffffffffcfffffffff7ffffffff7fffffff +ffffffff7ffffffff7fffffffffffffffffcfffffffff7ffffffff7fffffffffffffff7f +fffffff7fffffffffffffffffcffffffffefffffffffbffffffffffffffefffffffffbff +fffffffffffffffcffffffffefff8c6fffbffffffffffffffeff18fffffbffffffffffff +fffffcffffffffdfffdecfffdffffffffffffffdffbdfffffdfffffffffffffffffcffff +ffffdfffedafffdffffffffffffffdffdbfffffdfffffffffffffffffcffffffffdfffe3 +efffdffffffffffffffdffc74ffffdfffffffffffffffffcffffffffdffff3efffdfffff +fffffffffdffe7b7fffdfffffffffffffffffcffffffffdfffe9efffdffffffffffffffd +ffd3b7fffdfffffffffffffffffcffffffffdfffedefffdffffffffffffffdffdbb7fffd +fffffffffffffffffcffffffffdfffdeefffdffffffffffffffdffbdb7fffdffffffffff +fffffffcffffffffefff8c47ffbffffffffffffffeff1813fffbfffffffffffffffffcff +ffffffefffffffffbffffffffffffffefffffffffbfffffffffffffffffcfffffffff7ff +ffffff7fffffffffffffff7ffffffff7fffffffffffffffffcfffffffff7ffffffff7fff +ffffffffffff7ffffffff7fffffffffffffffffcfffffffffbfffffffeffffff3cf3ffff +ffbfffffffeffffffffffffffffffcfffffffffdfffffffdffffff3cf3ffffffdfffffff +dffffffffffffffffffcfffffffffefffffff9ffffffffffffffffefffffffbfffffffff +fffffffffcffffffffff3fffffe67ffffffffffffffff3fffffe7ffffffffffffffffffc +ffffffffffdfffffdf8ffffffffffffffffdfffffdfffffffffffffffffffcffffffffff +e3fffe3ff3fffffffffffffffe3fffe3fffffffffffffffffffcfffffffffffc7ff1fffc +7fffffffffffffffc7ff1ffffffffffffffffffffcfffffffffff7800f7fff9fffffffff +fffffff000f7fffffffffffffffffffcfffffffffff7ffff7fffe3ffffffffffffff87ff +fbfffffffffffffffffffcffffffffffefffffbffffcfffffffffffffe5ffffdffffffff +fffffffffffcffffffffffefffffbfffff1ffffffffffff13ffffeffffffffffffffffff +fcffffffffffdfffffdfffffe7ffffffffff8effffff7ffffffffffffffffffcffffffff +ffdfffffdffffff8fffffffffc7dffffffbffffffffffffffffffcffffffffffdfffffef +ffffff3ffffffff3f3ffffffdffffffffffffffffffcffffffffffbfffffefffffffc7ff +ffff8fefffffffeffffffffffffffffffcffffffffffbffffff7fffffff9fffffc7f9fff +fffff7fffffffffffffffffcffffffffffbffffff7fffffffe3fffe3ff7ffffffffbffff +fffffffffffffcffffffffff7ffffffbffffffffcfff9ffcfffffffffdffffffffffffff +fffcffffffffff7ffffffbfffffffff1fc7ffbfffffffffefffffffffffffffffcffffff +fffefffffffdfffffffffe63fff7ffffffffff7ffffffffffffffffcfffffffffeffffff +fdffffffffff0fffcfffffffffffbffffffffffffffffcfffffffffefffffffeffffffff +fcf3ffbfffffffffffdffffffffffffffffcfffffffffdfffffffeffffffffe3fc7e7fff +ffffffffeffffffffffffffffcfffffffffdffffffff7fffffff1fff9dfffffffffffff7 +fffffffffffffffcfffffffffdffffffff7ffffff8ffffe3fffffffffffffbffffffffff +fffffcfffffffffbffffffffbfffffe7ffffecfffffffffffffdfffffffffffffffcffff +fffffbffffffffbfffff1fffffdf1ffffffffffffefffffffffffffffcfffffffff7ffff +ffffdffff8ffffff3fe7ffffffffffff7ffffffffffffffcfffffffff7ffffffffdfffc7 +fffffefff8ffffffffffffbffffffffffffffcfffffffff7ffffffffefff3ffffff9ffff +3fffffffffffdffffffffffffffcffffffffefffffffffeff8fffffff7ffffc7ffffffff +ffeffffffffffffffcffffffffeffffffffff7c7ffffffcffffff9fffffffffff7ffffff +fffffffcffffffffeffffffffff63fffffffbffffffe3ffffffffffbfffffffffffffcff +ffffffdffffffffff9ffffffff7fffffffcffffffffffdfffffffffffffcffffffffdfff +ffffffc3fffffffcfffffffff1fffffffffefffffffffffffcffffffffbffffffffe3dff +ffffdbfffffffffe7fffffffff7ffffffffffffcffffffffbffffffff1fdffffffa7ffff +ffffff8fffffffffbffffffffffffcfffffffebfffffffcffebfffff5ffffffffffff3ff +ffffffdffffffffffffcfffffffe7ffffffe3ffebffffe27fffffffffffc7fffffffefff +fffffffffcfffffffe5ffffff1fffd3ffffe9fffffffffffff9ffffffff6fffffffffffc +fffffffe3fffff8ffffe5ffffc7fffffffffffffe3fffffffb7ffffffffffcfffffffcbf +fffe7fffff1ffff9fffffffffffffffcfffffffd7ffffffffffcfffffffc7fff71ffffff +1fffffffffffffffffffff1ffffff6bffffffffffcfffffffcfffc8fffffff9fffffffff +ffffffffffffe7fffff93ffffffffffcfffffffcfff07fffffffcfffffffffffffffffff +fff8fffffe1ffffffffffcfffffffdffc3ffffffffefffffffffffffffffffffff3fffff +9ffffffffffcffffffffff803fffffffffffffffffffffffffffffffc7ffffefffffffff +fcffffff001fffffffffffff800ffffffffffffffffffff9dffff800fffffffcfffff8ff +e3fffffffffffc7ff1fffffffffffffffffffe27ffc7ff1ffffffcffffc6eeec7fffffff +ffe377763fffffffffffffffffffcbfe377763fffffcffffbfffffbfffffffffdfffffdf +ffffffffffffffffff00fdfffffdfffffcfffe3bbfbb8fffffffff1ddfddc7ffffffffff +fffffffff871ddfddc7ffffcfffdfffffff7fffffffefffffffbffffffffffffffffffff +efffffffbffffcfffaeeeeeeebfffffffd77777775ffffffffffffffffffffd77777775f +fffcfff7fffffffdfffffffbfffffffeffffffffffffffffffffbfffffffeffffcffebfb +fbfbfafffffff5fdfdfdfd7fffffffffffffffffff5fdfdfdfd7fffcffeffffffffeffff +fff7ffffffff7fffffffffffffffffff7ffffffff7fffcffceeeeeeeee7fffffe7777777 +773ffffffffffffffffffe7777777773fffcffdfe38dffff7fffffeff1c47fffbfffffff +fffffffffffefe38fffffbfffcffbfb799bfbfbfffffdfdbcb9fdfdfffffffffffffffff +fdfd7dfdfdfdfffcffbffbb5ffffbfffffdffddfbfffdfffff3cf3fffffffffdffbbffff +fdfffcffaeecaceeeebfffffd7765737775fffff3cf3fffffffffd7753417775fffcffbf +fc7dffffbfffffdffe3f7fffdffffffffffffffffffdffc7b6fffdfffcffbbfaf9fbfbbf +ffffddfd7cfdfddffffffffffffffffffddfcf96dfddfffcffbffefdffffbfffffdfff7d +ffffdffffffffffffffffffdffefb6fffdfffcffaeeeeceeeebfffffd7777337775fffff +fffffffffffffd7767367775fffcffdffc78ffff7fffffeffe383fffbfffffffffffffff +fffeffc7127ffbfffcffdfbbbfbbbf7fffffefdddfdddfbffffffffffffffffffefdddfd +ddfbfffcffeffffffffefffffff7ffffffff7fffffffffffffffffff7ffffffff7fffcff +eeeeeeeeeefffffff7777777777fffffffffffffffffff7777777777fffcfff7fffffffd +fffffffbfffffffeffffffffffffffffffffbfffffffeffffcfffbfbfbfbfbfffffffdfd +fdfdfdffffffffffffffffffffdfdfdfdfdffffcfffdfffffff7fffffffefffffffbffff +ffffffffffffffffefffffffbffffcfffe6eeeeecfffffffff37777767ffffffffffffff +fffffff37777767ffffcffffbfffffbfffffffffdfffffdffffffffffffffffffffffdff +fffdfffffcffffc7bfbc7fffffffffe3dfde3ffffffffffffffffffffffe3dfde3fffffc +fffff8ffe3fffffffffffc7ff1ffffffffffffffffffffffffc7ff1ffffffcffffff001f +ffffffffffff800ffffffffffffffffffffffffff800fffffffcffffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fffcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fffffcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffff +fffffffffffff80fffffffffffe7ffeffffffffffffffffffffffffffcffffffffffffff +fffdeffffffffffff7fffffffffffffffffffffffffffffcfffffffffffffffffdffffbf +fffffff7fffffffffffffffffffffffffffffcfffffffffffffffffdde790cd3e69e74cc +4e3ffffffffffffffffffffffffcfffffffffffffffffc1db6bb67db6db6db6dbfffffff +fffffffffffffffffcfffffffffffffffffdde37bb6fe36e3699ecffffffffffffffffff +fffffffcfffffffffffffffffdfdb7bb6fdb6db6be6f3ffffffffffffffffffffffffcff +fffffffffffffffdfdb6bb6fdb6db73b6dbffffffffffffffffffffffffcffffffffffff +fffff87e59ccc7e4264378c47ffffffffffffffffffffffffcffffffffffffffffffffff +ffffffffff7ffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffe +fffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffffff +fffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc +fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffffc +end +%%PageTrailer +%%Trailer +%%EOF diff --git a/sourcecodes/bnt-master/docs/Figures/fa_scalar.fig b/sourcecodes/bnt-master/docs/Figures/fa_scalar.fig new file mode 100644 index 00000000..7a2c3e41 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_scalar.fig @@ -0,0 +1,41 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 300 375 3675 1950 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 975 600 300 225 975 600 1275 825 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 2475 600 300 225 2475 600 2775 825 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 600 1650 300 225 600 1650 900 1875 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3315 1654 300 225 3315 1654 3615 1879 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1575 1650 300 225 1575 1650 1875 1875 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 825 600 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1200 750 3075 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2400 825 750 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2625 825 3225 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 825 1425 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2400 825 1650 1350 +4 0 -1 0 0 0 12 0.0000 4 135 225 900 675 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2325 675 Xn\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 450 1725 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 270 3150 1725 Ym\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 1575 750 ...\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1425 1725 Y2\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 2175 1650 ...\001 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/fa_scalar.gif b/sourcecodes/bnt-master/docs/Figures/fa_scalar.gif new file mode 100644 index 00000000..5e5a2598 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_scalar.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/fa_scalar_caption.fig b/sourcecodes/bnt-master/docs/Figures/fa_scalar_caption.fig new file mode 100644 index 00000000..0862b150 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/fa_scalar_caption.fig @@ -0,0 +1,38 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 300 375 3675 1950 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 975 600 300 225 975 600 1275 825 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 2475 600 300 225 2475 600 2775 825 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 600 1650 300 225 600 1650 900 1875 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3315 1654 300 225 3315 1654 3615 1879 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1575 1650 300 225 1575 1650 1875 1875 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 825 600 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1200 750 3075 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2400 825 750 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2625 825 3225 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 825 1425 1425 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2400 825 1650 1350 +4 0 -1 0 0 0 12 0.0000 4 135 210 900 675 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 2325 675 Xn\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 450 1725 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 270 3150 1725 Ym\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 1575 750 ...\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1425 1725 Y2\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 2175 1650 ...\001 +-6 +4 0 -1 0 0 0 12 0.0000 4 180 1170 1200 2325 Factor analysis\001 diff --git a/sourcecodes/bnt-master/docs/Figures/factorial_hmm3.fig b/sourcecodes/bnt-master/docs/Figures/factorial_hmm3.fig new file mode 100644 index 00000000..a42ed6bb --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/factorial_hmm3.fig @@ -0,0 +1,81 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 1 0 0 100 0 -1 0.000 0 1 1 0 2587.500 3637.500 1500 2775 1200 3675 1500 4500 + 0 0 1.00 60.00 120.00 +5 1 0 1 0 0 100 0 -1 0.000 0 1 1 0 2772.051 3261.376 1500 1875 900 3450 1425 4575 + 0 0 1.00 60.00 120.00 +5 1 0 1 0 0 100 0 -1 0.000 0 1 1 0 5962.500 3637.500 4875 2775 4575 3675 4875 4500 + 0 0 1.00 60.00 120.00 +5 1 0 1 0 0 100 0 -1 0.000 0 1 1 0 4162.500 3637.500 3075 2775 2775 3675 3075 4500 + 0 0 1.00 60.00 120.00 +5 1 0 1 0 0 100 0 -1 0.000 0 1 1 0 4347.051 3261.376 3075 1875 2475 3450 3000 4575 + 0 0 1.00 60.00 120.00 +5 1 0 1 0 0 100 0 -1 0.000 0 1 1 0 6147.051 3261.376 4875 1875 4275 3450 4800 4575 + 0 0 1.00 60.00 120.00 +6 1425 4350 2100 4875 +1 1 0 1 0 0 100 0 5 0.000 1 0.0000 1749 4596 300 225 1749 4596 2049 4821 +4 0 0 100 0 0 20 0.0000 4 195 345 1575 4725 Y1\001 +-6 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1770 3645 300 225 1770 3645 2070 3870 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 3420 3645 300 225 3420 3645 3720 3870 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 5220 3645 300 225 5220 3645 5520 3870 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1770 2745 300 225 1770 2745 2070 2970 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 3420 2745 300 225 3420 2745 3720 2970 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 5220 2745 300 225 5220 2745 5520 2970 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 5220 1845 300 225 5220 1845 5520 2070 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 3420 1845 300 225 3420 1845 3720 2070 +1 1 0 1 0 7 100 0 -1 0.000 1 0.0000 1770 1845 300 225 1770 1845 2070 2070 +1 1 0 1 0 0 100 0 5 0.000 1 0.0000 3399 4596 300 225 3399 4596 3699 4821 +1 1 0 1 0 0 100 0 5 0.000 1 0.0000 5199 4521 300 225 5199 4521 5499 4746 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 1 + 4200 4200 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2100 3600 3150 3600 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3750 3600 4950 3600 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 1 + 4200 3300 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2100 2700 3150 2700 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3750 2700 4950 2700 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 1 + 4200 2400 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2100 1800 3150 1800 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3750 1800 4950 1800 +2 1 0 1 0 0 100 0 5 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1725 3900 1725 4350 +2 1 0 1 0 0 100 0 5 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5175 3825 5175 4275 +2 1 0 1 0 0 100 0 5 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 3900 3450 4350 +4 0 0 100 0 0 20 0.0000 4 195 345 1575 1950 A1\001 +4 0 0 100 0 0 20 0.0000 4 195 330 1575 2850 B1\001 +4 0 0 100 0 0 20 0.0000 4 195 330 1575 3750 C1\001 +4 0 0 100 0 0 20 0.0000 4 195 345 3225 1950 A2\001 +4 0 0 100 0 0 20 0.0000 4 195 330 3225 2850 B2\001 +4 0 0 100 0 0 20 0.0000 4 195 330 3225 3750 C2\001 +4 0 0 100 0 0 20 0.0000 4 195 345 5025 1950 A3\001 +4 0 0 100 0 0 20 0.0000 4 195 330 5025 2850 B3\001 +4 0 0 100 0 0 20 0.0000 4 195 330 5025 3750 C3\001 +4 0 0 100 0 0 30 0.0000 4 30 525 6375 3000 . . .\001 +4 0 0 100 0 0 20 0.0000 4 195 345 3225 4725 Y2\001 +4 0 0 100 0 0 20 0.0000 4 195 345 5025 4650 Y3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/filter.eps b/sourcecodes/bnt-master/docs/Figures/filter.eps new file mode 100644 index 00000000..27fd6ced --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/filter.eps @@ -0,0 +1,9861 @@ +%!PS-Adobe-3.0 EPSF-3.0 +%%Creator: (ImageMagick) +%%Title: (filter.eps) +%%CreationDate: (Tue Nov 16 20:04:33 2004) +%%BoundingBox: 0 0 657 525 +%%DocumentData: Clean7Bit +%%LanguageLevel: 1 +%%Pages: 1 +%%EndComments + +%%BeginDefaults +%%EndDefaults + +%%BeginProlog +% +% Display a color image. The image is displayed in color on +% Postscript viewers or printers that support color, otherwise +% it is displayed as grayscale. +% +/DirectClassPacket +{ + % + % Get a DirectClass packet. + % + % Parameters: + % red. + % green. + % blue. + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/DirectClassImage +{ + % + % Display a DirectClass image. + % + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { DirectClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayDirectClassPacket } image + } ifelse +} bind def + +/GrayDirectClassPacket +{ + % + % Get a DirectClass packet; convert to grayscale. + % + % Parameters: + % red + % green + % blue + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/GrayPseudoClassPacket +{ + % + % Get a PseudoClass packet; convert to grayscale. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassPacket +{ + % + % Get a PseudoClass packet. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassImage +{ + % + % Display a PseudoClass image. + % + % Parameters: + % class: 0-PseudoClass or 1-Grayscale. + % + currentfile buffer readline pop + token pop /class exch def pop + class 0 gt + { + currentfile buffer readline pop + token pop /depth exch def pop + /grays columns 8 add depth sub depth mul 8 idiv string def + columns rows depth + [ + columns 0 0 + rows neg 0 rows + ] + { currentfile grays readhexstring pop } image + } + { + % + % Parameters: + % colors: number of colors in the colormap. + % colormap: red, green, blue color packets. + % + currentfile buffer readline pop + token pop /colors exch def pop + /colors colors 3 mul def + /colormap colors string def + currentfile colormap readhexstring pop pop + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { PseudoClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayPseudoClassPacket } image + } ifelse + } ifelse +} bind def + +/DisplayImage +{ + % + % Display a DirectClass or PseudoClass image. + % + % Parameters: + % x & y translation. + % x & y scale. + % label pointsize. + % image label. + % image columns & rows. + % class: 0-DirectClass or 1-PseudoClass. + % compression: 0-none or 1-RunlengthEncoded. + % hex color packets. + % + gsave + /buffer 512 string def + /byte 1 string def + /color_packet 3 string def + /pixels 768 string def + + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + x y translate + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + currentfile buffer readline pop + token pop /pointsize exch def pop + /Times-Roman findfont pointsize scalefont setfont + x y scale + currentfile buffer readline pop + token pop /columns exch def + token pop /rows exch def pop + currentfile buffer readline pop + token pop /class exch def pop + currentfile buffer readline pop + token pop /compression exch def pop + class 0 gt { PseudoClassImage } { DirectClassImage } ifelse + grestore +} bind def +%%EndProlog +%%Page: 1 1 +%%PageBoundingBox: 0 0 657 525 +userdict begin +DisplayImage +0 0 +657 525 +12.000000 +657 525 +1 +1 +1 +8 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00000000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff00000000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff000000000000000000ffffffffffffffff0000ff00000000 +00000000ffffff0000000000ffffffffffffff0000ffffffffffffffff00ffffffffffff +ff0000ffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffffffff +ffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffffff00000000ffffffff +ff0000ffffffff00000000ffffffffffff0000ffffffffffffffff0000ffffffffffffff +ffffff00ffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffff +ff00000000ffffffffffffffffffffffffffffffffffffffff00ffffffffffffff00ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffff +ffffff000000ffffffff00ffffffffffff00000000ffffffffff00ffffffffffffffffff +00ffffffffffff00ffffffffff00ffffffffff0000ffffffffffffffffffffffffffffff +ffffff00ffffffffffffffff0000ffffffffffffffffffffffffffffff00ffffffffff00 +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffff0000ffffff00ffffffffffffffff000000ffffffff0000 +ffffffffffffffff00ffffffffffff0000ffffffffff0000ffffffff0000ffffffffffff +ffffffffffffffffffffff00ffffffffffffffffff0000ffffffffffffffffffffffffff +ff0000ffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff0000ffffffffffff0000ffff0000ffffffffffffffff +ff000000ffff0000ffffffffffffffff0000ffffffffffff0000ffffffffffff0000ffff +ff0000ffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffffff +ffffffffffffffffffff0000ffffffffffff0000ffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffff00 +00ffffffffffffffffffff0000ffff00ffffffffffffffffff0000ffffffff0000000000 +00ffffffff0000ffffff0000ffffff0000000000ffff000000ffffff0000ffffffffffff +ffffff0000ffffffffffffff0000ffffff000000000000ffffffff0000ffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +0000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffff +ffffff000000ff0000ffffffffffffffffffffff00000000ffffffffffffffffff0000ff +ffffffffffff0000ffffffffffff000000ffff0000ffffffff0000ffffffffff00ffffff +0000ffffffffffffffffffff0000ffffffffffffff000000ffffffff0000ffffffffffff +000000ffffffffff000000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff0000ffffffff000000ffff0000ffffffffffffffffffffffff000000ffffff +ffffffffffff0000ffffffffffffff0000ffffffffffffff0000ffff0000ffffffff0000 +ffffffffff00ffffff0000ffffffffffffffffffff0000ffffffffffffff0000ffffffff +ff0000ffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff00000000000000ffffffff0000ffffffffffffffffff +ffffff00000000ffffffffffffffff0000ffffffffffffff0000ffffffffffffff0000ff +ff0000ffffffff000000ffffff0000ffffff0000ffffffffffffffffffff0000ffffffff +ffffffffffffffffffff0000ffffffffffffff0000ffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000 +ffffffffffffffffffffff0000000000ffffffffffffffff0000ffffffffffffff0000ff +ffffffffffff0000ffff0000ffffffffff0000ffffff00ffffffff0000ffffffffffffff +ffffff0000ffffffffffffffffffffffffffff0000ffffffffffffff0000ffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffff +ffffffffffffff0000ffffffffffffffffffffff00ffff000000ffffffffffffff0000ff +ffffffffffff0000ffffffffffffff0000ffff0000ffffffffff000000ff0000ffffffff +0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffff +ff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffffffffff0000ffffffffffffffffffff00ffffffff000000 +ffffffffffff0000ffffffffffffff0000ffffffffffffff0000ffff0000ffffffffffff +0000ff0000ffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffff +ff0000ffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff00000000ff0000ffffff000000ffffffffffffffffffffffffffffffffff +ffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffff +00ffffffffff000000ffffffffffff0000ffffffffffffff0000ffffffffffffff0000ff +ff0000ffffffffffff00000000ffffffffff0000ffffffffffffffffffff0000ffffffff +ffffffffffffffffffff0000ffffffffffffff0000ffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff00ffff0000ff0000ffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000 +00ffffffffffffff0000ffffffffffff000000ffffffffff000000ffffffffffff0000ff +ffffffffffff0000ffff0000ffffffffffffff000000ffffffffff000000ffffffffffff +ffffff0000ffffffffffffff0000ffffffffff0000ffffffffffffff0000ffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffff +ffffffffffffffff0000ffffffffffff0000ffffffffffffff00000000ffffffffff0000 +ffffffffffff0000ff00ffffffff0000ffffff0000ffffffffffffff0000ffffffffffff +ff0000ffffffffffffffffff0000ffffffffffffff000000ffffffff0000ff00ffffffff +0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffffffffffff0000ffffffff00ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff000000000000ffffffffffffffff0000ffffffff000000000000ffffffff000000 +00000000ffffff0000ffffffffffffff000000ffffffff0000ffffff0000ffffffffffff +ffff00ffffffffffffff0000ffffffffffffff00000000000000ffffffff0000ffffffff +ffff000000ffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffffff00ffffffff0000ffffff0000ffffffffffffffffffffffff +ffffffffffffffffffff00ffffffff0000ffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffff0000ffffff +ffffffffffffffffffff0000ffffffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00000000000000ff000000ffffffff0000ffff0000000000ff +ffff000000000000ffff0000ff000000ff000000ffffff00000000000000ffffffffff00 +000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff00ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff +ffffffff00ffffffffffffffffffffffffffff00ffffffffffffffffffff00ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ffffffff00 +00ffffff0000ffffffff0000ffffff0000ffff000000ffff00ffff0000ffffffff0000ff +ffff0000ffffff00ffffff000000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +0000ffffffffffffffffffff00ffffffffffffffffffffffffffff0000ffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffff0000ffffffff0000ffffff0000ffffffff0000000000000000ff0000ffffffffffff +0000ffffffff0000ffffff0000ffffff0000ffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffff0000ffffffffffffffffffffff000000 +00ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffff0000ffffffff0000ffffff0000ffffff0000ffffffffffffff +ff0000ffffffffffff0000ffffffff0000ffffff0000ffffff0000ffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffff0000ffffffff0000ffffff0000ffffff +0000ffffffffffffffff0000ffffffffffff0000ffffffff0000ffffff0000ffffff0000 +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ffffffff00 +00ffffff0000ffffff000000ffffffffffffff0000ffffffffffff0000ffffffff0000ff +ffff0000ffffffff00000000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffff0000ffffffff0000ffffff0000ffffffff0000ffffffffff00ff0000ffffffffffff +0000ffffffff0000ffffff0000ffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffff0000ffffffff0000ffffff0000ffffffff00000000000000ff +ff0000ffffffffffff0000ffffffff0000ffffff0000ffffff00000000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000000000ffff00000000ffff0000000000ffff000000ff +ffff0000000000ffff0000000000ffffff00000000ffff00000000ff00000000ffff0000 +0000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffff0000000000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000000000ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff00ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +00000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff00ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff00ff0000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00ff0000ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00000000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000 +ffffffffff00000000ffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff00000000ffffffffffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff000000000000000000ffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffff00ffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff000000ffffffffffffffff0000ffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffff00000000ffffffffff0000 +ffffffffffffffffffffffffffffffffffff0000ffffffffff00000000ffffffffffffff +ffffffffffffffffffffffffffff00ffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffff0000ffffffffff00000000ffffffffffffffffff +ffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff000000ffffffffffffffff0000ffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff +000000ffffffff00ffffffffffffffffffffffffffffffffffffff00ffffffffffffffff +0000ffffffffffffffffffffffffffffff00ffffffffffff00ffffffffffffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff0000 +ffffffffffffffffffffffffffffff00ffffffffffff00ffffffffffffff00ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff000000ffffffffffffff00ffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff0000ffffffffffff0000ffffff00ffffffffffffffffffffffffffffffffffffff00 +ffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffff0000ff +ffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff00ffff +ffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffff0000ffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ff +ffffffffff0000ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff0000ffffffffffff0000ffff0000ffffffffffffffffffffff +ffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffffffffff0000 +ffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffff +ffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000ffffffffff0000ffffffffff0000ffffff000000000000ffffffff +0000000000ffffffff0000ff000000ffff0000ff00000000ffffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff0000000000ffffffffff0000ff000000ffffffff0000 +0000ffffffffffff0000ffff00000000ffff00000000ffffffffff0000000000ffffff00 +0000000000ff00000000ffffffffffffffffffff0000ffffffffffff0000ffff0000ffff +ffff000000000000ff00000000ffffff0000ffffffffffffffffff0000ffffffffffffff +0000ffffff000000000000ffffffffff0000ffffffffffffffffff0000ffffffffffffff +ff0000000000ffff000000ffffff0000ffffffffffffffffff0000ffffffffffffff0000 +ffffff000000000000ffffffffff0000ffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffff000000ffffffffff00ffffffffff000000ffffffff +ff0000ffffffffff0000ffffff0000ff0000000000000000ffff0000000000000000ffff +ff000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000ffffffff +ffffffffffffffffffffffffffffffffffffffffff000000ffff000000ffff0000000000 +000000ffffff0000ffff0000000000ff0000000000ffff0000000000ffff0000ffff0000 +00ffff000000ffffffff000000ffff0000ffffffffffffffffffffffff0000ffffffffff +000000ff0000ffffffffffffff000000ffff0000ffffffff0000ffffffffffffffffffff +0000ffffffffffffff000000ffffffff0000ffffffffffffff000000ffffffffffffffff +0000ffffffffffffffffff0000ffffffffff00ffffff0000ffffffffffffffffffff0000 +ffffffffffffff000000ffffffff0000ffffffffffffff000000ffffffffff000000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffff0000ffffff +ffffff0000ffffffffff0000ffffffff0000ffffffff000000ffff0000ffffffffffff00 +0000ffff00000000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffff +0000ffffffff0000ffffffffffff0000ffffff000000ffffffff000000ffffffff000000 +ffffff0000ffff000000ffffff0000ffffffffff0000ff0000ffffffffffffffffffffff +ffff0000ffffffff000000ffff0000ffffffffffffffff0000ff0000ffffffffff0000ff +ffffffffffffffffff0000ffffffffffffff0000ffffffffff0000ffffffffffffffff00 +00ffffffffffffffff0000ffffffffffffffffff0000ffffffffff00ffffff0000ffffff +ffffffffffffff0000ffffffffffffff0000ffffffffff0000ffffffffffffffff0000ff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +00ffffff00ffffffffffffff0000ffffffffff0000ffffffff000000000000000000ffff +0000ffffffffffff0000ffffffff000000ffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff00ffffffff0000ffffffff0000ffffffffffff0000ffffffff0000ffffffff00 +00ffffffffff0000ffffffff0000ffffff00ffffffff0000ffffffffffff000000ffffff +ffffffffffffffffffffff00000000000000ffffffff0000ffffffffffffffffff000000 +ffffffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000 +ffffffffffffffff0000ffffffffffffffff0000ffffffffffffffffff000000ffffff00 +00ffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffff +ffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff000000ff0000ffffffffffffff0000ffffffffff0000ffffffff0000 +ffffffffffffffffff0000ffffffffffff0000ffffffffff0000ffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00000000ffffffff0000ffffffffffff0000ffff +ffff0000ffffffff0000ffffffffff0000ffffffff0000ffffffffffff00000000ffffff +ffffff000000ffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffff +ffffffffffff000000ffffffffffff0000ffffffffffffffffffff0000ffffffffffffff +ffffffffffffff0000ffffffffffffffff0000ffffffffffffffff0000ffffffffffffff +ffffff0000ffffff00ffffffff0000ffffffffffffffffffff0000ffffffffffffffffff +ffffffffff0000ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff000000ff0000ffffffffffffff0000ffffffff +ff0000ffffff000000ffffffffffffffffff0000ffffffffffff0000ffffffffff0000ff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffff0000ffffffff0000ff +ffffffffff0000ffffffff0000ffffffff0000ffffffffff0000ffffffff0000ffffffff +0000ffff0000ffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffff +ffffffff0000ffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffff +0000ffffffffffffffffffffffffffff0000ffffffffffffffff0000ffffffffffffffff +0000ffffffffffffffffffff000000ff0000ffffffff0000ffffffffffffffffffff0000 +ffffffffffffffffffffffffffff0000ffffffffffffffff0000ffffffffffff0000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ffffffffff +ffffff0000ffffffffff0000ffffff000000ffffffffffffffffff0000ffffffffffff00 +00ffffffffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +0000ffffffff0000ffffffffffffff0000ffff0000ffffffffff0000ffffffffff0000ff +ffffff0000ffffff0000ffffff0000ffffffffffff00000000ffffffffffffffffffffff +ffff0000ffffffffffffffffff0000ffffffffffffffffff00000000ffffffffff0000ff +ffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffffffff00 +00ffffffffffffffff0000ffffffffffffffffffffff0000ff0000ffffffff0000ffffff +ffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffffffff0000ff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00000000ffffffffffffffff0000ffffffffff0000ffffffff0000ffffffffffff00ffff +0000ffffffffffff0000ffffffffff0000ffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffff0000ffffffff0000ffffffffffffff0000000000ffffffffffff00 +00ffffffffff0000ffffffff0000ffff0000ffffffff0000ffffffffff0000ff000000ff +ffffffffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffff0000ff00 +0000ffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000 +ffffffffffffffff0000ffffffffffffffff0000ffffffffffffffffffffff00000000ff +ffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffff +ffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffff0000ffffffffff0000ffffffff0000 +00ffffffff0000ffff0000ffffffffffff0000ffffffff0000ffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffff0000ffffffff0000ffffffffffff0000ffff +ffffffffffffffff0000ffffffffff0000ffffffff0000ffff0000ffffffff0000ffffff +ff0000ffffff0000ffffffffffffffffffffffff0000ffffffffffffffffff000000ffff +ffffffff0000ffffff0000ffffffff000000ffffffffffffffffff0000ffffffffffffff +0000ffffffffff0000ffffffffffffffff0000ffffffffffffffff0000ffffffffffffff +ffffffffff000000ffffffffff000000ffffffffffffffffff0000ffffffffffffff0000 +ffffffffff0000ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffffff +ff0000ff00ffffff00000000000000ffffff0000ffffffffffff0000ffffffff00ffffff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000ffff00000000ffffffff0000ff +ffffffffff00000000000000ffffffffff0000ffffffffff0000ffffffff0000ffff0000 +ffff00000000ffffffff00ffffffff000000ffffffffffffffffffffff0000ffffffffff +ffffffffff0000ffffffffffff00ffffffff000000ffffffff0000ffffffffffffffffff +0000ffffffffffffff000000ffffffff0000ff00ffffffffff0000ffffffffffffffffff +0000ffffffffffffffffffffffff0000ffffffffffffff0000ffffffffffffffffff0000 +ffffffffffffff000000ffffffff0000ff00ffffffffff0000ffffffffffff0000ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffff0000000000ffffffff000000ffffffff0000000000ffff000000000000ffffffffff +0000000000ffffffffff0000000000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ff +00000000000000000000ffffffff0000000000000000ffff000000000000ff0000000000 +00ff0000000000ff00000000ff00000000ff000000ffff000000000000ffffffffffffff +000000000000ffffffffffffffff0000ffffffffff000000ffff000000000000ffff0000 +ffffffffffffff00000000000000ffffffff0000ffffffffffff000000ffffffffff0000 +ffffffffffffffffff0000ffffffffffffffffffffffffff00ffffffffffffff0000ffff +ffffffffff00000000000000ffffffff0000ffffffffffff000000ffffffffff0000ffff +ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ff000000000000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff0000ffff +ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffff +ffffffffff00ffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffffffffffffff0000ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff000000ffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffffffffffffffffffffffffffffffffff00000000ffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00000000000000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffff +ffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffffffff00000000ffffffffff +ffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffff0000ffffffffffffffffffffffffffffff00ffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff00ffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffffffffff +0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff000000000000ff00000000ffffff0000ffffffffffffffffff0000ffffffffff +ffff0000ffffff000000000000ffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff000000ffff0000ffffffff0000ffffffffffffffff +ffff0000ffffffffffffff000000ffffffff0000ffffffffffff000000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff0000ff0000ffffffffff00 +00ffffffffffffffffffff0000ffffffffffffff0000ffffffffff0000ffffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +0000ffffffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff +0000ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff000000ffffffffffff0000ffffffffffffffffffff0000ffffffffff +ffffffffffffffffff0000ffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffffffffffffffff +ffff0000ffffffffffffffffffffffffffff0000ffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ffffffffff00 +00ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ff000000ffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff +0000ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff0000ffffff0000ffffffff000000ffffffffffffffffff0000ffffffffff +ffff0000ffffffffff0000ffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff00ffffffff000000ffffffff0000ffffffffffffff +ffff0000ffffffffffffff000000ffffffff0000ff00ffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff000000ffff000000000000ffff +0000ffffffffffffff00000000000000ffffffff0000ffffffffffff000000ffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00ff0000ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff00ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff00ff0000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff000000000000000000ffffffffffffffff0000ff00 +00000000000000ffffff0000000000ffffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff000000ffffffffffffffffffffffffffff000000 +ffffffffffffffffffffffffffffffffffffffffff00ffffffffffffff0000ffffffffff +ffffffffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffff00ffffffffffffff00ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff00000000ff +ffffffff0000ffffffff00000000ffffffffffff0000ffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff00ffffffff +ffff0000ffffffffffffffffffffffffffffffffff0000ffffffffff00000000ffffffff +ffffffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffff000000ffffffff00ffffffffffff00000000ffffffffff00ffffffffffff +ffffff00ffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffff0000ffffffffff00ffffffffffffffffffffffff +ffffffffff00ffffffffff0000ffffffffffffffffffffffffffffffffff00ffffffffff +ffffff0000ffffffffffffffffffffffffffffff00ffffffffffff00ffffffffffffff00 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffff0000ffffff00ffffffffffffffff000000ffffff +ff0000ffffffffffffffff00ffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffff0000ffffffff0000ffffff +ffffffffffffffffffffffffffff0000ffffffff0000ffffffffffffffffffffffffffff +ffff00ffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffff +0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffff0000ffffffffff +ffffffff000000ffff0000ffffffffffffffff0000ffffffffffff0000ffffffffffffff +ff0000ffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffff0000 +ffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffff0000ffffffffff +ffffffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffffffff +ff0000ffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000 +ffff0000ffffffffffffffffffff0000ffff00ffffffffffffffffff0000ffffffff0000 +00000000ffffffffffff0000ffffffffffffffffffff000000ff0000ffffffffff000000 +0000ffffffffff0000ffff000000000000ffffffff0000000000ffffffffffffff0000ff +ffff0000ffff0000000000ffff000000ffffff0000ffffffffffffffffff0000ffffffff +ffffff0000ffffff000000000000ffffffffff0000ffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffff000000ff0000ffffffffffffffffffffff00000000ffffffffffffffffff +0000ffffffffffffff0000ffffffffffffffff0000ffffffffffffffff0000ffffff0000 +00ffffffff0000ffffff0000ffffffff0000ffffffff0000ffffffff000000ffff000000 +ffffffffffff000000ffff0000ffffff0000ffffffffff00ffffff0000ffffffffffffff +ffffff0000ffffffffffffff000000ffffffff0000ffffffffffffff000000ffffffffff +000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffff000000ffff0000ffffffffffffffffffffffff000000 +ffffffffffffffffff0000ffffffffffffff0000ffffffffffffffff0000ffffffffffff +ffff0000ffffffff0000ffffff0000ffffffff000000ffffff0000ffffffff0000ffffff +ff000000ffffff0000ffffffffffffff0000ffff0000ffffff0000ffffffffff00ffffff +0000ffffffffffffffffffff0000ffffffffffffff0000ffffffffff0000ffffffffffff +ffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff00000000000000ffffffff0000ffffffffffff +ffffffffffff00000000ffffffffffffffff0000ffffffffffffff0000ffffffffffffff +ff0000ffffffffffffff0000ffffffffff0000ffffff000000000000000000ffffff0000 +ffffffff0000ffffffffff00ffffffff0000ffffffffffffff0000ffff0000ffffff0000 +00ffffff0000ffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffff +ff0000ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ff0000ffffffffffffffffffffff0000000000ffffffffffffffff0000ffffffffffffff +0000ffff00000000000000000000000000ffff0000ffffffffff0000ffffff0000ffffff +ffffffffffffff0000ffffffff0000ffffffffffffffff00000000ffffffffffffff0000 +ffff0000ffffffff0000ffffff00ffffffff0000ffffffffffffffffffff0000ffffffff +ffffffffffffffffffff0000ffffffffffffffff0000ffffffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffff0000ffffffffffffffffffffff00ffff000000ffffffffffffff +0000ffffffffffffff0000ffff00000000000000000000000000ffff0000ffffffffff00 +00ffff000000ffffffffffffffffffff0000ffffffff0000ffffffffffff0000ffff0000 +ffffffffffffff0000ffff0000ffffffff000000ff0000ffffffff0000ffffffffffffff +ffffff0000ffffffffffffffffffffffffffff0000ffffffffffffffff0000ffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000ffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffff00ffffffff +000000ffffffffffff0000ffffffffffffff0000ffffffffffffffff0000ffffffffffff +ff0000ffffffffff0000ffff000000ffffffffffffffffffff0000ffffffff0000ffffff +ffff0000ffffff0000ffffffffffffff0000ffff0000ffffffffff0000ff0000ffffffff +0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffff +ffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff000000ffffff0000ffffffffffffffffffffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffffffffff +ffffff00ffffffffff000000ffffffffffff0000ffffffffffffff0000ffffffffffffff +ff0000ffffffffffffff0000ffffffffff0000ffffff0000ffffffffffff00ffffff0000 +ffffffff0000ffffffff0000ffffffff0000ffffffffffffff0000ffff0000ffffffffff +00000000ffffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffff +ff0000ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffff0000ffffffffffffffffffffffff +ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ff000000ffffffffffffff0000ffffffffffff000000ffffffffff000000ffffffffffff +0000ffffffffffffffff0000ffffffffffffff000000ffffffff0000ffffff000000ffff +ffff0000ffffff0000ffffffff0000ffffffff0000ffffffff0000ffffffffffffff0000 +ffff0000ffffffffffff000000ffffffffff000000ffffffffffffffffff0000ffffffff +ffffff0000ffffffffff0000ffffffffffffffff0000ffffffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffff0000ffffffffffff0000ffffffffffffff00000000ffffffff +ff0000ffffffffffff0000ff00ffffffffffff0000ffffffffffffffff000000ffff0000 +000000ffff00000000000000ffffffff0000ffffffff0000ff00ffff0000ffff00000000 +ffffffffffff0000ffffff0000ffffffffffff0000ffffffffffffff0000ffffffffffff +ffffff0000ffffffffffffff000000ffffffff0000ff00ffffffffff0000ffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000000000ffffffffffffffff0000ffffffff000000000000ffffffff +00000000000000ffffff0000ffffffffffffff000000ffffffffffff0000ffffffffffff +ffffff0000000000000000ffffffff0000000000ffffffff0000000000ffffff000000ff +ffff00000000ff00000000ffffffff0000ffffff0000ffffffffffffff00ffffffffffff +ff0000ffffffffffffff00000000000000ffffffff0000ffffffffffff000000ffffffff +ff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffff00ffffffffffffffffffffffff0000ffffffffffff00 +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffff +ff0000ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ff00000000ffffffff0000ff0000 +00ff000000000000ffffffffff000000000000ffff000000ffffffff000000000000ff00 +00000000ffff000000ffffffffff0000000000ffffffff00000000000000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffff +ffffffffffffffffffff00ffffffffffffffffffff00ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffff00 +00ffffff000000ffff000000ffffff0000ffffff0000ffffffff0000ffffff0000ffffff +0000ffffff0000ffff0000ffffffffff0000ffffffff0000ffff000000ffffffff0000ff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff00ffffffffffffffffffffffffff0000ffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff0000ffffffff000000ffff0000ffffffff0000000000000000ffff00ffffffffff00 +00ffffff0000ffffff00ffffffffff00ffff0000ffffffffff0000ffffff0000ffffffff +000000ffffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffff00000000ffffffffffffffff +ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff0000ffffffffff0000ffff0000ffffff0000ffffffffffffff +ff0000ffffffffff0000ffffff0000ffff0000ffffffffffffffff0000ffffffffff0000 +ffffff0000ffffffffff0000ffffff0000ffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ffff0000ffffff +0000ffffffffffffffff0000ffffffffff0000ffffff0000ffff0000ffffffffffffffff +0000ffffffffff0000ffffff0000ffffffffff0000ffffff0000ffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff +0000ffff0000ffffff000000ffffffffffffff0000ffffffffff0000ffffff0000ffff00 +0000ffffffffffffff0000ffffffffff0000ffffff0000ffffffffff0000ffffff0000ff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff0000ffffffffff00ffffff0000ffffffff0000ffffffffff00ff000000ffffffff00 +00ffffff0000ffff00000000ffffff0000ff0000ffffffffff0000ffffff0000ffffffff +ff00ffffffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff000000ffffff0000ffffff0000ffffffff00000000000000ff +ffff000000ffffff000000ffff0000ffffff00000000000000ffff0000ffffffffff0000 +ffffffff0000ffffff0000ffffffff0000ffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff000000000000ffffffff0000000000ff +ffff0000000000ffffffffff00000000ff000000ff00000000ffffff0000000000ffffff +ff000000ffff00000000ffffffff00000000ffffffffff00000000ff00000000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffffffffffffff +ffff000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffff0000ffffff00ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffff0000ffff0000 +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff000000000000ffffff00000000ff +ffffff0000ff00000000ffffff00000000ffffffffffffffffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffff0000 +00ffff00ffffff0000ffffff0000ffff0000ffffff0000ffff0000ffffffffffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff0000ffffffff0000ffff000000000000ffffff0000ffff0000ffffff00ffffff0000 +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff0000ffffffff0000ff0000ffffffffffffffff0000ffff0000 +ffffffffff00000000ffffffffffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000ffffffff0000ff0000ffffffffff +ffffff0000ffff0000ffffffff00ffff0000ffffffffffffffffff00ff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffff00 +00ff0000ffffffffff00ffff0000ffff0000ffffff0000ffff0000ffffffffffffffffff +00ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffff0000ffff0000ffffff00ffffff0000ffff0000ffff000000ff000000 +ffffffffffffffffff00ff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff000000000000ffffff00000000ffffff000000ffff0000 +0000ff00000000000000ffffffffffffffff00ff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000 +ffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff000000000000000000000000000000000000000000000000000000000000000000 +0000000000ffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff00000000ffffffffffffffffffffffffffffffffffff +ffffffffffffffff00000000ffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00000000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff00000000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff000000000000000000ffffffffffffffff0000ff0000000000000000ff +ffff0000000000ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffff +ffffffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffff00000000ffffffffff0000ffff +ffff00000000ffffffffffff0000ffffffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff00ffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffff +ffffff00000000ffffffffffffffffffffffffffffffffffffffff00ffffffffffffff00 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000 +00ffffffff00ffffffffffff00000000ffffffffff00ffffffffffffffffff00ffffffff +ffff00ffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff00ffffffffff0000ffffffffffffffffffffffffff +ffffffffff00ffffffffffffffff0000ffffffffffffffffffffffffffffff00ffffffff +ff00ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +0000ffffffffffff0000ffffff00ffffffffffffffff000000ffffffff0000ffffffffff +ffffff00ffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff0000ffffffff +ffffffffffffffffffffffffff00ffffffffffffffffff0000ffffffffffffffffffffff +ffffff0000ffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff0000ffffffffffff0000ffff0000ffffffffffffffffff000000ff +ff0000ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffff0000ffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffff +ffffffffffffffffffffffff0000ffffffffffff0000ffffffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffff0000ffffffff +ffffffffffff0000ffff00ffffffffffffffffff0000ffffffff000000000000ffffffff +ffffffffffffffffffff000000000000ffffffff0000000000ffff00000000ffff000000 +00ffffffffffff0000ffffff0000ffffff0000000000ffff000000ffffff0000ffffffff +ffffffffff0000ffffffffffffff0000ffffff000000000000ffffffff0000ffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000 +00ff0000ffffffffffffffffffffff00000000ffffffffffffffffff0000ffffffffffff +ff0000ffffffffffffffffffffffffffffffffffff0000ffffffff000000ffff000000ff +ffff0000ffffffff0000ffffffffffff000000ffff0000ffffffff0000ffffffffff00ff +ffff0000ffffffffffffffffffff0000ffffffffffffff000000ffffffff0000ffffffff +ffff000000ffffffffff000000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +0000ffffffff000000ffff0000ffffffffffffffffffffffff000000ffffffffffffffff +ff0000ffffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffff +000000ffffff0000ffffff0000ffffffff0000ffffffffffffff0000ffff0000ffffffff +0000ffffffffff00ffffff0000ffffffffffffffffffff0000ffffffffffffff0000ffff +ffffff0000ffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff00000000000000ffffffff0000ffffffffffffffffffffffff0000 +0000ffffffffffffffff0000ffffffffffffff0000ffffffffffffffffffffffffffffff +ffffff0000ffffffffff00ffffffff0000ffffff0000ffffffff0000ffffffffffffff00 +00ffff0000ffffffff000000ffffff0000ffffff0000ffffffffffffffffffff0000ffff +ffffffffffffffffffffffff0000ffffffffffffff0000ffffffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffffffff +ffffffffffff0000000000ffffffffffffffff0000ffffffffffffff0000ffff00000000 +000000000000000000ffffff0000ffffffffffffffff00000000ffffff0000ffffffff00 +00ffffffffffffff0000ffff0000ffffffffff0000ffffff00ffffffff0000ffffffffff +ffffffffff0000ffffffffffffffffffffffffffff0000ffffffffffffff0000ffffffff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffff0000ffffffffffffffffffffff00ffff000000ffffffffffffff0000ffffffffffff +ff0000ffff00000000000000000000000000ffffff0000ffffffffffff0000ffff0000ff +ffff0000ffffffff0000ffffffffffffff0000ffff0000ffffffffff000000ff0000ffff +ffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffff0000ffffffff +ffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000ffffffffffffffffffffffffffffffffffffffffffff +0000ffffffffffffffffff0000ffffffffffffffffffff00ffffffff000000ffffffffff +ff0000ffffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffff +ff0000ffffff0000ffffff0000ffffffff0000ffffffffffffff0000ffff0000ffffffff +ffff0000ff0000ffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffff +ffffff0000ffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffff00ffffffff +ff000000ffffffffffff0000ffffffffffffff0000ffffffffffffffffffffffffffffff +ffffff0000ffffffff0000ffffffff0000ffffff0000ffffffff0000ffffffffffffff00 +00ffff0000ffffffffffff00000000ffffffffff0000ffffffffffffffffffff0000ffff +ffffffffffffffffffffffff0000ffffffffffffff0000ffffffffffff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff000000ffffffff +ffffff0000ffffffffffff000000ffffffffff000000ffffffffffff0000ffffffffffff +ffffffffffffffffffffffff0000ffffffff0000ffffffff0000ffffff0000ffffffff00 +00ffffffffffffff0000ffff0000ffffffffffffff000000ffffffffff000000ffffffff +ffffffffff0000ffffffffffffff0000ffffffffff0000ffffffffffffff0000ffffffff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffffff0000ffffffffffff0000ffffffffffffff00000000ffffffffff0000ffffffffff +ff0000ff00ffffffffffffffffffffffffffffffff0000ff00ffff0000ffff00000000ff +ffff000000ff0000000000ffffffffff0000ffffff0000ffffffffffffff0000ffffffff +ffffff0000ffffffffffffffffff0000ffffffffffffff000000ffffffff0000ff00ffff +ffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +00000000ffffffffffffffff0000ffffffff000000000000ffffffff00000000000000ff +ffff0000ffffffffffffff000000ffffffffffffffffffffffffffffffffff000000ffff +ff00000000ff00000000ffff00000000ff000000ffffffffff0000ffffff0000ffffffff +ffffffff00ffffffffffffff0000ffffffffffffff00000000000000ffffffff0000ffff +ffffffff000000ffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ff +ffffffffffffffffffffffff0000ffffffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff00000000ff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff000000ffffffffffffffffffffffffffff000000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffff +ffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff00ffffffffffffffffffffffffffff00ffffffffffffffffffff00ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffff +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00ffff0000ff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff00ffffffffffffffffffffffffffff0000ffffffffff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ff +ffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffff00 +000000ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff0000ffffffffffffffffffffffffffffff0000ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff000000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffffff00ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffff0000ffff +ffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00000000000000ff000000ffff000000 +0000ff000000ffffff000000000000ffffffffff000000000000ffffffffffffffffffff +ffffffffff0000ffffffff0000000000ffffffffff00000000000000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffff0000ffffffffff0000ff00ffffffff0000ffffff0000ffffff0000ffffffff0000ff +ffffffffffffffffffffffffffff0000ffffff0000ffffff0000ffffff00ffffff000000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffff0000ffffffffff000000ffffffffff0000000000000000ffff +00ffffffffff0000ffffffffffffffffffffffffffffff0000ffffff00ffffffff0000ff +ffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffff0000ffffffffffff0000ffffffff0000 +ffffffffffffffff0000ffffffffff0000ffff00000000000000000000ffffff0000ffff +ffffffff00000000ffffff0000ffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ffffffffff +ff0000ffffffff0000ffffffffffffffff0000ffffffffff0000ffffffffffffffffffff +ffffffffff0000ffffffff0000ffff0000ffffff0000ffff0000ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffff0000ffffffffff00ff0000ffffff000000ffffffffffffff0000ffffffffff0000ff +ffffffffffffffffffffffffffff0000ffffff0000ffffff0000ffffffff00000000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffff0000ffffffffff00ffff0000ffffff0000ffffffffff00ff00 +0000ffffffff0000ffffffffffffffffffffffffffffff0000ffffff0000ffffff0000ff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffff0000ffffffff00ffffff000000ffff00 +000000000000ffffff000000ffffff000000ffffffffffffffffffffffffffff0000ffff +ff0000ffff000000ffffff00000000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000000000ffff00000000ff000000 +00ffff00000000ffff0000000000ffffffffff00000000ff000000ffffffffffffffffff +ffffffff0000000000ff00000000ff000000ffff00000000000000ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffff00000000 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +0000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff00ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffff00ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff000000000000ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +0000ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00ffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +0000ffffffffffffffffffffff00ffffffff0000ffffffffffffffffffffffff00ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ff +ffff000000ff00000000ff00000000ffffffffffff0000000000ffffffffff0000000000 +ffffff0000000000ff0000ff00000000ffffffff000000ffffff00000000000000ffffff +ffff00000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffffffffffffffffffffffff00000000ffffff00000000ffffff0000ffff000000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff00ffffff0000ffffff0000ffffff0000ffffffff0000ffffffff0000ffff000000 +ffffff0000ffff000000ffffff0000ffffff000000ffffff0000ffffffff0000ffffffff +0000ffffff0000ffffff00ffffff000000ffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffff0000ffffff0000ffff0000ffff +0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffff0000ffffff0000ffffffff0000ffff +ff0000ffffffff000000ff0000ffffffff000000ffff0000ffffff0000ffffffff0000ff +ffffff0000ffffffff0000ffffff0000ffffff0000ffffff0000ffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffff0000ffff +ff00ffffff0000ffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff000000ffffffffff0000ffffff00 +00ffffffff0000ffffff0000ffffffffff0000ff0000ffffffffff0000ffff0000ffffff +0000ffffffff0000ffffffff0000ffffffff0000ffffff0000ffffff0000ffffff0000ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffff0000ffffffffff00000000ffff0000ffffff0000ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ff +ffffff0000ffffff0000ffffffff0000ffffff0000ffffffffff0000ff0000ffffffffff +0000ffff0000ffffff0000ffffffff0000ffffffff0000ffffffff0000ffffff0000ffff +ff0000ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff0000ffffffffff +ffffffffffffffffffffffffffff0000ffffffff00ffff0000ffff0000ffffff0000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000ffffff0000ffffff0000ffffffff0000ffffff0000ffffffffff00 +00ff0000ffffffffff0000ffff0000ffffff0000ffffffff0000ffffffff0000ffffffff +0000ffffff0000ffffffff00000000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ff0000ffffffffffffffffffffffffffffffffffffff0000ffffff0000ffff0000ffff +0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff00ffffffff00ffffff0000ffffff0000ffffffff0000ffff +ff0000ffffffffff00ffff0000ffffffffff00ffffff0000ffffff0000ffffffff0000ff +ffffff0000ffffffff0000ffffff0000ffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff00ff0000ffffffffffffffffffffffffffffffffffffff0000ffff +000000ff000000ffff0000ffff000000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff00ffffffff00ffffff0000ffffff00 +00ffffffff0000ffffffff0000ffffff0000ffffff0000ffffff0000ffffff0000ffffff +0000ffffffff0000ffffffff0000ffffffff0000ffffff0000ffffff00000000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff00ff0000ffffffffffffffffffffffffffff +ffffffffff00000000ff00000000000000ffff000000ff0000ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000ff +ff0000000000ff0000000000ff00000000ffffffff00000000ffffffffffff00000000ff +ffffffffff0000000000000000ff00000000ffff00000000ffff00000000ff00000000ff +ff00000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff00ffff0000000000ffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffff00000000ffffffffffffffffffffffffffffffffff +ffffffffffffffffff00000000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffff0000000000000000000000000000 +000000000000000000000000000000000000000000000000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffff000000 +00ffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000000000 +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff00ffffff0000ffffffff00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00000000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff00000000ffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff0000000000ffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff000000000000000000ff +ffffffffffffff0000ff0000000000000000ffffff0000000000ffffffffffffff0000ff +ffffffffffffff00ffffffffffffff0000ffffffffffffffffffffffffffffffffffffff +0000ffffffffffff0000ffffffffffffffffffffffff0000000000000000000000000000 +ff00ffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffff00000000ffffffffff0000ffffffff00000000ffffffffffff0000ffffff +ffffffffff0000ffffffffffffffffffff00ffffffffffff0000ffffffffffffffffffff +ffffffffffffffff0000ffffffffff00000000ffffffffffffffffffffffff0000ffffff +ff0000ffffffff0000ffff00ffffffffffffff00ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000ff0000ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff000000ffffffffffff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff000000ffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffff000000ffffffff00ffffffffffff00000000ff +ffffffff00ffffffffffffffffff00ffffffffffff00ffffffffff00ffffffffff0000ff +ffffffffffffffffffffffffffffffffff00ffffffffffffffff0000ffffffffffffffff +ffffffff0000ffffffff0000ffffffffff00ffffff00ffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ffff0000ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ffffff00ffffff +ffffffffff000000ffffffff0000ffffffffffffffff00ffffffffffff0000ffffffffff +0000ffffffff0000ffffffffffffffffffffffffffffffffff00ffffffffffffffffff00 +00ffffffffffffffffffffffff00ffffffffff0000ffffffffff00ffffff0000ffffffff +ffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +00ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +0000ffff0000ffffffffffffffffff000000ffff0000ffffffffffffffff0000ffffffff +ffff0000ffffffffffff0000ffffff0000ffffffffffffffffffffffffffffffff0000ff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffffffffff0000ffff0000ffffffffffffffffffff0000ffff00ffffffffffff +ffffff0000ffffffff000000000000ffffffff0000ffffff0000ffffff0000000000ffff +000000ffffff0000ffffffffffffffffff0000ffffffffffffff0000ffffffffffffffff +ff0000ffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffff00ffffffff0000ffffffffffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffff000000ff0000ffffffffffffffffffffff0000 +0000ffffffffffffffffff0000ffffffffffffff0000ffffffffffff000000ffff0000ff +ffffff0000ffffffffff00ffffff0000ffffffffffffffffffff0000ffffffffffffff00 +0000ffffffffffffffff0000ffffffffffffffffffff000000ffffffffff000000ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000 +00000000ff000000ffff0000000000ff000000ffffff000000000000ffffffffff000000 +000000ffffffffffffff000000ffffff00000000000000ffffff0000000000ffffff0000 +00000000ffff0000ff0000000000000000ffff000000ffff0000000000ffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffff000000ffff0000ffffffff +ffffffffffffffff000000ffffffffffffffffff0000ffffffffffffff0000ffffffffff +ffff0000ffff0000ffffffff0000ffffffffff00ffffff0000ffffffffffffffffffff00 +00ffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffff0000ffff +ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffff0000ffffffffff0000ff00ffffffff0000ffffff00 +00ffffff0000ffffffff0000ffffffffffffffff0000ffffffff0000ffffff0000ffffff +0000ffffffff0000ffffff0000ffff000000ffff00ffff0000ffffffff00ffff0000ffff +ff0000ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +00ffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000ff +ffffff0000ffffffffffffffffffffffff00000000ffffffffffffffff0000ffffffffff +ffff0000ffffffffffffff0000ffff0000ffffffff000000ffffff0000ffffff0000ffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffff0000ffffffffff000000ffff +ffffff0000000000000000ffff00ffffffffff0000ffffffffffffffff0000ffffffff00 +00ffffff0000ffffff0000ffffffff0000000000000000ff0000ffffffffffff0000ffff +ff0000ffff00ffffffff0000ffffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffffff0000ffffffffffffffffffffff0000000000ffffffffffff +ffff0000ffffffffffffff0000ffffffffffffff0000ffff0000ffffffffff0000ffffff +00ffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ff +ffffffffff0000ffffffff0000ffffffffffffffff0000ffffffffff0000ffffffffffff +ffff0000ffffffff0000ffffff0000ffffff0000ffffff0000ffffffffffffffff0000ff +ffffffffffff0000ffff00ffffffffffff00000000ffffffff0000ffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffff0000ffffffffffffffffffffff00ff +ff000000ffffffffffffff0000ffffffffffffff0000ffffffffffffff0000ffff0000ff +ffffffff000000ff0000ffffffff0000ffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffff0000ffffffffffff0000ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffff0000ffffffffffff0000ffffffff0000ffffffffffffffff0000ffffffff +ff0000ffffffffffffffff0000ffffffff0000ffffff0000ffffff0000ffffff0000ffff +ffffffffffff0000ffffffffffffff0000ff0000ffffffff0000ffff0000ffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffff0000ffffffff +ffffffffffff00ffffffff000000ffffffffffff0000ffffffffffffff0000ffffffffff +ffff0000ffff0000ffffffffffff0000ff0000ffffffff0000ffffffffffffffffffff00 +00ffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffff0000ffff +ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffff0000ffffffffff00ff0000ffffff000000ffffffff +ffffff0000ffffffffff0000ffffffffffffffff0000ffffffff0000ffffff0000ffffff +0000ffffff000000ffffffffffffff0000ffffffffffffffff000000ffffffff0000ffff +ff0000ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +00ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffff0000ffffffffffffffffff00ffffffffff000000ffffffffffff0000ffffffffff +ffff0000ffffffffffffff0000ffff0000ffffffffffff00000000ffffffffff0000ffff +ffffffffffffffff0000ffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffff0000ffffffffff00ffff0000 +ffffff0000ffffffffff00ff000000ffffffff0000ffffffffffffffff0000ffffffff00 +00ffffff0000ffffff0000ffffffff0000ffffffffff00ff0000ffffffffffffffff0000 +00ffffffff0000ffffff0000ffffffff0000ffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffffff000000ffffffffffffff0000ffffffffffff000000ffffff +ffff000000ffffffffffff0000ffffffffffffff0000ffff0000ffffffffffffff000000 +ffffffffff000000ffffffffffffffffff0000ffffffffffffff0000ffffffffffffffff +ff0000ffffffffffffffffffffff0000ffffffffffff0000ffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ff +ffffff00ffffff000000ffff00000000000000ffffff000000ffffff000000ffffffffff +ffff0000ffffffff0000ffffff0000ffffff0000ffffffff00000000000000ffff0000ff +ffffffffffffff000000ffffffff0000ffff000000ffffffff0000ffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffff0000ffffffffffff0000ffffffff +ffffff00000000ffffffffff0000ffffffffffff0000ff00ffffffff0000ffffff0000ff +ffffffffffff0000ffffffffffffff0000ffffffffffffffffff0000ffffffffffffff00 +0000ffffffffffffffff000000ffffffffffffffffff0000ffffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +000000ffff00000000ff00000000ffff00000000ffff0000000000ffffffffff00000000 +ff000000ffffffffffff00000000ffff00000000ff00000000ffffff000000ffffff0000 +000000ffff0000000000ffffffffffffff00ffffffffff00000000ff000000ffff000000 +0000ffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffff +ffffffffffffffffffffffffffffffffff000000000000ffffffffffffffff0000ffffff +ff000000000000ffffffff00000000000000ffffff0000ffffffffffffff000000ffffff +ff0000ffffff0000ffffffffffffffff00ffffffffffffff0000ffffffffffffff000000 +00000000ffffffff0000ffffffffffffff00000000000000ffffffffffffff0000ffffff +ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffffffffffffff0000ffffffffffffffffffffffffff0000ffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffff0000ffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffff +ffffffffff00ffffffffffffffffffffffff00ffffffffffffffffffffffffffff00ffff +ffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffff00ffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff +ffffffffffffffffffffffffffff0000ffffffffffffffffffff00ffffffffffffffffff +ffffffffff0000ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffff00ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffff007f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f00ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff0000 +ffffffffffffffffffffff00000000ffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff007f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f +00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff000000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffff000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffff0000ffffffffffffffffffff0000ffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffff0000ffffffffffffffffffffff00ffffffff0000ffffff +ffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff00000000ffffff000000ff00000000ff00000000ffffffffffff0000 +000000ffffffffff0000000000ffffff0000000000ff0000ff00000000ffffffff000000 +ffffff00000000000000ffffffffff00000000000000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff00ffffff0000ffffff0000ffffff0000ffffffff +0000ffffffff0000ffff000000ffffff0000ffff000000ffffff0000ffffff000000ffff +ff0000ffffffff0000ffffffff0000ffffff0000ffffff00ffffff000000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff0000 +ffffff0000ffffffff0000ffffff0000ffffffff000000ff0000ffffffff000000ffff00 +00ffffff0000ffffffff0000ffffffff0000ffffffff0000ffffff0000ffffff0000ffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +0000ffffffffff0000ffffff0000ffffffff0000ffffff0000ffffffffff0000ff0000ff +ffffffff0000ffff0000ffffff0000ffffffff0000ffffffff0000ffffffff0000ffffff +0000ffffff0000ffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff000000ffffffff0000ffffff0000ffffffff0000ffffff0000ffff +ffffff0000ff0000ffffffffff0000ffff0000ffffff0000ffffffff0000ffffffff0000 +ffffffff0000ffffff0000ffffff0000ffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff00ff0000ffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff000000ffffff0000ffffff0000ffffffff +0000ffffff0000ffffffffff0000ff0000ffffffffff0000ffff0000ffffff0000ffffff +ff0000ffffffff0000ffffffff0000ffffff0000ffffffff00000000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff0000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffff00ffffff0000 +ffffff0000ffffffff0000ffffff0000ffffffffff00ffff0000ffffffffff00ffffff00 +00ffffff0000ffffffff0000ffffffff0000ffffffff0000ffffff0000ffffff0000ffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +00ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffff00ffffff0000ffffff0000ffffffff0000ffffffff0000ffffff0000ffffff0000 +ffffff0000ffffff0000ffffff0000ffffffff0000ffffffff0000ffffffff0000ffffff +0000ffffff00000000ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffff00ff0000ffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff0000000000ffff0000000000ff0000000000ff00000000ffffffff0000 +0000ffffffffffff00000000ffffffffffff0000000000000000ff00000000ffff000000 +00ffff00000000ff00000000ffff00000000000000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffff00ffff0000000000ffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffff +ffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffff0000ffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffff000000000000ffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffff00000000ffffff0000 +0000000000ffffffff0000ffffffffffffffffffffffffffffffffffffffffffffff00ff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff00 +ffff0000ffff00ffff0000ff0000ffffffff0000ffffffffffffffffffffffffffffffff +ffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff +ffffffffffffff0000ffffffffff0000ffffffffff0000ffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00 +00ffffffffffffffffffffffffffffff0000ffffffffff0000ffffffffff0000ffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff0000ffffffffffffffffffffffffffffff0000ffffffffff0000ffff +ffffff0000ffffffffff00ffffffff0000ffffffffffffffffffffffffffffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff000000ffffffffffff0000000000ffff00000000 +00000000000000000000ffff0000ffffff000000ffffff00000000000000ffffffffff00 +0000000000ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffff0000ff +ff000000ffffff0000ffffffffff0000ffffffffff0000ffffffff0000ffffffff0000ff +ffff0000ffffff0000ffffff0000ffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffff0000ffffffff000000ffff0000ffffffffff0000ffffffffff0000ffffffff +0000ffffffff0000ffffff0000ffffff0000000000000000ffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffff0000ffffffffff0000ffffffffff0000ffff0000ffffffffff0000ffff +ffffff0000ffffffff0000ffffffff0000ffffff0000ffff0000ffffffffffffffffffff +ff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffff0000ffffffffff0000ffffffffff0000ffff0000 +ffffffffff0000ffffffffff0000ffffffff0000ffffffff0000ffffff0000ffff0000ff +ffffffffffffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffff0000ffff +ffffff0000ffff0000ffffffffff0000ffffffffff0000ffffffff0000ffffffff0000ff +ffff0000ffff000000ffffffffffffffffffff0000ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffff0000ffffffffff00ffffff0000ffffffffff0000ffffffffff0000ffffffff +0000ffffffff0000ffffff0000ffffff0000ffffffffff00ffffffff0000ffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffff00ffffffffffff0000ffffff0000ffffff0000ffffffffff0000ffff +ffffff0000ffffffff0000ffffffff0000ffffff0000ffffff00000000000000ffffffff +ff00ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffff0000ffffffffffff00000000ffffffff000000 +0000ffff0000000000ffff0000000000ff00000000ffff00000000ff00000000ffffff00 +00000000ffffffffff0000ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffff00ffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffff0000ffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff00ffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000 +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +ffffffffffffffffffff +end +%%PageTrailer +%%Trailer +%%EOF diff --git a/sourcecodes/bnt-master/docs/Figures/filter.fig b/sourcecodes/bnt-master/docs/Figures/filter.fig new file mode 100644 index 00000000..4e125ccf --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/filter.fig @@ -0,0 +1,74 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 2550 5100 6300 6375 +2 2 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 5 + 2550 5475 4050 5475 4050 5775 2550 5775 2550 5475 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 2 + 4050 5475 6300 5475 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3300 6375 3300 5925 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 3450 6225 4050 6225 +4 0 0 100 0 0 18 0.0000 4 165 90 4050 5325 t\001 +4 0 0 100 0 0 18 0.0000 4 165 345 3600 6075 tau\001 +-6 +2 2 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 5 + 2550 2025 4050 2025 4050 2325 2550 2325 2550 2025 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 2 + 4050 2025 6300 2025 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4050 2925 4050 2475 +2 2 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 5 + 2550 7200 6375 7200 6375 7500 2550 7500 2550 7200 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3975 8025 3975 7575 +2 2 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 5 + 2475 3525 3975 3525 3975 3825 2475 3825 2475 3525 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 2 + 3975 3525 6225 3525 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 3975 4275 4575 4275 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4650 4425 4650 3975 +2 2 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 5 + 2550 600 4050 600 4050 900 2550 900 2550 600 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 0 0 2 + 4050 600 6300 600 +2 1 0 1 0 0 100 0 10 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4050 1500 4050 1050 +4 0 0 100 0 0 18 0.0000 4 165 90 4050 1875 t\001 +4 0 0 50 0 0 24 0.0000 4 330 3540 6675 2175 argmax P(x(1:t) | y(1:t))\001 +4 0 0 50 0 0 24 0.0000 4 330 810 6975 2550 x(1:t)\001 +4 0 0 100 0 0 18 0.0000 4 165 90 3975 7125 t\001 +4 0 0 100 0 0 18 0.0000 4 195 180 6225 7125 T\001 +4 0 0 100 0 0 18 0.0000 4 165 90 3975 3375 t\001 +4 0 0 100 0 0 18 0.0000 4 195 540 3975 4125 delta\001 +4 0 0 100 0 0 18 0.0000 4 165 90 4050 450 t\001 +4 0 0 50 0 0 24 0.0000 4 330 1950 6675 750 P(X(t)|y(1:t))\001 +4 0 0 50 0 0 24 0.0000 4 330 2865 6450 3600 P(X(t+delta)|y(1:t))\001 +4 0 0 50 0 0 24 0.0000 4 330 2550 6600 5550 P(X(t-tau)|y(1:t))\001 +4 0 0 50 0 0 24 0.0000 4 330 2085 6750 7425 P(X(t)|y(1:T))\001 +4 0 0 50 0 0 24 0.0000 4 330 1140 600 900 filtering\001 +4 0 0 50 0 0 24 0.0000 4 330 1470 450 3750 prediction\001 +4 0 0 50 0 0 24 0.0000 4 255 1005 600 2175 Viterbi\001 +4 0 0 50 0 0 24 0.0000 4 330 1305 600 5775 fixed-lag\001 +4 0 0 50 0 0 24 0.0000 4 330 1575 525 7740 smoothing\001 +4 0 0 50 0 0 24 0.0000 4 330 1170 525 8055 (offline)\001 +4 0 0 50 0 0 24 0.0000 4 330 1575 600 6090 smoothing\001 +4 0 0 50 0 0 24 0.0000 4 255 1920 525 7425 fixed interval\001 diff --git a/sourcecodes/bnt-master/docs/Figures/filter.gif b/sourcecodes/bnt-master/docs/Figures/filter.gif new file mode 100644 index 00000000..bda3de61 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/filter.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/filter.tex b/sourcecodes/bnt-master/docs/Figures/filter.tex new file mode 100644 index 00000000..871f8dfd --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/filter.tex @@ -0,0 +1,51 @@ +%Latex +\documentstyle[fleqn,psfig,12pt,pstricks,pst-node,pst-tree]{article} + +%\documentclass[fleqn,12pt]{article} +%\usepackage{pstricks,pst-node,pst-tree} + +%latex2e +%\documentclass{article} +%\usepackage{epsfig,alltt,fancybox} + +\setlength{\textwidth}{6.5in} +\setlength{\oddsidemargin}{0in} +\setlength{\textheight}{8.5in} +\setlength{\headheight}{0in} +\setlength{\headsep}{-0.5in} +\setlength{\parindent}{0in} % block style +\setlength{\parskip}{0.3cm} + +\newcommand{\mytitle}[1]{\newpage \huge \begin{center}#1\vspace{0.8cm}\end{center} \LARGE} + + +\begin{document} + +\mytitle{Hello world} + +Hello world + +\centering +$ +\pstree[treemode=R]{\Tcircle{b}}{% + \pstree{\TC*^{a_1}}{% + \Tr{b_{11}}^{x_1} + \Tr{b_{12}}_{x_2} + } + \pstree{\TC*_{a_2}}{% + \Tr{b_{21}}^{x_1} + \Tr{b_{22}}_{x_2} + } +} +$ + + + +\mytitle{Hello world 2} + +\psline[linewidth=0.5cm](0,0)(2,0) +\psline[linewidth=0.05cm](2,-1)(6,-1) + + + +\end{document} diff --git a/sourcecodes/bnt-master/docs/Figures/gaussplot.png b/sourcecodes/bnt-master/docs/Figures/gaussplot.png new file mode 100644 index 00000000..29cb66ec --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/gaussplot.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hme.fig b/sourcecodes/bnt-master/docs/Figures/hme.fig new file mode 100644 index 00000000..dffda256 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hme.fig @@ -0,0 +1,35 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 225 150 2625 3825 +6 300 150 1800 3450 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 2212.500 2250.000 825 1425 600 2325 825 3075 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 3083.468 1905.242 750 525 375 2025 675 3150 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 -1 0 0 -1 0.000 0 0 1 0 679.747 1409.810 1200 450 1725 1725 1275 2325 + 0 0 1.00 60.00 120.00 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 975 375 300 225 975 375 1275 600 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1050 3225 300 225 1050 3225 1350 3450 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 825 1050 1200 1050 1200 1500 825 1500 825 1050 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 825 2100 1200 2100 1200 2550 825 2550 825 2100 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 600 1050 1050 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 1575 1050 2100 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 2625 1050 3000 +-6 +4 0 -1 0 0 0 12 0.0000 4 180 2370 225 3750 Hierarchical Mixture of Experts\001 +4 0 -1 0 0 0 12 0.0000 4 135 120 825 450 X\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 900 1350 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 900 2400 Q2\001 +4 0 -1 0 0 0 12 0.0000 4 135 135 900 3300 Y\001 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/hme.gif b/sourcecodes/bnt-master/docs/Figures/hme.gif new file mode 100644 index 00000000..85150fbb --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hme.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hme_dec_boundary.gif b/sourcecodes/bnt-master/docs/Figures/hme_dec_boundary.gif new file mode 100644 index 00000000..15cd6b1d --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hme_dec_boundary.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hme_dec_boundary.png b/sourcecodes/bnt-master/docs/Figures/hme_dec_boundary.png new file mode 100644 index 00000000..a12cb885 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hme_dec_boundary.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm3.fig b/sourcecodes/bnt-master/docs/Figures/hmm3.fig new file mode 100644 index 00000000..61578c4e --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm3.fig @@ -0,0 +1,39 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 525 1650 270 270 525 1650 675 1875 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1575 1650 270 270 1575 1650 1725 1875 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 2625 1650 270 270 2625 1650 2775 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 375 600 750 600 750 1050 375 1050 375 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 1050 525 1425 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 825 1350 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1425 600 1800 600 1800 1050 1425 1050 1425 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 1050 1575 1425 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1800 825 2325 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2400 600 2775 600 2775 1050 2400 1050 2400 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2550 1050 2550 1425 +4 0 0 100 0 2 12 0.0000 4 135 90 525 975 1\001 +4 0 0 100 0 2 12 0.0000 4 135 90 1575 975 3\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2475 900 5\001 +4 0 0 100 0 2 12 0.0000 4 135 90 450 1725 2\001 +4 0 0 100 0 2 12 0.0000 4 135 90 1425 1725 4\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2550 1725 6\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm3.gif b/sourcecodes/bnt-master/docs/Figures/hmm3.gif new file mode 100644 index 00000000..05a6f592 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm3.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm3letter.fig b/sourcecodes/bnt-master/docs/Figures/hmm3letter.fig new file mode 100644 index 00000000..3a190045 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm3letter.fig @@ -0,0 +1,39 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 525 1650 270 270 525 1650 675 1875 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1575 1650 270 270 1575 1650 1725 1875 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 2625 1650 270 270 2625 1650 2775 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 375 600 750 600 750 1050 375 1050 375 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 1050 525 1425 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 825 1350 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1425 600 1800 600 1800 1050 1425 1050 1425 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 1050 1575 1425 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1800 825 2325 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2400 600 2775 600 2775 1050 2400 1050 2400 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2550 1050 2550 1425 +4 0 0 100 0 0 12 0.0000 4 135 225 450 1725 Y1\001 +4 0 0 100 0 0 12 0.0000 4 135 225 1425 1725 Y2\001 +4 0 0 100 0 0 12 0.0000 4 135 225 2475 1725 Y3\001 +4 0 0 100 0 0 12 0.0000 4 135 225 450 900 X1\001 +4 0 0 100 0 0 12 0.0000 4 135 225 2475 900 X3\001 +4 0 0 100 0 0 12 0.0000 4 135 225 1500 900 X2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm3letter.gif b/sourcecodes/bnt-master/docs/Figures/hmm3letter.gif new file mode 100644 index 00000000..540fb75b --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm3letter.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm3letter.jpg b/sourcecodes/bnt-master/docs/Figures/hmm3letter.jpg new file mode 100644 index 00000000..29695dfd --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm3letter.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm4.fig b/sourcecodes/bnt-master/docs/Figures/hmm4.fig new file mode 100644 index 00000000..92066fb9 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm4.fig @@ -0,0 +1,55 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 600 1200 300 300 600 1200 900 1200 +1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 1200 300 300 1800 1200 2100 1200 +1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3000 1200 300 300 3000 1200 3300 1200 +1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 4200 1200 300 300 4200 1200 4500 1200 +1 3 0 2 0 0 50 0 7 0.000 1 0.0000 600 2400 300 300 600 2400 900 2400 +1 3 0 2 0 0 50 0 7 0.000 1 0.0000 3000 2400 300 300 3000 2400 3300 2400 +1 3 0 2 0 0 50 0 7 0.000 1 0.0000 4200 2400 300 300 4200 2400 4500 2400 +1 3 0 2 0 0 50 0 7 0.000 1 0.0000 1800 2400 300 300 1800 2400 2100 2400 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 600 1500 600 2100 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 900 1200 1500 1200 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 2100 1200 2700 1200 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 3375 1200 3900 1200 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 1800 1500 1800 2100 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 3000 1500 3000 2100 +2 1 0 2 0 0 50 0 7 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 4200 1500 4200 2100 +4 0 0 100 0 0 30 0.0000 4 60 600 5025 1200 . . .\001 +4 0 0 50 0 0 24 0.0000 4 255 270 450 1275 X\001 +4 0 0 50 0 0 24 0.0000 4 255 270 1650 1275 X\001 +4 0 0 50 0 0 24 0.0000 4 255 270 2850 1275 X\001 +4 0 0 50 0 0 24 0.0000 4 255 240 450 2475 Y\001 +4 0 0 50 0 0 24 0.0000 4 255 240 2850 2475 Y\001 +4 0 0 50 0 0 24 0.0000 4 255 240 4050 2475 Y\001 +4 0 0 50 0 0 24 0.0000 4 255 180 675 1425 1\001 +4 0 0 50 0 0 24 0.0000 4 255 180 1875 1350 2\001 +4 0 0 50 0 0 24 0.0000 4 255 180 3075 1350 3\001 +4 0 0 50 0 0 24 0.0000 4 255 180 4275 1425 4\001 +4 0 0 50 0 0 24 0.0000 4 255 180 600 2550 1\001 +4 0 0 50 0 0 24 0.0000 4 255 180 3000 2550 3\001 +4 0 0 50 0 0 24 0.0000 4 255 180 4200 2550 4\001 +4 0 0 50 0 0 24 0.0000 4 255 270 4050 1275 X\001 +4 0 0 50 0 0 24 0.0000 4 255 180 1800 2625 2\001 +4 0 0 50 0 0 24 0.0000 4 255 240 1650 2475 Y\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm4.gif b/sourcecodes/bnt-master/docs/Figures/hmm4.gif new file mode 100644 index 00000000..336bc5d6 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm4.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm4_params.fig b/sourcecodes/bnt-master/docs/Figures/hmm4_params.fig new file mode 100644 index 00000000..b254e1e5 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm4_params.fig @@ -0,0 +1,81 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 3150 2550 270 270 3150 2550 3300 2775 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 3975 2550 270 270 3975 2550 4125 2775 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1350 2550 270 270 1350 2550 1500 2775 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 2250 2550 270 270 2250 2550 2400 2775 +1 3 1 1 0 7 100 0 -1 4.000 1 0.0000 2625 3825 270 270 2625 3825 2775 4050 +1 3 1 1 0 7 100 0 -1 4.000 1 0.0000 1425 600 270 270 1425 600 1575 825 +1 3 1 1 0 7 100 0 -1 4.000 1 0.0000 3000 600 270 270 3000 600 3150 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2925 1500 3300 1500 3300 1950 2925 1950 2925 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 1950 3075 2325 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3750 1500 4125 1500 4125 1950 3750 1950 3750 1500 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3300 1725 3750 1725 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3975 1950 3975 2250 +2 1 0 1 0 0 100 0 20 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 1725 2100 1725 +2 1 0 1 0 0 100 0 20 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2475 1725 2925 1725 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1200 1500 1575 1500 1575 1950 1200 1950 1200 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1350 1950 1350 2325 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2100 1500 2475 1500 2475 1950 2100 1950 2100 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2250 1950 2250 2325 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2475 3600 1500 2775 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2550 3600 2400 2850 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2700 3525 3075 2775 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2850 3675 3825 2775 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1425 900 1425 1500 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2925 900 2400 1500 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 900 3075 1425 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3225 750 3900 1500 +4 0 0 100 0 0 12 0.0000 4 135 225 3000 2625 Y3\001 +4 0 0 100 0 0 12 0.0000 4 165 225 3825 1800 Q4\001 +4 0 0 100 0 0 12 0.0000 4 165 225 3000 1800 Q3\001 +4 0 0 100 0 0 12 0.0000 4 135 225 3825 2625 Y4\001 +4 0 0 100 0 0 12 0.0000 4 135 225 1275 2625 Y1\001 +4 0 0 100 0 0 12 0.0000 4 165 225 1275 1800 Q1\001 +4 0 0 100 0 0 12 0.0000 4 135 225 2100 2625 Y2\001 +4 0 0 100 0 0 12 0.0000 4 165 225 2175 1800 Q2\001 +4 0 0 100 0 0 12 0.0000 4 135 195 1275 675 P1\001 +4 0 0 100 0 0 12 0.0000 4 135 195 2850 675 P2\001 +4 0 0 100 0 0 12 0.0000 4 135 195 2550 3900 P3\001 +4 0 0 100 0 0 30 0.0000 4 30 525 4350 1800 . . .\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm4_params.gif b/sourcecodes/bnt-master/docs/Figures/hmm4_params.gif new file mode 100644 index 00000000..656bc13f --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm4_params.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm4_paramsX.fig b/sourcecodes/bnt-master/docs/Figures/hmm4_paramsX.fig new file mode 100644 index 00000000..5deec7e1 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm4_paramsX.fig @@ -0,0 +1,81 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 3150 2550 270 270 3150 2550 3300 2775 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 3975 2550 270 270 3975 2550 4125 2775 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1350 2550 270 270 1350 2550 1500 2775 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 2250 2550 270 270 2250 2550 2400 2775 +1 3 1 1 0 7 100 0 -1 4.000 1 0.0000 1425 600 270 270 1425 600 1575 825 +1 3 1 1 0 7 100 0 -1 4.000 1 0.0000 3000 600 270 270 3000 600 3150 825 +1 3 1 1 0 7 100 0 -1 4.000 1 0.0000 2615 3863 270 270 2615 3863 2765 4088 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2925 1500 3300 1500 3300 1950 2925 1950 2925 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 1950 3075 2325 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3750 1500 4125 1500 4125 1950 3750 1950 3750 1500 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3300 1725 3750 1725 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3975 1950 3975 2250 +2 1 0 1 0 0 100 0 20 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 1725 2100 1725 +2 1 0 1 0 0 100 0 20 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2475 1725 2925 1725 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1200 1500 1575 1500 1575 1950 1200 1950 1200 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1350 1950 1350 2325 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2100 1500 2475 1500 2475 1950 2100 1950 2100 1500 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2250 1950 2250 2325 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2475 3600 1500 2775 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2550 3600 2400 2850 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2700 3525 3075 2775 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2850 3675 3825 2775 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1425 900 1425 1500 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2925 900 2400 1500 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 900 3075 1425 +2 1 1 1 0 7 100 0 -1 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3225 750 3900 1500 +4 0 0 100 0 0 12 0.0000 4 135 225 3000 2625 Y3\001 +4 0 0 100 0 0 12 0.0000 4 135 225 3825 2625 Y4\001 +4 0 0 100 0 0 12 0.0000 4 135 225 1275 2625 Y1\001 +4 0 0 100 0 0 12 0.0000 4 135 225 2100 2625 Y2\001 +4 0 0 100 0 0 30 0.0000 4 60 600 4350 1800 . . .\001 +4 0 0 50 0 0 12 0.0000 4 135 225 1275 1800 X1\001 +4 0 0 50 0 0 12 0.0000 4 135 225 2175 1800 X2\001 +4 0 0 50 0 0 12 0.0000 4 135 225 3000 1800 X3\001 +4 0 0 50 0 0 12 0.0000 4 135 225 3825 1800 X4\001 +4 0 0 50 0 0 18 0.0000 4 195 180 2550 3975 B\001 +4 0 0 50 0 0 18 0.0000 4 255 210 1275 750 pi\001 +4 0 0 50 0 0 18 0.0000 4 195 225 2850 750 A\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm4_square.fig b/sourcecodes/bnt-master/docs/Figures/hmm4_square.fig new file mode 100644 index 00000000..9f0226f0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm4_square.fig @@ -0,0 +1,55 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +6 1425 600 2025 1950 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 1725 1650 270 270 1725 1650 1875 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1575 600 1950 600 1950 1050 1575 1050 1575 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1725 1050 1725 1425 +4 0 0 100 0 0 12 0.0000 4 135 225 1575 1725 Y2\001 +4 0 0 100 0 0 12 0.0000 4 165 225 1650 900 Q2\001 +-6 +6 525 600 1125 1950 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 825 1650 270 270 825 1650 975 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 600 1050 600 1050 1050 675 1050 675 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1050 825 1425 +4 0 0 100 0 0 12 0.0000 4 135 225 750 1725 Y1\001 +4 0 0 100 0 0 12 0.0000 4 165 225 750 900 Q1\001 +-6 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 2625 1650 270 270 2625 1650 2775 1875 +1 3 0 1 0 0 100 0 2 0.000 1 0.0000 3450 1650 270 270 3450 1650 3600 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2400 600 2775 600 2775 1050 2400 1050 2400 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2550 1050 2550 1425 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3225 600 3600 600 3600 1050 3225 1050 3225 600 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2775 825 3225 825 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 1050 3450 1350 +2 1 0 1 0 0 100 0 20 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 825 1575 825 +2 1 0 1 0 0 100 0 20 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 825 2400 825 +4 0 0 100 0 0 12 0.0000 4 135 225 2475 1725 Y3\001 +4 0 0 100 0 0 12 0.0000 4 165 225 3300 900 Q4\001 +4 0 0 100 0 0 12 0.0000 4 165 225 2475 900 Q3\001 +4 0 0 100 0 0 12 0.0000 4 135 225 3300 1725 Y4\001 +4 0 0 100 0 0 30 0.0000 4 30 525 3825 975 . . .\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_ar.fig b/sourcecodes/bnt-master/docs/Figures/hmm_ar.fig new file mode 100644 index 00000000..8aefe002 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_ar.fig @@ -0,0 +1,30 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 225 225 2025 1875 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 525 1275 300 225 525 1275 825 1500 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1575 1275 300 225 1575 1275 1875 1500 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 450 1350 450 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 375 225 750 225 750 675 375 675 375 225 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 675 525 1050 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1425 225 1800 225 1800 675 1425 675 1425 225 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 675 1575 1050 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1275 1275 1275 +4 0 -1 0 0 0 12 0.0000 4 165 225 450 525 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1500 525 Q2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 375 1350 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1425 1350 Y2\001 +4 0 -1 0 0 0 12 0.0000 4 180 1785 225 1800 Auto Regressive HMM\001 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_ar.gif b/sourcecodes/bnt-master/docs/Figures/hmm_ar.gif new file mode 100644 index 00000000..57d57c4d --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_ar.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_coupled.fig b/sourcecodes/bnt-master/docs/Figures/hmm_coupled.fig new file mode 100644 index 00000000..6eab0030 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_coupled.fig @@ -0,0 +1,65 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +6 300 225 1950 675 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 600 450 300 225 600 450 900 675 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1650 450 300 225 1650 450 1950 675 +-6 +6 450 1875 1875 2325 +6 450 1875 1875 2325 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 2100 1425 2100 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 450 1875 825 1875 825 2325 450 2325 450 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1500 1875 1875 1875 1875 2325 1500 2325 1500 1875 +-6 +-6 +6 450 1050 1875 1500 +6 450 1050 1875 1500 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1275 1425 1275 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 450 1050 825 1050 825 1500 450 1500 450 1050 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1500 1050 1875 1050 1875 1500 1500 1500 1500 1050 +-6 +-6 +6 300 2700 1950 3150 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 600 2925 300 225 600 2925 900 3150 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1650 2925 300 225 1650 2925 1950 3150 +-6 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 600 2325 600 2700 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1650 2325 1650 2700 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 600 1050 600 675 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1650 1050 1650 675 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 2100 1425 1350 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 900 1275 1350 2025 +4 0 -1 0 0 0 12 0.0000 4 135 210 450 525 C1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 525 1350 A1\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 525 2175 B1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 450 3000 D1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1500 3075 D2\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1500 525 C2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1575 1350 A2\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1575 2175 B2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_coupled.gif b/sourcecodes/bnt-master/docs/Figures/hmm_coupled.gif new file mode 100644 index 00000000..1dd2859c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_coupled.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_factorial.fig b/sourcecodes/bnt-master/docs/Figures/hmm_factorial.fig new file mode 100644 index 00000000..94e6010c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_factorial.fig @@ -0,0 +1,50 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 1501.355 1323.343 675 450 300 1275 525 2025 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 2263.600 1307.555 1650 525 1275 1200 1575 2025 + 0 0 1.00 60.00 120.00 +6 675 1050 2100 1500 +6 675 1050 2100 1500 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 1275 1650 1275 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 1050 1050 1050 1050 1500 675 1500 675 1050 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1725 1050 2100 1050 2100 1500 1725 1500 1725 1050 +-6 +-6 +6 675 225 2100 675 +6 675 225 2100 675 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 450 1650 450 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 225 1050 225 1050 675 675 675 675 225 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1725 225 2100 225 2100 675 1725 675 1725 225 +-6 +-6 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 825 2100 300 225 825 2100 1125 2325 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1875 2100 300 225 1875 2100 2175 2325 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1500 825 1875 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1875 1500 1875 1875 +4 0 -1 0 0 0 12 0.0000 4 135 225 675 2175 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1800 2250 Y2\001 +4 0 -1 0 0 0 12 0.0000 4 135 1185 675 2850 Factorial HMM\001 +4 0 0 100 0 0 12 0.0000 4 135 225 750 525 A1\001 +4 0 0 100 0 0 12 0.0000 4 135 225 1800 525 A2\001 +4 0 0 100 0 0 12 0.0000 4 135 210 750 1350 B1\001 +4 0 0 100 0 0 12 0.0000 4 135 210 1800 1350 B2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_factorial.gif b/sourcecodes/bnt-master/docs/Figures/hmm_factorial.gif new file mode 100644 index 00000000..6ee4d113 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_factorial.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_gauss.fig b/sourcecodes/bnt-master/docs/Figures/hmm_gauss.fig new file mode 100644 index 00000000..bb26e360 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_gauss.fig @@ -0,0 +1,27 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 225 600 1875 1875 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 525 1650 300 225 525 1650 825 1875 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1575 1650 300 225 1575 1650 1875 1875 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 825 1350 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 375 600 750 600 750 1050 375 1050 375 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 1050 525 1425 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1425 600 1800 600 1800 1050 1425 1050 1425 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 1050 1575 1425 +-6 +4 0 -1 0 0 0 12 0.0000 4 165 225 450 900 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 375 1725 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1500 1725 Y2\001 +4 0 -1 0 0 0 12 0.0000 4 180 2130 150 2250 HMM with Gaussian output\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1500 900 Q2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_gauss.gif b/sourcecodes/bnt-master/docs/Figures/hmm_gauss.gif new file mode 100644 index 00000000..d0aa117b --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_gauss.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_io.fig b/sourcecodes/bnt-master/docs/Figures/hmm_io.fig new file mode 100644 index 00000000..1f84aafd --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_io.fig @@ -0,0 +1,44 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 1725.000 1200.000 600 450 375 1275 600 1950 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 2859.375 1303.125 1650 525 1425 1200 1575 1950 + 0 0 1.00 60.00 120.00 +6 525 150 2175 2325 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1875 2100 300 225 1875 2100 2175 2325 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 825 2100 300 225 825 2100 1125 2325 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 900 375 300 225 900 375 1200 600 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1875 375 300 225 1875 375 2175 600 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 1275 1650 1275 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 1050 1050 1050 1050 1500 675 1500 675 1050 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1500 825 1875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1725 1050 2100 1050 2100 1500 1725 1500 1725 1050 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1875 1500 1875 1875 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 675 825 1050 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1875 675 1875 1050 +-6 +4 0 -1 0 0 0 12 0.0000 4 165 225 750 1350 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1800 1350 Q2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 750 450 U1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 750 2175 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1725 450 U2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1725 2175 Y2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_io.gif b/sourcecodes/bnt-master/docs/Figures/hmm_io.gif new file mode 100644 index 00000000..24dfc2da --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_io.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_mixgauss.fig b/sourcecodes/bnt-master/docs/Figures/hmm_mixgauss.fig new file mode 100644 index 00000000..cbfb1c63 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_mixgauss.fig @@ -0,0 +1,52 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +6 675 1650 1275 2100 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 975 1875 300 225 975 1875 1275 2100 +4 0 -1 0 0 0 12 0.0000 4 135 225 825 1950 Y1\001 +-6 +6 1725 1725 2325 2175 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 2025 1950 300 225 2025 1950 2325 2175 +4 0 -1 0 0 0 12 0.0000 4 135 225 1950 2025 Y2\001 +-6 +6 1500 900 1875 1350 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1500 900 1875 900 1875 1350 1500 1350 1500 900 +4 0 -1 0 0 0 12 0.0000 4 135 255 1575 1200 M2\001 +-6 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1275 450 1800 450 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 825 225 1200 225 1200 675 825 675 825 225 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1875 225 2250 225 2250 675 1875 675 1875 225 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 225 900 600 900 600 1350 225 1350 225 900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 675 450 900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 1350 750 1650 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 675 1050 1650 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1875 675 1725 900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1800 1350 1950 1725 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2100 675 2100 1725 +4 0 -1 0 0 0 12 0.0000 4 165 225 900 525 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 135 255 300 1200 M1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1950 525 Q2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_mixgauss.gif b/sourcecodes/bnt-master/docs/Figures/hmm_mixgauss.gif new file mode 100644 index 00000000..33d758b8 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_mixgauss.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_zoo.fig b/sourcecodes/bnt-master/docs/Figures/hmm_zoo.fig new file mode 100644 index 00000000..05d34578 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_zoo.fig @@ -0,0 +1,162 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 2784.375 4153.125 1575 3375 1350 4050 1500 4800 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 1650.000 4050.000 525 3300 300 4125 525 4800 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 6376.355 4323.343 5550 3450 5175 4275 5400 5025 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 7138.600 4307.555 6525 3525 6150 4200 6450 5025 + 1 1 1.00 60.00 120.00 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 750 4950 300 225 750 4950 1050 5175 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 1800 4950 300 225 1800 4950 2100 5175 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 825 3225 300 225 825 3225 1125 3450 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 1800 3225 300 225 1800 3225 2100 3450 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 3150 3150 300 225 3150 3150 3450 3375 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 4200 3150 300 225 4200 3150 4500 3375 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 3150 5625 300 225 3150 5625 3450 5850 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 4200 5625 300 225 4200 5625 4500 5850 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 5700 5100 300 225 5700 5100 6000 5325 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 6750 5100 300 225 6750 5100 7050 5325 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 3375 1800 300 225 3375 1800 3675 2025 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 1575 1650 300 225 1575 1650 1875 1875 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 525 1650 300 225 525 1650 825 1875 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 4425 1875 300 225 4425 1875 4725 2100 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 5550 1425 300 225 5550 1425 5850 1650 +1 1 0 2 7 0 50 0 20 0.000 1 0.0000 6600 1425 300 225 6600 1425 6900 1650 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3150 3750 3150 3375 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6750 4500 6750 4875 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 4275 150 4650 150 4650 600 4275 600 4275 150 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3225 600 2850 825 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 600 3900 975 3900 975 4350 600 4350 600 3900 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 750 4350 750 4725 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 750 3525 750 3900 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 1800 4350 1800 4725 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 1050 4125 1575 4125 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 1800 3525 1800 3900 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 1650 3900 2025 3900 2025 4350 1650 4350 1650 3900 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 3000 3750 3375 3750 3375 4200 3000 4200 3000 3750 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3450 3975 3900 4725 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3450 3975 3975 3975 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 3000 4575 3375 4575 3375 5025 3000 5025 3000 4575 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3150 5025 3150 5400 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3450 4800 3975 4050 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3450 4800 3975 4800 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 4050 4575 4425 4575 4425 5025 4050 5025 4050 4575 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 4050 3750 4425 3750 4425 4200 4050 4200 4050 3750 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 4200 3750 4200 3375 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 4200 5025 4200 5400 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 5550 4050 5925 4050 5925 4500 5550 4500 5550 4050 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 5700 4500 5700 4875 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 3450 6525 3450 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 4275 6525 4275 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 6600 3225 6975 3225 6975 3675 6600 3675 6600 3225 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 6600 4050 6975 4050 6975 4500 6600 4500 6600 4050 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 825 825 1350 825 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 1575 1050 1575 1425 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 375 600 750 600 750 1050 375 1050 375 600 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 525 1050 525 1425 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 1425 600 1800 600 1800 1050 1425 1050 1425 600 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 2625 825 3000 825 3000 1275 2625 1275 2625 825 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 2925 1275 3150 1575 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3450 600 3450 1575 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 3225 150 3600 150 3600 600 3225 600 3225 150 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 3675 375 4200 375 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 3900 825 4275 825 4275 1275 3900 1275 3900 825 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 4275 600 4125 825 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 4200 1275 4350 1650 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 4500 600 4500 1650 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 5550 825 5550 1200 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 5400 375 5775 375 5775 825 5400 825 5400 375 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 5850 600 6375 600 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 6450 375 6825 375 6825 825 6450 825 6450 375 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 6600 825 6600 1200 +2 1 0 2 0 0 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 1.00 60.00 120.00 + 5850 1425 6300 1425 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 5550 3225 5925 3225 5925 3675 5550 3675 5550 3225 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_zoo.gif b/sourcecodes/bnt-master/docs/Figures/hmm_zoo.gif new file mode 100644 index 00000000..c16ddffa --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_zoo.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/hmm_zoo_names.fig b/sourcecodes/bnt-master/docs/Figures/hmm_zoo_names.fig new file mode 100644 index 00000000..b53a7143 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_zoo_names.fig @@ -0,0 +1,169 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 1650.000 4050.000 525 3300 300 4125 525 4800 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 2784.375 4153.125 1575 3375 1350 4050 1500 4800 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 6376.355 4323.343 5550 3450 5175 4275 5400 5025 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 7138.600 4307.555 6525 3525 6150 4200 6450 5025 + 0 0 1.00 60.00 120.00 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 5550 1425 300 225 5550 1425 5850 1650 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 6600 1425 300 225 6600 1425 6900 1650 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 525 1650 300 225 525 1650 825 1875 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1800 4950 300 225 1800 4950 2100 5175 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 825 3225 300 225 825 3225 1125 3450 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1800 3225 300 225 1800 3225 2100 3450 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3150 3150 300 225 3150 3150 3450 3375 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 4200 3150 300 225 4200 3150 4500 3375 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3150 5625 300 225 3150 5625 3450 5850 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 4200 5625 300 225 4200 5625 4500 5850 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 5700 5100 300 225 5700 5100 6000 5325 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3375 1800 300 225 3375 1800 3675 2025 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 750 4950 300 225 750 4950 1050 5175 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 6750 5100 300 225 6750 5100 7050 5325 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 4425 1875 300 225 4425 1875 4725 2100 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1575 1650 300 225 1575 1650 1875 1875 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5850 600 6375 600 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 5400 375 5775 375 5775 825 5400 825 5400 375 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5550 825 5550 1200 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 6450 375 6825 375 6825 825 6450 825 6450 375 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6600 825 6600 1200 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5850 1425 6300 1425 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 825 1350 825 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 375 600 750 600 750 1050 375 1050 375 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 1050 525 1425 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1425 600 1800 600 1800 1050 1425 1050 1425 600 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 1050 1575 1425 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 4125 1575 4125 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 600 3900 975 3900 975 4350 600 4350 600 3900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 750 4350 750 4725 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1650 3900 2025 3900 2025 4350 1650 4350 1650 3900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1800 4350 1800 4725 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 750 3525 750 3900 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1800 3525 1800 3900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3150 5025 3150 5400 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4200 5025 4200 5400 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3150 3750 3150 3375 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4200 3750 4200 3375 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 4800 3975 4050 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 3975 3900 4725 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 4800 3975 4800 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3000 4575 3375 4575 3375 5025 3000 5025 3000 4575 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 4050 4575 4425 4575 4425 5025 4050 5025 4050 4575 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 3975 3975 3975 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3000 3750 3375 3750 3375 4200 3000 4200 3000 3750 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 4050 3750 4425 3750 4425 4200 4050 4200 4050 3750 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5700 4500 5700 4875 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6750 4500 6750 4875 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6000 3450 6525 3450 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 5550 3225 5925 3225 5925 3675 5550 3675 5550 3225 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 6600 3225 6975 3225 6975 3675 6600 3675 6600 3225 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6000 4275 6525 4275 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 5550 4050 5925 4050 5925 4500 5550 4500 5550 4050 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 6600 4050 6975 4050 6975 4500 6600 4500 6600 4050 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3675 375 4200 375 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3225 150 3600 150 3600 600 3225 600 3225 150 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 4275 150 4650 150 4650 600 4275 600 4275 150 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2625 825 3000 825 3000 1275 2625 1275 2625 825 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3225 600 2850 825 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2925 1275 3150 1575 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3450 600 3450 1575 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4275 600 4125 825 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4200 1275 4350 1650 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4500 600 4500 1650 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 3900 825 4275 825 4275 1275 3900 1275 3900 825 +4 0 -1 0 0 0 12 0.0000 4 180 2130 150 2400 HMM with Gaussian output\001 +4 0 -1 0 0 0 12 0.0000 4 180 1785 5100 2400 Auto Regressive HMM\001 +4 0 0 50 0 0 12 0.0000 4 180 1440 300 6075 Input-output HMM\001 +4 0 0 50 0 0 12 0.0000 4 180 1125 3000 6225 Coupled HMM\001 +4 0 -1 0 0 0 12 0.0000 4 135 1185 5475 6225 Factorial HMM\001 +4 0 0 50 0 0 12 0.0000 4 180 1530 2925 2625 of Gaussians output\001 +4 0 0 50 0 0 12 0.0000 4 135 1455 2925 2400 HMM with mixture\001 diff --git a/sourcecodes/bnt-master/docs/Figures/hmm_zoo_small.fig b/sourcecodes/bnt-master/docs/Figures/hmm_zoo_small.fig new file mode 100644 index 00000000..1fae5d57 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/hmm_zoo_small.fig @@ -0,0 +1,104 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +75.00 +Single +-2 +1200 2 +5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 10209.375 1303.125 9000 525 8775 1200 8925 1950 + 0 0 2.00 120.00 240.00 +5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 9075.000 1200.000 7950 450 7725 1275 7950 1950 + 0 0 2.00 120.00 240.00 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 525 1650 300 225 525 1650 825 1875 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 1590 1654 300 225 1590 1654 1890 1879 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 3375 1800 300 225 3375 1800 3675 2025 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 4427 1882 300 225 4427 1882 4727 2107 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 5925 1800 300 225 5925 1800 6225 2025 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 6975 1800 300 225 6975 1800 7275 2025 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 8173 2107 300 225 8173 2107 8473 2332 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 9225 2100 300 225 9225 2100 9525 2325 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 8250 375 300 225 8250 375 8550 600 +1 1 0 2 0 0 50 0 20 0.000 1 0.0000 9225 375 300 225 9225 375 9525 600 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 1575 1050 1575 1425 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 825 825 1350 825 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 525 1050 525 1425 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 375 600 750 600 750 1050 375 1050 375 600 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 1425 600 1800 600 1800 1050 1425 1050 1425 600 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 3450 600 3450 1575 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 3225 600 2850 825 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 2625 825 3000 825 3000 1275 2625 1275 2625 825 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 3225 150 3600 150 3600 600 3225 600 3225 150 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 3675 375 4200 375 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 3900 825 4275 825 4275 1275 3900 1275 3900 825 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 4200 1275 4350 1650 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 4500 600 4500 1650 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 4275 150 4650 150 4650 600 4275 600 4275 150 +2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 2.00 120.00 240.00 + 4275 600 3975 825 +2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 2.00 120.00 240.00 + 2925 1275 3225 1575 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 8175 675 8175 1050 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 9225 675 9225 1050 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 9225 1500 9225 1875 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 8475 1275 9000 1275 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 8175 1500 8175 1875 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 8025 1050 8400 1050 8400 1500 8025 1500 8025 1050 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 9075 1050 9450 1050 9450 1500 9075 1500 9075 1050 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 5925 1200 5925 1575 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 6225 975 6750 975 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 6225 1800 6675 1800 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 6975 1200 6975 1575 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 6825 750 7200 750 7200 1200 6825 1200 6825 750 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 5775 750 6150 750 6150 1200 5775 1200 5775 750 +4 0 0 50 0 0 24 0.0000 4 255 945 375 2400 HMM\001 +4 0 0 50 0 0 24 0.0000 4 255 2580 2625 2475 MixGauss HMM\001 +4 0 0 50 0 0 24 0.0000 4 255 1470 8025 2700 IO-HMM\001 +4 0 0 50 0 0 24 0.0000 4 255 1575 5625 2475 AR-HMM\001 diff --git a/sourcecodes/bnt-master/docs/Figures/ifa.eps b/sourcecodes/bnt-master/docs/Figures/ifa.eps new file mode 100644 index 00000000..f2ea58cb --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/ifa.eps @@ -0,0 +1,470 @@ +%!PS-Adobe-3.0 EPSF-3.0 +%%Creator: (ImageMagick) +%%Title: (ifa.eps) +%%CreationDate: (Tue Nov 16 19:52:10 2004) +%%BoundingBox: 0 0 246 221 +%%DocumentData: Clean7Bit +%%LanguageLevel: 1 +%%Pages: 1 +%%EndComments + +%%BeginDefaults +%%EndDefaults + +%%BeginProlog +% +% Display a color image. The image is displayed in color on +% Postscript viewers or printers that support color, otherwise +% it is displayed as grayscale. +% +/DirectClassPacket +{ + % + % Get a DirectClass packet. + % + % Parameters: + % red. + % green. + % blue. + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/DirectClassImage +{ + % + % Display a DirectClass image. + % + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { DirectClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayDirectClassPacket } image + } ifelse +} bind def + +/GrayDirectClassPacket +{ + % + % Get a DirectClass packet; convert to grayscale. + % + % Parameters: + % red + % green + % blue + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/GrayPseudoClassPacket +{ + % + % Get a PseudoClass packet; convert to grayscale. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassPacket +{ + % + % Get a PseudoClass packet. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassImage +{ + % + % Display a PseudoClass image. + % + % Parameters: + % class: 0-PseudoClass or 1-Grayscale. + % + currentfile buffer readline pop + token pop /class exch def pop + class 0 gt + { + currentfile buffer readline pop + token pop /depth exch def pop + /grays columns 8 add depth sub depth mul 8 idiv string def + columns rows depth + [ + columns 0 0 + rows neg 0 rows + ] + { currentfile grays readhexstring pop } image + } + { + % + % Parameters: + % colors: number of colors in the colormap. + % colormap: red, green, blue color packets. + % + currentfile buffer readline pop + token pop /colors exch def pop + /colors colors 3 mul def + /colormap colors string def + currentfile colormap readhexstring pop pop + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { PseudoClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayPseudoClassPacket } image + } ifelse + } ifelse +} bind def + +/DisplayImage +{ + % + % Display a DirectClass or PseudoClass image. + % + % Parameters: + % x & y translation. + % x & y scale. + % label pointsize. + % image label. + % image columns & rows. + % class: 0-DirectClass or 1-PseudoClass. + % compression: 0-none or 1-RunlengthEncoded. + % hex color packets. + % + gsave + /buffer 512 string def + /byte 1 string def + /color_packet 3 string def + /pixels 768 string def + + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + x y translate + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + currentfile buffer readline pop + token pop /pointsize exch def pop + /Times-Roman findfont pointsize scalefont setfont + x y scale + currentfile buffer readline pop + token pop /columns exch def + token pop /rows exch def pop + currentfile buffer readline pop + token pop /class exch def pop + currentfile buffer readline pop + token pop /compression exch def pop + class 0 gt { PseudoClassImage } { DirectClassImage } ifelse + grestore +} bind def +%%EndProlog +%%Page: 1 1 +%%PageBoundingBox: 0 0 246 221 +userdict begin +DisplayImage +0 0 +246 221 +12.000000 +246 221 +1 +1 +1 +1 +fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffc00000007f +ffffffffffffffffff00000001fffffffffffffffffcffffffffdfffffff7fffffffffff +ffffffff7ffffffdfffffffffffffffffcffffffffdfffffff7fffffffffffffffffff7f +fffffdfffffffffffffffffcffffffffdfffffff7fffffffffffffffffff7ffffffdffff +fffffffffffffcffffffffdfffffff7fffffffffffffffffff7ffffffdffffffffffffff +fffcffffffffdfffffff7fffffffffffffffffff7ffffffdfffffffffffffffffcffffff +ffdfffffff7fffffffffffffffffff7ffffffdfffffffffffffffffcffffffffdfffffff +7fffffffffffffffffff7ffffffdfffffffffffffffffcffffffffdfffffff7fffffffff +ffffffffff7ffffffdfffffffffffffffffcffffffffdfffffff7fffffffffffffffffff +7ffffffdfffffffffffffffffcffffffffdfffffff7fffffffffffffffffff7ffffffdff +fffffffffffffffcffffffffdf87dfff7fffffffffffffffffff7e1ffffdffffffffffff +fffffcffffffffdf339fff7fffffffffffffffffff7ccffffdfffffffffffffffffcffff +ffffdf7b5fff7fffffffffffffffffff7deffffdfffffffffffffffffcffffffffdefddf +ff7fffffffffffffffffff7bf53ffdfffffffffffffffffcffffffffdefddfff7fffffff +ffffffffffff7bf6dffdfffffffffffffffffcffffffffdefddfff7fffffffffffffffff +ff7bf6dffdfffffffffffffffffcffffffffdf7bdfff7fffffffffffffffffff7deedffd +fffffffffffffffffcffffffffdf33dfff7fffffffffffffffffff7ccedffdffffffffff +fffffffcffffffffdf878fff7fffffffffffffffffff7e1c4ffdfffffffffffffffffcff +ffffffdfe7ffff7fffffffffffffffffff7f9ffffdfffffffffffffffffcffffffffdff9 +ffff7fffffffffffffffffff7fe7fffdfffffffffffffffffcffffffffdfffffff7fffff +ffffffffffffff7ffffffdfffffffffffffffffcffffffffdfffffff7fffffffffffffff +ffff7ffffffdfffffffffffffffffcffffffffdfffffff7fffffffffffffffffff7fffff +fdfffffffffffffffffcffffffffdfffffff7fffffffffffffffffff7ffffffdffffffff +fffffffffcffffffffdfffffff7fffffffffffffffffff7ffffffdfffffffffffffffffc +ffffffffdfffffff7fffffffffffffffffff7ffffffdfffffffffffffffffcffffffffdf +ffffff7fffffffffffffffffff7ffffffdfffffffffffffffffcffffffffdfffffff7fff +ffffffffffffffff7ffffffdfffffffffffffffffcffffffffc00000007fffffffffffff +ffffff00000001fffffffffffffffffcffffffffffffbffffffffffffffffffffffffffe +fffffffffffffffffffffcffffffffffffbffffffffffffffffffffffffffeffffffffff +fffffffffffcffffffffffffbffffffffffffffffffffffffffeffffffffffffffffffff +fcffffffffffffbffffffffffffffffffffffffffefffffffffffffffffffffcffffffff +fffeaffffffffffffffffffffffffffabffffffffffffffffffffcfffffffffffeafffff +fffffffffffffffffffffabffffffffffffffffffffcfffffffffffe9fffffffffffffff +fffffffffffa7ffffffffffffffffffffcffffffffffff1fffffffffffffffffffffffff +fc7ffffffffffffffffffffcffffffffffff1ffffffffffffffffffffffffffc7fffffff +fffffffffffffcffffffffffff1ffffffffffffffffffffffffffc7fffffffffffffffff +fffcffffffffffff3ffffffffffffffffffffffffffcfffffffffffffffffffffcffffff +ffffffbffffffffffffffffffffffffffefffffffffffffffffffffcffffffffffffbfff +fffffffffffffffffffffffefffffffffffffffffffffcffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffffffcffffffffffff800fffffffffffffffffffff +f800fffffffffffffffffffffcfffffffffffc7ff1ffffffffffffffffffffc7ff1fffff +fffffffffffffffcffffffffffe3fffe3ffffffffffffffffffe3fffe3ffffffffffffff +fffffcffffffffffdfffffdffffffffffffffffffdfffffdfffffffffffffffffffcffff +ffffff3fffffe7fffffffffffffffff3fffffe7ffffffffffffffffffcfffffffffeffff +fffbffffffffffffffffefffffffbffffffffffffffffffcfffffffffdfffffffdffffff +ffffffffffdfffffffdffffffffffffffffffcfffffffffbfffffffeffffffffffffffff +bfffffffeffffffffffffffffffcfffffffff7ffffffff7fffffffffffffff7ffffffff7 +fffffffffffffffffcfffffffff7ffffffff7fffffffffffffff7ffffffff7ffffffffff +fffffffcffffffffefffffffffbffffffffffffffefffffffffbfffffffffffffffffcff +ffffffefff8c6fffbffffffffffffffeff18fffffbfffffffffffffffffcffffffffdfff +decfffdffffffffffffffdffbdfffffdfffffffffffffffffcffffffffdfffedafffdfff +fffffffffffdffdbfffffdfffffffffffffffffcffffffffdfffe3efffdfffffffffffff +fdffc74ffffdfffffffffffffffffcffffffffdffff3efffdffffffffffffffdffe7b7ff +fdfffffffffffffffffcffffffffdfffe9efffdffffffffffffffdffd3b7fffdffffffff +fffffffffcffffffffdfffedefffdffffffffffffffdffdbb7fffdfffffffffffffffffc +ffffffffdfffdeefffdffffffffffffffdffbdb7fffdfffffffffffffffffcffffffffef +ff8c47ffbffffffffffffffeff1813fffbfffffffffffffffffcffffffffefffffffffbf +fffffffffffffefffffffffbfffffffffffffffffcfffffffff7ffffffff7fffffffffff +ffff7ffffffff7fffffffffffffffffcfffffffff7ffffffff7fffffffffffffff7fffff +fff7fffffffffffffffffcfffffffffbfffffffeffffff3cf3ffffffbfffffffefffffff +fffffffffffcfffffffffdfffffffdffffff3cf3ffffffdfffffffdfffffffffffffffff +fcfffffffffefffffff9ffffffffffffffffefffffffbffffffffffffffffffcffffffff +ff3fffffe67ffffffffffffffff3fffffe7ffffffffffffffffffcffffffffffdfffffdf +8ffffffffffffffffdfffffdfffffffffffffffffffcffffffffffe3fffe3ff3ffffffff +fffffffe3fffe3fffffffffffffffffffcfffffffffffc7ff1fffc7fffffffffffffffc7 +ff1ffffffffffffffffffffcfffffffffff7800f7fff9ffffffffffffffff000f7ffffff +fffffffffffffcfffffffffff7ffff7fffe3ffffffffffffff87fffbffffffffffffffff +fffcffffffffffefffffbffffcfffffffffffffe5ffffdfffffffffffffffffffcffffff +ffffefffffbfffff1ffffffffffff13ffffefffffffffffffffffffcffffffffffdfffff +dfffffe7ffffffffff8effffff7ffffffffffffffffffcffffffffffdfffffdffffff8ff +fffffffc7dffffffbffffffffffffffffffcffffffffffdfffffefffffff3ffffffff3f3 +ffffffdffffffffffffffffffcffffffffffbfffffefffffffc7ffffff8fefffffffefff +fffffffffffffffcffffffffffbffffff7fffffff9fffffc7f9ffffffff7ffffffffffff +fffffcffffffffffbffffff7fffffffe3fffe3ff7ffffffffbfffffffffffffffffcffff +ffffff7ffffffbffffffffcfff9ffcfffffffffdfffffffffffffffffcffffffffff7fff +fffbfffffffff1fc7ffbfffffffffefffffffffffffffffcfffffffffefffffffdffffff +fffe63fff7ffffffffff7ffffffffffffffffcfffffffffefffffffdffffffffff0fffcf +ffffffffffbffffffffffffffffcfffffffffefffffffefffffffffcf3ffbfffffffffff +dffffffffffffffffcfffffffffdfffffffeffffffffe3fc7e7fffffffffffefffffffff +fffffffcfffffffffdffffffff7fffffff1fff9dfffffffffffff7fffffffffffffffcff +fffffffdffffffff7ffffff8ffffe3fffffffffffffbfffffffffffffffcfffffffffbff +ffffffbfffffe7ffffecfffffffffffffdfffffffffffffffcfffffffffbffffffffbfff +ff1fffffdf1ffffffffffffefffffffffffffffcfffffffff7ffffffffdffff8ffffff3f +e7ffffffffffff7ffffffffffffffcfffffffff7ffffffffdfffc7fffffefff8ffffffff +ffffbffffffffffffffcfffffffff7ffffffffefff3ffffff9ffff3fffffffffffdfffff +fffffffffcffffffffefffffffffeff8fffffff7ffffc7ffffffffffeffffffffffffffc +ffffffffeffffffffff7c7ffffffcffffff9fffffffffff7fffffffffffffcffffffffef +fffffffff63fffffffbffffffe3ffffffffffbfffffffffffffcffffffffdffffffffff9 +ffffffff7fffffffcffffffffffdfffffffffffffcffffffffdfffffffffc3fffffffcff +fffffff1fffffffffefffffffffffffcffffffffbffffffffe3dffffffdbfffffffffe7f +ffffffff7ffffffffffffcffffffffbffffffff1fdffffffa7ffffffffff8fffffffffbf +fffffffffffcfffffffebfffffffcffebfffff5ffffffffffff3ffffffffdfffffffffff +fcfffffffe7ffffffe3ffebffffe27fffffffffffc7fffffffeffffffffffffcfffffffe +5ffffff1fffd3ffffe9fffffffffffff9ffffffff6fffffffffffcfffffffe3fffff8fff +fe5ffffc7fffffffffffffe3fffffffb7ffffffffffcfffffffcbffffe7fffff1ffff9ff +fffffffffffffcfffffffd7ffffffffffcfffffffc7fff71ffffff1fffffffffffffffff +ffff1ffffff6bffffffffffcfffffffcfffc8fffffff9fffffffffffffffffffffe7ffff +f93ffffffffffcfffffffcfff07fffffffcffffffffffffffffffffff8fffffe1fffffff +fffcfffffffdffc3ffffffffefffffffffffffffffffffff3fffff9ffffffffffcffffff +ffff803fffffffffffffffffffffffffffffffc7ffffeffffffffffcffffff001fffffff +ffffff800ffffffffffffffffffff9dffff800fffffffcfffff8ffe3fffffffffffc7ff1 +fffffffffffffffffffe27ffc7ff1ffffffcffffc6eeec7fffffffffe377763fffffffff +ffffffffffcbfe377763fffffcffffbfffffbfffffffffdfffffdfffffffffffffffffff +00fdfffffdfffffcfffe3bbfbb8fffffffff1ddfddc7fffffffffffffffffff871ddfddc +7ffffcfffdfffffff7fffffffefffffffbffffffffffffffffffffefffffffbffffcfffa +eeeeeeebfffffffd77777775ffffffffffffffffffffd77777775ffffcfff7fffffffdff +fffffbfffffffeffffffffffffffffffffbfffffffeffffcffebfbfbfbfafffffff5fdfd +fdfd7fffffffffffffffffff5fdfdfdfd7fffcffeffffffffefffffff7ffffffff7fffff +ffffffffffffff7ffffffff7fffcffceeeeeeeee7fffffe7777777773fffffffffffffff +fffe7777777773fffcffdfe38dffff7fffffeff1c47fffbffffffffffffffffffefe38ff +fffbfffcffbfb799bfbfbfffffdfdbcb9fdfdffffffffffffffffffdfd7dfdfdfdfffcff +bffbb5ffffbfffffdffddfbfffdfffff3cf3fffffffffdffbbfffffdfffcffaeecaceeee +bfffffd7765737775fffff3cf3fffffffffd7753417775fffcffbffc7dffff8000001ffe +3f7fffdfffffffffffff800001ffc7b6fffdfffcffbbfaf9fbfbbfffffddfd7cfdfddfff +fffffffffffffffddfcf96dfddfffcffbffefdffffbfffffdfff7dffffdfffffffffffff +fffffdffefb6fffdfffcffaeeeeceeeebfffffd7777337775ffffffffffffffffffd7767 +367775fffcffdffc78ffff7fffffeffe383fffbffffffffffffffffffeffc7127ffbfffc +ffdfbbbfbbbf7fffffefdddfdddfbffffffffffffffffffefdddfdddfbfffcffefffffff +fefffffff7ffffffff7fffffffffffffffffff7ffffffff7fffcffeeeeeeeeeefffffff7 +777777777fffffffffffffffffff7777777777fffcfff7fffffffdfffffffbfffffffeff +ffffffffffffffffffbfffffffeffffcfffbfbfbfbfbfffffffdfdfdfdfdffffffffffff +ffffffffdfdfdfdfdffffcfffdfffffff7fffffffefffffffbffffffffffffffffffffef +ffffffbffffcfffe6eeeeecfffffffff37777763ffffffffffffffffffffe37777767fff +fcffffbfffffbfffffffffdfffffddffffffffffffffffffffddfffffdfffffcffffc7bf +bc7fffffffffe3dfde3e7fffffffffffffffffff3e3dfde3fffffcfffff8ffe3ffffffff +fffc7ff1ffbffffffffffffffffffeffc7ff1ffffffcffffff001effffffffffff800fff +cffffffffffffffffff9fff800fffffffcffffffffff7ffffffffffffffffff7ffffffff +fffffffff7ffff7ffffffffcffffffffff9ffffffffffffffffff9ffffffffffffffffcf +fffcfffffffffcffffffffffeffffffffffffffffffe7fffffffffffffff3ffffbffffff +fffcfffffffffff7ffffffffffffffffff9ffffffffffffffcfffff7fffffffffcffffff +fffff9ffffffffffffffffffe7fffffffffffff3ffffcffffffffffcfffffffffffeffff +fffffffffffffff9ffffffffffffcfffffbffffffffffcffffffffffff3fffffffffffff +fffffe3ffffffffffe3ffffe7ffffffffffcffffffffffffdfffffffffffffffffffc7ff +fffffff1fffffdfffffffffffcffffffffffffe7fffffffffffffffffff87fffffff0fff +fff3fffffffffffcfffffffffffffbffffffffffffffffffff83ffffe0ffffffefffffff +fffffcfffffffffffffcfffffffffffffffffffffc00001fffffff9ffffffffffffcffff +ffffffffff3ffffffffffffffffffffffffffffffffe7ffffffffffffcffffffffffffff +cffffffffffffffffffffffffffffffff9fffffffffffffcfffffffffffffff7ffffffff +fffffffffffffffffffffff7fffffffffffffcfffffffffffffff9ffffffffffffffffff +ffffffffffffcffffffffffffffcfffffffffffffffe7fffffffffffffffffffffffffff +ff3ffffffffffffffcffffffffffffffff8ffffffffffffffffffffffffffff8ffffffff +fffffffcfffffffffffffffff3ffffffffffffffffffffffffffe7fffffffffffffffcff +fffffffffffffffcffffffffffffffffffffffffff9ffffffffffffffffcffffffffffff +ffffff1ffffffffffffffffffffffffc7ffffffffffffffffcffffffffffffffffffe7ff +fffffffffffffffffffff3fffffffffffffffffcfffffffffffffffffff8ffffffffffff +ffffffffff8ffffffffffffffffffcffffffffffffffffffff1ffffffffffffffffffffc +7ffffffffffffffffffcffffffffffffffffffffe3ffffffffffffffffffe3ffffffffff +fffffffffcfffffffffffffffffffffc3ffffffffffffffffe1ffffffffffffffffffffc +ffffffffffffffffffffffc3ffffffffffffffe1fffffffffffffffffffffcffffffffff +fffffffffffffc3ffffffffffffe1ffffffffffffffffffffffcffffffffffffffffffff +ffffc0ffffffffff81fffffffffffffffffffffffcffffffffffffffffffffffffff00ff +ffff807ffffffffffffffffffffffffcffffffffffffffffffffffffffff0000007fffff +fffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffffffff +fffcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffff +fffff1ff9ffffffe7ffff80ffffffffbfffcfffdfffffffffffffffcfffffffffffbffdf +ffffff7ffffdeffffffffbfffefffffffffffffffffffcfffffffffffbffdfffffff7ffe +fdffffbffff5fffefffffffffffffffffffcfffffffffffb4f1cc3cd3c734c3dde790cd3 +f5d3ce9989c7fffffffffffffcfffffffffffbb6db6db6db6db6fc1db6bb67eeedb6db6d +b7fffffffffffffcfffffffffffbb6d86d86db61b6fdde37bb6fe0edc6d33d9fffffffff +fffffcfffffffffffbb6dbedbedb6fb6fdfdb7bb6feeedb6d7cde7fffffffffffffcffff +fffffffbb6d9ad9adb66b6fdfdb6bb6fdf6db6e76db7fffffffffffffcfffffffffff113 +2c63c44cb113387e59ccc78e04c86f188ffffffffffffffcffffffffffffffffefffffff +ffffffffffffffffffeffffffffffffffffffcffffffffffffffffefffffffffffffffff +ffffffffdffffffffffffffffffcffffffffffffffffc7ffffffffffffffffffffffff9f +fffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcff +fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffcffffffffffffffffffffffffffffffffffffffffffffffffffff +fffffffffcfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc +fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffff +fffffffffffffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffff +fffffffffffffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffff +fffffffffffffffffffffffffffffffcffffffffffffffffffffffffffffffffffffffff +fffffffffffffffffffffc +end +%%PageTrailer +%%Trailer +%%EOF diff --git a/sourcecodes/bnt-master/docs/Figures/ifa.fig b/sourcecodes/bnt-master/docs/Figures/ifa.fig new file mode 100644 index 00000000..864d5112 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/ifa.fig @@ -0,0 +1,59 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 0 0 2362.500 1537.500 1725 2325 2400 2550 3000 2325 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 0 0 1950.000 818.750 675 2400 1950 2850 3225 2400 +6 225 900 3600 2475 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 900 1125 300 225 900 1125 1200 1350 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 2400 1125 300 225 2400 1125 2700 1350 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 525 2175 300 225 525 2175 825 2400 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3240 2179 300 225 3240 2179 3540 2404 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1500 2175 300 225 1500 2175 1800 2400 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 750 1350 525 1950 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 1275 3000 2025 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2325 1350 675 1950 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2550 1350 3150 1950 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 1350 1350 1950 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2325 1350 1575 1875 +4 0 -1 0 0 0 12 0.0000 4 135 225 825 1200 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2250 1200 Xn\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 375 2250 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 270 3075 2250 Ym\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 1500 1275 ...\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1350 2250 Y2\001 +4 0 -1 0 0 0 24 0.0000 4 30 270 2100 2175 ...\001 +-6 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 600 225 1050 225 1050 675 600 675 600 225 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2250 225 2700 225 2700 675 2250 675 2250 225 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 675 825 900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2475 675 2475 900 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 825 2175 1200 2175 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 2625 2175 2925 2175 +4 0 -1 0 0 0 12 0.0000 4 165 225 675 525 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 2325 525 Qn\001 diff --git a/sourcecodes/bnt-master/docs/Figures/ifa.gif b/sourcecodes/bnt-master/docs/Figures/ifa.gif new file mode 100644 index 00000000..d77cd76a --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/ifa.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/kf.fig b/sourcecodes/bnt-master/docs/Figures/kf.fig new file mode 100644 index 00000000..a2e24c82 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf.fig @@ -0,0 +1,27 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 225 225 1950 2250 +6 225 225 1950 675 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 525 450 300 225 525 450 825 675 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1650 450 300 225 1650 450 1950 675 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 450 1350 450 +-6 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 525 1575 300 225 525 1575 825 1800 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1650 1575 300 225 1650 1575 1950 1800 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 750 525 1275 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1650 750 1650 1275 +4 0 -1 0 0 0 12 0.0000 4 135 1455 225 2250 Kalman filter model\001 +-6 +4 0 -1 0 0 0 12 0.0000 4 135 210 375 525 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1500 525 X2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 375 1650 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1500 1650 Y2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/kf.gif b/sourcecodes/bnt-master/docs/Figures/kf.gif new file mode 100644 index 00000000..9d0ae34e --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/kf_input.fig b/sourcecodes/bnt-master/docs/Figures/kf_input.fig new file mode 100644 index 00000000..e51d73fa --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf_input.fig @@ -0,0 +1,40 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 1 0 7 100 0 -1 0.000 0 1 1 0 3131.250 1462.500 1575 450 1275 1425 1575 2475 + 0 0 1.00 60.00 120.00 +5 1 0 1 0 7 100 0 -1 0.000 0 1 1 0 4272.606 1559.043 2775 525 2475 1275 2700 2475 + 0 0 1.00 60.00 120.00 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1875 2550 300 225 1875 2550 2175 2775 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3000 2550 300 225 3000 2550 3300 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1875 1425 300 225 1875 1425 2175 1650 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 3000 1425 300 225 3000 1425 3300 1650 +1 1 0 1 0 0 100 0 2 0.000 1 0.0000 1875 375 300 225 1875 375 2175 600 +1 1 0 1 0 0 100 0 2 0.000 1 0.0000 3000 375 300 225 3000 375 3300 600 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1875 1725 1875 2250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3000 1725 3000 2250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2175 1425 2700 1425 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1875 600 1875 1200 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3000 600 3000 1200 +4 0 -1 0 0 0 12 0.0000 4 135 225 1725 1500 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2850 1500 X2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1725 2625 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2850 2625 Y2\001 +4 0 0 100 0 0 12 0.0000 4 135 225 1725 450 U1\001 +4 0 0 100 0 0 12 0.0000 4 135 225 2850 450 U2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/kf_input.gif b/sourcecodes/bnt-master/docs/Figures/kf_input.gif new file mode 100644 index 00000000..595d5ae0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf_input.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/kf_notime.fig b/sourcecodes/bnt-master/docs/Figures/kf_notime.fig new file mode 100644 index 00000000..11b75c1f --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf_notime.fig @@ -0,0 +1,26 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 1 0 2 0 0 100 0 2 0.000 1 0.0000 525 1575 300 225 525 1575 825 1800 +1 1 0 2 0 0 100 0 2 0.000 1 0.0000 1650 1575 300 225 1650 1575 1950 1800 +1 1 0 2 0 0 100 0 -1 0.000 1 0.0000 525 450 300 225 525 450 825 675 +1 1 0 2 0 0 100 0 -1 0.000 1 0.0000 1650 450 300 225 1650 450 1950 675 +2 1 0 2 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 525 750 525 1275 +2 1 0 2 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 1650 750 1650 1275 +2 1 0 2 0 0 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 2.00 120.00 240.00 + 825 450 1350 450 +4 0 0 100 0 0 20 0.0000 4 195 210 375 525 X\001 +4 0 0 100 0 0 20 0.0000 4 195 210 1500 525 X\001 +4 0 0 100 0 0 20 0.0000 4 195 210 1500 1650 Y\001 +4 0 0 100 0 0 20 0.0000 4 195 210 375 1650 Y\001 diff --git a/sourcecodes/bnt-master/docs/Figures/kf_obs_track.fig b/sourcecodes/bnt-master/docs/Figures/kf_obs_track.fig new file mode 100644 index 00000000..b657f62e --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf_obs_track.fig @@ -0,0 +1,43 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 1500 335 335 1200 1200 1500 1800 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 2700 335 335 1200 2400 1500 3000 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 3900 335 335 1200 3600 1500 4200 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 5100 335 335 1200 4800 1500 5400 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 5100 335 335 2700 4800 3000 5400 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 3900 335 335 2700 3600 3000 4200 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 2700 335 335 2700 2400 3000 3000 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 1500 335 335 2700 1200 3000 1800 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 1500 2550 1500 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 2700 2550 1650 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 3900 2475 3900 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 5100 2625 4125 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 5100 2475 5100 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 2700 2550 2700 +4 0 0 100 0 0 20 0.0000 4 195 405 1125 2850 dx1\001 +4 0 0 100 0 0 20 0.0000 4 195 270 1125 4050 x2\001 +4 0 0 100 0 0 20 0.0000 4 195 405 1200 5175 dx2\001 +4 0 0 100 0 0 20 0.0000 4 195 270 1200 1650 x1\001 +4 0 0 100 0 0 20 0.0000 4 195 270 2700 1650 x1\001 +4 0 0 100 0 0 20 0.0000 4 195 405 2700 2850 dx1\001 +4 0 0 100 0 0 20 0.0000 4 195 270 2700 4050 x2\001 +4 0 0 100 0 0 20 0.0000 4 195 405 2700 5250 dx2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/kf_scalar_track.fig b/sourcecodes/bnt-master/docs/Figures/kf_scalar_track.fig new file mode 100644 index 00000000..27ac7892 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kf_scalar_track.fig @@ -0,0 +1,59 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 2726.786 5448.214 1125 4125 675 5775 1050 6675 + 0 0 4.00 120.00 240.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 6809.923 4980.705 1050 1725 225 5625 975 8100 + 0 0 4.00 120.00 240.00 +5 1 0 2 0 7 100 0 -1 0.000 0 0 1 0 1778.571 5400.000 3225 3900 3750 6075 3225 6900 + 0 0 4.00 120.00 240.00 +5 1 0 2 0 7 100 0 -1 0.000 0 0 1 0 -75.000 4837.500 3225 1500 4575 5475 3225 8175 + 0 0 4.00 120.00 240.00 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 1500 335 335 1200 1200 1500 1800 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 2700 335 335 1200 2400 1500 3000 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 3900 335 335 1200 3600 1500 4200 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 1350 5100 335 335 1200 4800 1500 5400 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 5100 335 335 2700 4800 3000 5400 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 3900 335 335 2700 3600 3000 4200 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 2700 335 335 2700 2400 3000 3000 +1 4 0 2 0 7 100 0 -1 0.000 1 0.0000 2850 1500 335 335 2700 1200 3000 1800 +1 4 0 2 0 0 100 0 2 0.000 1 0.0000 1350 6900 335 335 1200 6600 1500 7200 +1 4 0 2 0 0 100 0 2 0.000 1 0.0000 2850 6900 335 335 2700 6600 3000 7200 +1 4 0 2 0 0 100 0 2 0.000 1 0.0000 2850 8100 335 335 2700 7800 3000 8400 +1 4 0 2 0 0 100 0 2 0.000 1 0.0000 1350 8100 335 335 1200 7800 1500 8400 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 1500 2550 1500 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 2700 2550 1650 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 3900 2475 3900 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 5100 2625 4125 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 5100 2475 5100 +2 1 0 2 0 0 100 0 2 0.000 0 0 -1 1 0 2 + 0 0 4.00 120.00 240.00 + 1725 2700 2550 2700 +4 0 0 100 0 0 20 0.0000 4 195 405 1125 2850 dx1\001 +4 0 0 100 0 0 20 0.0000 4 195 270 1125 4050 x2\001 +4 0 0 100 0 0 20 0.0000 4 195 405 1200 5175 dx2\001 +4 0 0 100 0 0 20 0.0000 4 195 270 1200 1650 x1\001 +4 0 0 100 0 0 20 0.0000 4 195 270 2700 1650 x1\001 +4 0 0 100 0 0 20 0.0000 4 195 405 2700 2850 dx1\001 +4 0 0 100 0 0 20 0.0000 4 195 270 2700 4050 x2\001 +4 0 0 100 0 0 20 0.0000 4 195 405 2700 5250 dx2\001 +4 0 0 100 0 0 20 0.0000 4 255 270 1200 6975 y2\001 +4 0 0 100 0 0 20 0.0000 4 255 270 1200 8175 y1\001 +4 0 0 100 0 0 20 0.0000 4 255 270 2700 7050 y2\001 +4 0 0 100 0 0 20 0.0000 4 255 270 2700 8175 y1\001 diff --git a/sourcecodes/bnt-master/docs/Figures/kfhead.jpg b/sourcecodes/bnt-master/docs/Figures/kfhead.jpg new file mode 100644 index 00000000..2c1fc6f0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/kfhead.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mathbymatlab.gif b/sourcecodes/bnt-master/docs/Figures/mathbymatlab.gif new file mode 100644 index 00000000..0de8d7a0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mathbymatlab.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mcmc_accept.jpg b/sourcecodes/bnt-master/docs/Figures/mcmc_accept.jpg new file mode 100644 index 00000000..f383627e --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mcmc_accept.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mcmc_post.jpg b/sourcecodes/bnt-master/docs/Figures/mcmc_post.jpg new file mode 100644 index 00000000..bbe172b5 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mcmc_post.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mfa.eps b/sourcecodes/bnt-master/docs/Figures/mfa.eps new file mode 100644 index 00000000..9b40d66c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mfa.eps @@ -0,0 +1,347 @@ +%!PS-Adobe-3.0 EPSF-3.0 +%%Creator: (ImageMagick) +%%Title: (mfa.eps) +%%CreationDate: (Tue Nov 16 19:52:06 2004) +%%BoundingBox: 0 0 126 151 +%%DocumentData: Clean7Bit +%%LanguageLevel: 1 +%%Pages: 1 +%%EndComments + +%%BeginDefaults +%%EndDefaults + +%%BeginProlog +% +% Display a color image. The image is displayed in color on +% Postscript viewers or printers that support color, otherwise +% it is displayed as grayscale. +% +/DirectClassPacket +{ + % + % Get a DirectClass packet. + % + % Parameters: + % red. + % green. + % blue. + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/DirectClassImage +{ + % + % Display a DirectClass image. + % + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { DirectClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayDirectClassPacket } image + } ifelse +} bind def + +/GrayDirectClassPacket +{ + % + % Get a DirectClass packet; convert to grayscale. + % + % Parameters: + % red + % green + % blue + % length: number of pixels minus one of this color (optional). + % + currentfile color_packet readhexstring pop pop + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/GrayPseudoClassPacket +{ + % + % Get a PseudoClass packet; convert to grayscale. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + color_packet 0 get 0.299 mul + color_packet 1 get 0.587 mul add + color_packet 2 get 0.114 mul add + cvi + /gray_packet exch def + compression 0 eq + { + /number_pixels 1 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add def + } ifelse + 0 1 number_pixels 1 sub + { + pixels exch gray_packet put + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassPacket +{ + % + % Get a PseudoClass packet. + % + % Parameters: + % index: index into the colormap. + % length: number of pixels minus one of this color (optional). + % + currentfile byte readhexstring pop 0 get + /offset exch 3 mul def + /color_packet colormap offset 3 getinterval def + compression 0 eq + { + /number_pixels 3 def + } + { + currentfile byte readhexstring pop 0 get + /number_pixels exch 1 add 3 mul def + } ifelse + 0 3 number_pixels 1 sub + { + pixels exch color_packet putinterval + } for + pixels 0 number_pixels getinterval +} bind def + +/PseudoClassImage +{ + % + % Display a PseudoClass image. + % + % Parameters: + % class: 0-PseudoClass or 1-Grayscale. + % + currentfile buffer readline pop + token pop /class exch def pop + class 0 gt + { + currentfile buffer readline pop + token pop /depth exch def pop + /grays columns 8 add depth sub depth mul 8 idiv string def + columns rows depth + [ + columns 0 0 + rows neg 0 rows + ] + { currentfile grays readhexstring pop } image + } + { + % + % Parameters: + % colors: number of colors in the colormap. + % colormap: red, green, blue color packets. + % + currentfile buffer readline pop + token pop /colors exch def pop + /colors colors 3 mul def + /colormap colors string def + currentfile colormap readhexstring pop pop + systemdict /colorimage known + { + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { PseudoClassPacket } false 3 colorimage + } + { + % + % No colorimage operator; convert to grayscale. + % + columns rows 8 + [ + columns 0 0 + rows neg 0 rows + ] + { GrayPseudoClassPacket } image + } ifelse + } ifelse +} bind def + +/DisplayImage +{ + % + % Display a DirectClass or PseudoClass image. + % + % Parameters: + % x & y translation. + % x & y scale. + % label pointsize. + % image label. + % image columns & rows. + % class: 0-DirectClass or 1-PseudoClass. + % compression: 0-none or 1-RunlengthEncoded. + % hex color packets. + % + gsave + /buffer 512 string def + /byte 1 string def + /color_packet 3 string def + /pixels 768 string def + + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + x y translate + currentfile buffer readline pop + token pop /x exch def + token pop /y exch def pop + currentfile buffer readline pop + token pop /pointsize exch def pop + /Times-Roman findfont pointsize scalefont setfont + x y scale + currentfile buffer readline pop + token pop /columns exch def + token pop /rows exch def pop + currentfile buffer readline pop + token pop /class exch def pop + currentfile buffer readline pop + token pop /compression exch def pop + class 0 gt { PseudoClassImage } { DirectClassImage } ifelse + grestore +} bind def +%%EndProlog +%%Page: 1 1 +%%PageBoundingBox: 0 0 126 151 +userdict begin +DisplayImage +0 0 +126 151 +12.000000 +126 151 +1 +1 +1 +1 +fffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffff +fffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffffffffffff +fffffffcfffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffc +ff8000001fffffffffffff800ffffffcffbfffffdffffffffffffc7ff1fffffcffbfffff +dfffffffffffe3fffe3ffffcffbfffffdfffffffffffdfffffdffffcffbfffffdfffffff +ffff3fffffe7fffcffbfffffdffffffffffefffffffbfffcffbfffffdffffffffffdffff +fffdfffcffbfffffdffffffffffbfffffffefffcffbfffffdffffffffff7ffffffff7ffc +ffbfffffdffffffffff7ffffffff7ffcffbfffffdfffffffffefffffffffbffcffbf0fff +dfffffffffeff18fffffbffcffbe67ffdfffffffffdffbdfffffdffcffbef7ffdfffffff +ffdffdbfffffdffcffbdfbffdfffffffffdffc7fffffdffcffbdfbffdfffffffffdffe7f +ffffdffcffbdfbffdfffffffffdffd3fffffdffcffbef7ffdfffffffffdffdbfffffdffc +ffbe67ffdfffffffffdffbdfffffdffcffbf0fffdfffffffffeff18fffffbffcffbfcfff +dfffffffffefffffffffbffcffbff3ffdffffffffff7ffffffff7ffcffbfffffdfffffff +fff7ffffffff7ffcffbfffffdffffffffffbfffffffefffcffbfffffdffffffffffdffff +fffdfffcffbfffffdffffffffffefffffffbfffcffbfffffdfffffffffff3fffffe7fffc +ffbfffffdfffffffffffdfffffdffffcffbfffffdfffffffffffe3fffe3ffffcffbfffff +dffffffffffffc7ff1fffffcff8000001fffffffffffff800ffffffcfffffffdffffffff +ffffff7ffffffffcfffffffdffffffffffffff7ffffffffcfffffffefffffffffffffeff +fffffffcffffffff7ffffffffffffdfffffffffcffffffffbffffffffffffbfffffffffc +ffffffffbffffffffffffbfffffffffcffffffffdffffffffffff7fffffffffcffffffff +efffffffffffeffffffffffcffffffffefffffffffffeffffffffffcfffffffff7ffffff +ffffdffffffffffcfffffffffbffffffffffbffffffffffcfffffffffdffffffffff7fff +fffffffcfffffffffdffffffffff7ffffffffffcfffffffffefffffffffefffffffffffc +ffffffffff7ffffffffdfffffffffffcffffffffff7ffffffffdfffffffffffcffffffff +ffbffffffffbfffffffffffcffffffffffdffffffff7fffffffffffcffffffffffefffff +ffeffffffffffffcffffffffffefffffffeffffffffffffcfffffffffff7ffffffdfffff +fffffffcfffffffffffbffffffbffffffffffffcfffffffffffbffffffbffffffffffffc +fffffffffffdffffff7ffffffffffffcfffffffffffefffffefffffffffffffcffffffff +ffff5ffff5fffffffffffffcffffffffffff5ffff5fffffffffffffcfffffffffffeafff +eafffffffffffffcffffffffffff4fffe5fffffffffffffcffffffffffff8fffd3ffffff +fffffffcffffffffffffc7ffc7fffffffffffffcffffffffffffe7ffcffffffffffffffc +fffffffffffff3ff9ffffffffffffffcfffffffffffffbffbffffffffffffffcffffffff +fffffc007ffffffffffffffcffffffffffffe3ff8ffffffffffffffcffffffffffff1bbb +b1fffffffffffffcfffffffffffefffffefffffffffffffcfffffffffff8eefeee3fffff +fffffffcfffffffffff7ffffffdffffffffffffcffffffffffebbbbbbbaffffffffffffc +ffffffffffdffffffff7fffffffffffcffffffffffafefefefebfffffffffffcffffffff +ffbffffffffbfffffffffffcffffffffff3bbbbbbbb9fffffffffffcffffffffff7f8e3f +fffdfffffffffffcfffffffffefede7efefefffffffffffcfffffffffeffeefffffeffff +fffffffcfffffffffebbb2bbbbbafffffffffffcfffffffffefff1fffffefffffffffffc +fffffffffeefebefefeefffffffffffcfffffffffefffbfffffefffffffffffcffffffff +febbbbbbbbbafffffffffffcffffffffff7ff1fffffdfffffffffffcffffffffff7eeefe +eefdfffffffffffcffffffffffbffffffffbfffffffffffcffffffffffbbbbbbbbbbffff +fffffffcffffffffffdffffffff7fffffffffffcffffffffffefefefefeffffffffffffc +fffffffffff7ffffffdffffffffffffcfffffffffff9bbbbbb3ffffffffffffcffffffff +fffefffffefffffffffffffcffffffffffff1efef1fffffffffffffcffffffffffffe3ff +8ffffffffffffffcfffffffffffffc007ffffffffffffffcffffffffffffffffffffffff +fffffffcfffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffc +fffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffff +fffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffffffffffff +fffffffcfffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffc +fffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffff +fffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffffffffffff +fffffffcfffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffc +fffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffff3f97ffffff +ffee03effffffffcffffff9f3fffffffffdf7beffffffffcffffff9f3ffbffffffdf7fd7 +fffffffcffffffaea6402533f98f77d7c7fffffcffffffaeb75bb66df6df07bbb7fffffc +ffffffb5b7bbb6e1f6df77839ffffffcffffffb5b7bbb6eff6df7fbbe7fffffcffffffbb +b75bb6e6f6df7f7db7fffffcffffff1b024cc871f9de1e388ffffffcffffffffffffffff +fffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffffffffffff +fffffffcfffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffc +fffffffffffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffff +fffffffffffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffff +fffffffffffffffcfffffffffffffffffffffffffffffffcffffffffffffffffffffffff +fffffffc +end +%%PageTrailer +%%Trailer +%%EOF diff --git a/sourcecodes/bnt-master/docs/Figures/mfa.fig b/sourcecodes/bnt-master/docs/Figures/mfa.fig new file mode 100644 index 00000000..90662e74 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mfa.fig @@ -0,0 +1,24 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 225 225 1800 1650 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1500 450 300 225 1500 450 1800 675 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 975 1425 300 225 975 1425 1275 1650 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 225 225 600 225 600 675 225 675 225 225 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 525 675 900 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1425 675 1050 1200 +-6 +4 0 -1 0 0 0 12 0.0000 4 135 135 825 1500 Y\001 +4 0 -1 0 0 0 12 0.0000 4 165 135 300 525 Q\001 +4 0 -1 0 0 0 12 0.0000 4 135 135 1350 525 X\001 diff --git a/sourcecodes/bnt-master/docs/Figures/mfa.gif b/sourcecodes/bnt-master/docs/Figures/mfa.gif new file mode 100644 index 00000000..3325b3a4 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mfa.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mixexp.fig b/sourcecodes/bnt-master/docs/Figures/mixexp.fig new file mode 100644 index 00000000..6999ae55 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mixexp.fig @@ -0,0 +1,23 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 525 300 1575 2925 +5 1 0 1 -1 7 0 0 -1 0.000 0 1 1 0 2055.000 1650.000 975 675 600 1650 975 2625 + 0 0 1.00 60.00 120.00 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1275 525 300 225 1275 525 1575 750 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1275 2700 300 225 1275 2700 1575 2925 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1050 1350 1425 1350 1425 1800 1050 1800 1050 1350 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1275 825 1275 1350 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1275 1800 1275 2475 +-6 +4 0 -1 0 0 0 12 0.0000 4 180 1425 450 3300 Mixture of Experts\001 +4 0 -1 0 0 0 12 0.0000 4 135 120 1200 600 X\001 +4 0 -1 0 0 0 12 0.0000 4 165 135 1125 1650 Q\001 +4 0 -1 0 0 0 12 0.0000 4 135 135 1200 2775 Y\001 diff --git a/sourcecodes/bnt-master/docs/Figures/mixexp.gif b/sourcecodes/bnt-master/docs/Figures/mixexp.gif new file mode 100644 index 00000000..1b76e58c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mixexp.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mixexp_after.gif b/sourcecodes/bnt-master/docs/Figures/mixexp_after.gif new file mode 100644 index 00000000..06525bf4 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mixexp_after.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mixexp_before.gif b/sourcecodes/bnt-master/docs/Figures/mixexp_before.gif new file mode 100644 index 00000000..6eef2add --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mixexp_before.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/mixexp_data.gif b/sourcecodes/bnt-master/docs/Figures/mixexp_data.gif new file mode 100644 index 00000000..7a770216 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/mixexp_data.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/model_select.png b/sourcecodes/bnt-master/docs/Figures/model_select.png new file mode 100644 index 00000000..25ee5da4 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/model_select.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/qmr.fig b/sourcecodes/bnt-master/docs/Figures/qmr.fig new file mode 100644 index 00000000..0d852876 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/qmr.fig @@ -0,0 +1,50 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 750 2700 309 309 750 2700 1050 2775 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 3225 2700 309 309 3225 2700 3525 2775 +1 3 0 3 0 0 100 0 5 0.000 1 0.0000 4500 2700 309 309 4500 2700 4800 2775 +1 3 0 3 0 0 100 0 -1 0.000 1 0.0000 1950 2700 309 309 1950 2700 2025 3000 +1 3 0 3 0 0 100 0 -1 0.000 1 0.0000 5775 2700 309 309 5775 2700 5850 3000 +1 3 0 3 0 0 100 0 -1 0.000 1 0.0000 4125 1200 309 309 4125 1200 4200 1500 +1 3 0 3 0 0 100 0 -1 0.000 1 0.0000 3000 1200 309 309 3000 1200 3075 1500 +1 3 0 3 0 0 100 0 -1 0.000 1 0.0000 1875 1200 309 309 1875 1200 1950 1500 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 1 2 + 0 0 3.00 180.00 360.00 + 0 0 3.00 180.00 360.00 + 450 3750 6150 3750 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 1 2 + 0 0 3.00 180.00 360.00 + 0 0 3.00 180.00 360.00 + 1350 525 4650 525 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 1725 1500 900 2475 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 2850 1500 1050 2475 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 2925 1500 2100 2400 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 3975 1500 2100 2400 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 3075 1575 3150 2325 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 3150 1575 4200 2400 +2 1 0 3 0 7 100 0 -1 0.000 0 0 7 1 0 2 + 0 0 3.00 180.00 360.00 + 4200 1500 5475 2475 +4 0 0 100 0 0 24 0.0000 4 255 720 1950 4200 4000\001 +4 0 0 100 0 0 24 0.0000 4 330 1575 2850 4200 Symptoms\001 +4 0 0 100 0 0 24 0.0000 4 255 540 2100 375 600\001 +4 0 0 100 0 0 24 0.0000 4 255 1290 2775 375 Diseases\001 diff --git a/sourcecodes/bnt-master/docs/Figures/qmr.gif b/sourcecodes/bnt-master/docs/Figures/qmr.gif new file mode 100644 index 00000000..f32294fe --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/qmr.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/qmr.jpg b/sourcecodes/bnt-master/docs/Figures/qmr.jpg new file mode 100644 index 00000000..40a6c57c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/qmr.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/qmr.rnd.jpg b/sourcecodes/bnt-master/docs/Figures/qmr.rnd.jpg new file mode 100644 index 00000000..57026cd7 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/qmr.rnd.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/rainer_dbn.jpg b/sourcecodes/bnt-master/docs/Figures/rainer_dbn.jpg new file mode 100644 index 00000000..a546ff4f --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/rainer_dbn.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/rainer_tied.fig b/sourcecodes/bnt-master/docs/Figures/rainer_tied.fig new file mode 100644 index 00000000..49cf565c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/rainer_tied.fig @@ -0,0 +1,88 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 3196.875 3000.000 1275 1875 975 3150 1275 4125 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 5596.875 3000.000 3675 1875 3375 3150 3675 4125 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 0 1 0 3849.948 3549.006 2475 3375 3075 2400 4500 2325 + 1 1 1.00 60.00 120.00 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 1500 1875 237 237 1500 1875 1575 2100 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 1500 2775 237 237 1500 2775 1575 3000 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2400 2325 237 237 2400 2325 2475 2550 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2325 3600 237 237 2325 3600 2400 3825 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 1500 4275 237 237 1500 4275 1575 4500 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 3900 1875 237 237 3900 1875 3975 2100 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 3900 2775 237 237 3900 2775 3975 3000 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4800 2325 237 237 4800 2325 4875 2550 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4725 3600 237 237 4725 3600 4800 3825 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 3900 4275 237 237 3900 4275 3975 4500 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 2625 1875 1725 300 2625 1875 4350 2175 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 2625 4275 1800 300 2625 4275 4425 4575 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 3375 3600 1725 300 3375 3600 5100 3900 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 1500 2775 375 300 1500 2775 1875 3075 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 3900 2775 375 300 3900 2775 4275 3075 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 4800 2325 375 300 4800 2325 5175 2625 +1 1 1 2 0 7 100 0 -1 6.000 1 0.0000 2400 2325 375 225 2400 2325 2775 2550 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 1725 1950 2175 2250 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 2400 1725 2700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2400 2550 2400 3375 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 1575 3000 2100 3450 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 3 + 1 1 1.00 60.00 120.00 + 1725 1950 2175 3375 2175 3375 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 3750 1650 4125 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4125 1950 4575 2250 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4575 2400 4125 2700 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4800 2550 4800 3375 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 3975 3000 4500 3450 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 3 + 1 1 1.00 60.00 120.00 + 4125 1950 4575 3375 4575 3375 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4575 3750 4050 4125 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2700 2325 3675 2700 +4 0 0 100 0 2 12 0.0000 4 135 90 1425 1950 1\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2325 2400 2\001 +4 0 0 100 0 2 12 0.0000 4 135 90 1425 2850 3\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2250 3675 4\001 +4 0 0 100 0 2 12 0.0000 4 135 90 1425 4350 5\001 +4 0 0 100 0 2 12 0.0000 4 135 90 3750 1950 6\001 +4 0 0 100 0 2 12 0.0000 4 135 90 4650 2400 7\001 +4 0 0 100 0 2 12 0.0000 4 135 90 3750 2850 8\001 +4 0 0 100 0 2 12 0.0000 4 135 90 4650 3675 9\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3750 4350 10\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2475 2700 E2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 1350 4725 E5\001 +4 0 0 100 0 2 12 0.0000 4 135 210 1725 3900 E4\001 +4 0 0 100 0 2 12 0.0000 4 135 210 1350 3300 E3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 1350 2325 E1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3750 3225 E7\001 +4 0 0 100 0 2 12 0.0000 4 135 210 4800 2850 E6\001 diff --git a/sourcecodes/bnt-master/docs/Figures/rainer_tied.gif b/sourcecodes/bnt-master/docs/Figures/rainer_tied.gif new file mode 100644 index 00000000..19cb8279 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/rainer_tied.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/sar.fig b/sourcecodes/bnt-master/docs/Figures/sar.fig new file mode 100644 index 00000000..f6f54d1d --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sar.fig @@ -0,0 +1,37 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 375 225 2175 2325 +6 375 1350 2100 1800 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 675 1575 300 225 675 1575 975 1800 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1800 1575 300 225 1800 1575 2100 1800 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 975 1575 1500 1575 +-6 +6 450 225 1950 675 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 450 225 825 225 825 675 450 675 450 225 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1575 225 1950 225 1950 675 1575 675 1575 225 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 450 1575 450 +-6 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 675 675 675 1350 +2 1 0 1 -1 0 0 0 2 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1800 675 1800 1350 +2 1 1 1 -1 0 0 0 2 4.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 975 1425 1500 525 +4 0 -1 0 0 0 12 0.0000 4 180 1530 600 2250 Switching AR model\001 +-6 +4 0 -1 0 0 0 12 0.0000 4 165 225 525 525 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1650 525 Q2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 525 1650 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1650 1650 Y2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/sar.gif b/sourcecodes/bnt-master/docs/Figures/sar.gif new file mode 100644 index 00000000..d22e3e2d --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sar.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/skf.fig b/sourcecodes/bnt-master/docs/Figures/skf.fig new file mode 100644 index 00000000..85ff267f --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/skf.fig @@ -0,0 +1,48 @@ +#FIG 3.1 +Landscape +Center +Inches +1200 2 +6 300 300 2550 3375 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 3718.581 1734.122 1725 600 1425 1725 1575 2550 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 2652.330 1587.076 600 600 375 1575 525 2400 + 0 0 1.00 60.00 120.00 +6 1650 1200 2250 2775 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1950 2550 300 225 1950 2550 2250 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1950 1425 300 225 1950 1425 2250 1650 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 1725 1950 2250 +-6 +6 675 300 2175 750 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 300 1050 300 1050 750 675 750 675 300 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1800 300 2175 300 2175 750 1800 750 1800 300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 525 1800 525 +-6 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 825 2550 300 225 825 2550 1125 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 825 1425 300 225 825 1425 1125 1650 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 750 825 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 750 1950 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 1425 1650 1425 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1725 825 2250 +4 0 -1 0 0 0 12 0.0000 4 180 1740 750 3300 Switching Kalman filter\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 750 600 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1875 600 Q2\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 675 1500 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 210 1800 1500 X2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 675 2625 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1800 2625 Y2\001 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/skf.gif b/sourcecodes/bnt-master/docs/Figures/skf.gif new file mode 100644 index 00000000..aa784e11 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/skf.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/skf3.fig b/sourcecodes/bnt-master/docs/Figures/skf3.fig new file mode 100644 index 00000000..5940436c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/skf3.fig @@ -0,0 +1,71 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 300 300 3750 3450 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 2100.000 1950.000 675 975 375 1875 675 2925 + 1 1 2.00 120.00 240.00 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 3300.000 1950.000 1875 975 1575 1875 1875 2925 + 1 1 2.00 120.00 240.00 +5 1 0 2 0 0 50 0 -1 0.000 0 1 1 0 4500.000 1950.000 3075 975 2775 1875 3075 2925 + 1 1 2.00 120.00 240.00 +1 3 0 2 0 0 50 0 -1 0.000 1 0.0000 975 1875 335 335 975 1875 1125 2175 +1 3 0 2 0 0 50 0 -1 0.000 1 0.0000 3375 1875 335 335 3375 1875 3525 2175 +1 3 0 2 0 0 50 0 20 0.000 1 0.0000 975 3075 335 335 975 3075 1125 3375 +1 3 0 2 0 0 50 0 20 0.000 1 0.0000 2175 3075 335 335 2175 3075 2325 3375 +1 3 0 2 0 0 50 0 20 0.000 1 0.0000 3375 3075 335 335 3375 3075 3525 3375 +1 3 0 2 0 0 50 0 -1 0.000 1 0.0000 2175 1875 335 335 2175 1875 2325 2175 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 675 375 1275 375 1275 975 675 975 675 375 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 1875 375 2475 375 2475 975 1875 975 1875 375 +2 2 0 2 0 0 50 0 -1 0.000 0 0 7 0 0 5 + 3075 375 3675 375 3675 975 3075 975 3075 375 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 975 975 975 1575 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 975 2175 975 2775 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 2175 975 2175 1575 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 2175 2175 2175 2775 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 3375 975 3375 1575 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 3375 2175 3375 2775 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 1275 1875 1875 1875 +2 1 0 2 0 0 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 120.00 240.00 + 2475 1875 3075 1875 +4 0 0 50 0 0 24 0.0000 4 255 225 750 750 Z\001 +4 0 0 50 0 0 24 0.0000 4 255 225 1950 750 Z\001 +4 0 0 50 0 0 24 0.0000 4 255 180 2175 900 2\001 +4 0 0 50 0 0 24 0.0000 4 255 225 3150 750 Z\001 +4 0 0 50 0 0 24 0.0000 4 255 180 3375 900 3\001 +4 0 0 50 0 0 24 0.0000 4 255 270 825 2025 X\001 +4 0 0 50 0 0 24 0.0000 4 255 180 1050 2175 1\001 +4 0 0 50 0 0 24 0.0000 4 255 180 2250 2175 2\001 +4 0 0 50 0 0 24 0.0000 4 255 180 3450 2175 3\001 +4 0 0 50 0 0 24 0.0000 4 255 270 3225 2025 X\001 +4 0 0 50 0 0 24 0.0000 4 255 270 2025 2025 X\001 +4 0 7 50 0 0 24 0.0000 4 255 240 825 3150 Y\001 +4 0 7 50 0 0 24 0.0000 4 255 240 2025 3150 Y\001 +4 0 7 50 0 0 24 0.0000 4 255 240 3225 3150 Y\001 +4 0 7 50 0 0 24 0.0000 4 255 180 1050 3300 1\001 +4 0 7 50 0 0 24 0.0000 4 255 180 2175 3375 2\001 +4 0 7 50 0 0 24 0.0000 4 255 180 3375 3300 3\001 +4 0 0 50 0 0 24 0.0000 4 255 180 975 900 1\001 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/skf3_nobold.fig b/sourcecodes/bnt-master/docs/Figures/skf3_nobold.fig new file mode 100644 index 00000000..0d071170 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/skf3_nobold.fig @@ -0,0 +1,66 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 2652.330 1587.076 600 600 375 1575 525 2400 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 3718.581 1734.122 1725 600 1425 1725 1575 2550 + 0 0 1.00 60.00 120.00 +5 1 0 1 -1 -1 0 0 -1 0.000 0 1 1 0 4843.581 1734.122 2850 600 2550 1725 2700 2550 + 0 0 1.00 60.00 120.00 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 825 2550 300 225 825 2550 1125 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 825 1425 300 225 825 1425 1125 1650 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1950 2550 300 225 1950 2550 2250 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1950 1425 300 225 1950 1425 2250 1650 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 3075 1425 300 225 3075 1425 3375 1650 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3075 2550 300 225 3075 2550 3375 2775 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 750 825 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1725 825 2250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 750 1950 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 1425 1650 1425 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 750 3075 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2250 1425 2775 1425 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 300 1050 300 1050 750 675 750 675 300 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1800 300 2175 300 2175 750 1800 750 1800 300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 525 1800 525 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 1725 1950 2250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 1725 3075 2250 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2850 300 3225 300 3225 750 2850 750 2850 300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2100 525 2850 525 +4 0 -1 0 0 0 12 0.0000 4 135 225 675 1500 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 675 2625 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1800 1500 X2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1800 2625 Y2\001 +4 0 0 50 0 0 12 0.0000 4 135 225 2850 1500 X3\001 +4 0 0 0 0 0 12 0.0000 4 135 225 2925 2625 Y3\001 +4 0 0 50 0 0 12 0.0000 4 135 210 750 600 Z1\001 +4 0 0 50 0 0 12 0.0000 4 135 210 1875 600 Z2\001 +4 0 0 50 0 0 12 0.0000 4 135 210 2925 600 Z3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/skf3_nosolid.fig b/sourcecodes/bnt-master/docs/Figures/skf3_nosolid.fig new file mode 100644 index 00000000..60609cb2 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/skf3_nosolid.fig @@ -0,0 +1,66 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 2652.330 1587.076 600 600 375 1575 525 2400 + 1 1 2.00 120.00 240.00 +5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 3718.581 1734.122 1725 600 1425 1725 1575 2550 + 1 1 2.00 120.00 240.00 +5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 4843.581 1734.122 2850 600 2550 1725 2700 2550 + 1 1 2.00 120.00 240.00 +1 1 0 2 0 7 50 0 -1 0.000 1 0.0000 1950 1425 300 225 1950 1425 2250 1650 +1 1 0 2 0 7 50 0 -1 0.000 1 0.0000 825 1425 300 225 825 1425 1125 1650 +1 1 0 2 0 0 50 0 7 0.000 1 0.0000 825 2550 300 225 825 2550 1125 2775 +1 1 0 2 0 0 50 0 7 0.000 1 0.0000 3075 2550 300 225 3075 2550 3375 2775 +1 1 0 2 0 0 50 0 7 0.000 1 0.0000 1950 2550 300 225 1950 2550 2250 2775 +1 1 0 2 0 7 50 0 -1 0.000 1 0.0000 3075 1425 300 225 3075 1425 3375 1650 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 825 1725 825 2250 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 1125 1425 1650 1425 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 1800 300 2175 300 2175 750 1800 750 1800 300 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 2100 525 2850 525 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 2850 300 3225 300 3225 750 2850 750 2850 300 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 1950 1725 1950 2250 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 2250 1425 2775 1425 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 1950 750 1950 1200 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 3075 750 3075 1200 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 3075 1725 3075 2250 +2 1 0 2 0 7 50 0 -1 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 825 750 825 1200 +2 2 0 2 0 7 50 0 -1 0.000 0 0 7 0 0 5 + 675 300 1050 300 1050 750 675 750 675 300 +2 1 0 2 0 0 50 0 7 0.000 0 0 7 1 0 2 + 1 1 2.00 120.00 240.00 + 1050 525 1800 525 +4 0 0 50 0 2 12 0.0000 4 135 225 675 2625 Y1\001 +4 0 0 50 0 2 12 0.0000 4 135 225 2925 2625 Y3\001 +4 0 0 50 0 2 12 0.0000 4 135 225 675 1500 X1\001 +4 0 0 50 0 2 12 0.0000 4 135 225 1800 1500 X2\001 +4 0 0 50 0 2 12 0.0000 4 135 225 2850 1500 X3\001 +4 0 0 50 0 2 12 0.0000 4 135 210 750 600 Z1\001 +4 0 0 50 0 2 12 0.0000 4 135 210 1875 600 Z2\001 +4 0 0 50 0 2 12 0.0000 4 135 210 2925 600 Z3\001 +4 0 0 50 0 2 12 0.0000 4 135 225 1875 2625 Y2\001 diff --git a/sourcecodes/bnt-master/docs/Figures/skf3_polytree.fig b/sourcecodes/bnt-master/docs/Figures/skf3_polytree.fig new file mode 100644 index 00000000..eea5df04 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/skf3_polytree.fig @@ -0,0 +1,60 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 825 2550 300 225 825 2550 1125 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 825 1425 300 225 825 1425 1125 1650 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 1950 2550 300 225 1950 2550 2250 2775 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 1950 1425 300 225 1950 1425 2250 1650 +1 1 0 1 -1 -1 0 0 -1 0.000 1 0.0000 3075 1425 300 225 3075 1425 3375 1650 +1 1 0 1 -1 0 0 0 2 0.000 1 0.0000 3075 2550 300 225 3075 2550 3375 2775 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 750 825 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 825 1725 825 2250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 750 1950 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1125 1425 1650 1425 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 750 3075 1200 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2250 1425 2775 1425 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 675 300 1050 300 1050 750 675 750 675 300 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 1800 300 2175 300 2175 750 1800 750 1800 300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1050 525 1800 525 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 1725 1950 2250 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3075 1725 3075 2250 +2 2 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 5 + 2850 300 3225 300 3225 750 2850 750 2850 300 +2 1 0 1 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2100 525 2850 525 +4 0 -1 0 0 0 12 0.0000 4 165 225 750 600 Q1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 675 1500 X1\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 675 2625 Y1\001 +4 0 -1 0 0 0 12 0.0000 4 165 225 1875 600 Q2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1800 1500 X2\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 1800 2625 Y2\001 +4 0 0 50 0 0 12 0.0000 4 165 225 2925 600 Q3\001 +4 0 0 50 0 0 12 0.0000 4 135 225 2850 1500 X3\001 +4 0 0 0 0 0 12 0.0000 4 135 225 2925 2625 Y3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler.fig b/sourcecodes/bnt-master/docs/Figures/sprinkler.fig new file mode 100644 index 00000000..6e343f52 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler.fig @@ -0,0 +1,73 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 3675 1275 6000 3450 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 4125 2400 450 225 4125 2400 4575 2625 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 4800 1500 450 225 4800 1500 5250 1725 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 5550 2400 450 225 5550 2400 6000 2625 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 4875 3225 450 225 4875 3225 5325 3450 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4650 1725 4200 2175 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5025 1725 5400 2175 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4350 2625 4800 3000 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5475 2700 5100 3000 +4 0 -1 0 0 0 12 0.0000 4 180 525 4575 1575 Cloudy\001 +4 0 -1 0 0 0 12 0.0000 4 180 675 3825 2475 Sprinkler\001 +4 0 -1 0 0 0 12 0.0000 4 135 345 5400 2475 Rain\001 +4 0 -1 0 0 0 12 0.0000 4 135 795 4500 3300 WetGrass\001 +-6 +6 3975 3900 6000 5775 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 3975 4275 6000 4275 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 1 + 4425 3975 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 4485 3960 4485 5730 +4 0 -1 0 0 0 12 0.0000 4 135 300 4050 4500 F F\001 +4 0 -1 0 0 0 12 0.0000 4 135 810 4650 4500 1.0 0.0\001 +4 0 -1 0 0 0 12 0.0000 4 135 300 4050 4875 T F\001 +4 0 -1 0 0 0 12 0.0000 4 135 300 4050 5250 F T\001 +4 0 -1 0 0 0 12 0.0000 4 135 345 4050 5625 T T\001 +4 0 -1 0 0 0 12 0.0000 4 135 855 4650 4875 0.1 0.9\001 +4 0 -1 0 0 0 12 0.0000 4 135 855 4650 5250 0.1 0.9\001 +4 0 -1 0 0 0 12 0.0000 4 135 900 4650 5625 0.01 0.99\001 +4 0 -1 0 0 0 12 0.0000 4 180 1785 4125 4230 S R P(W=F) P(W=T)\001 +-6 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 4125 525 5700 525 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 1650 2625 3375 2625 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 2025 2325 2025 3300 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 6525 2700 8250 2700 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 0 0 2 + 6900 2400 6900 3450 +4 0 -1 0 0 0 12 0.0000 4 180 1260 4290 450 P(C=F) P(C=T)\001 +4 0 -1 0 0 0 12 0.0000 4 135 855 4425 825 0.5 0.5\001 +4 0 -1 0 0 0 12 0.0000 4 135 105 1725 2925 F\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2175 2925 0.5\001 +4 0 -1 0 0 0 12 0.0000 4 135 225 2775 2925 0.5\001 +4 0 -1 0 0 0 12 0.0000 4 135 105 1725 3225 T\001 +4 0 -1 0 0 0 12 0.0000 4 135 900 2175 3225 0.9 0.1\001 +4 0 -1 0 0 0 12 0.0000 4 135 120 6600 2625 C\001 +4 0 -1 0 0 0 12 0.0000 4 135 105 6600 3000 F\001 +4 0 -1 0 0 0 12 0.0000 4 135 105 6525 3375 T\001 +4 0 -1 0 0 0 12 0.0000 4 135 945 7050 3375 0.2 0.8\001 +4 0 -1 0 0 0 12 0.0000 4 135 945 7050 3000 0.8 0.2\001 +4 0 0 100 0 0 12 0.0000 4 135 120 1650 2550 C\001 +4 0 0 100 0 0 12 0.0000 4 180 1185 2100 2550 P(S=F) P(S=T)\001 +4 0 0 100 0 0 12 0.0000 4 180 1215 6975 2625 P(R=F) P(R=T)\001 diff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler.gif b/sourcecodes/bnt-master/docs/Figures/sprinkler.gif new file mode 100644 index 00000000..c3520ef7 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler.jpg b/sourcecodes/bnt-master/docs/Figures/sprinkler.jpg new file mode 100644 index 00000000..65ebf1e3 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler.noparams.fig b/sourcecodes/bnt-master/docs/Figures/sprinkler.noparams.fig new file mode 100644 index 00000000..d71d8102 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler.noparams.fig @@ -0,0 +1,31 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +6 600 300 2925 2475 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 1050 1425 450 225 1050 1425 1500 1650 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 1725 525 450 225 1725 525 2175 750 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 2475 1425 450 225 2475 1425 2925 1650 +1 1 0 1 -1 7 0 0 -1 0.000 1 0.0000 1800 2250 450 225 1800 2250 2250 2475 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1575 750 1125 1200 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1950 750 2325 1200 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 1275 1650 1725 2025 +2 1 0 1 -1 7 0 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2400 1725 2025 2025 +4 0 -1 0 0 0 12 0.0000 4 180 525 1500 600 Cloudy\001 +4 0 -1 0 0 0 12 0.0000 4 180 675 750 1500 Sprinkler\001 +4 0 -1 0 0 0 12 0.0000 4 135 345 2325 1500 Rain\001 +4 0 -1 0 0 0 12 0.0000 4 135 795 1425 2325 WetGrass\001 +-6 diff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.gif b/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.gif new file mode 100644 index 00000000..25a444a2 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.jpg b/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.jpg new file mode 100644 index 00000000..07894e5f --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.jpg Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.png b/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.png new file mode 100644 index 00000000..6194915e --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/sprinkler_bar.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/water3.cts.fig b/sourcecodes/bnt-master/docs/Figures/water3.cts.fig new file mode 100644 index 00000000..da9022b4 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3.cts.fig @@ -0,0 +1,266 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 3229.069 1880.389 1950 3450 1275 1350 1875 375 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 1 1 0 4200.000 6900.000 1950 5700 1650 6900 1950 8100 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 1 1 0 3787.500 6900.000 1950 4950 1125 7200 1950 8850 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 7232.812 1950.000 5775 3450 5175 1575 5775 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 1 1 0 7387.500 6975.000 5775 5700 5400 7500 5775 8250 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 1 1 0 7188.461 6911.539 5775 4950 5175 8250 5850 8925 + 1 1 1.00 60.00 120.00 +6 2100 75 4425 9225 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 5282.812 1950.000 3825 3450 3225 1575 3825 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 1 1 0 5437.500 6975.000 3825 5700 3450 7500 3825 8250 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 1 1 0 5238.461 6911.539 3825 4950 3225 8250 3900 8925 + 1 1 1.00 60.00 120.00 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4125 8250 237 237 4125 8250 4200 8475 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4125 8925 237 237 4125 8925 4200 9150 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4050 375 237 237 4050 375 4125 600 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4050 1050 237 237 4050 1050 4125 1275 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7050 3825 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7125 3825 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7200 3825 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6525 3825 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6450 3825 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5700 3825 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5025 3825 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5625 3825 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2850 3825 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4350 3825 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5550 3825 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4950 3825 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3600 3825 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4200 3825 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4800 3825 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3525 3825 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4125 3825 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3450 3825 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2775 3825 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2025 3825 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2700 3825 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 1950 3825 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2175 4275 1725 3825 1725 3825 2175 4275 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2925 4275 2475 3825 2475 3825 2925 4275 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 3675 4275 3225 3825 3225 3825 3675 4275 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 4425 4275 3975 3825 3975 3825 4425 4275 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5175 4275 4725 3825 4725 3825 5175 4275 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5925 4275 5475 3825 5475 3825 5925 4275 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 6675 4275 6225 3825 6225 3825 6675 4275 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 7425 4275 6975 3825 6975 3825 7425 4275 7425 +2 1 0 2 0 0 100 0 3 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 1650 2175 1200 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 1725 4050 1275 +-6 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 8250 237 237 2175 8250 2250 8475 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 8925 237 237 2175 8925 2250 9150 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 300 237 237 2175 300 2250 525 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 975 237 237 2175 975 2250 1200 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6075 8250 237 237 6075 8250 6150 8475 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6075 8925 237 237 6075 8925 6150 9150 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6000 375 237 237 6000 375 6075 600 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6000 1050 237 237 6000 1050 6075 1275 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 7425 2400 6975 1950 6975 1950 7425 2400 7425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 6675 2400 6225 1950 6225 1950 6675 2400 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5925 2400 5475 1950 5475 1950 5925 2400 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5175 2400 4725 1950 4725 1950 5175 2400 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 4425 2400 3975 1950 3975 1950 4425 2400 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 3675 2400 3225 1950 3225 1950 3675 2400 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2175 2400 1725 1950 1725 1950 2175 2400 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2925 2400 2475 1950 2475 1950 2925 2400 2925 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7050 5775 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7125 5775 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7200 5775 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6525 5775 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6450 5775 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5700 5775 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5025 5775 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5625 5775 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2850 5775 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4350 5775 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5550 5775 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4950 5775 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3600 5775 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4200 5775 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4800 5775 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3525 5775 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4125 5775 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3450 5775 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2775 5775 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2025 5775 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2700 5775 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 1950 5775 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2175 6225 1725 5775 1725 5775 2175 6225 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2925 6225 2475 5775 2475 5775 2925 6225 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 3675 6225 3225 5775 3225 5775 3675 6225 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 4425 6225 3975 5775 3975 5775 4425 6225 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5175 6225 4725 5775 4725 5775 5175 6225 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5925 6225 5475 5775 5475 5775 5925 6225 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 6675 6225 6225 5775 6225 5775 6675 6225 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 7425 6225 6975 5775 6975 5775 7425 6225 7425 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 1725 6000 1275 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 2025 A1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 2025 A2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 2025 A3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 2775 B1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 2775 B2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 2775 B3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 3525 C1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 3525 C2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 3525 C3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 4275 D1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 4275 D2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 4275 D3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 5025 E1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 5025 E2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 5025 E3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 5775 F2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 5775 F1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 5775 F3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 6525 G1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3975 6525 G2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 6525 G3\001 +4 0 0 100 0 2 12 0.0000 4 135 240 3900 7275 H2\001 +4 0 0 100 0 2 12 0.0000 4 135 240 5925 7275 H3\001 +4 0 0 100 0 2 12 0.0000 4 135 240 2025 7275 H1\001 +4 0 0 100 0 2 12 0.0000 4 135 360 1950 1050 YA1\001 +4 0 0 100 0 2 12 0.0000 4 135 360 3900 1125 YA2\001 +4 0 0 100 0 2 12 0.0000 4 135 360 5850 1125 YA3\001 +4 0 0 100 0 2 12 0.0000 4 135 345 1950 375 YC1\001 +4 0 0 100 0 2 12 0.0000 4 135 345 3900 450 YC2\001 +4 0 0 100 0 2 12 0.0000 4 135 345 5850 450 YC3\001 +4 0 0 100 0 2 12 0.0000 4 135 345 2025 8325 YE1\001 +4 0 0 100 0 2 12 0.0000 4 135 345 3975 8325 YE2\001 +4 0 0 100 0 2 12 0.0000 4 135 345 5925 8325 YE3\001 +4 0 0 100 0 2 12 0.0000 4 135 345 3975 9000 YF2\001 +4 0 0 100 0 2 12 0.0000 4 135 345 5925 9000 YF3\001 +4 0 0 100 0 2 12 0.0000 4 135 345 2025 9000 YF1\001 diff --git a/sourcecodes/bnt-master/docs/Figures/water3.fig b/sourcecodes/bnt-master/docs/Figures/water3.fig new file mode 100644 index 00000000..23bc3e66 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3.fig @@ -0,0 +1,279 @@ +#FIG 3.2 +Portrait +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 7232.812 1950.000 5775 3450 5175 1575 5775 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 5282.812 1950.000 3825 3450 3225 1575 3825 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 5445.833 6562.500 1950 4200 1275 7200 1950 8925 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 7595.833 6637.500 3825 4275 3150 6825 3825 9000 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 8419.351 6558.096 5700 4200 5100 7950 5775 9000 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 3332.812 2025.000 1875 3525 1275 1650 1875 525 + 1 1 1.00 60.00 120.00 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 7425 2400 6975 1950 6975 1950 7425 2400 7425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 6675 2400 6225 1950 6225 1950 6675 2400 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5925 2400 5475 1950 5475 1950 5925 2400 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5175 2400 4725 1950 4725 1950 5175 2400 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 4425 2400 3975 1950 3975 1950 4425 2400 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 3675 2400 3225 1950 3225 1950 3675 2400 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2175 2400 1725 1950 1725 1950 2175 2400 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2925 2400 2475 1950 2475 1950 2925 2400 2925 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7050 5775 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7125 5775 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7200 5775 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6525 5775 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6450 5775 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5700 5775 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5025 5775 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5625 5775 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2850 5775 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4350 5775 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5550 5775 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4950 5775 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3600 5775 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4200 5775 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4800 5775 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3525 5775 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4125 5775 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3450 5775 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2775 5775 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2025 5775 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2700 5775 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 1950 5775 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2175 6225 1725 5775 1725 5775 2175 6225 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2925 6225 2475 5775 2475 5775 2925 6225 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 3675 6225 3225 5775 3225 5775 3675 6225 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 4425 6225 3975 5775 3975 5775 4425 6225 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5175 6225 4725 5775 4725 5775 5175 6225 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5925 6225 5475 5775 5475 5775 5925 6225 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 6675 6225 6225 5775 6225 5775 6675 6225 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 7425 6225 6975 5775 6975 5775 7425 6225 7425 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 1725 6000 1275 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7050 3825 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7125 3825 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7200 3825 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6525 3825 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6450 3825 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5700 3825 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5025 3825 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5625 3825 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2850 3825 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4350 3825 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5550 3825 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4950 3825 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3600 3825 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4200 3825 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4800 3825 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3525 3825 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4125 3825 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3450 3825 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2775 3825 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2025 3825 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2700 3825 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 1950 3825 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2175 4275 1725 3825 1725 3825 2175 4275 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2925 4275 2475 3825 2475 3825 2925 4275 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 3675 4275 3225 3825 3225 3825 3675 4275 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 4425 4275 3975 3825 3975 3825 4425 4275 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5175 4275 4725 3825 4725 3825 5175 4275 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5925 4275 5475 3825 5475 3825 5925 4275 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 6675 4275 6225 3825 6225 3825 6675 4275 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 7425 4275 6975 3825 6975 3825 7425 4275 7425 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 1725 4050 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 1275 6225 825 5775 825 5775 1275 6225 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 600 6225 150 5775 150 5775 600 6225 600 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 600 4275 150 3825 150 3825 600 4275 600 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 1275 4275 825 3825 825 3825 1275 4275 1275 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 7425 2175 7950 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 8400 2400 7950 1950 7950 1950 8400 2400 8400 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 8400 4275 7950 3825 7950 3825 8400 4275 8400 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 8400 6225 7950 5775 7950 5775 8400 6225 8400 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 7425 4050 7950 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 7425 6000 7950 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 9150 2400 8700 1950 8700 1950 9150 2400 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 9150 4275 8700 3825 8700 3825 9150 4275 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 9150 6225 8700 5775 8700 5775 9150 6225 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 1200 2400 750 1950 750 1950 1200 2400 1200 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 600 2400 150 1950 150 1950 600 2400 600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 1650 2175 1200 +4 0 0 100 0 2 12 0.0000 4 135 90 2100 2025 1\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2100 2775 2\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2100 3525 3\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2025 4275 4\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2025 5025 5\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2025 5775 6\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2025 6525 7\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2025 7275 8\001 +4 0 0 100 0 2 12 0.0000 4 135 180 2025 9000 11\001 +4 0 0 100 0 2 12 0.0000 4 135 180 2100 8250 12\001 +4 0 0 100 0 2 12 0.0000 4 135 90 2100 1050 9\001 +4 0 0 100 0 2 12 0.0000 4 135 180 2100 450 10\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 2025 13\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 2775 14\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 3525 15\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 4275 16\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 5025 17\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 5775 18\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 6525 19\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 7275 20\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 1125 21\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3975 450 22\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 9000 23\001 +4 0 0 100 0 2 12 0.0000 4 135 180 3900 8250 24\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 2025 25\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 2775 26\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 3525 27\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 4275 28\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 5025 29\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 5775 30\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 6525 31\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 7275 32\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 1125 33\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5925 450 34\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 8250 36\001 +4 0 0 100 0 2 12 0.0000 4 135 180 5850 9000 35\001 diff --git a/sourcecodes/bnt-master/docs/Figures/water3.gif b/sourcecodes/bnt-master/docs/Figures/water3.gif new file mode 100644 index 00000000..9e8ecadd --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/water3_75.gif b/sourcecodes/bnt-master/docs/Figures/water3_75.gif new file mode 100644 index 00000000..e4e477eb --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3_75.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/Figures/water3_circle.fig b/sourcecodes/bnt-master/docs/Figures/water3_circle.fig new file mode 100644 index 00000000..a29b9a8b --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3_circle.fig @@ -0,0 +1,231 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 3229.069 1880.389 1950 3450 1275 1350 1875 375 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 7232.812 1950.000 5775 3450 5175 1575 5775 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 5282.812 1950.000 3825 3450 3225 1575 3825 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 6931.999 6519.041 1875 4275 1425 7050 1950 8925 + 0 0 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 7187.500 6562.500 3825 4275 3150 6075 3825 8850 + 0 0 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 9787.500 6592.500 5775 4275 5175 6150 5700 8775 + 0 0 1.00 60.00 120.00 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 8925 237 237 2175 8925 2250 9150 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 300 237 237 2175 300 2250 525 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 975 237 237 2175 975 2250 1200 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2100 1950 237 237 2100 1950 2175 2175 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2175 2700 237 237 2175 2700 2250 2925 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2175 3450 237 237 2175 3450 2250 3675 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2100 4200 237 237 2100 4200 2175 4425 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2175 4950 237 237 2175 4950 2250 5175 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2175 5700 237 237 2175 5700 2250 5925 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2175 6450 237 237 2175 6450 2250 6675 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 2175 7125 237 237 2175 7125 2250 7350 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4050 375 237 237 4050 375 4125 600 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4050 1050 237 237 4050 1050 4125 1275 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 1950 237 237 4050 1950 4125 2175 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 2700 237 237 4050 2700 4125 2925 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 3450 237 237 4050 3450 4125 3675 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 4200 237 237 4050 4200 4125 4425 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 4950 237 237 4050 4950 4125 5175 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 5700 237 237 4050 5700 4125 5925 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 6450 237 237 4050 6450 4125 6675 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 4050 7200 237 237 4050 7200 4125 7425 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 1950 237 237 6000 1950 6075 2175 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 2700 237 237 6000 2700 6075 2925 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 4200 237 237 6000 4200 6075 4425 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 4875 237 237 6000 4875 6075 5100 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 5700 237 237 6000 5700 6075 5925 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 6450 237 237 6000 6450 6075 6675 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 7200 237 237 6000 7200 6075 7425 +1 3 0 2 0 7 100 0 -1 0.000 1 0.0000 6000 3450 237 237 6000 3450 6075 3675 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 2175 8250 237 237 2175 8250 2250 8475 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4050 8250 237 237 4050 8250 4125 8475 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 4050 8925 237 237 4050 8925 4125 9150 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6000 8175 237 237 6000 8175 6075 8400 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6000 8850 237 237 6000 8850 6075 9075 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6000 375 237 237 6000 375 6075 600 +1 3 0 2 0 0 100 0 3 0.000 1 0.0000 6000 1050 237 237 6000 1050 6075 1275 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7050 5775 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7125 5775 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7200 5775 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6525 5775 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6450 5775 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5700 5775 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5025 5775 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5625 5775 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2850 5775 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4350 5775 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5550 5775 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4950 5775 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3600 5775 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4200 5775 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4800 5775 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3525 5775 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4125 5775 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3450 5775 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2775 5775 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2025 5775 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2700 5775 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 1950 5775 1950 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 1725 6000 1275 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7050 3825 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7125 3825 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7200 3825 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6525 3825 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6450 3825 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5700 3825 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5025 3825 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5625 3825 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2850 3825 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4350 3825 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5550 3825 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4950 3825 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3600 3825 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4200 3825 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4800 3825 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3525 3825 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4125 3825 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3450 3825 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2775 3825 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2025 3825 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2700 3825 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 1950 3825 1950 +2 1 0 2 0 0 100 0 3 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 1650 2175 1200 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 1725 4050 1275 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 2175 7350 2175 7950 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4050 7425 4050 8025 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6000 7425 6000 7950 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 2025 A1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 2025 A2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 2025 A3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 2775 B1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 2775 B2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 2775 B3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 3525 C1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 3525 C2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 3525 C3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 4275 D1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 4275 D2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 4275 D3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 5025 E1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 5025 E2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 5025 E3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 5775 F2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 5775 F1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 5775 F3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 6525 G1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3975 6525 G2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 6525 G3\001 +4 0 0 100 0 2 12 0.0000 4 135 240 3900 7275 H2\001 +4 0 0 100 0 2 12 0.0000 4 135 240 5925 7275 H3\001 +4 0 0 100 0 2 12 0.0000 4 135 240 2025 7275 H1\001 diff --git a/sourcecodes/bnt-master/docs/Figures/water3_named_nodes.fig b/sourcecodes/bnt-master/docs/Figures/water3_named_nodes.fig new file mode 100644 index 00000000..362c4aaa --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3_named_nodes.fig @@ -0,0 +1,279 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 3229.069 1880.389 1950 3450 1275 1350 1875 375 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 7232.812 1950.000 5775 3450 5175 1575 5775 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 5282.812 1950.000 3825 3450 3225 1575 3825 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 5445.833 6562.500 1950 4200 1275 7200 1950 8925 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 7595.833 6637.500 3825 4275 3150 6825 3825 9000 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 8419.351 6558.096 5700 4200 5100 7950 5775 9000 + 1 1 1.00 60.00 120.00 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 7425 2400 6975 1950 6975 1950 7425 2400 7425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 6675 2400 6225 1950 6225 1950 6675 2400 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5925 2400 5475 1950 5475 1950 5925 2400 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5175 2400 4725 1950 4725 1950 5175 2400 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 4425 2400 3975 1950 3975 1950 4425 2400 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 3675 2400 3225 1950 3225 1950 3675 2400 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2175 2400 1725 1950 1725 1950 2175 2400 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2925 2400 2475 1950 2475 1950 2925 2400 2925 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7050 5775 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7125 5775 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7200 5775 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6525 5775 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6450 5775 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5700 5775 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5025 5775 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5625 5775 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2850 5775 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4350 5775 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5550 5775 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4950 5775 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3600 5775 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4200 5775 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4800 5775 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3525 5775 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4125 5775 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3450 5775 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2775 5775 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2025 5775 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2700 5775 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 1950 5775 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2175 6225 1725 5775 1725 5775 2175 6225 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2925 6225 2475 5775 2475 5775 2925 6225 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 3675 6225 3225 5775 3225 5775 3675 6225 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 4425 6225 3975 5775 3975 5775 4425 6225 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5175 6225 4725 5775 4725 5775 5175 6225 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5925 6225 5475 5775 5475 5775 5925 6225 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 6675 6225 6225 5775 6225 5775 6675 6225 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 7425 6225 6975 5775 6975 5775 7425 6225 7425 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 1725 6000 1275 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7050 3825 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7125 3825 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7200 3825 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6525 3825 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6450 3825 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5700 3825 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5025 3825 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5625 3825 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2850 3825 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4350 3825 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5550 3825 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4950 3825 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3600 3825 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4200 3825 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4800 3825 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3525 3825 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4125 3825 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3450 3825 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2775 3825 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2025 3825 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2700 3825 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 1950 3825 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2175 4275 1725 3825 1725 3825 2175 4275 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2925 4275 2475 3825 2475 3825 2925 4275 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 3675 4275 3225 3825 3225 3825 3675 4275 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 4425 4275 3975 3825 3975 3825 4425 4275 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5175 4275 4725 3825 4725 3825 5175 4275 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5925 4275 5475 3825 5475 3825 5925 4275 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 6675 4275 6225 3825 6225 3825 6675 4275 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 7425 4275 6975 3825 6975 3825 7425 4275 7425 +2 1 0 2 0 0 100 0 3 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 1650 2175 1200 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 1725 4050 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 1275 6225 825 5775 825 5775 1275 6225 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 600 6225 150 5775 150 5775 600 6225 600 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 600 4275 150 3825 150 3825 600 4275 600 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 1275 4275 825 3825 825 3825 1275 4275 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2325 1200 2325 750 1875 750 1875 1200 2325 1200 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2325 525 2325 75 1875 75 1875 525 2325 525 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 7425 2175 7950 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 8400 2400 7950 1950 7950 1950 8400 2400 8400 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 8400 4275 7950 3825 7950 3825 8400 4275 8400 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 8400 6225 7950 5775 7950 5775 8400 6225 8400 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 7425 4050 7950 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 7425 6000 7950 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 9150 2400 8700 1950 8700 1950 9150 2400 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 9150 4275 8700 3825 8700 3825 9150 4275 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 9150 6225 8700 5775 8700 5775 9150 6225 9150 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 2025 A1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 2025 A2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 2025 A3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 2775 B1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 2775 B2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 2775 B3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 3525 C1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 3525 C2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 3525 C3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 4275 D1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3900 4275 D2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 4275 D3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 5025 E1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 5025 E2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 5025 E3\001 +4 0 0 100 0 2 12 0.0000 4 135 210 3900 5775 F2\001 +4 0 0 100 0 2 12 0.0000 4 135 210 2025 5775 F1\001 +4 0 0 100 0 2 12 0.0000 4 135 210 5850 5775 F3\001 +4 0 0 100 0 2 12 0.0000 4 135 225 2025 6525 G1\001 +4 0 0 100 0 2 12 0.0000 4 135 225 3975 6525 G2\001 +4 0 0 100 0 2 12 0.0000 4 135 225 5850 6525 G3\001 +4 0 0 100 0 2 12 0.0000 4 135 240 3900 7275 H2\001 +4 0 0 100 0 2 12 0.0000 4 135 240 5925 7275 H3\001 +4 0 0 100 0 2 12 0.0000 4 135 240 2025 7275 H1\001 +4 0 0 100 0 2 12 0.0000 4 135 360 1950 1050 YA1\001 +4 0 0 100 0 2 12 0.0000 4 135 360 3900 1125 YA2\001 +4 0 0 100 0 2 12 0.0000 4 135 360 5850 1125 YA3\001 +4 0 0 100 0 2 12 0.0000 4 135 345 1950 375 YC1\001 +4 0 0 100 0 2 12 0.0000 4 135 345 3900 450 YC2\001 +4 0 0 100 0 2 12 0.0000 4 135 345 5850 450 YC3\001 +4 0 0 100 0 2 12 0.0000 4 135 375 2025 8250 YH1\001 +4 0 0 100 0 2 12 0.0000 4 135 375 3900 8250 YH2\001 +4 0 0 100 0 2 12 0.0000 4 135 375 5850 8250 YH3\001 +4 0 0 100 0 2 12 0.0000 4 135 360 2025 9000 YD1\001 +4 0 0 100 0 2 12 0.0000 4 135 360 3900 9000 YD2\001 +4 0 0 100 0 2 12 0.0000 4 135 360 5850 9000 YD3\001 diff --git a/sourcecodes/bnt-master/docs/Figures/water3_nolabels.fig b/sourcecodes/bnt-master/docs/Figures/water3_nolabels.fig new file mode 100644 index 00000000..2e64a25c --- /dev/null +++ b/sourcecodes/bnt-master/docs/Figures/water3_nolabels.fig @@ -0,0 +1,243 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +100.00 +Single +-2 +1200 2 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 7232.812 1950.000 5775 3450 5175 1575 5775 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 5282.812 1950.000 3825 3450 3225 1575 3825 450 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 5445.833 6562.500 1950 4200 1275 7200 1950 8925 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 7595.833 6637.500 3825 4275 3150 6825 3825 9000 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 7 100 0 -1 0.000 0 1 1 0 8419.351 6558.096 5700 4200 5100 7950 5775 9000 + 1 1 1.00 60.00 120.00 +5 1 0 2 0 0 100 0 -1 0.000 0 0 1 0 3332.812 2025.000 1875 3525 1275 1650 1875 525 + 1 1 1.00 60.00 120.00 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 7425 2400 6975 1950 6975 1950 7425 2400 7425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 6675 2400 6225 1950 6225 1950 6675 2400 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5925 2400 5475 1950 5475 1950 5925 2400 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 5175 2400 4725 1950 4725 1950 5175 2400 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 4425 2400 3975 1950 3975 1950 4425 2400 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 3675 2400 3225 1950 3225 1950 3675 2400 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2175 2400 1725 1950 1725 1950 2175 2400 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 2400 2925 2400 2475 1950 2475 1950 2925 2400 2925 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7050 5775 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7125 5775 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 7200 5775 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6525 5775 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 6450 5775 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5700 5775 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5025 5775 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5625 5775 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2850 5775 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4350 5775 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 5550 5775 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4950 5775 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3600 5775 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4200 5775 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4800 5775 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3525 5775 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 4125 5775 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 3450 5775 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2775 5775 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2025 5775 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 2700 5775 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 4350 1950 5775 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2175 6225 1725 5775 1725 5775 2175 6225 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 2925 6225 2475 5775 2475 5775 2925 6225 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 3675 6225 3225 5775 3225 5775 3675 6225 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 4425 6225 3975 5775 3975 5775 4425 6225 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5175 6225 4725 5775 4725 5775 5175 6225 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 5925 6225 5475 5775 5475 5775 5925 6225 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 6675 6225 6225 5775 6225 5775 6675 6225 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 6225 7425 6225 6975 5775 6975 5775 7425 6225 7425 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 1725 6000 1275 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7050 3825 5850 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7125 3825 6525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 7200 3825 7200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6525 3825 7125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 6450 3825 6450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5700 3825 5700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5025 3825 5625 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5625 3825 5025 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2850 3825 6300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4350 3825 5550 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 5550 3825 4350 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4950 3825 4950 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3600 3825 4800 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4200 3825 4200 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4800 3825 3600 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3525 3825 4125 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 4125 3825 3525 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 3450 3825 3450 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2775 3825 3375 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2025 3825 3300 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 2700 3825 2700 +2 1 0 3 -1 -1 0 0 -1 0.000 0 0 -1 1 0 2 + 3 1 1.00 60.00 120.00 + 2400 1950 3825 1950 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2175 4275 1725 3825 1725 3825 2175 4275 2175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 2925 4275 2475 3825 2475 3825 2925 4275 2925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 3675 4275 3225 3825 3225 3825 3675 4275 3675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 4425 4275 3975 3825 3975 3825 4425 4275 4425 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5175 4275 4725 3825 4725 3825 5175 4275 5175 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 5925 4275 5475 3825 5475 3825 5925 4275 5925 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 6675 4275 6225 3825 6225 3825 6675 4275 6675 +2 2 0 3 -1 -1 0 0 -1 0.000 0 0 7 0 0 5 + 4275 7425 4275 6975 3825 6975 3825 7425 4275 7425 +2 1 0 2 0 0 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 1725 4050 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 1275 6225 825 5775 825 5775 1275 6225 1275 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 600 6225 150 5775 150 5775 600 6225 600 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 600 4275 150 3825 150 3825 600 4275 600 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 1275 4275 825 3825 825 3825 1275 4275 1275 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 7425 2175 7950 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 8400 2400 7950 1950 7950 1950 8400 2400 8400 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 8400 4275 7950 3825 7950 3825 8400 4275 8400 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 8400 6225 7950 5775 7950 5775 8400 6225 8400 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 4050 7425 4050 7950 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 6000 7425 6000 7950 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 9150 2400 8700 1950 8700 1950 9150 2400 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 4275 9150 4275 8700 3825 8700 3825 9150 4275 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 6225 9150 6225 8700 5775 8700 5775 9150 6225 9150 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 1200 2400 750 1950 750 1950 1200 2400 1200 +2 2 0 2 0 0 100 0 3 0.000 0 0 7 0 0 5 + 2400 600 2400 150 1950 150 1950 600 2400 600 +2 1 0 2 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2175 1650 2175 1200 diff --git a/sourcecodes/bnt-master/docs/GR03~1.PDF b/sourcecodes/bnt-master/docs/GR03~1.PDF new file mode 100644 index 00000000..8ad8cf49 --- /dev/null +++ b/sourcecodes/bnt-master/docs/GR03~1.PDF Binary files differdiff --git a/sourcecodes/bnt-master/docs/Talks/BNT_mathworks.ppt b/sourcecodes/bnt-master/docs/Talks/BNT_mathworks.ppt new file mode 100644 index 00000000..fb41110b --- /dev/null +++ b/sourcecodes/bnt-master/docs/Talks/BNT_mathworks.ppt Binary files differdiff --git a/sourcecodes/bnt-master/docs/Talks/gR03.ppt b/sourcecodes/bnt-master/docs/Talks/gR03.ppt new file mode 100644 index 00000000..fe7b6f74 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Talks/gR03.ppt Binary files differdiff --git a/sourcecodes/bnt-master/docs/Talks/stair_BNT_mathworks.ppt b/sourcecodes/bnt-master/docs/Talks/stair_BNT_mathworks.ppt new file mode 100644 index 00000000..a400eba0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/Talks/stair_BNT_mathworks.ppt Binary files differdiff --git a/sourcecodes/bnt-master/docs/adj2pajek2.m b/sourcecodes/bnt-master/docs/adj2pajek2.m new file mode 100644 index 00000000..5276a4e3 --- /dev/null +++ b/sourcecodes/bnt-master/docs/adj2pajek2.m @@ -0,0 +1,86 @@ +% ADJ2PAJEK2 Converts an adjacency matrix representation to a Pajek .net read format +% adj2pajek2(adj, filename-stem, 'argname1', argval1, ...) +% +% Set A(i,j)=-1 to get a dotted line +% +% Optional arguments +% +% nodeNames - cell array, defaults to {'v1','v2,...} +% shapes - cell array, defaults to {'ellipse','ellipse',...} +% Choices are 'ellipse', 'box', 'diamond', 'triangle', 'cross', 'empty' +% partition - vector of integers, defaults to [1 1 ... 1] +% This will automatically color-code the vertices by their partition +% +% Run pajek (available from http://vlado.fmf.uni-lj.si/pub/networks/pajek/) +% Choose File->Network->Read from the menu +% Then press ctrl-G (Draw->Draw) +% Optional: additionally load the partition file then press ctrl-P (Draw->partition) +% +% Examples +% A=zeros(5,5);A(1,2)=-1;A(2,1)=-1;A(1,[3 4])=1;A(2,5)=1; +% adj2pajek2(A,'foo') % makes foo.net +% +% adj2pajek2(A,'foo','partition',[1 1 2 2 2]) % makes foo.net and foo.clu +% +% adj2pajek2(A,'foo',... +% 'nodeNames',{'TF1','TF2','G1','G2','G3'},... +% 'shapes',{'box','box','ellipse','ellipse','ellipse'}); +% +% +% The file format is documented on p68 of the pajek manual +% and good examples are on p58, p72 +% +% Written by Kevin Murphy, 30 May 2007 +% Based on adj2pajek by Gergana Bounova +% http://stuff.mit.edu/people/gerganaa/www/matlab/routines.html +% Fixes a small bug (opens files as 'wt' instead of 'w' so it works in windows) +% Also, simplified her code and added some features. + +function []=adj2pajek2(adj,filename, varargin) + +N = length(adj); +for i=1:N + nodeNames{i} = strcat('"v',num2str(i),'"'); + shapes{i} = 'ellipse'; +end + +[nodeNames, shapes, partition] = process_options(varargin, ... + 'nodeNames', nodeNames, 'shapes', shapes, 'partition', []); + +if ~isempty(partition) + fid = fopen(sprintf('%s.clu', filename),'wt','native'); + fprintf(fid,'*Vertices %6i\n',N); + for i=1:N + fprintf(fid, '%d\n', partition(i)); + end + fclose(fid); +end + +fid = fopen(sprintf('%s.net', filename),'wt','native'); + +fprintf(fid,'*Vertices %6i\n',N); +for i=1:N + fprintf(fid,'%3i %s %s\n', i, nodeNames{i}, shapes{i}); +end + +%fprintf(fid,'*Edges\n'); +fprintf(fid,'*Arcs\n'); % directed +for i=1:N + for j=1:N + if adj(i,j) ~= 0 + fprintf(fid,' %4i %4i %2i\n',i,j,adj(i,j)); + end + end +end +fclose(fid) + + +if 0 +adj2pajek2(A,'foo',... + 'nodeNames',{'TF1','TF2','G1','G2','G3'},... + 'shapes',{'box','box','ellipse','ellipse','ellipse'}); + +N = 100; part = ones(1,N); part(intersect(reg.tfidxTest,1:N))=2; +G = reg.Atest(1:N, 1:N)'; +adj2pajek2(G, 'Ecoli100', 'partition', part) +end diff --git a/sourcecodes/bnt-master/docs/bnsoftOld.html b/sourcecodes/bnt-master/docs/bnsoftOld.html new file mode 100644 index 00000000..9da92fa1 --- /dev/null +++ b/sourcecodes/bnt-master/docs/bnsoftOld.html @@ -0,0 +1,1078 @@ +<head> +<TITLE>Software Packages for Graphical Models / Bayesian Networks</TITLE> +</head> + +<body> + +<h1> +Software Packages for Graphical Models / Bayesian Networks +</h1> +<p> +Written by Kevin Murphy. +<br> +Last updated 31 October 2005. + +<h3>Remarks</h3> +<ul> + +<li> +A much more detailed comparison of some of these software packages is +available from Appendix B of +<a href="http://www.csse.monash.edu.au/bai/">Bayesian AI</a>, by +Ann Nicholson and Kevin Korb. +This appendix is +available +<a +href="http://www.csse.monash.edu.au/bai/book/appendix_b.pdf">here</a>, +and is based on the online comparison below. + +<li> +An online <a +href="http://servasi.insa-rouen.fr/enseignement/siteUV/rna/BNT/software.html">French +version</a> of this page is also available (not necessarily up-to-date). + +<!-- +<li> +Matt Turk has made a +<a href="http://ilab.cs.ucsb.edu/BayesNets/Bayesia_Software.pdf">pdf +version of this page</a> (not necessarily up-to-date) with some extra +comments on the packages. +--> +<!-- +<br> +(He also has a +<a href="http://ilab.cs.ucsb.edu/BayesNets">cached copy</a> of several of +the packages listed below, which you can look at if links are down.) +--> + +</ul> + +<h3>What do the headers in the table mean?</h3> + +<ul> +<li> Src = source code included? (N=no) If so, what language? + +<li> API = application program interface included? +(N means the program cannot be integrated into your code, i.e., it +must be run as a standalone executable.) + +<li> Exec = Executable runs on W = Windows (95/98/NT), U = Unix, M = +Mac, or - = any machine with a compiler. + +<li> Cts = are continuous (latent) nodes supported? +G = (conditionally) Gaussians nodes supported analytically, +Cs = continuous nodes supported by sampling, +Cd = continuous nodes supported by discretization, +Cx = continuous nodes supported by some unspecified method, +D = only discrete nodes supported. + + +<li> GUI = Graphical User Interface included? + +<li> Learns parameters? + +<li> Learns structure? CI = means uses conditional independency tests + +<!-- +<li> Sample = sampling methods (e.g., likelihood weighting, MCMC) supported? +--> + +<li> Utility = utility and decision nodes (i.e., influence diagrams) +supported? + +<li> Free? +0 = free (although possibly only for academic use). +$ = commercial software (although most have free versions +which are restricted in +various ways, e.g., the model size is limited, or models cannot be +saved, or there is no API.) + + +<li> Undir? +What kind of graphs are supported? +U = only undirected graphs, +D = only directed graphs, +UD = both undirected and directed, +CG = chain graphs (mixed directed/undirected). + + +<li> Inference = which inference algorithm is used? +jtree = junction tree, +varelim = variable (bucket) elimination, +MH = Metropols Hastings, +G = Gibbs sampling, +IS = importance sampling, +sampling = some other Monte Carlo method, +polytree = Pearl's algorithm restricted to a graph with no cycles, +none = no inference supported (hence the program is only designed for +structure learning from completely observed data) + +<li> Comments. +If in "quotes", I am quoting the authors at their request. + +</ul> + + + +<!-- +The following table is automatically generated as follows +bnsoft_to_html.pl < bnsoft.txt > ! soft.html +Then soft.html is inserted and the last line is tidied up. +--> + +<table border> +<tr> +<td> Name +<td> Authors +<td> Src +<td> API +<td> Exec +<td> Cts +<td> GUI +<td> Params +<td> Struct +<td> Utility +<td> Free +<td> Undir +<td> Inference +<td> Comments + + +<tr> +<td> <!--Name--> <a href=http://www.agenarisk.com > AgenaRisk</a> +<td> <!--Authors-->Agena +<td> <!--Src-->N +<td> <!--API-->Y +<td> <!--Exec-->W,U +<td> <!--Cts-->Cx +<td> <!--GUI-->Y +<td> <!--Params-->Y +<td> <!--Struct-->N +<td> <!--Utility-->N +<td> <!--Free-->$ +<td> <!--Undir-->D +<td> <!--Inference-->JTree +<td> <!--Comments-->Simulation by Dynamic discretisation + +<tr> +<td> <!--Name--> <a href=http://www.lumina.com > Analytica</a> +<td> <!--Authors-->Lumina +<td> <!--Src-->N +<td> <!--API-->Y +<td> <!--Exec-->W,M +<td> <!--Cts-->G +<td> <!--GUI-->Y +<td> <!--Params-->N +<td> <!--Struct-->N +<td> <!--Utility-->Y +<td> <!--Free-->$ +<td> <!--Undir-->D +<td> <!--Inference-->sampling +<td> <!--Comments-->spread sheet compatible + +<tr> +<td> <!--Name--> <a href=http://www.cs.duke.edu/~amink/software/banjo/> Banjo</a> +<td> <!--Authors-->Hartemink +<td> <!--Src-->Java +<td> <!--API-->Y +<td> <!--Exec-->W,U,M +<td> <!--Cts-->Cd +<td> <!--GUI-->N +<td> <!--Params-->N +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->none +<td> <!--Comments--> structure learning of +static or dynamic networks of discrete variables + + +<tr> +<td><a href=http://www.cs.Helsinki.FI/research/fdk/bassist/> Bassist</a> +<td> U. Helsinki +<td> C++ +<td> Y +<td> U +<td> G +<td> N +<td> Y +<td> N +<td> N +<td> 0 +<td> D +<td> MH +<td> Generates C++ for MCMC. + +<tr> +<td><a href=http://www.cs.Helsinki.FI/research/cosco/Projects/NONE/SW/ > Bayda</a> +<td> U. Helsinki +<td> Java +<td> Y +<td> WUM +<td> G +<td> Y +<td> Y +<td> N +<td> N +<td> 0 +<td> D +<td> ? +<td> Bayesian Naive Bayes classifier. + +<tr> +<td><a href=http://www.mbfys.kun.nl/snn/Research/bayesbuilder/ > BayesBuilder</a> +<td> Nijman (U. Nijmegen) +<td> N +<td> N +<td> W +<td> D +<td> Y +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> ? +<td> - + +<tr> +<td> <!--Name--> <a +href="http://www.bayesia.com">BayesiaLab</a> +<td> <!--Authors-->Bayesia Ltd +<td> <!--Src-->N +<td> <!--API-->N +<td> <!--Exec-->- +<td> <!--Cts-->Cd +<td> <!--GUI-->Y +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->$ +<td> <!--Undir-->CG +<td> <!--Inference-->jtree,G +<td> <!--Comments--> +<!-- +Structural learning (association discovery, +classification), Bayesian clustering with analysis report generation, +missing values +processing, essential graphs, analysis toolbox, adaptive +questionnaires, dynamic models +--> +Structural learning, adaptive +questionnaires, dynamic models + + + + +<tr> + +<td><a href=http://www.bayesware.com>Bayesware Discoverer</a> +<td> Bayesware +<td> N +<td> N +<td> WUM +<td> Cd +<td> Y +<td> Y +<td> Y +<td> N +<td> $ +<td> D +<td> ? +<td> Uses bound and collapse for learning with missing data. + +<tr> +<td><a href=http://B-Course.hiit.fi/> B-course</a> +<td> U. Helsinki +<td> N +<td> N +<td> WUM +<td> Cd +<td> Y +<td> Y +<td> Y +<td> N +<td> 0 +<td> D +<td> ? +<td> Runs on their server: view results using a web browser. + +<!-- +<tr> +<td><a href="http://www.aist.go.jp/ETL/etl/suri/motomura/BN/bn-java.html">Bayonnet</a> +<td> Motomura (ETL) +<td> Java +<td> Y +<td> WUM +<td> NN +<td> Cx +<td> Y +<td> N +<td> N +<td> 0 +<td> D +<td> ? +<td> For learning, represents BN as a neural net. +--> + +<tr> +<td><a href=http://www.cs.ualberta.ca/~jcheng/bnpc.htm > Belief net power constructor</a> +<td> Cheng (U.Alberta) +<td> N +<td> W +<td> W +<td> D +<td> Y +<td> Y +<td> CI +<td> N +<td> 0 +<td> D +<td> ? +<td> - + +<tr> +<td><a href=http://www.cs.berkeley.edu/~murphyk/Bayes/bnt.html > BNT</a> +<td> Murphy (U.C.Berkeley) +<td> Matlab/C +<td> Y +<td> WUM +<td> G +<td> N +<td> Y +<td> Y +<td> Y +<td> 0 +<td> D,U +<td> Many +<td> Also handles dynamic models, like HMMs and Kalman filters. + + +<tr> +<td> <!--Name--> <a href="http://bndev.sourceforge.net/">BNJ</a> +<td> <!--Authors-->Hsu (Kansas) +<td> <!--Src-->Java +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->D +<td> <!--GUI-->Y +<td> <!--Params-->N +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->jtree, IS +<td> <!--Comments-->- + +<!-- +<tr> +<td><a href=http://94.parkview-court.net/project/ > BN Toolkit</a> +<td> Gowans (Imperial) +<td> Visual Basic +<td> Y +<td> W +<td> D +<td> Y +<td> N +<td> Y +<td> N +<td> 0 +<td> D +<td> Polytree +<td> Parser and GUI for the XML-BIF format. +--> + +<tr> +<td><a href=http://www.ics.uci.edu/~irinar > BucketElim</a> +<td> Rish (U.C.Irvine) +<td> C++ +<td> Y +<td> WU +<td> D +<td> N +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> Varelim +<td> - + +<tr> +<td><a href=http://www.mrc-bsu.cam.ac.uk/bugs > BUGS</a> +<td> MRC/Imperial College +<td> N +<td> N +<td> WU +<td> Cs +<td> W +<td> Y +<td> N +<td> N +<td> 0 +<td> D +<td> Gibbs +<td> - + +<tr> +<td><a href=http://www.data-digest.com > Business Navigator 5</a> +<td> Data Digest Corp +<td> N +<td> N +<td> W +<td> Cd +<td> Y +<td> Y +<td> Y +<td> N +<td> $ +<td> D +<td> Jtree +<td> - + +<tr> +<td><a href=http://www-pcd.stanford.edu/cousins/caben-1.1.tar.gz > CABeN</a> +<td> Cousins et al. (Wash. U.) +<td> C +<td> Y +<td> WU +<td> D +<td> N +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> 5 Sampling methods +<td> - + +<tr> +<td> <!--Name--> <a href= +http://discover1.mc.vanderbilt.edu/discover/public/>Causal discoverer</a> +<td> <!--Authors-->Vanderbilt +<td> <!--Src-->N +<td> <!--API-->N +<td> <!--Exec-->W +<td> <!--Cts-->- +<td> <!--GUI-->- +<td> <!--Params-->N +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->- +<td> <!--Comments-->structure learning only + + +<tr> +<td><a href=http://www.math.auc.dk/~jhb/CoCo/information.html > CoCo+Xlisp</a> +<td> Badsberg (U. Aalborg) +<td> C/lisp +<td> Y +<td> U +<td> D +<td> Y +<td> Y +<td> CI +<td> N +<td> 0 +<td> U +<td> Jtree +<td> Designed for contingency tables. + +<tr> +<td><a href=http://www.cs.ubc.ca/labs/lci/CIspace/ > CIspace</a> +<td> Poole et al. (UBC) +<td> Java +<td> N +<td> WU +<td> D +<td> Y +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> Varelim +<td> - + +<tr> +<td> <!--Name--> <a href="http://www.robots.ox.ac.uk/~parg/software.html">DBNbox</a> +<td> <!--Authors-->Roberts et al +<td> <!--Src-->Matlab +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->Y +<td> <!--GUI-->N +<td> <!--Params-->Y +<td> <!--Struct-->N +<td> <!--Utility-->N + +<td> <!--Free-->Y +<td> <!--Undir-->D +<td> <!--Inference-->Various +<td> <!--Comments-->DBNs + +<tr> +<td> <!--Name--><a href=http://www.math.auc.dk/novo/deal>Deal</a> +<td> <!--Authors-->Bottcher et al +<td> <!--Src-->R +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->G +<td> <!--GUI-->Y +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->None +<td> <!--Comments--> +Structure learning. + +<tr> +<td><a href=http://www.deriveit.com >DeriveIt</a> +<td> <!--Authors-->DeriveIt LLC +<td> <!--Src-->N +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->? +<td> <!--GUI-->? +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->? +<td> <!--Free-->$ +<td> <!--Undir-->D +<td> <!--Inference-->Jtree +<td> <!--Comments--> +Exploits local structure in CPDs. + + +<tr> +<td><a href=http://www.noeticsystems.com>Ergo</> +<td> <!--Authors-->Noetic systems +<td> <!--Src-->N +<td> <!--API-->Y +<td> <!--Exec-->W,M +<td> <!--Cts-->D +<td> <!--GUI-->Y +<td> <!--Params-->N +<td> <!--Struct-->N +<td> <!--Utility-->N +<td> <!--Free-->$ +<td> <!--Undir-->D +<td> <!--Inference-->jtree +<td> <!--Comments-->- + +<!-- +<tr> +<td><a href=http://www.ens-lyon.fr/~jnarboux/Floue/index.html>FLoUE/BIFtoN</a> +<td> ENS Lyon +<td> Java +<td> Y +<td> WUM +<td> D +<td> N +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> Jtree +<td> - +--> + +<tr> +<td><a href=http://www.staff.ncl.ac.uk/d.j.wilkinson/software/gdagsim/ > GDAGsim</a> +<td> Wilkinson (U. Newcastle) +<td> C +<td> Y +<td> WUM +<td> G +<td> N +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> Exact +<td> Bayesian analysis of large linear Gaussian directed models. + + +<tr> +<td><a href=http://www2.sis.pitt.edu/~genie > Genie</a> +<td> U. Pittsburgh +<td> N +<td> WU +<td> WU +<td> D +<td> W +<td> N +<td> N +<td> Y +<td> 0 +<td> D +<td> Jtree +<td> - + +<tr> +<td><a href=http://www.math.ntnu.no/~hrue/GMRFsim/ > GMRFsim</a> +<td> Rue (U. Trondheim) +<td> C +<td> Y +<td> WUM +<td> G +<td> N +<td> N +<td> N +<td> N +<td> 0 +<td> U +<td> MCMC +<td> Bayesian analysis of large linear Gaussian undirected models. + + +<tr> +<td> <!--Name--> <a href=http://ssli.ee.washington.edu/~bilmes/gmtk/>GMTk</a> +<td> <!--Authors-->Bilmes (UW), Zweig (IBM) +<td> <!--Src-->N +<td> <!--API-->Y +<td> <!--Exec-->U +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->Jtree +<td> <!--Comments--> +Designed for speech recognition. + +<tr> +<td> <!--Name--> <a href=http://www.r-project.org/gR>gR</a> +<td> <!--Authors-->Lauritzen et al. +<td> <!--Src-->R +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->- +<td> <!--GUI-->- +<td> <!--Params-->- +<td> <!--Struct-->- +<td> <!--Utility-->- +<td> <!--Free-->0 +<td> <!--Undir-->- +<td> <!--Inference-->- +<td> <!--Comments-->Currently vaporware + + +<tr> +<td> <!--Name--> <a href=http://www.stats.bris.ac.uk/~peter/Grappa/>Grappa</a> +<td> <!--Authors-->Green (Bristol) +<td> <!--Src-->R +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->N) +<td> <!--Struct-->N +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->Jtree +<td> <!--Comments-->- + + +<tr> +<td><a href=http://www.hugin.com > Hugin Expert</a> +<td> Hugin +<td> N +<td> Y +<td> W +<td> G +<td> W +<td> Y +<td> CI +<td> Y +<td> $ +<td> CG +<td> Jtree +<td> - + + +<tr> +<td> <!--Name--> <a href=http://www.warnes.net/GregsSoftwareLinks/index_html?Tab=MCMC>Hydra</a> +<td> <!--Authors-->Warnes (U.Wash.) +<td> <!--Src-->Java +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->Cs +<td> <!--GUI-->Y +<td> <!--Params-->Y +<td> <!--Struct-->N +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->U,D +<td> <!--Inference-->MCMC +<td> <!--Comments-->- + + +<tr> +<!-- http://www.rpal.rockwell.com/ideal.html --> +<td><a href=http://yoda.cis.temple.edu:8080/ideal>Ideal</a> +<td> Rockwell +<td> Lisp +<td> Y +<td> WUM +<td> D +<td> Y +<td> N +<td> N +<td> Y +<td> 0 +<td> D +<td> Jtree +<td> GUI requires Allegro Lisp. + +<tr> +<td><a href=http://www.cs.cmu.edu/~javabayes/Home/ > Java Bayes</a> +<td> Cozman (CMU) +<td> Java +<td> Y +<td> WUM +<td> D +<td> Y +<td> N +<td> N +<td> Y +<td> 0 +<td> D +<td> Varelim, jtree +<td> - + + +<tr> +<td> <!--Name--><a href="http://www.codeas.com/kbaseai.php" >KBaseAI</a> +<td> <!--Authors-->Codeas +<td> <!--Src-->N +<td> <!--API-->Y +<td> <!--Exec-->W,U +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->N +<td> <!--Struct-->N +<td> <!--Utility-->N +<td> <!--Free-->$ +<td> <!--Undir-->D +<td> <!--Inference-->varelim +<td> <!--Comments-->client/server architecture, multiple users, access +control, query language + + +<tr> +<td> <!--Name--> <a href=http://www.cs.huji.ac.il/labs/compbio/LibB>LibB</a> +<td> <!--Authors-->Friedman (Hebrew U) +<td> <!--Src-->N +<td> <!--API-->Y +<td> <!--Exec-->W +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->none +<td> <!--Comments--> +Structure learning + +<tr> +<td><a href=http://www.hypergraph.dk/ > MIM</a> +<td> HyperGraph Software +<td> N +<td> N +<td> W +<td> G +<td> Y +<td> Y +<td> Y +<td> N +<td> $ +<td> CG +<td> Jtree +<td> Up to 52 variables. + +<tr> +<td><a href=http://research.microsoft.com/adapt/MSBNx/ > MSBNx</a> +<td> Microsoft +<td> N +<td> Y +<td> W +<td> D +<td> W +<td> N +<td> N +<td> Y +<td> 0 +<td> D +<td> Jtree +<td> - + +<tr> +<td><a href=http://www.norsys.com > Netica</a> +<td> Norsys +<td> N +<td> WUM +<td> W +<td> G +<td> W +<td> Y +<td> N +<td> Y +<td> $ +<td> D +<td> jtree +<td> - + +<tr> +<td> <!--Name--> <a href="http://www.autonlab.org/autonweb/showSoftware/149/">Optimal +Reinsertion</a> +<td> <!--Authors-->Moore, Wong (CMU) +<td> <!--Src-->N +<td> <!--API-->N +<td> <!--Exec-->W,U +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->none +<td> <!--Comments-->structure learning + + +<tr> +<td> <!--Name--> <a href="http://people.bu.edu/vladimir/pmt/index.html">PMT</a> +<td> <!--Authors-->Pavlovic (BU) +<td> <!--Src-->Matlab/C +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->Y +<td> <!--Struct-->N +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->special purpose +<td> <!--Comments-->- + + + +<tr> +<td> <!--Name--> <a href=http://www.intel.com/research/mrl/pnl/>PNL</a> +<td> <!--Authors-->Eruhimov (Intel) +<td> <!--Src-->C++ +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->D +<td> <!--GUI-->N +<td> <!--Params-->Y +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->U,D +<td> <!--Inference-->Jtree +<td> <!--Comments--> +A C++ version of BNT; will be released 12/03. + + +<tr> +<td><a href=http://iridia.ulb.ac.be/pulcinella/Welcome.html > Pulcinella</a> +<td> IRIDIA +<td> Lisp +<td> Y +<td> WUM +<td> D +<td> Y +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> ? +<td> Uses valuation systems for non-probabilistic calculi. + +<tr> +<!-- http://civil.colorado.edu/~dodier/sonero-mirror/index.html --> +<td><a href=http://sourceforge.net/projects/riso>RISO</a> +<td> Dodier (U.Colorado) +<td> Java +<td> Y +<td> WUM +<td> G +<td> Y +<td> N +<td> N +<td> N +<td> 0 +<td> D +<td> Polytree +<td> Distributed implementation. + + + +<tr> +<td> <!--Name--> <a href=http://reasoning.cs.ucla.edu/samiam/> Sam Iam</a> +<td> <!--Authors-->Darwiche (UCLA) +<td> <!--Src-->N +<td> <!--API-->N ? +<td> <!--Exec-->WU ? (Java executable) +<td> <!--Cts-->G ? +<td> <!--GUI-->Y +<td> <!--Params-->Y +<td> <!--Struct-->N ? +<td> <!--Utility-->Y +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->Recursive conditioning +<td> <!--Comments-->Also does sensitivity Analysis + + +<tr> +<td><a href=http://www.phil.cmu.edu/tetrad/>Tetrad</a> +<td> CMU +<td> N +<td> N +<td> WU +<td> G +<td> N +<td> Y +<td> CI +<td> N +<td> 0 +<td> U,D +<td> None +<td> - + + +<tr> +<td> <!--Name--> <a href="https://sourceforge.net/projects/unbbayes/">UnBBayes</a> +<td> <!--Authors-->? +<td> <!--Src-->Java +<td> <!--API-->- +<td> <!--Exec-->- +<td> <!--Cts-->D +<td> <!--GUI-->Y +<td> <!--Params-->N +<td> <!--Struct-->Y +<td> <!--Utility-->N +<td> <!--Free-->0 +<td> <!--Undir-->D +<td> <!--Inference-->jtree +<td> <!--Comments-->K2 for struct learning + + +<tr> +<td><a href=http://www.inference.phy.cam.ac.uk/jmw39/>Vibes</a> +<td> Winn & Bishop (U. Cambridge) +<td> Java +<td> Y +<td> WU +<td> Cx +<td> Y +<td> Y +<td> N +<td> N +<td> 0 +<td> D +<td> Variational +<td> +Not yet available. + + +<tr> +<td><a href=http://snowhite.cis.uoguelph.ca/faculty_info/yxiang/ww3/> Web Weaver</a> +<td> Xiang (U.Regina) +<td> Java +<td> Y +<td> WUM +<td> D +<td> Y +<td> N +<td> N +<td> Y +<td> 0 +<td> D +<td> ? +<td> - + +<tr> +<td><a href=http://research.microsoft.com/~dmax/WinMine/tooldoc.htm > WinMine</a> +<td> Microsoft +<td> N +<td> N +<td> W +<td> Cx +<td> Y +<td> Y +<td> Y +<td> N +<td> 0 +<td> U,D +<td> None +<td> Learns BN or dependency net structure. + +<tr> +<td><a href=http://www.staff.city.ac.uk/~rgc/webpages/xbpage.html > XBAIES 2.0</a> +<td> Cowell (City U.) +<td> N +<td> N +<td> W +<td> G +<td> Y +<td> Y +<td> N +<td> Y +<td> 0 +<td> CG +<td> Jtree +<td> - + +<tr> + +</table> + + +<p> +<h2>Other sites related to (software for) graphical models</h2> +<ul> + + +<li> +<a +href="http://directory.google.com/Top/Computers/Artificial_Intelligence/Belief_Networks/Software/"> +Google's list</a> of Bayes net software. + +<!-- +<li> <a href="http://www.sis.pitt.edu/~dsl/da-software.html"> +Decision analysis programs</a>, list maintained by +Marek Druzdzel. +--> + +<li> <a href="http://www.cs.cmu.edu/~lorens/papers/mscthesis.html"> +Comparison of decision analysis software packages</a> by +Håkan L. Younes. The emphasis is on real-time decision making. + +<li> <a href="http://bayes.stat.washington.edu/almond/belief.html"> +Older belief net programs</a> (c 1996), a list created (but no longer maintained) +by Russ Almond. + +<li> +<a +href="http://www.cs.huji.ac.il/labs/compbio/Repository"> +Repository of Bayes nets</a>. + +</document> diff --git a/sourcecodes/bnt-master/docs/bnt.html b/sourcecodes/bnt-master/docs/bnt.html new file mode 100644 index 00000000..d43e9fda --- /dev/null +++ b/sourcecodes/bnt-master/docs/bnt.html @@ -0,0 +1,359 @@ +<html> <head> +<title>Bayes Net Toolbox for Matlab</title> +</head> + +<body> +<!--<body bgcolor="#FFFFFF"> --> + +<h1>Bayes Net Toolbox for Matlab</h1> +Written by Kevin Murphy, 1997--2002. +Last updated: 19 October 2007. + +<P><P> +<table> +<tr> +<td> +<img align=left src="Figures/mathbymatlab.gif" alt="Matlab logo"> +<!-- <img align=left src="toolbox.gif" alt="Toolbox logo">--> +<td> +<!--<center>--> +<a href="http://groups.yahoo.com/group/BayesNetToolbox/join"> +<img src="http://groups.yahoo.com/img/ui/join.gif" border=0><br> +Click to subscribe to the BNT email list</a> +<br> +(<a href="http://groups.yahoo.com/group/BayesNetToolbox"> +http://groups.yahoo.com/group/BayesNetToolbox</a>) +<!--</center>--> +</table> + + +<p> +<ul> +<li> <a href="changelog.html">Changelog</a> + +<li> <a +href="http://www.cs.ubc.ca/~murphyk/Software/BNT/FullBNT-1.0.4.zip">Download +zip file</a>. + +<li> <a href="install.html">Installation</a> + +<li> <a href="license.gpl">Terms and conditions of use (GNU Library GPL)</a> + + +<li> <a href="usage.html">How to use the toolbox</a> + +<li> <a href="whyNotSourceforge.html">Why I closed the sourceforge +site</a>. + +<!-- +<li> <a href="Talks/BNT_mathworks.ppt">Powerpoint slides on graphical models +and BNT</a>, presented to the Mathworks, June 2003 + + +<li> <a href="Talks/gR03.ppt">Powerpoint slides on BNT and object +recognition</a>, presented at the <a +href="http://www.math.auc.dk/gr/gr2003.html">gR</a> workshop, +September 2003. +--> + +<!-- +<li> <a href="gR03.pdf">Proposed design for gR, a graphical models +toolkit in R</a>, September 2003. +(For more information on the gR project, +click <a href="http://www.r-project.org/gR/">here</a>.) +--> + +<li> +<!-- +<img src = "../new.gif" alt="new"> +--> + +<a href="../../Papers/bnt.pdf">Invited paper on BNT</a>, +published in +Computing Science and Statistics, 2001. + +<li> <a href="../bnsoft.html">Other Bayes net software</a> + +<!--<li> <a href="software.html">Other Matlab software</a>--> + +<li> <a href="../../Bayes/bnintro.html">A brief introduction to +Bayesian Networks</a> + + +<li> <a href="#features">Major features</a> +<li> <a href="#models">Supported models</a> +<!--<li> <a href="#future">Future work</a>--> +<li> <a href="#give_away">Why do I give the code away?</a> +<li> <a href="#why_matlab">Why Matlab?</a> +<li> <a href="#ack">Acknowledgments</a> +</ul> +<p> + + + +<h2><a name="features">Major features</h2> +<ul> + +<li> BNT supports many types of +<b>conditional probability distributions</b> (nodes), +and it is easy to add more. +<ul> +<li>Tabular (multinomial) +<li>Gaussian +<li>Softmax (logistic/ sigmoid) +<li>Multi-layer perceptron (neural network) +<li>Noisy-or +<li>Deterministic +</ul> +<p> + +<li> BNT supports <b>decision and utility nodes</b>, as well as chance +nodes, +i.e., influence diagrams as well as Bayes nets. +<p> + +<li> BNT supports static and dynamic BNs (useful for modelling dynamical systems +and sequence data). +<p> + +<li> BNT supports many different <b>inference algorithms</b>, +and it is easy to add more. + +<ul> +<li> Exact inference for static BNs: +<ul> +<li>junction tree +<li>variable elimination +<li>brute force enumeration (for discrete nets) +<li>linear algebra (for Gaussian nets) +<li>Pearl's algorithm (for polytrees) +<li>quickscore (for QMR) +</ul> + +<p> +<li> Approximate inference for static BNs: +<ul> +<li>likelihood weighting +<li> Gibbs sampling +<li>loopy belief propagation +</ul> + +<p> +<li> Exact inference for DBNs: +<ul> +<li>junction tree +<li>frontier algorithm +<li>forwards-backwards (for HMMs) +<li>Kalman-RTS (for LDSs) +</ul> + +<p> +<li> Approximate inference for DBNs: +<ul> +<li>Boyen-Koller +<li>factored-frontier/loopy belief propagation +</ul> + +</ul> +<p> + +<li> +BNT supports several methods for <b>parameter learning</b>, +and it is easy to add more. +<ul> + +<li> Batch MLE/MAP parameter learning using EM. +(Each node type has its own M method, e.g. softmax nodes use IRLS,<br> +and each inference engine has its own E method, so the code is fully modular.) + +<li> Sequential/batch Bayesian parameter learning (for fully observed tabular nodes only). +</ul> + + +<p> +<li> +BNT supports several methods for <b>regularization</b>, +and it is easy to add more. +<ul> +<li> Any node can have its parameters clamped (made non-adjustable). +<li> Any set of compatible nodes can have their parameters tied (c.f., +weight sharing in a neural net). +<li> Some node types (e.g., tabular) supports priors for MAP estimation. +<li> Gaussian covariance matrices can be declared full or diagonal, and can +be tied across states of their discrete parents (if any). +</ul> + +<p> +<li> +BNT supports several methods for <b>structure learning</b>, +and it is easy to add more. +<ul> + +<li> Bayesian structure learning, +using MCMC or local search (for fully observed tabular nodes only). + +<li> Constraint-based structure learning (IC/PC and IC*/FCI). +</ul> + + +<p> +<li> The source code is extensively documented, object-oriented, and free, making it +an excellent tool for teaching, research and rapid prototyping. + +</ul> + + + +<h2><a name="models">Supported probabilistic models</h2> +<p> +It is trivial to implement all of +the following probabilistic models using the toolbox. +<ul> +<li>Static +<ul> +<li> Linear regression, logistic regression, hierarchical mixtures of experts + +<li> Naive Bayes classifiers, mixtures of Gaussians, +sigmoid belief nets + +<li> Factor analysis, probabilistic +PCA, probabilistic ICA, mixtures of these models + +</ul> + +<li>Dynamic +<ul> + +<li> HMMs, Factorial HMMs, coupled HMMs, input-output HMMs, DBNs + +<li> Kalman filters, ARMAX models, switching Kalman filters, +tree-structured Kalman filters, multiscale AR models + +</ul> + +<li> Many other combinations, for which there are (as yet) no names! + +</ul> + + +<!-- +<h2><a name="future">Future work</h2> + +I have a long <a href="wish.txt">wish list</a> +of features I would like to add to BNT +at some point in the future. +Please email me (<a +href="mailto:murphyk@cs.berkeley.edu">murphyk@cs.berkeley.edu</a>) +if you are interested in contributing! +--> + + + +<h2><a name="give_away">Why do I give the code away?</h2> + +<ul> + +<li> +I was hoping for a Linux-style effect, whereby people would contribute +their own Matlab code so that the package would grow. With a few +exceptions, this has not happened, +although several people have provided bug-fixes (see the <a +href="#ack">acknowledgements</a>). +Perhaps the <a +href="http://www.cs.berkeley.edu/~murphyk/OpenBayes/index.html">Open +Bayes Project</a> will be more +succesful in this regard, although the evidence to date is not promising. + +<p> +<li> +Knowing that someone else might read your code forces one to +document it properly, a good practice in any case, as anyone knows who +has revisited old code. +In addition, by having many "eye balls", it is easier to spot bugs. + + +<p> +<li> +I believe in the concept of +<a href="http://www-stat.stanford.edu/~donoho/Reports/1995/wavelab.pdf"> +reproducible research</a>. +Good science requires that other people be able +to replicate your experiments. +Often a paper does not give enough details about how exactly an +algorithm was implemented (e.g., how were the parameters chosen? what +initial conditions were used?), and these can make a big difference in +practice. +Hence one should release the code that +was actually used to generate the results in one's paper. +This also prevents re-inventing the wheel. + +<p> +<li> +I was fed up with reading papers where all people do is figure out how +to do exact inference and/or learning +in a model which is just a trivial special case of a general Bayes net, e.g., +input-output HMMs, coupled-HMMs, auto-regressive HMMs. +My hope is that, by releasing general purpose software, the field can +move on to more interesting questions. +As Alfred North Whitehead said in 1911, +"Civilization advances by extending the number of important operations +that we can do without thinking about them." + +</ul> + + + + + +<h2><a name="why_matlab">Why Matlab?</h2> + +Matlab is an interactive, matrix-oriented programming language that +enables one to express one's (mathematical) ideas very concisely and directly, +without having to worry about annoying details like memory allocation +or type checking. This considerably reduces development time and +keeps code short, readable and fully portable. +Matlab has excellent built-in support for many data analysis and +visualization routines. In addition, there are many useful toolboxes, e.g., for +neural networks, signal and image processing. +The main disadvantages of Matlab are that it can be slow (which is why +we are currently rewriting parts of BNT in C), and that the commercial +license is expensive (although the student version is only $100 in the US). +<p> +Many people ask me why I did not use +<a href="http://www.octave.org/">Octave</a>, +an open-source Matlab clone. +The reason is that +Octave does not support multi-dimensional arrays, +cell arrays, objects, etc. +<p> +Click <a href="../which_language.html">here</a> for a more detailed +comparison of matlab and other languages. + + + +<h2><a name="ack">Acknowledgments</h2> + +I would like to thank numerous people for bug fixes, including: +Rainer Deventer, Michael Robert James, Philippe Leray, Pedrito Maynard-Reid II, Andrew Ng, +Ron Parr, Ilya Shpitser, Xuejing Sun, Ursula Sondhauss. +<p> +I would like to thank the following people for contributing code: +Pierpaolo Brutti, Ali Taylan Cemgil, Tamar Kushnir, +Tom Murray, +Nicholas Saunier, +Ken Shan, +Yair Weiss, +Bob Welch, +Ron Zohar. +<p> +The following Intel employees have also contributed code: +Qian Diao, Shan Huang, Yimin Zhang and especially Wei Hu. + +<p> +I would like to thank Stuart Russell for funding me over the years as +I developed BNT, and Gary Bradksi for hiring me as an intern at Intel, +which has supported much of the recent developments of BNT. + + +</body> + diff --git a/sourcecodes/bnt-master/docs/bnt_download.html b/sourcecodes/bnt-master/docs/bnt_download.html new file mode 100644 index 00000000..83d32bf4 --- /dev/null +++ b/sourcecodes/bnt-master/docs/bnt_download.html @@ -0,0 +1,16 @@ +<title>Download BNT</title> + +Download BNT from <a href="http://bnt.sourceforge.net/">BNT sourceforge site</a> + +<!-- +Number of hits since 13 November 2002: +<!-- Begin of WebCounter.com code. Do not modify --> +<applet codebase="http://vc.webcounter.com/" code="vc5.class" width=88 height=31> +<param name=a value=37322><a href="http://www.webcounter.com/"> +<img src="http://vc.webcounter.com/0/t?a=37322&s=&g=" width=88 height=31></a></applet> +<!-- End of WebCounter.com code. Do not modify --> + +<p> +<a href="../FullBNT.zip">Download BNT</a> +<p> +--> diff --git a/sourcecodes/bnt-master/docs/bnt_pre_sf.html b/sourcecodes/bnt-master/docs/bnt_pre_sf.html new file mode 100644 index 00000000..9b1fe2cb --- /dev/null +++ b/sourcecodes/bnt-master/docs/bnt_pre_sf.html @@ -0,0 +1,350 @@ +<html> <head> +<title>Bayes Net Toolbox for Matlab</title> +</head> + +<body> +<!--<body bgcolor="#FFFFFF"> --> + +<h1>Bayes Net Toolbox for Matlab</h1> +Written by Kevin Murphy. +<br> +<b>BNT is now available from <a href="http://bnt.sourceforge.net/">sourceforge</a>!</b> +<!-- +Last updated on 9 June 2004 (<a href="changelog.html">Detailed +changelog</a>) +--> +<p> + +<P><P> +<table> +<tr> +<td> +<img align=left src="Figures/mathbymatlab.gif" alt="Matlab logo"> +<!-- <img align=left src="toolbox.gif" alt="Toolbox logo">--> +<td> +<!--<center>--> +<a href="http://groups.yahoo.com/group/BayesNetToolbox/join"> +<img src="http://groups.yahoo.com/img/ui/join.gif" border=0><br> +Click to subscribe to the BNT email list</a> +<br> +(<a href="http://groups.yahoo.com/group/BayesNetToolbox"> +http://groups.yahoo.com/group/BayesNetToolbox</a>) +<!--</center>--> +</table> + + +<p> +<ul> +<!--<li> <a href="bnt_download.html">Download toolbox</a>--> +<li> Download BNT from <a href="http://bnt.sourceforge.net/">BNT sourceforge site</a> + +<li> <a href="license.gpl">Terms and conditions of use (GNU Library GPL)</a> + +<li> <a href="usage.html">How to use the toolbox</a> + + +<li> <a href="Talks/BNT_mathworks.ppt">Powerpoint slides on graphical models +and BNT</a>, presented to the Mathworks, June 2003 + +<!-- +<li> <a href="Talks/gR03.ppt">Powerpoint slides on BNT and object +recognition</a>, presented at the <a +href="http://www.math.auc.dk/gr/gr2003.html">gR</a> workshop, +September 2003. +--> + +<li> <a href="gR03.pdf">Proposed design for gR, a graphical models +toolkit in R</a>, September 2003. +<!-- +(For more information on the gR project, +click <a href="http://www.r-project.org/gR/">here</a>.) +--> + +<li> +<!-- +<img src = "../new.gif" alt="new"> +--> +<a href="../../Papers/bnt.pdf">Invited paper on BNT</a>, +published in +Computing Science and Statistics, 2001. + +<li> <a href="bnsoft.html">Other Bayes net software</a> + +<!--<li> <a href="software.html">Other Matlab software</a>--> + +<li> <a href="../../Bayes/bnintro.html">A brief introduction to +Bayesian Networks</a> + +<li> <a href="#features">Major features</a> +<li> <a href="#models">Supported models</a> +<!--<li> <a href="#future">Future work</a>--> +<li> <a href="#give_away">Why do I give the code away?</a> +<li> <a href="#why_matlab">Why Matlab?</a> +<li> <a href="#ack">Acknowledgments</a> +</ul> +<p> + + + +<h2><a name="features">Major features</h2> +<ul> + +<li> BNT supports many types of +<b>conditional probability distributions</b> (nodes), +and it is easy to add more. +<ul> +<li>Tabular (multinomial) +<li>Gaussian +<li>Softmax (logistic/ sigmoid) +<li>Multi-layer perceptron (neural network) +<li>Noisy-or +<li>Deterministic +</ul> +<p> + +<li> BNT supports <b>decision and utility nodes</b>, as well as chance +nodes, +i.e., influence diagrams as well as Bayes nets. +<p> + +<li> BNT supports static and dynamic BNs (useful for modelling dynamical systems +and sequence data). +<p> + +<li> BNT supports many different <b>inference algorithms</b>, +and it is easy to add more. + +<ul> +<li> Exact inference for static BNs: +<ul> +<li>junction tree +<li>variable elimination +<li>brute force enumeration (for discrete nets) +<li>linear algebra (for Gaussian nets) +<li>Pearl's algorithm (for polytrees) +<li>quickscore (for QMR) +</ul> + +<p> +<li> Approximate inference for static BNs: +<ul> +<li>likelihood weighting +<li> Gibbs sampling +<li>loopy belief propagation +</ul> + +<p> +<li> Exact inference for DBNs: +<ul> +<li>junction tree +<li>frontier algorithm +<li>forwards-backwards (for HMMs) +<li>Kalman-RTS (for LDSs) +</ul> + +<p> +<li> Approximate inference for DBNs: +<ul> +<li>Boyen-Koller +<li>factored-frontier/loopy belief propagation +</ul> + +</ul> +<p> + +<li> +BNT supports several methods for <b>parameter learning</b>, +and it is easy to add more. +<ul> + +<li> Batch MLE/MAP parameter learning using EM. +(Each node type has its own M method, e.g. softmax nodes use IRLS,<br> +and each inference engine has its own E method, so the code is fully modular.) + +<li> Sequential/batch Bayesian parameter learning (for fully observed tabular nodes only). +</ul> + + +<p> +<li> +BNT supports several methods for <b>regularization</b>, +and it is easy to add more. +<ul> +<li> Any node can have its parameters clamped (made non-adjustable). +<li> Any set of compatible nodes can have their parameters tied (c.f., +weight sharing in a neural net). +<li> Some node types (e.g., tabular) supports priors for MAP estimation. +<li> Gaussian covariance matrices can be declared full or diagonal, and can +be tied across states of their discrete parents (if any). +</ul> + +<p> +<li> +BNT supports several methods for <b>structure learning</b>, +and it is easy to add more. +<ul> + +<li> Bayesian structure learning, +using MCMC or local search (for fully observed tabular nodes only). + +<li> Constraint-based structure learning (IC/PC and IC*/FCI). +</ul> + + +<p> +<li> The source code is extensively documented, object-oriented, and free, making it +an excellent tool for teaching, research and rapid prototyping. + +</ul> + + + +<h2><a name="models">Supported probabilistic models</h2> +<p> +It is trivial to implement all of +the following probabilistic models using the toolbox. +<ul> +<li>Static +<ul> +<li> Linear regression, logistic regression, hierarchical mixtures of experts + +<li> Naive Bayes classifiers, mixtures of Gaussians, +sigmoid belief nets + +<li> Factor analysis, probabilistic +PCA, probabilistic ICA, mixtures of these models + +</ul> + +<li>Dynamic +<ul> + +<li> HMMs, Factorial HMMs, coupled HMMs, input-output HMMs, DBNs + +<li> Kalman filters, ARMAX models, switching Kalman filters, +tree-structured Kalman filters, multiscale AR models + +</ul> + +<li> Many other combinations, for which there are (as yet) no names! + +</ul> + + +<!-- +<h2><a name="future">Future work</h2> + +I have a long <a href="wish.txt">wish list</a> +of features I would like to add to BNT +at some point in the future. +Please email me (<a +href="mailto:murphyk@cs.berkeley.edu">murphyk@cs.berkeley.edu</a>) +if you are interested in contributing! +--> + + + +<h2><a name="give_away">Why do I give the code away?</h2> + +<ul> + +<li> +I was hoping for a Linux-style effect, whereby people would contribute +their own Matlab code so that the package would grow. With a few +exceptions, this has not happened, +although several people have provided bug-fixes (see the <a +href="#ack">acknowledgements</a>). +Perhaps the <a +href="http://www.cs.berkeley.edu/~murphyk/OpenBayes/index.html">Open +Bayes Project</a> will be more +succesful in this regard, although the evidence to date is not promising. + +<p> +<li> +Knowing that someone else might read your code forces one to +document it properly, a good practice in any case, as anyone knows who +has revisited old code. +In addition, by having many "eye balls", it is easier to spot bugs. + + +<p> +<li> +I believe in the concept of +<a href="http://www-stat.stanford.edu/~donoho/Reports/1995/wavelab.pdf"> +reproducible research</a>. +Good science requires that other people be able +to replicate your experiments. +Often a paper does not give enough details about how exactly an +algorithm was implemented (e.g., how were the parameters chosen? what +initial conditions were used?), and these can make a big difference in +practice. +Hence one should release the code that +was actually used to generate the results in one's paper. +This also prevents re-inventing the wheel. + +<p> +<li> +I was fed up with reading papers where all people do is figure out how +to do exact inference and/or learning +in a model which is just a trivial special case of a general Bayes net, e.g., +input-output HMMs, coupled-HMMs, auto-regressive HMMs. +My hope is that, by releasing general purpose software, the field can +move on to more interesting questions. +As Alfred North Whitehead said in 1911, +"Civilization advances by extending the number of important operations +that we can do without thinking about them." + +</ul> + + + + + +<h2><a name="why_matlab">Why Matlab?</h2> + +Matlab is an interactive, matrix-oriented programming language that +enables one to express one's (mathematical) ideas very concisely and directly, +without having to worry about annoying details like memory allocation +or type checking. This considerably reduces development time and +keeps code short, readable and fully portable. +Matlab has excellent built-in support for many data analysis and +visualization routines. In addition, there are many useful toolboxes, e.g., for +neural networks, signal and image processing. +The main disadvantages of Matlab are that it can be slow (which is why +we are currently rewriting parts of BNT in C), and that the commercial +license is expensive (although the student version is only $100 in the US). +<p> +Many people ask me why I did not use +<a href="http://www.octave.org/">Octave</a>, +an open-source Matlab clone. +The reason is that +Octave does not support multi-dimensional arrays, +cell arrays, objects, etc. +<p> +Click <a href="../which_language.html">here</a> for a more detailed +comparison of matlab and other languages. + + + +<h2><a name="ack">Acknowledgments</h2> + +I would like to thank numerous people for bug fixes, including: +Rainer Deventer, Michael Robert James, Philippe Leray, Pedrito Maynard-Reid II, Andrew Ng, +Ron Parr, Ilya Shpitser, Xuejing Sun, Ursula Sondhauss. +<p> +I would like to thank the following people for contributing code: +Pierpaolo Brutti, Ali Taylan Cemgil, Tamar Kushnir, Ken Shan, +<a href="http://www.cs.berkeley.edu/~yweiss">Yair Weiss</a>, +Ron Zohar. +<p> +The following Intel employees have also contributed code: +Qian Diao, Shan Huang, Yimin Zhang and especially Wei Hu. + +<p> +I would like to thank Stuart Russell for funding me over the years as +I developed BNT, and Gary Bradksi for hiring me as an intern at Intel, +which has supported much of the recent developments of BNT. + + +</body> + diff --git a/sourcecodes/bnt-master/docs/cellarray.html b/sourcecodes/bnt-master/docs/cellarray.html new file mode 100644 index 00000000..19a8da21 --- /dev/null +++ b/sourcecodes/bnt-master/docs/cellarray.html @@ -0,0 +1,59 @@ +Cell arrays are a little tricky in Matlab. +Consider this example. + +C=num2cell(rand(2,3)) +C = + [0.4565] [0.8214] [0.6154] + [0.0185] [0.4447] [0.7919] + +C{1,2} % this is the contents of this cell (could be a vector or a string) +ans = + 0.8214 + +C{1:2,2} % this is the contents of these cells - returns multiple +answers! +ans = + 0.8214 +ans = + 0.4447 + +A = C(1:2,2) % this is a slice of the cell array +ans = + [0.8214] + [0.4447] + +A{1} % A is itself a cell array +ans = + 0.8214 + + + +>> C(1:2,2)=0 % can't assign a scalar to a cell array +C(1:2,2)=0 +??? Conversion to cell from double is not possible. + + +>> C(1:2,2)={0;0} % can assign a cell array to a cell array +C(1:2,2)={0;0} +C = + [0.4565] [0] [0.6154] + [0.0185] [0] [0.7919] + + +BTW, I use cell arrays for evidence for 2 reasons: + +1. [] indicates missing values +2. it can easily represent vector-valued nodes + +The following example makes this clear + +C{1,1} = [] +C{2,1} = rand(3,1) + +C = + [] [0] [0.6154] + [3x1 double] [0] [0.7919] + + +Hope this helps, +Kevin diff --git a/sourcecodes/bnt-master/docs/changelog.html b/sourcecodes/bnt-master/docs/changelog.html new file mode 100644 index 00000000..23f7a9a3 --- /dev/null +++ b/sourcecodes/bnt-master/docs/changelog.html @@ -0,0 +1,1617 @@ +<title>History of changes to BNT</title> +<h1>History of changes to BNT</h1> + + +<h2>Changes since 4 Oct 2007</h2> + +<pre> +- 19 Oct 07 murphyk + +* BNT\CPDs\@noisyor_CPD\CPD_to_CPT.m: 2nd half of the file is a repeat +of the first half and was deleted (thanks to Karl Kuschner) + +* KPMtools\myismember.m should return logical for use in "assert" so add line at end + p=logical(p); this prevents "assert" from failing on an integer input. +(thanks to Karl Kuschner) + + + +- 17 Oct 07 murphyk + +* Updated subv2ind and ind2subv in KPMtools to Tom Minka's implementation. +His ind2subv is faster (vectorized), but I had to modify it so it +matched the behavior of my version when called with siz=[]. +His subv2inv is slightly simpler than mine because he does not treat +the siz=[2 2 ... 2] case separately. +Note: there is now no need to ever use the C versions of these +functions (or any others, for that matter). + +* removed BNT/add_BNT_to_path since no longer needed. + + + +- 4 Oct 07 murphyk + +* moved code from sourceforge to UBC website, made version 1.0.4 + +* @pearl_inf_engine/pearl_inf_engine line 24, default +argument for protocol changed from [] to 'parallel'. +Also, changed private/parallel_protocol so it doesn't write to an +empty file id (Matlab 7 issue) + +* added foptions (Matlab 7 issue) + +* changed genpathKPM to exclude svn. Put it in toplevel directory to +massively simplify the installation process. + +</pre> + + +<h2>Sourceforge changelog</h2> + +BNT was first ported to sourceforge on 28 July 2001 by yozhik. +BNT was removed from sourceforge on 4 October 2007 by Kevin Murphy; +that version is cached as <a +href="FullBNT-1.0.3.zip">FullBNT-1.0.3.zip</a>. +See <a href="ChangeLog.Sourceforge.txt">Changelog from +sourceforge</a> for a history of that version of the code, +which formed the basis of the branch currently on Murphy's web page. + + +<h2> Changes from August 1998 -- July 2004</h2> + +Kevin Murphy made the following changes to his own private copy. +(Other small changes were made between July 2004 and October 2007, but were +not documented.) +These may or may not be reflected in the sourceforge version of the +code (which was independently maintained). + + +<ul> +<li> 9 June 2004 +<ul> +<li> Changed tabular_CPD/learn_params back to old syntax, to make it +compatible with gaussian_CPD/learn_params (and re-enabled +generic_CPD/learn_params). +Modified learning/learn_params.m and learning/score_family +appropriately. +(In particular, I undid the change Sonia Leach had to make to +score_family to handle this asymmetry.) +Added examples/static/gaussian2 to test this new functionality. + +<li> Added bp_mrf2 (for generic pairwise MRFs) to +inference/static/@bp_belprop_mrf2_inf_engine. [MRFs are not +"officially" supported in BNT, so this code is just for expert +hackers.] + +<li> Added examples/static/nodeorderExample.m to illustrate importance +of using topological ordering. + +<li> Ran dos2unix on all *.c files within BNT to eliminate compiler +warnings. + +</ul> + +<li> 7 June 2004 +<ul> +<li> Replaced normaliseC with normalise in HMM/fwdback, for maximum +portability (and negligible loss in speed). +<li> Ensured FullBNT versions of HMM, KPMstats etc were as up-to-date +as stand-alone versions. +<li> Changed add_BNT_to_path so it no longer uses addpath(genpath()), +which caused Old versions of files to mask new ones. +</ul> + +<li> 18 February 2004 +<ul> +<li> A few small bug fixes to BNT, as posted to the Yahoo group. +<li> Several new functions added to KPMtools, KPMstats and Graphviz +(none needed by BNT). +<li> Added CVS to some of my toolboxes. +</ul> + +<li> 30 July 2003 +<ul> +<li> qian.diao fixed @mpot/set_domain_pot and @cgpot/set_domain_pot +<li> Marco Grzegorczyk found, and Sonia Leach fixed, a bug in +do_removal inside learn_struct_mcmc +</ul> + + +<li> 28 July 2003 +<ul> +<li> Sebastian Luehr provided 2 minor bug fixes, to HMM/fwdback (if any(scale==0)) +and FullBNT\HMM\CPDs\@hhmmQ_CPD\update_ess.m (wrong transpose). +</ul> + +<li> 8 July 2003 +<ul> +<li> Removed buggy BNT/examples/static/MRF2/Old/mk_2D_lattice.m which was +masking correct graph/mk_2D_lattice. +<li> Fixed bug in graph/mk_2D_lattice_slow in the non-wrap-around case +(line 78) +</ul> + + +<li> 2 July 2003 +<ul> +<li> Sped up normalize(., 1) in KPMtools by avoiding general repmat +<li> Added assign_cols and marginalize_table to KPMtools +</ul> + + +<li> 29 May 2003 +<ul> +<li> Modified KPMstats/mixgauss_Mstep so it repmats Sigma in the tied +covariance case (bug found by galt@media.mit.edu). + +<li> Bob Welch found bug in gaussian_CPDs/maximize_params in the way +cpsz was computed. + +<li> Added KPMstats/mixgauss_em, because my code is easier to +understand/modify than netlab's (at least for me!). + +<li> Modified BNT/examples/dynamic/viterbi1 to call multinomial_prob +instead of mk_dhmm_obs_lik. + +<li> Moved parzen window and partitioned models code to KPMstats. + +<li> Rainer Deventer fixed some bugs in his scgpot code, as follows: +1. complement_pot.m +Problems occured for probabilities equal to zero. The result is an +division by zero error. +<br> +2. normalize_pot.m +This function is used during the calculation of the log-likelihood. +For a probability of zero a warning "log of zero" occurs. I have not +realy fixed the bug. As a workaround I suggest to calculate the +likelihhod based on realmin (the smallest real number) instead of +zero. +<br> +3. recursive_combine_pots +At the beginning of the function there was no test for the trivial case, +which defines the combination of two potentials as equal to the direct +combination. The result might be an infinite recursion which leads to +a stack overflow in matlab. +</ul> + + + +<li> 11 May 2003 +<ul> +<li> Fixed bug in gaussian_CPD/maximize_params so it is compatible +with the new clg_Mstep routine +<li> Modified KPMstats/cwr_em to handle single cluster case +separately. +<li> Fixed bug in netlab/gmminit. +<li> Added hash tables to KPMtools. +</ul> + + +<li> 4 May 2003 +<ul> +<li> +Renamed many functions in KPMstats so the name of the +distribution/model type comes first, +Mstep_clg -> clg_Mstep, +Mstep_cond_gauss -> mixgauss_Mstep. +Also, renamed eval_pdf_xxx functions to xxx_prob, e.g. +eval_pdf_cond_mixgauss -> mixgauss_prob. +This is simpler and shorter. + +<li> +Renamed many functions in HMM toolbox so the name of the +distribution/model type comes first, +log_lik_mhmm -> mhmm_logprob, etc. +mk_arhmm_obs_lik has finally been re-implemented in terms of clg_prob +and mixgauss_prob (for slice 1). +Removed the Demos directory, and put them in the main directory. +This code is not backwards compatible. + +<li> Removed some of the my_xxx functions from KPMstats (these were +mostly copies of functions from the Mathworks stats toolbox). + + +<li> Modified BNT to take into account changes to KPMstats and +HMM toolboxes. + +<li> Fixed KPMstats/Mstep_clg (now called clg_Mstep) for spherical Gaussian case. +(Trace was wrongly parenthesised, and I used YY instead of YTY. +The spherical case now gives the same result as the full case +for cwr_demo.) +Also, mixgauss_Mstep now adds 0.01 to the ML estimate of Sigma, +to act as a regularizer (it used to add 0.01 to E[YY'], but this was +ignored in the spherical case). + +<li> Added cluster weighted regression to KPMstats. + +<li> Added KPMtools/strmatch_substr. +</ul> + + + +<li> 28 Mar 03 +<ul> +<li> Added mc_stat_distrib and eval_pdf_cond_prod_parzen to KPMstats +<li> Fixed GraphViz/arrow.m incompatibility with matlab 6.5 +(replace all NaN's with 0). +Modified GraphViz/graph_to_dot so it also works on windows. +<li> I removed dag_to_jtree and added graph_to_jtree to the graph +toolbox; the latter expects an undirected graph as input. +<li> I added triangulate_2Dlattice_demo.m to graph. +<li> Rainer Deventer fixed the stable conditional Gaussian potential +classes (scgpot and scgcpot) and inference engine +(stab_cond_gauss_inf_engine). +<li> Rainer Deventer added (stable) higher-order Markov models (see +inference/dynamic/@stable_ho_inf_engine). +</ul> + + +<li> 14 Feb 03 +<ul> +<li> Simplified learning/learn_params so it no longer returns BIC +score. Also, simplified @tabular_CPD/learn_params so it only takes +local evidence. +Added learn_params_dbn, which does ML estimation of fully observed +DBNs. +<li> Vectorized KPMstats/eval_pdf_cond_mixgauss for tied Sigma +case (much faster!). +Also, now works in log-domain to prevent underflow. +eval_pdf_mixgauss now calls eval_pdf_cond_mixgauss and inherits these benefits. +<li> add_BNT_to_path now calls genpath with 2 arguments if using +matlab version 5. +</ul> + + +<li> 30 Jan 03 +<ul> +<li> Vectorized KPMstats/eval_pdf_cond_mixgauss for scalar Sigma +case (much faster!) +<li> Renamed mk_dotfile_from_hmm to draw_hmm and moved it to the +GraphViz library. +<li> Rewrote @gaussian_CPD/maximize_params.m so it calls +KPMstats/Mstep_clg. +This fixes bug when using clamped means (found by Rainer Deventer +and Victor Eruhimov) +and a bug when using a Wishart prior (no gamma term in the denominator). +It is also easier to read. +I rewrote the technical report re-deriving all the equations in a +clearer notation, making the solution to the bugs more obvious. +(See www.ai.mit.edu/~murphyk/Papers/learncg.pdf) +Modified Mstep_cond_gauss to handle priors. +<li> Fixed bug reported by Ramgopal Mettu in which add_BNT_to_path +calls genpath with only 1 argument, whereas version 5 requires 2. +<li> Fixed installC and uninstallC to search in FullBNT/BNT. +</ul> + + +<li> 24 Jan 03 +<ul> +<li> Major simplification of HMM code. +The API is not backwards compatible. +No new functionality has been added, however. +There is now only one fwdback function, instead of 7; +different behaviors are controlled through optional arguments. +I renamed 'evaluate observation likelihood' (local evidence) +to 'evaluate conditional pdf', since this is more general. +i.e., renamed +mk_dhmm_obs_lik to eval_pdf_cond_multinomial, +mk_ghmm_obs_lik to eval_pdf_cond_gauss, +mk_mhmm_obs_lik to eval_pdf_cond_mog. +These functions have been moved to KPMstats, +so they can be used by other toolboxes. +ghmm's have been eliminated, since they are just a special case of +mhmm's with M=1 mixture component. +mixgauss HMMs can now handle a different number of +mixture components per state. +init_mhmm has been eliminated, and replaced with init_cond_mixgauss +(in KPMstats) and mk_leftright/rightleft_transmat. +learn_dhmm can no longer handle inputs (although this is easy to add back). +</ul> + + + + + +<li> 20 Jan 03 +<ul> +<li> Added arrow.m to GraphViz directory, and commented out line 922, +in response to a bug report. +</ul> + +<li> 18 Jan 03 +<ul> +<li> Major restructuring of BNT file structure: +all code that is not specific to Bayes nets has been removed; +these packages must be downloaded separately. (Or just download FullBNT.) +This makes it easier to ensure different toolboxes are consistent. +misc has been slimmed down and renamed KPMtools, so it can be shared by other toolboxes, +such as HMM and Kalman; some of the code has been moved to BNT/general. +The Graphics directory has been slimmed down and renamed GraphViz. +The graph directory now has no dependence on BNT (dag_to_jtree has +been renamed graph_to_jtree and has a new API). +netlab2 no longer contains any netlab files, only netlab extensions. +None of the functionality has changed. +</ul> + + + +<li> 11 Jan 03 +<ul> +<li> jtree_dbn_inf_engine can now support soft evidence. + +<li> Rewrote graph/dfs to make it clearer. +Return arguments have changed, as has mk_rooted_tree. +The acyclicity check for large undirected graphs can cause a stack overflow. +It turns out that this was not a bug, but is because Matlab's stack depth +bound is very low by default. + +<li> Renamed examples/dynamic/filter2 to filter_test1, so it does not +conflict with the filter2 function in the image processing toolbox. + +<li> Ran test_BNT on various versions of matlab to check compatibility. +On matlab 6.5 (r13), elapsed time = 211s, cpu time = 204s. +On matlab 6.1 (r12), elapsed time = 173s, cpu time = 164s. +On matlab 5.3 (r11), elapsed time = 116s, cpu time = 114s. +So matlab is apparently getting slower with time!! +(All results were with a linux PIII machine.) +</ul> + + +<li> 14 Nov 02 +<ul> +<li> Removed all ndx inference routines, since they are only +marginally faster on toy problems, +and are slower on large problems due to having to store and lookup +the indices (causes cache misses). +In particular, I removed jtree_ndx_inf_eng and jtree_ndx_dbn_inf_eng, all the *ndx* +routines from potentials/Tables, and all the UID stuff from +add_BNT_to_path, +thus simplifying the code. +This required fixing hmm_(2TBN)_inf_engine/marginal_nodes\family, +and updating installC. + + +<li> Removed jtree_C_inf_engine and jtree_C_dbn_inf_engine. +The former is basically the same as using jtree_inf_engine with +mutliply_by_table.c and marginalize_table.c. +The latter benefited slightly by assuming potentials were tables +(arrays not objects), but these negligible savings don't justify the +complexity and code duplication. + +<li> Removed stab_cond_gauss_inf_engine and +scg_unrolled_dbn_inf_engine, +written by shan.huang@intel.com, since the code was buggy. + +<li> Removed potential_engine, which was only experimental anyway. + +</ul> + + + +<li> 13 Nov 02 +<ul> +<li> <b>Released version 5</b>. +The previous version, released on 7/28/02, is available +<a href="BNT4.zip">here</a>. + +<li> Moved code and documentation to MIT. + +<li> Added repmat.c from Thomas Minka's lightspeed library. +Modified it so it can return an empty matrix. + +<li> Tomas Kocka fixed bug in the BDeu option for tabular_CPD, +and contributed graph/dag_to_eg, to convert to essential graphs. + +<!--<li> Wrote a <a href="../Papers/fastmult.pdf">paper</a> which explains +the ndx methods and the ndx cache BNT uses for fast +multiplication/ marginalization of multi-dimensional arrays. +--> + +<li> Modified definition of hhmmQ_CPD, so that Qps can now accept +parents in either the current or previous slice. + +<li> Added hhmm2Q_CPD class, which is simpler than hhmmQ (no embedded +sub CPDs, etc), and which allows the conditioning parents, Qps, to +be before (in the topological ordering) the F or Q(t-1) nodes. +See BNT/examples/dynamic/HHMM/Map/mk_map_hhmm for an example. +</ul> + + +<li> 7/28/02 +<ul> +<li> Changed graph/best_first_elim_order from min-fill to min-weight. +<li> Ernest Chan fixed bug in Kalman/sample_lds (G{i} becomes G{m} in +line 61). +<li> Tal Blum <bloom@cs.huji.ac.il> fixed bug in HMM/init_ghmm (Q +becomes K, the number of states). +<li> Fixed jtree_2tbn_inf_engine/set_fields so it correctly sets the +maximize flag to 1 even in subengines. +<li> Gary Bradksi did a simple mod to the PC struct learn alg so you can pass it an +adjacency matrix as a constraint. Also, CovMat.m reads a file and +produces a covariance matrix. +<li> KNOWN BUG in CPDs/@hhmmQ_CPD/update_ess.m at line 72 caused by +examples/dynamic/HHMM/Square/learn_square_hhmm_cts.m at line 57. +<li> +The old version is available from www.cs.berkeley.edu/~murphyk/BNT.24june02.zip +</ul> + + +<li> 6/24/02 +<ul> +<li> Renamed dag_to_dot as graph_to_dot and added support for +undirected graphs. +<li> Changed syntax for HHMM CPD constructors: no need to specify d/D +anymore,so they can be used for more complex models. +<li> Removed redundant first argument to mk_isolated_tabular_CPD. +</ul> + + +<li> 6/19/02 +<ul> +<li> +Fixed most probable explanation code. +Replaced calc_mpe with find_mpe, which is now a method of certain +inference engines, e.g., jtree, belprop. +calc_mpe_global has become the find_mpe method of global_joint. +calc_mpe_bucket has become the find_mpe method of var_elim. +calc_mpe_dbn has become the find_mpe method of smoother. +These routines now correctly find the jointly most probable +explanation, instead of the marginally most probable assignments. +See examples/static/mpe1\mpe2 and examples/dynamic/viterbi1 +for examples. +Removed maximize flag from constructor and enter_evidence +methods, since this no longer needs to be specified by the user. + +<li> Rainer Deventer fixed in a bug in +CPDs/@gaussian_CPD/udpate_ess.m: +now, hidden_cps = any(hidden_bitv(cps)), whereas it used to be +hidden_cps = all(hidden_bitv(cps)). + +</ul> + + +<li> 5/29/02 +<ul> +<li> CPDs/@gaussian_CPD/udpate_ess.m fixed WX,WXX,WXY (thanks to Rainer Deventer and +Yohsuke Minowa for spotting the bug). Does the C version work?? +<li> potentials/@cpot/mpot_to_cpot fixed K==0 case (thanks to Rainer Deventer). +<li> CPDs/@gaussian_CPD/log_prob_node now accepts non-cell array data +on self (thanks to rishi <rishi@capsl.udel.edu> for catching this). +</ul> + + +<li> 5/19/02 +<ul> + +<!-- +<li> Finally added <a href="../Papers/wei_ndx.ps.gz">paper</a> by Wei Hu (written +November 2001) +describing ndxB, ndxD, and ndxSD. +--> + +<li> Wei Hu made the following changes. +<ul> +<li> Memory leak repair: + a. distribute_evidence.c in static/@jtree_C directory + b. distribute_evidence.c in static/@jtree_ndx directory + c. marg_tablec. in Tables dir + +<li> Add "@jtree_ndx_2TBN_inf_engine" in inference/online dir + +<li> Add "@jtree_sparse_inf_engine" in inference/static dir + +<li> Add "@jtree_sparse_2TBN_inf_engine" in inference/online dir + +<li> Modify "tabular_CPD.m" in CPDs/@tabular_CPD dir , used for sparse + +<li> In "@discrete_CPD" dir: + a. modify "convert_to_pot.m", used for sparse + b. add "convert_to_sparse_table.c" + +<li> In "potentials/@dpot" dir: + a. remove "divide_by_pot.c" and "multiply_by_pot.c" + b. add "divide_by_pot.m" and "multiply_by_pot.m" + c. modify "dpot.m", "marginalize_pot.m" and "normalize_pot.m" + +<li> In "potentials/Tables" dir: + a. modify mk_ndxB.c;(for speedup) + b. add "mult_by_table.m", + "divide_by_table.m", + "divide_by_table.c", + "marg_sparse_table.c", + "mult_by_sparse_table.c", + "divide_by_sparse_table.c". + +<li> Modify "normalise.c" in misc dir, used for sparse. + +<li>And, add discrete2, discrete3, filter2 and filter3 as test applications in test_BNT.m +Modify installC.m +</ul> + +<li> Kevin made the following changes related to strong junction +trees: +<ul> +<li> jtree_inf_engin line 75: +engine.root_clq = length(engine.cliques); +the last clq is guaranteed to be a strong root + +<li> dag_to_jtree line 38: [jtree, root, B, w] = +cliques_to_jtree(cliques, ns); +never call cliques_to_strong_jtree + +<li> strong_elim_order: use Ilya's code instead of topological sorting. +</ul> + +<li> Kevin fixed CPDs/@generic_CPD/learn_params, so it always passes +in the correct hidden_bitv field to update_params. + +</ul>. + + +<li> 5/8/02 +<ul> + +<li> Jerod Weinman helped fix some bugs in HHMMQ_CPD/maximize_params. + +<li> Removed broken online inference from hmm_inf_engine. +It has been replaced filter_inf_engine, which can take hmm_inf_engine +as an argument. + +<li> Changed graph visualization function names. +'draw_layout' is now 'draw_graph', +'draw_layout_dbn' is now 'draw_dbn', +'plotgraph' is now 'dag_to_dot', +'plothmm' is now 'hmm_to_dot', +added 'dbn_to_dot', +'mkdot' no longer exists': its functioality has been subsumed by dag_to_dot. +The dot functions now all take optional args in string/value format. +</ul> + + +<li> 4/1/02 +<ul> +<li> Added online inference classes. +See BNT/inference/online and BNT/examples/dynamic/filter1. +This is work in progress. +<li> Renamed cmp_inference to cmp_inference_dbn, and made its + interface and behavior more similar to cmp_inference_static. +<li> Added field rep_of_eclass to bnet and dbn, to simplify +parameter tying (see ~murphyk/Bayes/param_tieing.html). +<li> Added gmux_CPD (Gaussian mulitplexers). +See BNT/examples/dynamic/SLAM/skf_data_assoc_gmux for an example. +<li> Modified the forwards sampling routines. +general/sample_dbn and sample_bnet now take optional arguments as +strings, and can sample with pre-specified evidence. +sample_bnet can only generate a single sample, and it is always a cell +array. +sample_node can only generate a single sample, and it is always a +scalar or vector. +This eliminates the false impression that the function was +ever vectorized (which was only true for tabular_CPDs). +(Calling sample_bnet inside a for-loop is unlikely to be a bottleneck.) +<li> Updated usage.html's description of CPDs (gmux) and inference +(added gibbs_sampling and modified the description of pearl). +<li> Modified BNT/Kalman/kalman_filter\smoother so they now optionally +take an observed input (control) sequence. +Also, optional arguments are now passed as strings. +<li> Removed BNT/examples/static/uci_data to save space. +</ul> + +<li> 3/14/02 +<ul> +<li> pearl_inf_engine now works for (vector) Gaussian nodes, as well +as discrete. compute_pi has been renamed CPD_to_pi. compute_lambda_msg + has been renamed CPD_to_lambda_msg. These are now implemented for + the discrete_CPD class instead of tabular_CPD. noisyor and + Gaussian have their own private implemenations. +Created examples/static/Belprop subdirectory. +<li> Added examples/dynamic/HHMM/Motif. +<li> Added Matt Brand's entropic prior code. +<li> cmp_inference_static has changed. It no longer returns err. It + can check for convergence. It can accept 'observed'. +</ul> + + +<li> 3/4/02 +<ul> +<li> Fixed HHMM code. Now BNT/examples/dynamic/HHMM/mk_abcd_hhmm +implements the example in the NIPS paper. See also +Square/sample_square_hhmm_discrete and other files. + +<li> Included Bhaskara Marthi's gibbs_sampling_inf_engine. Currently +this only works if all CPDs are tabular and if you call installC. + +<li> Modified Kalman/tracking_demo so it calls plotgauss2d instead of + gaussplot. + +<li> Included Sonia Leach's speedup of mk_rnd_dag. +My version created all NchooseK subsets, and then picked among them. Sonia +reorders the possible parents randomly and choose +the first k. This saves on having to enumerate the large number of +possible subsets before picking from one. + +<li> Eliminated BNT/inference/static/Old, which contained some old +.mexglx files which wasted space. +</ul> + + + +<li> 2/15/02 +<ul> +<li> Removed the netlab directory, since most of it was not being +used, and it took up too much space (the goal is to have BNT.zip be +less than 1.4MB, so if fits on a floppy). +The required files have been copied into netlab2. +</ul> + +<li> 2/14/02 +<ul> +<li> Shan Huang fixed most (all?) of the bugs in his stable CG code. +scg1-3 now work, but scg_3node and scg_unstable give different + behavior than that reported in the Cowell book. + +<li> I changed gaussplot so it plots an ellipse representing the + eigenvectors of the covariance matrix, rather than numerically + evaluating the density and using a contour plot; this + is much faster and gives better pictures. The new function is + called plotgauss2d in BNT/Graphics. + +<li> Joni Alon <jalon@cs.bu.edu> fixed some small bugs: + mk_dhmm_obs_lik called forwards with the wrong args, and + add_BNT_to_path should quote filenames with spaces. + +<li> I added BNT/stats2/myunidrnd which is called by learn_struct_mcmc. + +<li> I changed BNT/potentials/@dpot/multiply_by_dpot so it now says +Tbig.T(:) = Tbig.T(:) .* Ts(:); +</ul> + + +<li> 2/6/02 +<ul> +<li> Added hierarchical HMMs. See BNT/examples/dynamic/HHMM and +CPDs/@hhmmQ_CPD and @hhmmF_CPD. +<li> sample_dbn can now sample until a certain condition is true. +<li> Sonia Leach fixed learn_struct_mcmc and changed mk_nbrs_of_digraph +so it only returns DAGs. +Click <a href="sonia_mcmc.txt">here</a> for details of her changes. +</ul> + + +<li> 2/4/02 +<ul> +<li> Wei Hu fixed a bug in +jtree_ndx_inf_engine/collect\distribute_evidence.c which failed when +maximize=1. +<li> +I fixed various bugs to do with conditional Gaussians, +so mixexp3 now works (thansk to Gerry Fung <gerry.fung@utoronto.ca> + for spotting the error). Specifically: +Changed softmax_CPD/convert_to_pot so it now puts cts nodes in cdom, and no longer inherits + this function from discrete_CPD. + Changed root_CPD/convert_to_put so it puts self in cdom. +</ul> + + +<li> 1/31/02 +<ul> +<li> Fixed log_lik_mhmm (thanks to ling chen <real_lingchen@yahoo.com> +for spotting the typo) +<li> Now many scripts in examples/static call cmp_inference_static. +Also, SCG scripts have been simplified (but still don't work!). +<li> belprop and belprop_fg enter_evidence now returns [engine, ll, + niter], with ll=0, so the order of the arguments is compatible with other engines. +<li> Ensured that all enter_evidence methods support optional + arguments such as 'maximize', even if they ignore them. +<li> Added Wei Hu's potentials/Tables/rep_mult.c, which is used to + totally eliminate all repmats from gaussian_CPD/update_ess. +</ul> + + +<li> 1/30/02 +<ul> +<li> update_ess now takes hidden_bitv instead of hidden_self and +hidden_ps. This allows gaussian_CPD to distinguish hidden discrete and +cts parents. Now learn_params_em, as well as learn_params_dbn_em, + passes in this info, for speed. + +<li> gaussian_CPD update_ess is now vectorized for any case where all + the continuous nodes are observed (eg., Gaussian HMMs, AR-HMMs). + +<li> mk_dbn now automatically detects autoregressive nodes. + +<li> hmm_inf_engine now uses indexes in marginal_nodes/family for + speed. Marginal_ndoes can now only handle single nodes. + (SDndx is hard-coded, to avoid the overhead of using marg_ndx, + which is slow because of the case and global statements.) + +<li> add_ev_to_dmarginal now retains the domain field. + +<li> Wei Hu wrote potentials/Tables/repmat_and_mult.c, which is used to + avoid some of the repmat's in gaussian_CPD/update_ess. + +<li> installC now longer sets the global USEC, since USEC is set to 0 + by add_BNT_to_path, even if the C files have already been compiled + in a previous session. Instead, gaussian_CPD checks to + see if repmat_and_mult exists, and (bat1, chmm1, water1, water2) + check to see if jtree_C_inf_engine/collect_evidence exists. + Note that checking if a file exists is slow, so we do the check + inside the gaussian_CPD constructor, not inside update_ess. + +<li> uninstallC now deletes both .mex and .dll files, just in case I + accidently ship a .zip file with binaries. It also deletes mex + files from jtree_C_inf_engine. + +<li> Now marginal_family for both jtree_limid_inf_engine and + global_joint_inf_engine returns a marginal structure and + potential, as required by solve_limid. + Other engines (eg. jtree_ndx, hmm) are not required to return a potential. +</ul> + + + +<li> 1/22/02 +<ul> +<li> Added an optional argument to mk_bnet and mk_dbn which lets you +add names to nodes. This uses the new assoc_array class. + +<li> Added Yimin Zhang's (unfinished) classification/regression tree +code to CPDs/tree_CPD. + +</ul> + + + +<li> 1/14/02 +<ul> +<li> Incorporated some of Shan Huang's (still broken) stable CG code. +</ul> + + +<li> 1/9/02 +<ul> +<li> Yimin Zhang vectorized @discrete_CPD/prob_node, which speeds up +structure learning considerably. I fixed this to handle softmax CPDs. + +<li> Shan Huang changed the stable conditional Gaussian code to handle +vector-valued nodes, but it is buggy. + +<li> I vectorized @gaussian_CPD/update_ess for a special case. + +<li> Removed denom=min(1, ... Z) from gaussian_CPD/maximize_params +(added to cope with negative temperature for entropic prior), which +gives wrong results on mhmm1. +</ul> + + +<li> 1/7/02 + +<ul> +<li> Removed the 'xo' typo from mk_qmr_bnet. + +<li> convert_dbn_CPDs_to_tables has been vectorized; it is now +substantially faster to compute the conditional likelihood for long sequences. + +<li> Simplified constructors for tabular_CPD and gaussian_CPD, so they +now both only take the form CPD(bnet, i, ...) for named arguments - +the CPD('self', i, ...) format is gone. Modified mk_fgraph_given_ev +to use mk_isolated_tabular_CPD instead. + +<li> Added entropic prior to tabular and Gaussian nodes. +For tabular_CPD, changed name of arguments to the constructor to +distinguish Dirichlet and entropic priors. In particular, +tabular_CPD(bnet, i, 'prior', 2) is now +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_weight', 2). + +<li> Added deterministic annealing to learn_params_dbn_em for use with +entropic priors. The old format learn(engine, cases, max_iter) has +been replaced by learn(engine, cases, 'max_iter', max_iter). + +<li> Changed examples/dynamic/bat1 and kjaerulff1, since default +equivalence classes have changed from untied to tied. +</ul> + +<li> 12/30/01 +<ul> +<li> DBN default equivalence classes for slice 2 has changed, so that +now parameters are tied for nodes with 'equivalent' parents in slices +1 and 2 (e.g., observed leaf nodes). This essentially makes passing in +the eclass arguments redundant (hooray!). +</ul> + + +<li> 12/20/01 +<ul> +<li> <b>Released version 4</b>. +Version 4 is considered a major new release +since it is not completely backwards compatible with V3. +Observed nodes are now specified when the bnet/dbn is created, +not when the engine is created. This changes the interface to many of +the engines, making the code no longer backwards compatible. +Hence support for non-named optional arguments (BNT2 style) has also +been removed; hence mk_dbn etc. requires arguments to be passed by name. + +<li> Ilya Shpitser's C code for triangulation now compiles under +Windows as well as Unix, thanks to Wei Hu. + +<li> All the ndx engines have been combined, and now take an optional +argument specifying what kind of index to use. + +<li> learn_params_dbn_em is now more efficient: +@tabular_CPD/update_ess for nodes whose families +are hidden does not need need to call add_evidence_to_dmarginal, which +is slow. + +<li> Wei Hu fixed bug in jtree_ndxD, so now the matlab and C versions +both work. + +<li> dhmm_inf_engine replaces hmm_inf_engine, since the former can +handle any kind of topology and is slightly more efficient. dhmm is +extended to handle Gaussian, as well as discrete, +observed nodes. The new hmm_inf_engine no longer supports online +inference (which was broken anyway). + +<li> Added autoregressive HMM special case to hmm_inf_engine for +speed. + +<li> jtree_ndxSD_dbn_inf_engine now computes likelihood of the +evidence in a vectorized manner, where possible, just like +hmm_inf_engine. + +<li> Added mk_limid, and hence simplified mk_bnet and mk_dbn. + + +<li> Gaussian_CPD now uses 0.01*I prior on covariance matrix by +default. To do ML estimation, set 'cov_prior_weight' to 0. + +<li> Gaussian_CPD and tabular_CPD +optional binary arguments are now set using 0/1 rather no 'no'/'yes'. + +<li> Removed Shan Huang's PDAG and decomposable graph code, which will +be put in a separate structure learning library. +</ul> + + +<li> 12/11/01 +<ul> +<li> Wei Hu fixed jtree_ndx*_dbn_inf_engine and marg_table.c. + +<li> Shan Huang contributed his implementation of stable conditional +Gaussian code (Lauritzen 1999), and methods to search through the +space of PDAGs (Markov equivalent DAGs) and undirected decomposable +graphs. The latter is still under development. +</ul> + + +<li> 12/10/01 +<ul> +<li> Included Wei Hu's new versions of the ndx* routines, which use +integers instead of doubles. The new versions are about 5 times faster +in C. In general, ndxSD is the best choice. + +<li> Fixed misc/add_ev_to_dmarginal so it works with the ndx routines +in bat1. + +<li> Added calc_mpe_dbn to do Viterbi parsing. + +<li> Updated dhmm_inf_engine so it computes marginals. +</ul> + + + +<li> 11/23/01 +<ul> +<li> learn_params now does MAP estimation (i.e., uses Dirichlet prior, +if define). Thanks to Simon Keizer skeizer@cs.utwente.nl for spotting +this. +<li> Changed plotgraph so it calls ghostview with the output of dotty, +instead of converting from .ps to .tif. The resulting image is much +easier to read. +<li> Fixed cgpot/multiply_by_pots.m. +<li> Wei Hu fixed ind2subv.c. +<li> Changed arguments to compute_joint_pot. +</ul> + + +<li> 11/1/01 +<ul> +<li> Changed sparse to dense in @dpot/multiply_pots, because sparse +arrays apparently cause a bug in the NT version of Matlab. + +<li> Fixed the bug in gaussian_CPD/log_prob_node.m which +incorrectly called the vectorized gaussian_prob with different means +when there were continuous parents and more than one case. +(Thanks to Dave Andre for finding this.) + +<li> Fixed the bug in root_CPD/convert_to_pot which did not check for +pot_type='g'. +(Thanks to Dave Andre for finding this.) + +<li> Changed calc_mpe and calc_mpe_global so they now return a cell array. + +<li> Combine pearl and loopy_pearl into a single inference engine +called 'pearl_inf_engine', which now takes optional arguments passed +in using the name/value pair syntax. +marginal_nodes/family now takes the optional add_ev argument (same as +jtree), which is the opposite of the previous shrink argument. + +<li> Created pearl_unrolled_dbn_inf_engine and "resurrected" +pearl_dbn_inf_engine in a simplified (but still broken!) form. + +<li> Wei Hi fixed the bug in ind2subv.c, so now ndxSD works. +He also made C versions of ndxSD and ndxB, and added (the unfinished) ndxD. + +</ul> + + +<li> 10/20/01 + +<ul> +<li> Removed the use_ndx option from jtree_inf, +and created 2 new inference engines: jtree_ndxSD_inf_engine and +jtree_ndxB_inf_engine. +The former stores 2 sets of indices for the small and difference +domains; the latter stores 1 set of indices for the big domain. +In Matlab, the ndxB version is often significantly faster than ndxSD +and regular jree, except when the clique size is large. +When compiled to C, the difference between ndxB and ndxSD (in terms of +speed) vanishes; again, both are faster than compiled jtree, except +when the clique size is large. +Note: ndxSD currently has a bug in it, so it gives the wrong results! +(The DBN analogs are jtree_dbn_ndxSD_inf_engine and +jtree_dbn_ndxB_inf_engine.) + +<li> Removed duplicate files from the HMM and Kalman subdirectories. +e.g., normalise is now only in BNT/misc, so when compiled to C, it +masks the unique copy of the Matlab version. +</ul> + + + +<li> 10/17/01 +<ul> +<li> Fixed bugs introduced on 10/15: +Renamed extract_gaussian_CPD_params_given_ev_on_dps.m to +gaussian_CPD_params_given_dps.m since Matlab can't cope with such long +names (this caused cg1 to fail). Fixed bug in +gaussian_CPD/convert_to_pot, which now calls convert_to_table in the +discrete case. + +<li> Fixed bug in bk_inf_engine/marginal_nodes. +The test 'if nodes < ss' is now +'if nodes <= ss' (bug fix due to Stephen seg_ma@hotmail.com) + +<li> Simplified uninstallC. +</ul> + + +<li> 10/15/01 +<ul> + +<li> Added use_ndx option to jtree_inf and jtree_dbn_inf. +This pre-computes indices for multiplying, dividing and marginalizing +discrete potentials. +This is like the old jtree_fast_inf_engine, but we use an extra level +of indirection to reduce the number of indices needed (see +uid_generator object). +Sometimes this is faster than the original way... +This is work in progress. + +<li> The constructor for dpot no longer calls myreshape, which is very +slow. +But new dpots still must call myones. +Hence discrete potentials are only sometimes 1D vectors (but should +always be thought of as multi-D arrays). This is work in progress. +</ul> + + +<li> 10/6/01 +<ul> +<li> Fixed jtree_dbn_inf_engine, and added kjaerulff1 to test this. +<li> Added option to jtree_inf_engine/marginal_nodes to return "full +sized" marginals, even on observed nodes. +<li> Clustered BK in examples/dynamic/bat1 seems to be broken, +so it has been commented out. +BK will be re-implemented on top of jtree_dbn, which should much more +efficient. +</ul> + +<li> 9/25/01 +<ul> +<li> jtree_dbn_inf_engine is now more efficient than calling BK with +clusters = exact, since it only uses the interface nodes, instead of +all of them, to maintain the belief state. +<li> Uninstalled the broken C version of strong_elim_order. +<li> Changed order of arguments to unroll_dbn_topology, so that intra1 +is no longer required. +<li> Eliminated jtree_onepass, which can be simulated by calling +collect_evidence on jtree. +<li> online1 is no longer in the test_BNT suite, since there is some +problem with online prediction with mixtures of Gaussians using BK. +This functionality is no longer supported, since doing it properly is +too much work. +</ul> +</li> + +<li> 9/7/01 +<ul> +<li> Added Ilya Shpitser's C triangulation code (43x faster!). +Currently this only compiles under linux; windows support is being added. +</ul> + + +<li> 9/5/01 +<ul> +<li> Fixed typo in CPDs/@tabular_kernel/convert_to_table (thanks, +Philippe!) +<li> Fixed problems with clamping nodes in tabular_CPD, learn_params, +learn_params_tabular, and bayes_update_params. See +BNT/examples/static/learn1 for a demo. +</ul> + + +<li> 9/3/01 +<ul> +<li> Fixed typo on line 87 of gaussian_CPD which caused error in cg1.m +<li> Installed Wei Hu's latest version of jtree_C_inf_engine, which +can now compute marginals on any clique/cluster. +<li> Added Yair Weiss's code to compute the Bethe free energy +approximation to the log likelihood in loopy_pearl (still need to add +this to belprop). The return arguments are now: engine, loglik and +niter, which is different than before. +</ul> + + + +<li> 8/30/01 +<ul> +<li> Fixed bug in BNT/examples/static/id1 which passed hard-coded +directory name to belprop_inf_engine. + +<li> Changed tabular_CPD and gaussian_CPD so they can now be created +without having to pass in a bnet. + +<li> Finished mk_fgraph_given_ev. See the fg* files in examples/static +for demos of factor graphs (work in progress). +</ul> + + + +<li> 8/22/01 +<ul> + +<li> Removed jtree_compiled_inf_engine, +since the C code it generated was so big that it would barf on large +models. + +<li> Tidied up the potentials/Tables directory. +Removed mk_marg/mult_ndx.c, +which have been superceded by the much faster mk_marg/mult_index.c +(written by Wei Hu). +Renamed the Matlab versions mk_marginalise/multiply_table_ndx.m +to be mk_marg/mult_index.m to be compatible with the C versions. +Note: nobody calls these routines anymore! +(jtree_C_inf_engine/enter_softev.c has them built-in.) +Removed mk_ndx.c, which was only used by jtree_compiled. +Removed mk_cluster_clq_ndx.m, mk_CPD_clq_ndx, and marginalise_table.m +which were not used. +Moved shrink_obs_dims_in_table.m to misc. + +<li> In potentials/@dpot directory: removed multiply_by_pot_C_old.c. +Now marginalize_pot.c can handle maximization, +and divide_by_pot.c has been implmented. +marginalize/multiply/divide_by_pot.m no longer have useC or genops options. +(To get the C versions, use installC.m) + +<li> Removed useC and genops options from jtree_inf_engine.m +To use the C versions, install the C code. + +<li> Updated BNT/installC.m. + +<li> Added fclose to @loopy_pearl_inf/enter_evidence. + +<li> Changes to MPE routines in BNT/general. +The maximize parameter is now specified inside enter_evidence +instead of when the engine is created. +Renamed calc_mpe_given_inf_engine to just calc_mpe. +Added Ron Zohar's optional fix to handle the case of ties. +Now returns log-likelihood instead of likelihood. +Added calc_mpe_global. +Removed references to genops in calc_mpe_bucket.m +Test file is now called mpe1.m + +<li> For DBN inference, filter argument is now passed by name, +as is maximize. This is NOT BACKWARDS COMPATIBLE. + +<li> Removed @loopy_dbn_inf_engine, which will was too complicated. +In the future, a new version, which applies static loopy to the +unrolled DBN, will be provided. + +<li> discrete_CPD class now contains the family sizes and supports the +method dom_sizes. This is because it could not access the child field +CPD.sizes, and mysize(CPT) may give the wrong answer. + +<li> Removed all functions of the form CPD_to_xxx, where xxx = dpot, cpot, +cgpot, table, tables. These have been replaced by convert_to_pot, +which takes a pot_type argument. +@discrete_CPD calls convert_to_table to implement a default +convert_to_pot. +@discrete_CPD calls CPD_to_CPT to implement a default +convert_to_table. +The convert_to_xxx routines take fewer arguments (no need to pass in +the globals node_sizes and cnodes!). +Eventually, convert_to_xxx will be vectorized, so it will operate on +all nodes in the same equivalence class "simultaneously", which should +be significantly quicker, at least for Gaussians. + +<li> Changed discrete_CPD/sample_node and prob_node to use +convert_to_table, instead of CPD_to_CPT, so mlp/softmax nodes can +benefit. + +<li> Removed @tabular_CPD/compute_lambda_msg_fast and +private/prod_CPD_and_pi_msgs_fast, since no one called them. + +<li> Renamed compute_MLE to learn_params, +by analogy with bayes_update_params (also because it may compute an +MAP estimate). + +<li> Renamed set_params to set_fields +and get_params to get_field for CPD and dpot objects, to +avoid confusion with the parameters of the CPD. + +<li> Removed inference/doc, which has been superceded +by the web page. + +<li> Removed inference/static/@stab_cond_gauss_inf_engine, which is +broken, and all references to stable CG. + +</ul> + + + + + +<li> 8/12/01 +<ul> +<li> I removed potentials/@dpot/marginalize_pot_max. +Now marginalize_pot for all potential classes take an optional third +argument, specifying whether to sum out or max out. +The dpot class also takes in optional arguments specifying whether to +use C or genops (the global variable USE_GENOPS has been eliminated). + +<li> potentials/@dpot/marginalize_pot has been simplified by assuming +that 'onto' is always in ascending order (i.e., we remove +Maynard-Reid's patch). This is to keep the code identical to the C +version and the other class implementations. + +<li> Added Ron Zohar's general/calc_mpe_bucket function, +and my general/calc_mpe_given_inf_engine, for calculating the most +probable explanation. + + +<li> Added Wei Hu's jtree_C_inf_engine. +enter_softev.c is about 2 times faster than enter_soft_evidence.m. + +<li> Added the latest version of jtree_compiled_inf_engine by Wei Hu. +The 'C' ndx_method now calls potentials/Tables/mk_marg/mult_index, +and the 'oldC' ndx_method calls potentials/Tables/mk_marg/mult_ndx. + +<li> Added potentials/@dpot/marginalize_pot_C.c and +multiply_by_pot_C.c by Wei Hu. +These can be called by setting the 'useC' argument in +jtree_inf_engine. + +<li> Added BNT/installC.m to compile all the mex files. + +<li> Renamed prob_fully_instantiated_bnet to log_lik_complete. + +<li> Added Shan Huang's unfinished stable conditional Gaussian +inference routines. +</ul> + + + +<li> 7/13/01 +<ul> +<li> Added the latest version of jtree_compiled_inf_engine by Wei Hu. +<li> Added the genops class by Doug Schwarz (see +BNT/genopsfun/README). This provides a 1-2x speed-up of +potentials/@dpot/multiply_by_pot and divide_by_pot. +<li> The function BNT/examples/static/qmr_compiled compares the +performance gains of these new functions. +</ul> + +<li> 7/6/01 +<ul> +<li> Made bk_inf_engine use the name/value argument syntax. This can +now do max-product (Viterbi) as well as sum-product +(forward-backward). +<li> Changed examples/static/mfa1 to use the new name/value argument +syntax. +</ul> + + +<li> 6/28/01 + +<ul> + +<li> <b>Released version 3</b>. +Version 3 is considered a major new release +since it is not completely backwards compatible with V2. +V3 supports decision and utility nodes, loopy belief propagation on +general graphs (including undirected), structure learning for non-tabular nodes, +a simplified way of handling optional +arguments to functions, +and many other features which are described below. +In addition, the documentation has been substantially rewritten. + +<li> The following functions can now take optional arguments specified +as name/value pairs, instead of passing arguments in a fixed order: +mk_bnet, jtree_inf_engine, tabular_CPD, gaussian_CPD, softmax_CPD, mlp_CPD, +enter_evidence. +This is very helpful if you want to use default values for most parameters. +The functions remain backwards compatible with BNT2. + +<li> dsoftmax_CPD has been renamed softmax_CPD, replacing the older +version of softmax. The directory netlab2 has been updated, and +contains weighted versions of some of the learning routines in netlab. +(This code is still being developed by P. Brutti.) + +<li> The "fast" versions of the inference engines, which generated +matlab code, have been removed. +@jtree_compiled_inf_engine now generates C code. +(This feature is currently being developed by Wei Hu of Intel (China), +and is not yet ready for public use.) + +<li> CPD_to_dpot, CPD_to_cpot, CPD_to_cgpot and CPD_to_upot +are in the process of being replaced by convert_to_pot. + +<li> determine_pot_type now takes as arguments (bnet, onodes) +instead of (onodes, cnodes, dag), +so it can detect the presence of utility nodes as well as continuous +nodes. +Hence this function is not backwards compatible with BNT2. + +<li> The structure learning code (K2, mcmc) now works with any node +type, not just tabular. +mk_bnets_tabular has been eliminated. +bic_score_family and dirichlet_score_family will be replaced by score_family. +Note: learn_struct_mcmc has a new interface that is not backwards +compatible with BNT2. + +<li> update_params_complete has been renamed bayes_update_params. +Also, learn_params_tabular has been replaced by learn_params, which +works for any CPD type. + +<li> Added decision/utility nodes. +</ul> + + +<li> 6/6/01 +<ul> +<li> Added soft evidence to jtree_inf_engine. +<li> Changed the documentation slightly (added soft evidence and +parameter tying, and separated parameter and structure learning). +<li> Changed the parameters of determine_pot_type, so it no longer +needs to be passed a DAG argument. +<li> Fixed parameter tying in mk_bnet (num. CPDs now equals num. equiv +classes). +<li> Made learn_struct_mcmc work in matlab version 5.2 (thanks to +Nimrod Megiddo for finding this bug). +<li> Made 'acyclic.m' work for undirected graphs. +</ul> + + +<li> 5/23/01 +<ul> +<li> Added Tamar Kushnir's code for the IC* algorithm +(learn_struct_pdag_ic_star). This learns the +structure of a PDAG, and can identify the presence of latent +variables. + +<li> Added Yair Weiss's code for computing the MAP assignment using +junction tree (i.e., a new method called @dpot/marginalize_pot_max +instead of marginalize_pot.) + +<li> Added @discrete_CPD/prob_node in addition to log_prob_node to handle +deterministic CPDs. +</ul> + + +<li> 5/12/01 +<ul> +<li> Pierpaolo Brutti updated his mlp and dsoftmax CPD classes, +and improved the HME code. + +<li> HME example now added to web page. (The previous example was non-hierarchical.) + +<li> Philippe Leray (author of the French documentation for BNT) +pointed out that I was including netlab.tar unnecessarily. +</ul> + + +<li> 5/4/01 +<ul> +<li> Added mlp_CPD which defines a CPD as a (conditional) multi-layer perceptron. +This class was written by Pierpaolo Brutti. + +<li> Added hierarchical mixtures of experts demo (due to Pierpaolo Brutti). + +<li> Fixed some bugs in dsoftmax_CPD. + +<li> Now the BNT distribution includes the whole +<a href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a> library in a +subdirectory. +It also includes my HMM and Kalman filter toolboxes, instead of just +fragments of them. +</ul> + + +<li> 5/2/01 +<ul> +<li> gaussian_inf_engine/enter_evidence now correctly returns the +loglik, even if all nodes are instantiated (bug fix due to +Michael Robert James). + +<li> Added dsoftmax_CPD which allows softmax nodes to have discrete +and continuous parents; the discrete parents act as indices into the +parameters for the continuous node, by analogy with conditional +Gaussian nodes. This class was written by Pierpaolo Brutti. +</ul> + + +<li> 3/27/01 +<ul> +<li> learn_struct_mcmc no longer returns sampled_bitv. +<li> Added mcmc_sample_to_hist to post-process the set of samples. +</ul> + +<li> 3/21/01 +<ul> +<li> Changed license from UC to GNU Library GPL. + +<li> Made all CPD constructors accept 0 arguments, so now bnets can be +saved to and loaded from files. + +<li> Improved the implementation of sequential and batch Bayesian +parameter learning for tabular CPDs with completely observed data (see +log_marg_lik_complete and update_params_complete). This code also +handles interventional data. + +<li> Added MCMC structure learning for completely observed, discrete, +static BNs. + +<li> Started implementing Bayesian estimation of linear Gaussian +nodes. See root_gaussian_CPD and +linear_gaussian_CPD. The old gaussian_CPD class has not been changed. + +<li> Renamed evaluate_CPD to log_prob_node, and simplified its +arguments. + +<li> Renamed sample_CPD to sample_node, simplified its +arguments, and vectorized it. + +<li> Renamed "learn_params_tabular" to "update_params_complete". +This does Bayesian updating, but no longer computes the BIC score. + +<li> Made routines for completely observed networks (sampling, +complete data likelihood, etc.) handle cell arrays or regular arrays, +which are faster. +If some nodes are not scalars, or are hidden, you must use cell arrays. +You must convert to a cell array before passing to an inference routine. + +<li> Fixed bug in gaussian_CPD constructor. When creating CPD with +more than 1 discrete parent with random parameters, the matrices were +the wrong shape (Bug fix due to Xuejing Sun). +</ul> + + + +<li> 11/24/00 +<ul> +<li> Renamed learn_params and learn_params_dbn to learn_params_em/ +learn_params_dbn_em. The return arguments are now [bnet, LLtrace, +engine] instead of [engine, LLtrace]. +<li> Added structure learning code for static nets (K2, PC). +<li> Renamed learn_struct_inter_full_obs as learn_struct_dbn_reveal, +and reimplemented it to make it simpler and faster. +<li> Added sequential Bayesian parameter learning (learn_params_tabular). +<li> Major rewrite of the documentation. +</ul> + +<!-- +<li> 6/1/00 +<ul> +<li> Subtracted 1911 off the counter, so now it counts hits from +5/22/00. (The initial value of 1911 was a conservative lower bound on the number of +hits from the time the page was created.) +</ul> +--> + +<li> 5/22/00 +<ul> +<li> Added online filtering and prediction. +<li> Added the factored frontier and loopy_dbn algorithms. +<li> Separated the online user manual into two, for static and dynamic +networks. +<!-- +<li> Added a counter to the BNT web page, and initialized it to 1911, +which is the number of people who have downloaded my software (BNT and +other toolboxes) since 8/24/98. +--> +<li> Added a counter to the BNT web page. +<!-- +Up to this point, 1911 people had downloaded my software (BNT and +other toolboxes) since 8/24/98. +--> +</ul> + + +<li> 4/27/00 +<ul> +<li> Fixed the typo in bat1.m +<li> Added preliminary code for online inference in DBNs +<li> Added coupled HMM example +</ul> + +<li> 4/23/00 +<ul> +<li> Fixed the bug in the fast inference routines where the indices +are empty (arises in bat1.m). +<li> Sped up marginal_family for the fast engines by precomputing indices. +</ul> + +<li> 4/17/00 +<ul> +<li> Simplified implementation of BK_inf_engine by using soft evidence. +<li> Added jtree_onepass_inf_engine (which computes a single marginal) +and modified jtree_dbn_fast to use it. +</ul> + +<li> 4/14/00 +<ul> +<li> Added fast versions of jtree and BK, which are +designed for models where the division into hidden/observed is fixed, +and all hidden variables are discrete. These routines are 2-3 times +faster than their non-fast counterparts. + +<li> Added graph drawing code +contributed by Ali Taylan Cemgil from the University of Nijmegen. +</ul> + +<li> 4/10/00 +<ul> +<li> Distinguished cnodes and cnodes_slice in DBNs so that kalman1 +works with BK. +<li> Removed dependence on cellfun (which only exists in matlab 5.3) +by adding isemptycell. Now the code works in 5.2. +<li> Changed the UC copyright notice. +</ul> + + + +<li> 3/29/00 +<ul> +<li><b>Released BNT 2.0</b>, now with objects! +Here are the major changes. + +<li> There are now 3 classes of objects in BNT: +Conditional Probability Distributions, potentials (for junction tree), +and inference engines. +Making an inference algorithm (junction tree, sampling, loopy belief +propagation, etc.) an object might seem counter-intuitive, but in +fact turns out to be a good idea, since the code and documentation +can be made modular. +(In Java, each algorithm would be a class that implements the +inferenceEngine interface. Since Matlab doesn't support interfaces, +inferenceEngine is an abstract (virtual) base class.) + +<p> +<li> +In version 1, instead of Matlab's built-in objects, +I used structs and a + simulated dispatch mechanism based on the type-tag system in the + classic textbook by Abelson + and Sussman ("Structure and Interpretation of Computer Programs", + MIT Press, 1985). This required editing the dispatcher every time a + new object type was added. It also required unique (and hence long) + names for each method, and allowed the user unrestricted access to + the internal state of objects. + +<p> +<li> The Bayes net itself is now a lightweight struct, and can be +used to specify a model independently of the inference algorithm used +to process it. +In version 1, the inference engine was stored inside the Bayes net. + +<!-- +See the list of <a href="differences2.html">changes from version +1</a>. +--> +</ul> + + + +<li> 11/24/99 +<ul> +<li> Added fixed lag smoothing, online EM and the ability to learn +switching HMMs (POMDPs) to the HMM toolbox. +<li> Renamed the HMM toolbox function 'mk_dhmm_obs_mat' to +'mk_dhmm_obs_lik', and similarly for ghmm and mhmm. Updated references +to these functions in BNT. +<li> Changed the order of return params from kalman_filter to make it +more natural. Updated references to this function in BNT. +</ul> + + + +<li>10/27/99 +<ul> +<li>Fixed line 42 of potential/cg/marginalize_cgpot and lines 32-39 of bnet/add_evidence_to_marginal +(thanks to Rainer Deventer for spotting these bugs!) +</ul> + + +<li>10/21/99 +<ul> +<li>Completely changed the blockmatrix class to make its semantics +more sensible. The constructor is not backwards compatible! +</ul> + +<li>10/6/99 +<ul> +<li>Fixed all_vals = cat(1, vals{:}) in user/enter_evidence +<li>Vectorized ind2subv and sub2indv and removed the C versions. +<li>Made mk_CPT_from_mux_node much faster by having it call vectorized +ind2subv +<li>Added Sondhauss's bug fix to line 68 of bnet/add_evidence_to_marginal +<li>In dbn/update_belief_state, instead of adding eps to likelihood if 0, +we leave it at 0, and set the scale factor to 0 instead of dividing. +</ul> + +<li>8/19/99 +<ul> +<li>Added Ghahramani's mfa code to examples directory to compare with +fa1, which uses BNT +<li>Changed all references of assoc to stringmatch (e.g., in +examples/mk_bat_topology) +</ul> + +<li>June 1999 +<ul> +<li><b>Released BNT 1.0</b> on the web. +</ul> + + +<li>August 1998 +<ul> +<li><b>Released BNT 0.0</b> via email. +</ul> + + +<li>October 1997 +<ul> +<li>First started working on Matlab version of BNT. +</ul> + +<li>Summer 1997 +<ul> +<li> First started working on C++ version of BNT while working at DEC (now Compaq) CRL. +</ul> + +<!-- +<li>Fall 1996 +<ul> +<li>Made a C++ program that generates DBN-specific C++ code +for inference using the frontier algorithm. +</ul> + +<li>Fall 1995 +<ul> +<li>Arrive in Berkeley, and first learn about Bayes Nets. Start using +Geoff Zweig's C++ code. +</ul> +--> + +</ul> diff --git a/sourcecodes/bnt-master/docs/dbn_hmm_demo.m b/sourcecodes/bnt-master/docs/dbn_hmm_demo.m new file mode 100644 index 00000000..30933b8f --- /dev/null +++ b/sourcecodes/bnt-master/docs/dbn_hmm_demo.m @@ -0,0 +1,37 @@ + Example due to Wang Hee Lin" <engp1622@nus.edu.sg + + +intra = zeros(2); +intra(1,2) = 1; +inter = zeros(2); +inter(1,1) = 1; + +Q = 2; % num hidden states +O = 2; % num observable symbols +ns = [Q O];%number of states +dnodes = 1:2; +%onodes = [1:2]; % only possible with jtree, not hmm +onodes = [2]; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'observed', onodes); +for i=1:4 + bnet.CPD{i} = tabular_CPD(bnet, i); +end + +prior0 = normalise(rand(Q,1)); +transmat0 = mk_stochastic(rand(Q,Q)); +obsmat0 = mk_stochastic(rand(Q,O)); + +%engine = smoother_engine(hmm_2TBN_inf_engine(bnet)); +engine = smoother_engine(jtree_2TBN_inf_engine(bnet)); + +ss = 2;%slice size(ss) +ncases = 10;%number of examples +T=10; +max_iter=2;%iterations for EM +cases = cell(1, ncases); +for i=1:ncases + ev = sample_dbn(bnet, T); + cases{i} = cell(ss,T); + cases{i}(onodes,:) = ev(onodes, :); +end +[bnet2, LLtrace] = learn_params_dbn_em(engine, cases, 'max_iter', 4); diff --git a/sourcecodes/bnt-master/docs/gr03.pdf b/sourcecodes/bnt-master/docs/gr03.pdf new file mode 100644 index 00000000..8ad8cf49 --- /dev/null +++ b/sourcecodes/bnt-master/docs/gr03.pdf Binary files differdiff --git a/sourcecodes/bnt-master/docs/graph_to_dot.m b/sourcecodes/bnt-master/docs/graph_to_dot.m new file mode 100644 index 00000000..c9f692d1 --- /dev/null +++ b/sourcecodes/bnt-master/docs/graph_to_dot.m @@ -0,0 +1,86 @@ +function graph_to_dot(adj, varargin) +%GRAPH_TO_DOT Makes a GraphViz (AT&T) file representing an adjacency matrix +% graph_to_dot(adj, ...) writes to the specified filename. +% +% Optional arguments can be passed as name/value pairs: [default] +% +% 'filename' - if omitted, writes to 'tmp.dot' +% 'arc_label' - arc_label{i,j} is a string attached to the i-j arc [""] +% 'node_label' - node_label{i} is a string attached to the node i ["i"] +% 'width' - width in inches [10] +% 'height' - height in inches [10] +% 'leftright' - 1 means layout left-to-right, 0 means top-to-bottom [0] +% 'directed' - 1 means use directed arcs, 0 means undirected [1] +% +% For details on graphviz, See http://www.research.att.com/sw/tools/graphviz +% +% See also dot_to_graph and draw_dot. + +% First version written by Kevin Murphy 2002. +% Modified by Leon Peshkin, Jan 2004. +% Bugfix by Tom Minka, Mar 2004. + +node_label = []; arc_label = []; % set default args +width = 10; height = 10; +leftright = 0; directed = 1; filename = 'tmp.dot'; + +for i = 1:2:nargin-1 % get optional args + switch varargin{i} + case 'filename', filename = varargin{i+1}; + case 'node_label', node_label = varargin{i+1}; + case 'arc_label', arc_label = varargin{i+1}; + case 'width', width = varargin{i+1}; + case 'height', height = varargin{i+1}; + case 'leftright', leftright = varargin{i+1}; + case 'directed', directed = varargin{i+1}; + end +end +% minka +if ~directed + adj = triu(adj | adj'); +end + +fid = fopen(filename, 'w'); +if directed + fprintf(fid, 'digraph G {\n'); + arctxt = '->'; + if isempty(arc_label) + labeltxt = ''; + else + labeltxt = '[label="%s"]'; + end +else + fprintf(fid, 'graph G {\n'); + arctxt = '--'; + if isempty(arc_label) + labeltxt = '[dir=none]'; + else + labeltext = '[label="%s",dir=none]'; + end +end +edgeformat = strcat(['%d ',arctxt,' %d ',labeltxt,';\n']); +fprintf(fid, 'center = 1;\n'); +fprintf(fid, 'size=\"%d,%d\";\n', width, height); +if leftright + fprintf(fid, 'rankdir=LR;\n'); +end +Nnds = length(adj); +for node = 1:Nnds % process nodes + if isempty(node_label) + fprintf(fid, '%d;\n', node); + else + fprintf(fid, '%d [ label = "%s" ];\n', node, node_label{node}); + end +end +for node1 = 1:Nnds % process edges + arcs = find(adj(node1,:)); % children(adj, node); + for node2 = arcs + if ~isempty(arc_label) + fprintf(fid, edgeformat,node1,node2,arc_label{node1,node2}); + else + fprintf(fid, edgeformat, node1, node2); + end + end +end +fprintf(fid, '}'); +fclose(fid); diff --git a/sourcecodes/bnt-master/docs/graphviz.html b/sourcecodes/bnt-master/docs/graphviz.html new file mode 100644 index 00000000..5ad3e777 --- /dev/null +++ b/sourcecodes/bnt-master/docs/graphviz.html @@ -0,0 +1,66 @@ +<h1>Visualizing graph structures in matlab</h1> + +We discuss some methods for visualizing graphs/ networks, including automatic +layout of the nodes. +We assume the graph is represented as an adjacency matrix. +If using BNT, you can access the DAG using +<pre> +G = bnet.dag; +</pre> + +<h2>Matlab's biograph function</h2> + +The Mathworks computational biology toolbox +has many useful graph related functions, including visualization. +<br> +Click +<a href="http://www.mathworks.com/products/bioinfo/demos.html?file=/products/demos/shipping/bioinfo/graphtheorydemo.html#4"> +here</a> +for a demo. + + + +<h2>Cemgil's draw_graph</h2> + +You can visualize an arbitrary graph (such as one learned using the +structure learning routines) with Matlab code written by +<a href="http://www-sigproc.eng.cam.ac.uk/~atc27/matlab/layout.html"> +Ali Taylan Cemgil</a> +from the University of Cambridge. +A modified version of this code +is <a href="GraphViz.zip">here</a> +(this is already bundled with BNT). +Just type +<pre> +draw_graph(G); +</pre> +For example, this is the output produced on a +<a href="http://www.cs.ubc.ca/~murphyk/Software/BNT/usage.html#qmr">random QMR-like model</a>: +<p> +<img src="Figures/qmr.rnd.jpg"> +<p> + +<h2>Pajek</h2> + +<a href="http://vlado.fmf.uni-lj.si/pub/networks/pajek">Pajek</a> +is an excellent, free Windows program for graph layout. +Use <a href="adj2pajek2.m">adj2pajek2.m</a> to convert a graph to the +Pajek file format. +<br> +Then Choose File->Network->Read from the menu. + +<h2>AT&T Graphviz</h2> + +<a href="http://www.research.att.com/sw/tools/graphviz">graphhviz</a> +is an +open-source graph visualization package from AT&T. +Use +<a href="graph_to_dot.m">graph_to_dot</a> +to convert an adjacency matrix to +the AT&T file format (the "dot" format). +You then use dot to convert it to postscript: +<pre> +graph_to_dot(G, 'filename', 'foo.dot'); +dot -Tps foo.dot -o foo.ps +ghostview foo.ps & +</pre> diff --git a/sourcecodes/bnt-master/docs/install.html b/sourcecodes/bnt-master/docs/install.html new file mode 100644 index 00000000..989dd8ff --- /dev/null +++ b/sourcecodes/bnt-master/docs/install.html @@ -0,0 +1,32 @@ +To install, just unzip FullBNT.zip, start Matlab, and proceed as +follows +<pre> +>> cd C:\kmurphy\FullBNT\FullBNT-1.0.4 +>> addpath(genpathKPM(pwd)) + +Warning: Function C:\kmurphy\FullBNT\FullBNT-1.0.4\KPMtools\isvector.m has the same name as a MATLAB builtin. We suggest you rename the function to avoid a potential name conflict. +> In path at 110 + In addpath at 89 +Warning: Function C:\kmurphy\FullBNT\FullBNT-1.0.4\KPMtools\isscalar.m has the same name as a MATLAB builtin. We suggest you rename the function to avoid a potential name conflict. +> In path at 110 + In addpath at 89 +Warning: Function C:\kmurphy\FullBNT\FullBNT-1.0.4\KPMtools\assert.m has the same name as a MATLAB builtin. We suggest you rename the function to avoid a potential name conflict. +> In path at 110 + In addpath at 89 + +>> test_BNT +</pre> +The genpathKPM function is like the builtin genpath function, but it +does not add directories called 'Old' to the path, thus preventing old +versions of functions accidently shadowing new ones. +The warnings occur because Matlab 7 added functions with the same +names as my functions. The BNT versions will shadow the built-in ones, +but this should be harmless. +<p> +Note: the functions installC_BNT etc. are not needed anymore: all C +code has either been removed or is unnecessary. +<p> Note: as mentioned above, with new versions of matlab you +will get lots of warning messages, but everything should still work. +<p> Note: Cory Reith tells me that, as of +15 Sep 2009, octave 3.2.2 can run most of BNT. +Please send email to crieth@ucsd.edu if you have questions. diff --git a/sourcecodes/bnt-master/docs/join.gif b/sourcecodes/bnt-master/docs/join.gif new file mode 100644 index 00000000..04692115 --- /dev/null +++ b/sourcecodes/bnt-master/docs/join.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/license.gpl b/sourcecodes/bnt-master/docs/license.gpl new file mode 100644 index 00000000..6a2d5712 --- /dev/null +++ b/sourcecodes/bnt-master/docs/license.gpl @@ -0,0 +1,450 @@ +This library is free software; you can redistribute it and/or +modify it under the terms of the GNU Library General Public +License version 2 as published by the Free Software Foundation. + +This library is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + +GNU Library General Public License + +---------------------------------------------------------------------------- + +Table of Contents + + * GNU LIBRARY GENERAL PUBLIC LICENSE + o Preamble + o TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + +---------------------------------------------------------------------------- + +GNU LIBRARY GENERAL PUBLIC LICENSE + +Version 2, June 1991 + +Copyright (C) 1991 Free Software Foundation, Inc. +675 Mass Ave, Cambridge, MA 02139, USA +Everyone is permitted to copy and distribute verbatim copies +of this license document, but changing it is not allowed. + +[This is the first released version of the library GPL. It is + numbered 2 because it goes with version 2 of the ordinary GPL.] + +Preamble + +The licenses for most software are designed to take away your freedom to +share and change it. By contrast, the GNU General Public Licenses are +intended to guarantee your freedom to share and change free software--to +make sure the software is free for all its users. + +This license, the Library General Public License, applies to some specially +designated Free Software Foundation software, and to any other libraries +whose authors decide to use it. You can use it for your libraries, too. + +When we speak of free software, we are referring to freedom, not price. Our +General Public Licenses are designed to make sure that you have the freedom +to distribute copies of free software (and charge for this service if you +wish), that you receive source code or can get it if you want it, that you +can change the software or use pieces of it in new free programs; and that +you know you can do these things. + +To protect your rights, we need to make restrictions that forbid anyone to +deny you these rights or to ask you to surrender the rights. These +restrictions translate to certain responsibilities for you if you distribute +copies of the library, or if you modify it. + +For example, if you distribute copies of the library, whether gratis or for +a fee, you must give the recipients all the rights that we gave you. You +must make sure that they, too, receive or can get the source code. If you +link a program with the library, you must provide complete object files to +the recipients so that they can relink them with the library, after making +changes to the library and recompiling it. And you must show them these +terms so they know their rights. + +Our method of protecting your rights has two steps: (1) copyright the +library, and (2) offer you this license which gives you legal permission to +copy, distribute and/or modify the library. + +Also, for each distributor's protection, we want to make certain that +everyone understands that there is no warranty for this free library. If the +library is modified by someone else and passed on, we want its recipients to +know that what they have is not the original version, so that any problems +introduced by others will not reflect on the original authors' reputations. + +Finally, any free program is threatened constantly by software patents. We +wish to avoid the danger that companies distributing free software will +individually obtain patent licenses, thus in effect transforming the program +into proprietary software. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + +Most GNU software, including some libraries, is covered by the ordinary GNU +General Public License, which was designed for utility programs. This +license, the GNU Library General Public License, applies to certain +designated libraries. This license is quite different from the ordinary one; +be sure to read it in full, and don't assume that anything in it is the same +as in the ordinary license. + +The reason we have a separate public license for some libraries is that they +blur the distinction we usually make between modifying or adding to a +program and simply using it. Linking a program with a library, without +changing the library, is in some sense simply using the library, and is +analogous to running a utility program or application program. However, in a +textual and legal sense, the linked executable is a combined work, a +derivative of the original library, and the ordinary General Public License +treats it as such. + +Because of this blurred distinction, using the ordinary General Public +License for libraries did not effectively promote software sharing, because +most developers did not use the libraries. We concluded that weaker +conditions might promote sharing better. + +However, unrestricted linking of non-free programs would deprive the users +of those programs of all benefit from the free status of the libraries +themselves. This Library General Public License is intended to permit +developers of non-free programs to use free libraries, while preserving your +freedom as a user of such programs to change the free libraries that are +incorporated in them. (We have not seen how to achieve this as regards +changes in header files, but we have achieved it as regards changes in the +actual functions of the Library.) The hope is that this will lead to faster +development of free libraries. + +The precise terms and conditions for copying, distribution and modification +follow. Pay close attention to the difference between a "work based on the +library" and a "work that uses the library". The former contains code +derived from the library, while the latter only works together with the +library. + +Note that it is possible for a library to be covered by the ordinary General +Public License rather than by this special one. + +TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + +0. This License Agreement applies to any software library which contains a +notice placed by the copyright holder or other authorized party saying it +may be distributed under the terms of this Library General Public License +(also called "this License"). Each licensee is addressed as "you". + +A "library" means a collection of software functions and/or data prepared so +as to be conveniently linked with application programs (which use some of +those functions and data) to form executables. + +The "Library", below, refers to any such software library or work which has +been distributed under these terms. A "work based on the Library" means +either the Library or any derivative work under copyright law: that is to +say, a work containing the Library or a portion of it, either verbatim or +with modifications and/or translated straightforwardly into another +language. (Hereinafter, translation is included without limitation in the +term "modification".) + +"Source code" for a work means the preferred form of the work for making +modifications to it. For a library, complete source code means all the +source code for all modules it contains, plus any associated interface +definition files, plus the scripts used to control compilation and +installation of the library. + +Activities other than copying, distribution and modification are not covered +by this License; they are outside its scope. The act of running a program +using the Library is not restricted, and output from such a program is +covered only if its contents constitute a work based on the Library +(independent of the use of the Library in a tool for writing it). Whether +that is true depends on what the Library does and what the program that uses +the Library does. + +1. You may copy and distribute verbatim copies of the Library's complete +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the notices +that refer to this License and to the absence of any warranty; and +distribute a copy of this License along with the Library. + +You may charge a fee for the physical act of transferring a copy, and you +may at your option offer warranty protection in exchange for a fee. + +2. You may modify your copy or copies of the Library or any portion of it, +thus forming a work based on the Library, and copy and distribute such +modifications or work under the terms of Section 1 above, provided that you +also meet all of these conditions: + + o a) The modified work must itself be a software library. + + o b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + o c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + o d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If identifiable +sections of that work are not derived from the Library, and can be +reasonably considered independent and separate works in themselves, then +this License, and its terms, do not apply to those sections when you +distribute them as separate works. But when you distribute the same sections +as part of a whole which is a work based on the Library, the distribution of +the whole must be on the terms of this License, whose permissions for other +licensees extend to the entire whole, and thus to each and every part +regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest your +rights to work written entirely by you; rather, the intent is to exercise +the right to control the distribution of derivative or collective works +based on the Library. + +In addition, mere aggregation of another work not based on the Library with +the Library (or with a work based on the Library) on a volume of a storage +or distribution medium does not bring the other work under the scope of this +License. + +3. You may opt to apply the terms of the ordinary GNU General Public License +instead of this License to a given copy of the Library. To do this, you must +alter all the notices that refer to this License, so that they refer to the +ordinary GNU General Public License, version 2, instead of to this License. +(If a newer version than version 2 of the ordinary GNU General Public +License has appeared, then you can specify that version instead if you +wish.) Do not make any other change in these notices. + +Once this change is made in a given copy, it is irreversible for that copy, +so the ordinary GNU General Public License applies to all subsequent copies +and derivative works made from that copy. + +This option is useful when you wish to copy part of the code of the Library +into a program that is not a library. + +4. You may copy and distribute the Library (or a portion or derivative of +it, under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you accompany it with the complete +corresponding machine-readable source code, which must be distributed under +the terms of Sections 1 and 2 above on a medium customarily used for +software interchange. + +If distribution of object code is made by offering access to copy from a +designated place, then offering equivalent access to copy the source code +from the same place satisfies the requirement to distribute the source code, +even though third parties are not compelled to copy the source along with +the object code. + +5. A program that contains no derivative of any portion of the Library, but +is designed to work with the Library by being compiled or linked with it, is +called a "work that uses the Library". Such a work, in isolation, is not a +derivative work of the Library, and therefore falls outside the scope of +this License. + +However, linking a "work that uses the Library" with the Library creates an +executable that is a derivative of the Library (because it contains portions +of the Library), rather than a "work that uses the library". The executable +is therefore covered by this License. Section 6 states terms for +distribution of such executables. + +When a "work that uses the Library" uses material from a header file that is +part of the Library, the object code for the work may be a derivative work +of the Library even though the source code is not. Whether this is true is +especially significant if the work can be linked without the Library, or if +the work is itself a library. The threshold for this to be true is not +precisely defined by law. + +If such an object file uses only numerical parameters, data structure +layouts and accessors, and small macros and small inline functions (ten +lines or less in length), then the use of the object file is unrestricted, +regardless of whether it is legally a derivative work. (Executables +containing this object code plus portions of the Library will still fall +under Section 6.) + +Otherwise, if the work is a derivative of the Library, you may distribute +the object code for the work under the terms of Section 6. Any executables +containing that work also fall under Section 6, whether or not they are +linked directly with the Library itself. + +6. As an exception to the Sections above, you may also compile or link a +"work that uses the Library" with the Library to produce a work containing +portions of the Library, and distribute that work under terms of your +choice, provided that the terms permit modification of the work for the +customer's own use and reverse engineering for debugging such modifications. + +You must give prominent notice with each copy of the work that the Library +is used in it and that the Library and its use are covered by this License. +You must supply a copy of this License. If the work during execution +displays copyright notices, you must include the copyright notice for the +Library among them, as well as a reference directing the user to the copy of +this License. Also, you must do one of these things: + + o a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood that + the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + o b) Accompany the work with a written offer, valid for at least + three years, to give the same user the materials specified in + Subsection 6a, above, for a charge no more than the cost of + performing this distribution. + + o c) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + o d) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + +For an executable, the required form of the "work that uses the Library" +must include any data and utility programs needed for reproducing the +executable from it. However, as a special exception, the source code +distributed need not include anything that is normally distributed (in +either source or binary form) with the major components (compiler, kernel, +and so on) of the operating system on which the executable runs, unless that +component itself accompanies the executable. + +It may happen that this requirement contradicts the license restrictions of +other proprietary libraries that do not normally accompany the operating +system. Such a contradiction means you cannot use both them and the Library +together in an executable that you distribute. + +7. You may place library facilities that are a work based on the Library +side-by-side in a single library together with other library facilities not +covered by this License, and distribute such a combined library, provided +that the separate distribution of the work based on the Library and of the +other library facilities is otherwise permitted, and provided that you do +these two things: + + o a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + o b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + +8. You may not copy, modify, sublicense, link with, or distribute the +Library except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense, link with, or distribute the Library +is void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under this +License will not have their licenses terminated so long as such parties +remain in full compliance. + +9. You are not required to accept this License, since you have not signed +it. However, nothing else grants you permission to modify or distribute the +Library or its derivative works. These actions are prohibited by law if you +do not accept this License. Therefore, by modifying or distributing the +Library (or any work based on the Library), you indicate your acceptance of +this License to do so, and all its terms and conditions for copying, +distributing or modifying the Library or works based on it. + +10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the original +licensor to copy, distribute, link with or modify the Library subject to +these terms and conditions. You may not impose any further restrictions on +the recipients' exercise of the rights granted herein. You are not +responsible for enforcing compliance by third parties to this License. + +11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot distribute so +as to satisfy simultaneously your obligations under this License and any +other pertinent obligations, then as a consequence you may not distribute +the Library at all. For example, if a patent license would not permit +royalty-free redistribution of the Library by all those who receive copies +directly or indirectly through you, then the only way you could satisfy both +it and this License would be to refrain entirely from distribution of the +Library. + +If any portion of this section is held invalid or unenforceable under any +particular circumstance, the balance of the section is intended to apply, +and the section as a whole is intended to apply in other circumstances. + +It is not the purpose of this section to induce you to infringe any patents +or other property right claims or to contest validity of any such claims; +this section has the sole purpose of protecting the integrity of the free +software distribution system which is implemented by public license +practices. Many people have made generous contributions to the wide range of +software distributed through that system in reliance on consistent +application of that system; it is up to the author/donor to decide if he or +she is willing to distribute software through any other system and a +licensee cannot impose that choice. + +This section is intended to make thoroughly clear what is believed to be a +consequence of the rest of this License. + +12. If the distribution and/or use of the Library is restricted in certain +countries either by patents or by copyrighted interfaces, the original +copyright holder who places the Library under this License may add an +explicit geographical distribution limitation excluding those countries, so +that distribution is permitted only in or among countries not thus excluded. +In such case, this License incorporates the limitation as if written in the +body of this License. + +13. The Free Software Foundation may publish revised and/or new versions of +the Library General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Library does not specify a license version +number, you may choose any version ever published by the Free Software +Foundation. + +14. If you wish to incorporate parts of the Library into other free programs +whose distribution conditions are incompatible with these, write to the +author to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals of +preserving the free status of all derivatives of our free software and of +promoting the sharing and reuse of software generally. + +NO WARRANTY + +15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR +THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO +THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD THE LIBRARY +PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR +CORRECTION. + +16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE LIBRARY (INCLUDING BUT NOT LIMITED TO +LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR +THIRD PARTIES OR A FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER +SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + +END OF TERMS AND CONDITIONS + +This library is free software; you can redistribute it and/or +modify it under the terms of the GNU Library General Public +License version 2 as published by the Free Software Foundation. + +This library is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + + diff --git a/sourcecodes/bnt-master/docs/majorFeatures.html b/sourcecodes/bnt-master/docs/majorFeatures.html new file mode 100644 index 00000000..11005a85 --- /dev/null +++ b/sourcecodes/bnt-master/docs/majorFeatures.html @@ -0,0 +1,113 @@ + +<h2><a name="features">Major features</h2> +<ul> + +<li> BNT supports many types of +<b>conditional probability distributions</b> (nodes), +and it is easy to add more. +<ul> +<li>Tabular (multinomial) +<li>Gaussian +<li>Softmax (logistic/ sigmoid) +<li>Multi-layer perceptron (neural network) +<li>Noisy-or +<li>Deterministic +</ul> +<p> + +<li> BNT supports <b>decision and utility nodes</b>, as well as chance +nodes, +i.e., influence diagrams as well as Bayes nets. +<p> + +<li> BNT supports static and dynamic BNs (useful for modelling dynamical systems +and sequence data). +<p> + +<li> BNT supports many different <b>inference algorithms</b>, +and it is easy to add more. + +<ul> +<li> Exact inference for static BNs: +<ul> +<li>junction tree +<li>variable elimination +<li>brute force enumeration (for discrete nets) +<li>linear algebra (for Gaussian nets) +<li>Pearl's algorithm (for polytrees) +<li>quickscore (for QMR) +</ul> + +<p> +<li> Approximate inference for static BNs: +<ul> +<li>likelihood weighting +<li> Gibbs sampling +<li>loopy belief propagation +</ul> + +<p> +<li> Exact inference for DBNs: +<ul> +<li>junction tree +<li>frontier algorithm +<li>forwards-backwards (for HMMs) +<li>Kalman-RTS (for LDSs) +</ul> + +<p> +<li> Approximate inference for DBNs: +<ul> +<li>Boyen-Koller +<li>factored-frontier/loopy belief propagation +</ul> + +</ul> +<p> + +<li> +BNT supports several methods for <b>parameter learning</b>, +and it is easy to add more. +<ul> + +<li> Batch MLE/MAP parameter learning using EM. +(Each node type has its own M method, e.g. softmax nodes use IRLS,<br> +and each inference engine has its own E method, so the code is fully modular.) + +<li> Sequential/batch Bayesian parameter learning (for fully observed tabular nodes only). +</ul> + + +<p> +<li> +BNT supports several methods for <b>regularization</b>, +and it is easy to add more. +<ul> +<li> Any node can have its parameters clamped (made non-adjustable). +<li> Any set of compatible nodes can have their parameters tied (c.f., +weight sharing in a neural net). +<li> Some node types (e.g., tabular) supports priors for MAP estimation. +<li> Gaussian covariance matrices can be declared full or diagonal, and can +be tied across states of their discrete parents (if any). +</ul> + +<p> +<li> +BNT supports several methods for <b>structure learning</b>, +and it is easy to add more. +<ul> + +<li> Bayesian structure learning, +using MCMC or local search (for fully observed tabular nodes only). + +<li> Constraint-based structure learning (IC/PC and IC*/FCI). +</ul> + + +<p> +<li> The source code is extensively documented, object-oriented, and free, making it +an excellent tool for teaching, research and rapid prototyping. + +</ul> + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/docs/mathbymatlab.gif b/sourcecodes/bnt-master/docs/mathbymatlab.gif new file mode 100644 index 00000000..0de8d7a0 --- /dev/null +++ b/sourcecodes/bnt-master/docs/mathbymatlab.gif Binary files differdiff --git a/sourcecodes/bnt-master/docs/matlab_comparison.html b/sourcecodes/bnt-master/docs/matlab_comparison.html new file mode 100644 index 00000000..bdf75aae --- /dev/null +++ b/sourcecodes/bnt-master/docs/matlab_comparison.html @@ -0,0 +1,62 @@ +<html> <head> +<title>Comparison of Matlab, R/S/Splus, Gauss, etc.</title> +</head> + +<body> +<!--<body bgcolor="#FFFFFF"> --> + +<h1>Comparison of Matlab, R/S/Splus, Gauss, etc.</h1> + +<ul> +<li> <a href="http://www.scientificweb.com/ncrunch/">Comparison of +mathematical programs for data analysis</a>, +Stefan Steinhaus, tech report, 2000. +<br> +This is a very detailed comparison of features and speed of several +interactive scientific programming environments, e.g. Matlab, +Mathematica, Splus. + + +<!-- +<p> +<li> <a href="Papers/gauss.econ.review.ps">Econometric +programming environments: Gauss, Ox and S-PLUS</a>, +Francisco Cribari-Neto. +J. of Applied Econometrics, 12(1):77-89, 1997 +<br> +Ox can not be used interactively, and has a C-style syntax (it even +requires users to pre-declare variables!). Its only advantage is speed. +S-Plus has tons of features and good documentation, but is slow. +Gauss is somewhere in between. + + +<p> +<li> <a href="Papers/matlab.econ.review.ps">MATLAB as an econometric +programming environment</a>, +Francisco Cribari-Neto and Mark J. Jensen. +J. of Applied Econometrics, 12(6):735-432, 1997. +<br> +The basic conclusion is that Matlab has excellent graphics and +sparse-matrix facilities, but is slower than Gauss/Ox (especially on +code +with loops), and has few statistical routines built-in (one must buy the +stats toolbox). + + + +<p> +<li> <a href="Papers/R.econ.review.ps">R: Yet another econometric +programming environment</a>, +Francisco Cribari-Neto and S. Zarkos. +J. of Applied Econometrics, 14(3):319-329, 1999. +<br> +The basic conclusion is that R is much faster than Splus on +code with loops, but a little bit slower on vectorized code. (Gauss/ +Ox is much faster than both; in my experience, R and Matlab have about +the same speed.) +However, R has much better memory management than Splus, and R is free. Otherwise, R/S/Splus +are essentially the same. +--> + + +</ul> diff --git a/sourcecodes/bnt-master/docs/numDAGsEqn.png b/sourcecodes/bnt-master/docs/numDAGsEqn.png new file mode 100644 index 00000000..3524947b --- /dev/null +++ b/sourcecodes/bnt-master/docs/numDAGsEqn.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/numDAGsEqn2.png b/sourcecodes/bnt-master/docs/numDAGsEqn2.png new file mode 100644 index 00000000..0898c969 --- /dev/null +++ b/sourcecodes/bnt-master/docs/numDAGsEqn2.png Binary files differdiff --git a/sourcecodes/bnt-master/docs/param_tieing.html b/sourcecodes/bnt-master/docs/param_tieing.html new file mode 100644 index 00000000..b426fe30 --- /dev/null +++ b/sourcecodes/bnt-master/docs/param_tieing.html @@ -0,0 +1,47 @@ + +<h2>Example of more complex parameter tieing</h2> + +We now give a more complex pattern of parameter tieing. +(This example is due to Rainer Deventer.) +The structure is as follows: +<p> +<img src="Figures/rainer_tied.gif" height="400"> +<!--<img src="rainer_dbn.jpg" height="600">--> +<p> +Since nodes 2 and 3 in slice 2 (N7 and N8) +have different parents than their counterparts in slice 1 (N2 and N3), +they must be put into different equivalence classes. +Hence we define +<pre> +eclass1 = [1 2 3 4 5]; +eclass2 = [1 6 7 4 5]; +</pre> +The dotted bubbles represent the equivalence classes. +Node 7 is the representative node for equivalence class +6, and node 8 is the rep. for class 7, so we need to write +<pre> +bnet.CPD{6} = xxx_CPD(bnet, 7, xxx); +bnet.CPD{7} = xxx_CPD(bnet, 8, xxx); +</pre> +In general, you can use the following code fragment: +<pre> +eclass = bnet.equiv_class(:); +for e=1:max(eclass) + i = bnet.rep_of_eclass(e); + bnet.CPD{e} = xxx_CPD(bnet,i); +end +</pre> +<!-- +which is equivalent to +<pre> +E = max(eclass); +rep = zeros(1,E); +for e=1:E + mems = find(eclass==e); + rep(e) = mems(1); +end +for e=1:E + bnet.CPD{e} = xxx_CPD(bnet, rep(e)); +end +</pre> +--> diff --git a/sourcecodes/bnt-master/docs/supportedModels.html b/sourcecodes/bnt-master/docs/supportedModels.html new file mode 100644 index 00000000..af29d18e --- /dev/null +++ b/sourcecodes/bnt-master/docs/supportedModels.html @@ -0,0 +1,32 @@ + <h2><a name="models">Supported probabilistic models</h2> +<p> +It is trivial to implement all of +the following probabilistic models using the toolbox. +<ul> +<li>Static +<ul> +<li> Linear regression, logistic regression, hierarchical mixtures of experts + +<li> Naive Bayes classifiers, mixtures of Gaussians, +sigmoid belief nets + +<li> Factor analysis, probabilistic +PCA, probabilistic ICA, mixtures of these models + +</ul> + +<li>Dynamic +<ul> + +<li> HMMs, Factorial HMMs, coupled HMMs, input-output HMMs, DBNs + +<li> Kalman filters, ARMAX models, switching Kalman filters, +tree-structured Kalman filters, multiscale AR models + +</ul> + +<li> Many other combinations, for which there are (as yet) no names! + +</ul> + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/docs/usage.html b/sourcecodes/bnt-master/docs/usage.html new file mode 100644 index 00000000..018301e3 --- /dev/null +++ b/sourcecodes/bnt-master/docs/usage.html @@ -0,0 +1,3072 @@ +<HEAD> +<TITLE>How to use the Bayes Net Toolbox</TITLE> +</HEAD> + +<BODY BGCOLOR="#FFFFFF"> +<!-- white background is better for the pictures and equations --> + +<h1>How to use the Bayes Net Toolbox</h1> + +This documentation was last updated on 29 October 2007. +<br> +Click +<a href="http://bnt.insa-rouen.fr/">here</a> +for a French version of this documentation (last updated in 2005). +<p> + +<ul> +<li> <a href="install.html">Installation</a> + +<li> <a href="#basics">Creating your first Bayes net</a> + <ul> + <li> <a href="#basics">Creating a model by hand</a> + <li> <a href="#file">Loading a model from a file</a> + <li> <a href="#GUI">Creating a model using a GUI</a> + <li> <a href="graphviz.html">Graph visualization</a> + </ul> + +<li> <a href="#inference">Inference</a> + <ul> + <li> <a href="#marginal">Computing marginal distributions</a> + <li> <a href="#joint">Computing joint distributions</a> + <li> <a href="#soft">Soft/virtual evidence</a> + <li> <a href="#mpe">Most probable explanation</a> + </ul> + +<li> <a href="#cpd">Conditional Probability Distributions</a> + <ul> + <li> <a href="#tabular">Tabular (multinomial) nodes</a> + <li> <a href="#noisyor">Noisy-or nodes</a> + <li> <a href="#deterministic">Other (noisy) deterministic nodes</a> + <li> <a href="#softmax">Softmax (multinomial logit) nodes</a> + <li> <a href="#mlp">Neural network nodes</a> + <li> <a href="#root">Root nodes</a> + <li> <a href="#gaussian">Gaussian nodes</a> + <li> <a href="#glm">Generalized linear model nodes</a> + <li> <a href="#dtree">Classification/regression tree nodes</a> + <li> <a href="#nongauss">Other continuous distributions</a> + <li> <a href="#cpd_summary">Summary of CPD types</a> + </ul> + +<li> <a href="#examples">Example models</a> + <ul> + <li> <a + href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt"> +Gaussian mixture models</a> + <li> <a href="#pca">PCA, ICA, and all that</a> + <li> <a href="#mixep">Mixtures of experts</a> + <li> <a href="#hme">Hierarchical mixtures of experts</a> + <li> <a href="#qmr">QMR</a> + <li> <a href="#cg_model">Conditional Gaussian models</a> + <li> <a href="#hybrid">Other hybrid models</a> + </ul> + +<li> <a href="#param_learning">Parameter learning</a> + <ul> + <li> <a href="#load_data">Loading data from a file</a> + <li> <a href="#mle_complete">Maximum likelihood parameter estimation from complete data</a> + <li> <a href="#prior">Parameter priors</a> + <li> <a href="#bayes_learn">(Sequential) Bayesian parameter updating from complete data</a> + <li> <a href="#em">Maximum likelihood parameter estimation with missing values (EM)</a> + <li> <a href="#tying">Parameter tying</a> + </ul> + +<li> <a href="#structure_learning">Structure learning</a> + <ul> + <li> <a href="#enumerate">Exhaustive search</a> + <li> <a href="#K2">K2</a> + <li> <a href="#hill_climb">Hill-climbing</a> + <li> <a href="#mcmc">MCMC</a> + <li> <a href="#active">Active learning</a> + <li> <a href="#struct_em">Structural EM</a> + <li> <a href="#graphdraw">Visualizing the learned graph structure</a> + <li> <a href="#constraint">Constraint-based methods</a> + </ul> + + +<li> <a href="#engines">Inference engines</a> + <ul> + <li> <a href="#jtree">Junction tree</a> + <li> <a href="#varelim">Variable elimination</a> + <li> <a href="#global">Global inference methods</a> + <li> <a href="#quickscore">Quickscore</a> + <li> <a href="#belprop">Belief propagation</a> + <li> <a href="#sampling">Sampling (Monte Carlo)</a> + <li> <a href="#engine_summary">Summary of inference engines</a> + </ul> + + +<li> <a href="#influence">Influence diagrams/ decision making</a> + + +<li> <a href="usage_dbn.html">DBNs, HMMs, Kalman filters and all that</a> +</ul> + +</ul> + + + +<h1><a name="basics">Creating your first Bayes net</h1> + +To define a Bayes net, you must specify the graph structure and then +the parameters. We look at each in turn, using a simple example +(adapted from Russell and +Norvig, "Artificial Intelligence: a Modern Approach", Prentice Hall, +1995, p454). + + +<h2>Graph structure</h2> + + +Consider the following network. + +<p> +<center> +<IMG SRC="Figures/sprinkler.gif"> +</center> +<p> + +<P> +To specify this directed acyclic graph (dag), we create an adjacency matrix: +<PRE> +N = 4; +dag = zeros(N,N); +C = 1; S = 2; R = 3; W = 4; +dag(C,[R S]) = 1; +dag(R,W) = 1; +dag(S,W)=1; +</PRE> +<P> +We have numbered the nodes as follows: +Cloudy = 1, Sprinkler = 2, Rain = 3, WetGrass = 4. +<b>The nodes must always be numbered in topological order, i.e., +ancestors before descendants.</b> +For a more complicated graph, this is a little inconvenient: we will +see how to get around this <a href="usage_dbn.html#bat">below</a>. +<p> +In Matlab 6, you can use logical arrays instead of double arrays, +which are 4 times smaller: +<pre> +dag = false(N,N); +dag(C,[R S]) = true; +... +</pre> +However, <b>some graph functions (eg acyclic) do not work on +logical arrays</b>! +<p> +You can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>. +For details on GUIs, +click <a href="#GUI">here</a>. + +<h2>Creating the Bayes net shell</h2> + +In addition to specifying the graph structure, +we must specify the size and type of each node. +If a node is discrete, its size is the +number of possible values +each node can take on; if a node is continuous, +it can be a vector, and its size is the length of this vector. +In this case, we will assume all nodes are discrete and binary. +<PRE> +discrete_nodes = 1:N; +node_sizes = 2*ones(1,N); +</pre> +If the nodes were not binary, you could type e.g., +<pre> +node_sizes = [4 2 3 5]; +</pre> +meaning that Cloudy has 4 possible values, +Sprinkler has 2 possible values, etc. +Note that these are cardinal values, not ordinal, i.e., +they are not ordered in any way, like 'low', 'medium', 'high'. +<p> +We are now ready to make the Bayes net: +<pre> +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes); +</PRE> +By default, all nodes are assumed to be discrete, so we can also just +write +<pre> +bnet = mk_bnet(dag, node_sizes); +</PRE> +You may also specify which nodes will be observed. +If you don't know, or if this not fixed in advance, +just use the empty list (the default). +<pre> +onodes = []; +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes, 'observed', onodes); +</PRE> +Note that optional arguments are specified using a name/value syntax. +This is common for many BNT functions. +In general, to find out more about a function (e.g., which optional +arguments it takes), please see its +documentation string by typing +<pre> +help mk_bnet +</pre> +See also other <a href="matlab_tips.html">useful Matlab tips</a>. +<p> +It is possible to associate names with nodes, as follows: +<pre> +bnet = mk_bnet(dag, node_sizes, 'names', {'cloudy','S','R','W'}, 'discrete', 1:4); +</pre> +You can then refer to a node by its name: +<pre> +C = bnet.names('cloudy'); % bnet.names is an associative array +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +</pre> +This feature uses my own associative array class. + + +<h2><a name="cpt">Parameters</h2> + +A model consists of the graph structure and the parameters. +The parameters are represented by CPD objects (CPD = Conditional +Probability Distribution), which define the probability distribution +of a node given its parents. +(We will use the terms "node" and "random variable" interchangeably.) +The simplest kind of CPD is a table (multi-dimensional array), which +is suitable when all the nodes are discrete-valued. Note that the discrete +values are not assumed to be ordered in any way; that is, they +represent categorical quantities, like male and female, rather than +ordinal quantities, like low, medium and high. +(We will discuss CPDs in more detail <a href="#cpd">below</a>.) +<p> +Tabular CPDs, also called CPTs (conditional probability tables), +are stored as multidimensional arrays, where the dimensions +are arranged in the same order as the nodes, e.g., the CPT for node 4 +(WetGrass) is indexed by Sprinkler (2), Rain (3) and then WetGrass (4) itself. +Hence the child is always the last dimension. +If a node has no parents, its CPT is a column vector representing its +prior. +Note that in Matlab (unlike C), arrays are indexed +from 1, and are layed out in memory such that the first index toggles +fastest, e.g., the CPT for node 4 (WetGrass) is as follows +<P> +<P><IMG ALIGN=BOTTOM SRC="Figures/CPTgrass.gif"><P> +<P> +where we have used the convention that false==1, true==2. +We can create this CPT in Matlab as follows +<PRE> +CPT = zeros(2,2,2); +CPT(1,1,1) = 1.0; +CPT(2,1,1) = 0.1; +... +</PRE> +Here is an easier way: +<PRE> +CPT = reshape([1 0.1 0.1 0.01 0 0.9 0.9 0.99], [2 2 2]); +</PRE> +In fact, we don't need to reshape the array, since the CPD constructor +will do that for us. So we can just write +<pre> +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</pre> +The other nodes are created similarly (using the old syntax for +optional parameters) +<PRE> +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +bnet.CPD{R} = tabular_CPD(bnet, R, [0.8 0.2 0.2 0.8]); +bnet.CPD{S} = tabular_CPD(bnet, S, [0.5 0.9 0.5 0.1]); +bnet.CPD{W} = tabular_CPD(bnet, W, [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</PRE> + + +<h2><a name="rnd_cpt">Random Parameters</h2> + +If we do not specify the CPT, random parameters will be +created, i.e., each "row" of the CPT will be drawn from the uniform distribution. +To ensure repeatable results, use +<pre> +rand('state', seed); +randn('state', seed); +</pre> +To control the degree of randomness (entropy), +you can sample each row of the CPT from a Dirichlet(p,p,...) distribution. +If p << 1, this encourages "deterministic" CPTs (one entry near 1, the rest near 0). +If p = 1, each entry is drawn from U[0,1]. +If p >> 1, the entries will all be near 1/k, where k is the arity of +this node, i.e., each row will be nearly uniform. +You can do this as follows, assuming this node +is number i, and ns is the node_sizes. +<pre> +k = ns(i); +ps = parents(dag, i); +psz = prod(ns(ps)); +CPT = sample_dirichlet(p*ones(1,k), psz); +bnet.CPD{i} = tabular_CPD(bnet, i, 'CPT', CPT); +</pre> + + +<h2><a name="file">Loading a network from a file</h2> + +If you already have a Bayes net represented in the XML-based +<a href="http://www.cs.cmu.edu/afs/cs/user/fgcozman/www/Research/InterchangeFormat/"> +Bayes Net Interchange Format (BNIF)</a> (e.g., downloaded from the +<a +href="http://www.cs.huji.ac.il/labs/compbio/Repository"> +Bayes Net repository</a>), +you can convert it to BNT format using +the +<a href="http://www.digitas.harvard.edu/~ken/bif2bnt/">BIF-BNT Java +program</a> written by Ken Shan. +(This is not necessarily up-to-date.) +<p> +<b>It is currently not possible to save/load a BNT matlab object to +file</b>, but this is easily fixed if you modify all the constructors +for all the classes (see matlab documentation). + +<h2><a name="GUI">Creating a model using a GUI</h2> + +<ul> +<li>Senthil Nachimuthu +has started (Oct 07) an open source +GUI for BNT called +<a href="http://projeny.sourceforge.net">projeny</a> +using Java. This is a successor to BNJ. + +<li> +Philippe LeRay has written (Sep 05) +a +<a href="http://banquiseasi.insa-rouen.fr/projects/bnt-editor/"> +BNT GUI</a> in matlab. + +<li> +<a +href="http://www.dataonstage.com/BNT/PACKAGES/LinkStrength/index.html"> +LinkStrength</a>, + a package by Imme Ebert-Uphoff for visualizing the strength of +dependencies between nodes. + +</ul> + +<h2>Graph visualization</h2> + +Click <a href="graphviz.html">here</a> for more information +on graph visualization. + + +<h1><a name="inference">Inference</h1> + +Having created the BN, we can now use it for inference. +There are many different algorithms for doing inference in Bayes nets, +that make different tradeoffs between speed, +complexity, generality, and accuracy. +BNT therefore offers a variety of different inference +"engines". We will discuss these +in more detail <a href="#engines">below</a>. +For now, we will use the junction tree +engine, which is the mother of all exact inference algorithms. +This can be created as follows. +<pre> +engine = jtree_inf_engine(bnet); +</pre> +The other engines have similar constructors, but might take +additional, algorithm-specific parameters. +All engines are used in the same way, once they have been created. +We illustrate this in the following sections. + + +<h2><a name="marginal">Computing marginal distributions</h2> + +Suppose we want to compute the probability that the sprinker was on +given that the grass is wet. +The evidence consists of the fact that W=2. All the other nodes +are hidden (unobserved). We can specify this as follows. +<pre> +evidence = cell(1,N); +evidence{W} = 2; +</pre> +We use a 1D cell array instead of a vector to +cope with the fact that nodes can be vectors of different lengths. +In addition, the value [] can be used +to denote 'no evidence', instead of having to specify the observation +pattern as a separate argument. +(Click <a href="cellarray.html">here</a> for a quick tutorial on cell +arrays in matlab.) +<p> +We are now ready to add the evidence to the engine. +<pre> +[engine, loglik] = enter_evidence(engine, evidence); +</pre> +The behavior of this function is algorithm-specific, and is discussed +in more detail <a href="#engines">below</a>. +In the case of the jtree engine, +enter_evidence implements a two-pass message-passing scheme. +The first return argument contains the modified engine, which +incorporates the evidence. The second return argument contains the +log-likelihood of the evidence. (Not all engines are capable of +computing the log-likelihood.) +<p> +Finally, we can compute p=P(S=2|W=2) as follows. +<PRE> +marg = marginal_nodes(engine, S); +marg.T +ans = + 0.57024 + 0.42976 +p = marg.T(2); +</PRE> +We see that p = 0.4298. +<p> +Now let us add the evidence that it was raining, and see what +difference it makes. +<PRE> +evidence{R} = 2; +[engine, loglik] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, S); +p = marg.T(2); +</PRE> +We find that p = P(S=2|W=2,R=2) = 0.1945, +which is lower than +before, because the rain can ``explain away'' the +fact that the grass is wet. +<p> +You can plot a marginal distribution over a discrete variable +as a barchart using the built 'bar' function: +<pre> +bar(marg.T) +</pre> +This is what it looks like + +<p> +<center> +<IMG SRC="Figures/sprinkler_bar.gif"> +</center> +<p> + +<h2><a name="observed">Observed nodes</h2> + +What happens if we ask for the marginal on an observed node, e.g. P(W|W=2)? +An observed discrete node effectively only has 1 value (the observed + one) --- all other values would result in 0 probability. +For efficiency, BNT treats observed (discrete) nodes as if they were + set to 1, as we see below: +<pre> +evidence = cell(1,N); +evidence{W} = 2; +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, W); +m.T +ans = + 1 +</pre> +This can get a little confusing, since we assigned W=2. +So we can ask BNT to add the evidence back in by passing in an optional argument: +<pre> +m = marginal_nodes(engine, W, 1); +m.T +ans = + 0 + 1 +</pre> +This shows that P(W=1|W=2) = 0 and P(W=2|W=2) = 1. + + + +<h2><a name="joint">Computing joint distributions</h2> + +We can compute the joint probability on a set of nodes as in the +following example. +<pre> +evidence = cell(1,N); +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]); +</pre> +m is a structure. The 'T' field is a multi-dimensional array (in +this case, 3-dimensional) that contains the joint probability +distribution on the specified nodes. +<pre> +>> m.T +ans(:,:,1) = + 0.2900 0.0410 + 0.0210 0.0009 +ans(:,:,2) = + 0 0.3690 + 0.1890 0.0891 +</pre> +We see that P(S=1,R=1,W=2) = 0, since it is impossible for the grass +to be wet if both the rain and sprinkler are off. +<p> +Let us now add some evidence to R. +<pre> +evidence{R} = 2; +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]) +m = + domain: [2 3 4] + T: [2x1x2 double] +>> m.T +m.T +ans(:,:,1) = + 0.0820 + 0.0018 +ans(:,:,2) = + 0.7380 + 0.1782 +</pre> +The joint T(i,j,k) = P(S=i,R=j,W=k|evidence) +should have T(i,1,k) = 0 for all i,k, since R=1 is incompatible +with the evidence that R=2. +Instead of creating large tables with many 0s, BNT sets the effective +size of observed (discrete) nodes to 1, as explained above. +This is why m.T has size 2x1x2. +To get a 2x2x2 table, type +<pre> +m = marginal_nodes(engine, [S R W], 1) +m = + domain: [2 3 4] + T: [2x2x2 double] +>> m.T +m.T +ans(:,:,1) = + 0 0.082 + 0 0.0018 +ans(:,:,2) = + 0 0.738 + 0 0.1782 +</pre> + +<p> +Note: It is not always possible to compute the joint on arbitrary +sets of nodes: it depends on which inference engine you use, as discussed +in more detail <a href="#engines">below</a>. + + +<h2><a name="soft">Soft/virtual evidence</h2> + +Sometimes a node is not observed, but we have some distribution over +its possible values; this is often called "soft" or "virtual" +evidence. +One can use this as follows +<pre> +[engine, loglik] = enter_evidence(engine, evidence, 'soft', soft_evidence); +</pre> +where soft_evidence{i} is either [] (if node i has no soft evidence) +or is a vector representing the probability distribution over i's +possible values. +For example, if we don't know i's exact value, but we know its +likelihood ratio is 60/40, we can write evidence{i} = [] and +soft_evidence{i} = [0.6 0.4]. +<p> +Currently only jtree_inf_engine supports this option. +It assumes that all hidden nodes, and all nodes for +which we have soft evidence, are discrete. +For a longer example, see BNT/examples/static/softev1.m. + + +<h2><a name="mpe">Most probable explanation</h2> + +To compute the most probable explanation (MPE) of the evidence (i.e., +the most probable assignment, or a mode of the joint), use +<pre> +[mpe, ll] = calc_mpe(engine, evidence); +</pre> +mpe{i} is the most likely value of node i. +This calls enter_evidence with the 'maximize' flag set to 1, which +causes the engine to do max-product instead of sum-product. +The resulting max-marginals are then thresholded. +If there is more than one maximum probability assignment, we must take + care to break ties in a consistent manner (thresholding the + max-marginals may give the wrong result). To force this behavior, + type +<pre> +[mpe, ll] = calc_mpe(engine, evidence, 1); +</pre> +Note that computing the MPE is someties called abductive reasoning. + +<p> +You can also use <tt>calc_mpe_bucket</tt> written by Ron Zohar, +that does a forwards max-product pass, and then a backwards traceback +pass, which is how Viterbi is traditionally implemented. + + + +<h1><a name="cpd">Conditional Probability Distributions</h1> + +A Conditional Probability Distributions (CPD) +defines P(X(i) | X(Pa(i))), where X(i) is the i'th node, and X(Pa(i)) +are the parents of node i. There are many ways to represent this +distribution, which depend in part on whether X(i) and X(Pa(i)) are +discrete, continuous, or a combination. +We will discuss various representations below. + + +<h2><a name="tabular">Tabular nodes</h2> + +If the CPD is represented as a table (i.e., if it is a multinomial +distribution), it has a number of parameters that is exponential in +the number of parents. See the example <a href="#cpt">above</a>. + + +<h2><a name="noisyor">Noisy-or nodes</h2> + +A noisy-OR node is like a regular logical OR gate except that +sometimes the effects of parents that are on get inhibited. +Let the prob. that parent i gets inhibited be q(i). +Then a node, C, with 2 parents, A and B, has the following CPD, where +we use F and T to represent off and on (1 and 2 in BNT). +<pre> +A B P(C=off) P(C=on) +--------------------------- +F F 1.0 0.0 +T F q(A) 1-q(A) +F T q(B) 1-q(B) +T T q(A)q(B) 1-q(A)q(B) +</pre> +Thus we see that the causes get inhibited independently. +It is common to associate a "leak" node with a noisy-or CPD, which is +like a parent that is always on. This can account for all other unmodelled +causes which might turn the node on. +<p> +The noisy-or distribution is similar to the logistic distribution. +To see this, let the nodes, S(i), have values in {0,1}, and let q(i,j) +be the prob. that j inhibits i. Then +<pre> +Pr(S(i)=1 | parents(S(i))) = 1 - prod_{j} q(i,j)^S(j) +</pre> +Now define w(i,j) = -ln q(i,j) and rho(x) = 1-exp(-x). Then +<pre> +Pr(S(i)=1 | parents(S(i))) = rho(sum_j w(i,j) S(j)) +</pre> +For a sigmoid node, we have +<pre> +Pr(S(i)=1 | parents(S(i))) = sigma(-sum_j w(i,j) S(j)) +</pre> +where sigma(x) = 1/(1+exp(-x)). Hence they differ in the choice of +the activation function (although both are monotonically increasing). +In addition, in the case of a noisy-or, the weights are constrained to be +positive, since they derive from probabilities q(i,j). +In both cases, the number of parameters is <em>linear</em> in the +number of parents, unlike the case of a multinomial distribution, +where the number of parameters is exponential in the number of parents. +We will see an example of noisy-OR nodes <a href="#qmr">below</a>. + + +<h2><a name="deterministic">Other (noisy) deterministic nodes</h2> + +Deterministic CPDs for discrete random variables can be created using +the deterministic_CPD class. It is also possible to 'flip' the output +of the function with some probability, to simulate noise. +The boolean_CPD class is just a special case of a +deterministic CPD, where the parents and child are all binary. +<p> +Both of these classes are just "syntactic sugar" for the tabular_CPD +class. + + + +<h2><a name="softmax">Softmax nodes</h2> + +If we have a discrete node with a continuous parent, +we can define its CPD using a softmax function +(also known as the multinomial logit function). +This acts like a soft thresholding operator, and is defined as follows: +<pre> + exp(w(:,i)'*x + b(i)) +Pr(Q=i | X=x) = ----------------------------- + sum_j exp(w(:,j)'*x + b(j)) + +</pre> +The parameters of a softmax node, w(:,i) and b(i), i=1..|Q|, have the +following interpretation: w(:,i)-w(:,j) is the normal vector to the +decision boundary between classes i and j, +and b(i)-b(j) is its offset (bias). For example, suppose +X is a 2-vector, and Q is binary. Then +<pre> +w = [1 -1; + 0 0]; + +b = [0 0]; +</pre> +means class 1 are points in the 2D plane with positive x coordinate, +and class 2 are points in the 2D plane with negative x coordinate. +If w has large magnitude, the decision boundary is sharp, otherwise it +is soft. +In the special case that Q is binary (0/1), the softmax function reduces to the logistic +(sigmoid) function. +<p> +Fitting a softmax function can be done using the iteratively reweighted +least squares (IRLS) algorithm. +We use the implementation from +<a href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +Note that since +the softmax distribution is not in the exponential family, it does not +have finite sufficient statistics, and hence we must store all the +training data in uncompressed form. +If this takes too much space, one should use online (stochastic) gradient +descent (not implemented in BNT). +<p> +If a softmax node also has discrete parents, +we use a different set of w/b parameters for each combination of +parent values, as in the <a href="#gaussian">conditional linear +Gaussian CPD</a>. +This feature was implemented by Pierpaolo Brutti. +He is currently extending it so that discrete parents can be treated +as if they were continuous, by adding indicator variables to the X +vector. +<p> +We will see an example of softmax nodes <a href="#mixexp">below</a>. + + +<h2><a name="mlp">Neural network nodes</h2> + +Pierpaolo Brutti has implemented the mlp_CPD class, which uses a multi layer perceptron +to implement a mapping from continuous parents to discrete children, +similar to the softmax function. +(If there are also discrete parents, it creates a mixture of MLPs.) +It uses code from <a +href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +This is work in progress. + +<h2><a name="root">Root nodes</h2> + +A root node has no parents and no parameters; it can be used to model +an observed, exogeneous input variable, i.e., one which is "outside" +the model. +This is useful for conditional density models. +We will see an example of root nodes <a href="#mixexp">below</a>. + + +<h2><a name="gaussian">Gaussian nodes</h2> + +We now consider a distribution suitable for the continuous-valued nodes. +Suppose the node is called Y, its continuous parents (if any) are +called X, and its discrete parents (if any) are called Q. +The distribution on Y is defined as follows: +<pre> +- no parents: Y ~ N(mu, Sigma) +- cts parents : Y|X=x ~ N(mu + W x, Sigma) +- discrete parents: Y|Q=i ~ N(mu(:,i), Sigma(:,:,i)) +- cts and discrete parents: Y|X=x,Q=i ~ N(mu(:,i) + W(:,:,i) * x, Sigma(:,:,i)) +</pre> +where N(mu, Sigma) denotes a Normal distribution with mean mu and +covariance Sigma. Let |X|, |Y| and |Q| denote the sizes of X, Y and Q +respectively. +If there are no discrete parents, |Q|=1; if there is +more than one, then |Q| = a vector of the sizes of each discrete parent. +If there are no continuous parents, |X|=0; if there is more than one, +then |X| = the sum of their sizes. +Then mu is a |Y|*|Q| vector, Sigma is a |Y|*|Y|*|Q| positive +semi-definite matrix, and W is a |Y|*|X|*|Q| regression (weight) +matrix. +<p> +We can create a Gaussian node with random parameters as follows. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i); +</pre> +We can specify the value of one or more of the parameters as in the +following example, in which |Y|=2, and |Q|=1. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i, 'mean', [0; 0], 'weights', randn(Y,X), 'cov', eye(Y)); +</pre> +<p> +We will see an example of conditional linear Gaussian nodes <a +href="#cg_model">below</a>. +<p> +<b>When learning Gaussians from data</b>, it is helpful to ensure the +data has a small magnitde +(see e.g., KPMstats/standardize) to prevent numerical problems. +Unless you have a lot of data, it is also a very good idea to use +diagonal instead of full covariance matrices. +(BNT does not currently support spherical covariances, although it +would be easy to add, since KPMstats/clg_Mstep supports this option; +you would just need to modify gaussian_CPD/update_ess to accumulate +weighted inner products.) + + + +<h2><a name="nongauss">Other continuous distributions</h2> + +Currently BNT does not support any CPDs for continuous nodes other +than the Gaussian. +However, you can use a mixture of Gaussians to +approximate other continuous distributions. We will see some an example +of this with the IFA model <a href="#pca">below</a>. + + +<h2><a name="glm">Generalized linear model nodes</h2> + +In the future, we may incorporate some of the functionality of +<a href = +"http://www.sci.usq.edu.au/staff/dunn/glmlab/glmlab.html">glmlab</a> +into BNT. + + +<h2><a name="dtree">Classification/regression tree nodes</h2> + +We plan to add classification and regression trees to define CPDs for +discrete and continuous nodes, respectively. +Trees have many advantages: they are easy to interpret, they can do +feature selection, they can +handle discrete and continuous inputs, they do not make strong +assumptions about the form of the distribution, the number of +parameters can grow in a data-dependent way (i.e., they are +semi-parametric), they can handle missing data, etc. +However, they are not yet implemented. +<!-- +Yimin Zhang is currently (Feb '02) implementing this. +--> + + +<h2><a name="cpd_summary">Summary of CPD types</h2> + +We list all the different types of CPDs supported by BNT. +For each CPD, we specify if the child and parents can be discrete (D) or +continuous (C) (Binary (B) nodes are a special case). +We also specify which methods each class supports. +If a method is inherited, the name of the parent class is mentioned. +If a parent class calls a child method, this is mentioned. +<p> +The <tt>CPD_to_CPT</tt> method converts a CPD to a table; this +requires that the child and all parents are discrete. +The CPT might be exponentially big... +<tt>convert_to_table</tt> evaluates a CPD with evidence, and +represents the the resulting potential as an array. +This requires that the child is discrete, and any continuous parents +are observed. +<tt>convert_to_pot</tt> evaluates a CPD with evidence, and +represents the resulting potential as a dpot, gpot, cgpot or upot, as +requested. (d=discrete, g=Gaussian, cg = conditional Gaussian, u = +utility). + +<p> +When we sample a node, all the parents are observed. +When we compute the (log) probability of a node, all the parents and +the child are observed. +<p> +We also specify if the parameters are learnable. +For learning with EM, we require +the methods <tt>reset_ess</tt>, <tt>update_ess</tt> and +<tt>maximize_params</tt>. +For learning from fully observed data, we require +the method <tt>learn_params</tt>. +By default, all classes inherit this from generic_CPD, which simply +calls <tt>update_ess</tt> N times, once for each data case, followed +by <tt>maximize_params</tt>, i.e., it is like EM, without the E step. +Some classes implement a batch formula, which is quicker. +<p> +Bayesian learning means computing a posterior over the parameters +given fully observed data. +<p> +Pearl means we implement the methods <tt>compute_pi</tt> and +<tt>compute_lambda_msg</tt>, used by +<tt>pearl_inf_engine</tt>, which runs on directed graphs. +<tt>belprop_inf_engine</tt> only needs <tt>convert_to_pot</tt>.H +The pearl methods can exploit special properties of the CPDs for +computing the messages efficiently, whereas belprop does not. +<p> +The only method implemented by generic_CPD is <tt>adjustable_CPD</tt>, +which is not shown, since it is not very interesting. + + +<p> + + +<table> +<table border units = pixels><tr> +<td align=center>Name +<td align=center>Child +<td align=center>Parents +<td align=center>Comments +<td align=center>CPD_to_CPT +<td align=center>conv_to_table +<td align=center>conv_to_pot +<td align=center>sample +<td align=center>prob +<td align=center>learn +<td align=center>Bayes +<td align=center>Pearl + + +<tr> +<!-- Name--><td> +<!-- Child--><td> +<!-- Parents--><td> +<!-- Comments--><td> +<!-- CPD_to_CPT--><td> +<!-- conv_to_table--><td> +<!-- conv_to_pot--><td> +<!-- sample--><td> +<!-- prob--><td> +<!-- learn--><td> +<!-- Bayes--><td> +<!-- Pearl--><td> + +<tr> +<!-- Name--><td>boolean +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>deterministic +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>Discrete +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Calls CPD_to_CPT +<!-- conv_to_pot--><td>Calls conv_to_table +<!-- sample--><td>Calls conv_to_table +<!-- prob--><td>Calls conv_to_table +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>Gaussian +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>gmux +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>multiplexer +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>MLP +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>multi layer perceptron +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>noisy-or +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>root +<!-- Child--><td>C/D +<!-- Parents--><td>none +<!-- Comments--><td>no params +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>softmax +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>generic +<!-- Child--><td>C/D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>N +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>Tabular +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>Y +<!-- Pearl--><td>Y + +</table> + + + +<h1><a name="examples">Example models</h1> + + +<h2>Gaussian mixture models</h2> + +Richard W. DeVaul has made a detailed tutorial on how to fit mixtures +of Gaussians using BNT. Available +<a href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt">here</a>. + + +<h2><a name="pca">PCA, ICA, and all that </h2> + +In Figure (a) below, we show how Factor Analysis can be thought of as a +graphical model. Here, X has an N(0,I) prior, and +Y|X=x ~ N(mu + Wx, Psi), +where Psi is diagonal and W is called the "factor loading matrix". +Since the noise on both X and Y is diagonal, the components of these +vectors are uncorrelated, and hence can be represented as individual +scalar nodes, as we show in (b). +(This is useful if parts of the observations on the Y vector are occasionally missing.) +We usually take k=|X| << |Y|=D, so the model tries to explain +many observations using a low-dimensional subspace. + + +<center> +<table> +<tr> +<td><img src="Figures/fa.gif"> +<td><img src="Figures/fa_scalar.gif"> +<td><img src="Figures/mfa.gif"> +<td><img src="Figures/ifa.gif"> +<tr> +<td align=center> (a) +<td align=center> (b) +<td align=center> (c) +<td align=center> (d) +</table> +</center> + +<p> +We can create this model in BNT as follows. +<pre> +ns = [k D]; +dag = zeros(2,2); +dag(1,2) = 1; +bnet = mk_bnet(dag, ns, 'discrete', []); +bnet.CPD{1} = gaussian_CPD(bnet, 1, 'mean', zeros(k,1), 'cov', eye(k), ... + 'cov_type', 'diag', 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(D,1), 'cov', diag(Psi0), 'weights', W0, ... + 'cov_type', 'diag', 'clamp_mean', 1); +</pre> + +The root node is clamped to the N(0,I) distribution, so that we will +not update these parameters during learning. +The mean of the leaf node is clamped to 0, +since we assume the data has been centered (had its mean subtracted +off); this is just for simplicity. +Finally, the covariance of the leaf node is constrained to be +diagonal. W0 and Psi0 are the initial parameter guesses. + +<p> +We can fit this model (i.e., estimate its parameters in a maximum +likelihood (ML) sense) using EM, as we +explain <a href="#em">below</a>. +Not surprisingly, the ML estimates for mu and Psi turn out to be +identical to the +sample mean and variance, which can be computed directly as +<pre> +mu_ML = mean(data); +Psi_ML = diag(cov(data)); +</pre> +Note that W can only be identified up to a rotation matrix, because of +the spherical symmetry of the source. + +<p> +If we restrict Psi to be spherical, i.e., Psi = sigma*I, +there is a closed-form solution for W as well, +i.e., we do not need to use EM. +In particular, W contains the first |X| eigenvectors of the sample covariance +matrix, with scalings determined by the eigenvalues and sigma. +Classical PCA can be obtained by taking the sigma->0 limit. +For details, see + +<ul> +<li> <a href="ftp://hope.caltech.edu/pub/roweis/Empca/empca.ps"> +"EM algorithms for PCA and SPCA"</a>, Sam Roweis, NIPS 97. +(<a href="ftp://hope.caltech.edu/pub/roweis/Code/empca.tar.gz"> +Matlab software</a>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +By adding a hidden discrete variable, we can create mixtures of FA +models, as shown in (c). +Now we can explain the data using a set of subspaces. +We can create this model in BNT as follows. +<pre> +ns = [M k D]; +dag = zeros(3); +dag(1,3) = 1; +dag(2,3) = 1; +bnet = mk_bnet(dag, ns, 'discrete', 1); +bnet.CPD{1} = tabular_CPD(bnet, 1, Pi0); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(k, 1), 'cov', eye(k), 'cov_type', 'diag', ... + 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{3} = gaussian_CPD(bnet, 3, 'mean', Mu0', 'cov', repmat(diag(Psi0), [1 1 M]), ... + 'weights', W0, 'cov_type', 'diag', 'tied_cov', 1); +</pre> +Notice how the covariance matrix for Y is the same for all values of +Q; that is, the noise level in each sub-space is assumed the same. +However, we allow the offset, mu, to vary. +For details, see +<ul> + +<LI> +<a HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/tr-96-1.ps.gz"> The EM +Algorithm for Mixtures of Factor Analyzers </A>, +Ghahramani, Z. and Hinton, G.E. (1996), +University of Toronto +Technical Report CRG-TR-96-1. +(<A HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/mfa.tar.gz">Matlab software</A>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +I have included Zoubin's specialized MFA code (with his permission) +with the toolbox, so you can check that BNT gives the same results: +see 'BNT/examples/static/mfa1.m'. + +<p> +Independent Factor Analysis (IFA) generalizes FA by allowing a +non-Gaussian prior on each component of X. +(Note that we can approximate a non-Gaussian prior using a mixture of +Gaussians.) +This means that the likelihood function is no longer rotationally +invariant, so we can uniquely identify W and the hidden +sources X. +IFA also allows a non-diagonal Psi (i.e. correlations between the components of Y). +We recover classical Independent Components Analysis (ICA) +in the Psi -> 0 limit, and by assuming that |X|=|Y|, so that the +weight matrix W is square and invertible. +For details, see +<ul> +<li> +<a href="http://www.gatsby.ucl.ac.uk/~hagai/ifa.ps">Independent Factor +Analysis</a>, H. Attias, Neural Computation 11: 803--851, 1998. +</ul> + + + +<h2><a name="mixexp">Mixtures of experts</h2> + +As an example of the use of the softmax function, +we introduce the Mixture of Experts model. +<!-- +We also show +the Hierarchical Mixture of Experts model, where the hierarchy has two +levels. +(This is essentially a probabilistic decision tree of height two.) +--> +As before, +circles denote continuous-valued nodes, +squares denote discrete nodes, clear +means hidden, and shaded means observed. +<p> +<center> +<table> +<tr> +<td><img src="Figures/mixexp.gif"> +<!-- +<td><img src="Figures/hme.gif"> +--> +</table> +</center> +<p> +X is the observed +input, Y is the output, and +the Q nodes are hidden "gating" nodes, which select the appropriate +set of parameters for Y. During training, Y is assumed observed, +but for testing, the goal is to predict Y given X. +Note that this is a <em>conditional</em> density model, so we don't +associate any parameters with X. +Hence X's CPD will be a root CPD, which is a way of modelling +exogenous nodes. +If the output is a continuous-valued quantity, +we assume the "experts" are linear-regression units, +and set Y's CPD to linear-Gaussian. +If the output is discrete, we set Y's CPD to a softmax function. +The Q CPDs will always be softmax functions. + +<p> +As a concrete example, consider the mixture of experts model where X and Y are +scalars, and Q is binary. +This is just piecewise linear regression, where +we have two line segments, i.e., +<P> +<IMG ALIGN=BOTTOM SRC="Eqns/lin_reg_eqn.gif"> +<P> +We can create this model with random parameters as follows. +(This code is bundled in BNT/examples/static/mixexp2.m.) +<PRE> +X = 1; +Q = 2; +Y = 3; +dag = zeros(3,3); +dag(X,[Q Y]) = 1 +dag(Q,Y) = 1; +ns = [1 2 1]; % make X and Y scalars, and have 2 experts +onodes = [1 3]; +bnet = mk_bnet(dag, ns, 'discrete', 2, 'observed', onodes); + +rand('state', 0); +randn('state', 0); +bnet.CPD{1} = root_CPD(bnet, 1); +bnet.CPD{2} = softmax_CPD(bnet, 2); +bnet.CPD{3} = gaussian_CPD(bnet, 3); +</PRE> +Now let us fit this model using <a href="#em">EM</a>. +First we <a href="#load_data">load the data</a> (1000 training cases) and plot them. +<P> +<PRE> +data = load('/examples/static/Misc/mixexp_data.txt', '-ascii'); +plot(data(:,1), data(:,2), '.'); +</PRE> +<p> +<center> +<IMG SRC="Figures/mixexp_data.gif"> +</center> +<p> +This is what the model looks like before training. +(Thanks to Thomas Hofman for writing this plotting routine.) +<p> +<center> +<IMG SRC="Figures/mixexp_before.gif"> +</center> +<p> +Now let's train the model, and plot the final performance. +(We will discuss how to train models in more detail <a href="#param_learning">below</a>.) +<P> +<PRE> +ncases = size(data, 1); % each row of data is a training case +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); % each column of cases is a training case +engine = jtree_inf_engine(bnet); +max_iter = 20; +[bnet2, LLtrace] = learn_params_em(engine, cases, max_iter); +</PRE> +(We specify which nodes will be observed when we create the engine. +Hence BNT knows that the hidden nodes are all discrete. +For complex models, this can lead to a significant speedup.) +Below we show what the model looks like after 16 iterations of EM +(with 100 IRLS iterations per M step), when it converged +using the default convergence tolerance (that the +fractional change in the log-likelihood be less than 1e-3). +Before learning, the log-likelihood was +-322.927442; afterwards, it was -13.728778. +<p> +<center> +<IMG SRC="Figures/mixexp_after.gif"> +</center> +(See BNT/examples/static/mixexp2.m for details of the code.) + + + +<h2><a name="hme">Hierarchical mixtures of experts</h2> + +A hierarchical mixture of experts (HME) extends the mixture of experts +model by having more than one hidden node. A two-level example is shown below, along +with its more traditional representation as a neural network. +This is like a (balanced) probabilistic decision tree of height 2. +<p> +<center> +<IMG SRC="Figures/HMEforMatlab.jpg"> +</center> +<p> +<a href="mailto:pbrutti@stat.cmu.edu">Pierpaolo Brutti</a> +has written an extensive set of routines for HMEs, +which are bundled with BNT: see the examples/static/HME directory. +These routines allow you to choose the number of hidden (gating) +layers, and the form of the experts (softmax or MLP). +See the file hmemenu, which provides a demo. +For example, the figure below shows the decision boundaries learned +for a ternary classification problem, using a 2 level HME with softmax +gates and softmax experts; the training set is on the left, the +testing set on the right. +<p> +<center> +<!--<IMG SRC="Figures/hme_dec_boundary.gif">--> +<IMG SRC="Figures/hme_dec_boundary.png"> +</center> +<p> + + +<p> +For more details, see the following: +<ul> + +<li> <a href="http://www.cs.berkeley.edu/~jordan/papers/hierarchies.ps.Z"> +Hierarchical mixtures of experts and the EM algorithm</a> +M. I. Jordan and R. A. Jacobs. Neural Computation, 6, 181-214, 1994. + +<li> <a href = +"http://www.cs.berkeley.edu/~dmartin/software">David Martin's +matlab code for HME</a> + +<li> <a +href="http://www.cs.berkeley.edu/~jordan/papers/uai.ps.Z">Why the +logistic function? A tutorial discussion on +probabilities and neural networks.</a> M. I. Jordan. MIT Computational +Cognitive Science Report 9503, August 1995. + +<li> "Generalized Linear Models", McCullagh and Nelder, Chapman and +Halll, 1983. + +<li> +"Improved learning algorithms for mixtures of experts in multiclass +classification". +K. Chen, L. Xu, H. Chi. +Neural Networks (1999) 12: 1229-1252. + +<li> <a href="http://www.oigeeza.com/steve/"> +Classification Using Hierarchical Mixtures of Experts</a> +S.R. Waterhouse and A.J. Robinson. +In Proc. IEEE Workshop on Neural Network for Signal Processing IV (1994), pp. 177-186 + +<li> <a href="http://www.idiap.ch/~perry/"> +Localized mixtures of experts</a>, +P. Moerland, 1998. + +<li> "Nonlinear gated experts for time series", +A.S. Weigend and M. Mangeas, 1995. + +</ul> + + +<h2><a name="qmr">QMR</h2> + +Bayes nets originally arose out of an attempt to add probabilities to +expert systems, and this is still the most common use for BNs. +A famous example is +QMR-DT, a decision-theoretic reformulation of the Quick Medical +Reference (QMR) model. +<p> +<center> +<IMG ALIGN=BOTTOM SRC="Figures/qmr.gif"> +</center> +Here, the top layer represents hidden disease nodes, and the bottom +layer represents observed symptom nodes. +The goal is to infer the posterior probability of each disease given +all the symptoms (which can be present, absent or unknown). +Each node in the top layer has a Bernoulli prior (with a low prior +probability that the disease is present). +Since each node in the bottom layer has a high fan-in, we use a +noisy-OR parameterization; each disease has an independent chance of +causing each symptom. +The real QMR-DT model is copyright, but +we can create a random QMR-like model as follows. +<pre> +function bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% MK_QMR_BNET Make a QMR model +% bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% +% G(i,j) = 1 iff there is an arc from disease i to finding j +% inhibit(i,j) = inhibition probability on i->j arc +% leak(j) = inhibition prob. on leak->j arc +% prior(i) = prob. disease i is on + +[Ndiseases Nfindings] = size(inhibit); +N = Ndiseases + Nfindings; +finding_node = Ndiseases+1:N; +ns = 2*ones(1,N); +dag = zeros(N,N); +dag(1:Ndiseases, finding_node) = G; +bnet = mk_bnet(dag, ns, 'observed', finding_node); + +for d=1:Ndiseases + CPT = [1-prior(d) prior(d)]; + bnet.CPD{d} = tabular_CPD(bnet, d, CPT'); +end + +for i=1:Nfindings + fnode = finding_node(i); + ps = parents(G, i); + bnet.CPD{fnode} = noisyor_CPD(bnet, fnode, leak(i), inhibit(ps, i)); +end +</pre> +In the file BNT/examples/static/qmr1, we create a random bipartite +graph G, with 5 diseases and 10 findings, and random parameters. +(In general, to create a random dag, use 'mk_random_dag'.) +We can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>, with the +following results: +<p> +<img src="Figures/qmr.rnd.jpg"> + +<p> +Now let us put some random evidence on all the leaves except the very +first and very last, and compute the disease posteriors. +<pre> +pos = 2:floor(Nfindings/2); +neg = (pos(end)+1):(Nfindings-1); +onodes = myunion(pos, neg); +evidence = cell(1, N); +evidence(findings(pos)) = num2cell(repmat(2, 1, length(pos))); +evidence(findings(neg)) = num2cell(repmat(1, 1, length(neg))); + +engine = jtree_inf_engine(bnet); +[engine, ll] = enter_evidence(engine, evidence); +post = zeros(1, Ndiseases); +for i=diseases(:)' + m = marginal_nodes(engine, i); + post(i) = m.T(2); +end +</pre> +Junction tree can be quite slow on large QMR models. +Fortunately, it is possible to exploit properties of the noisy-OR +function to speed up exact inference using an algorithm called +<a href="#quickscore">quickscore</a>, discussed below. + + + + + +<h2><a name="cg_model">Conditional Gaussian models</h2> + +A conditional Gaussian model is one in which, conditioned on all the discrete +nodes, the distribution over the remaining (continuous) nodes is +multivariate Gaussian. This means we can have arcs from discrete (D) +to continuous (C) nodes, but not vice versa. +(We <em>are</em> allowed C->D arcs if the continuous nodes are observed, +as in the <a href="#mixexp">mixture of experts</a> model, +since this distribution can be represented with a discrete potential.) +<p> +We now give an example of a CG model, from +the paper "Propagation of Probabilities, Means amd +Variances in Mixed Graphical Association Models", Steffen Lauritzen, +JASA 87(420):1098--1108, 1992 (reprinted in the book "Probabilistic Networks and Expert +Systems", R. G. Cowell, A. P. Dawid, S. L. Lauritzen and +D. J. Spiegelhalter, Springer, 1999.) + +<h3>Specifying the graph</h3> + +Consider the model of waste emissions from an incinerator plant shown below. +We follow the standard convention that shaded nodes are observed, +clear nodes are hidden. +We also use the non-standard convention that +square nodes are discrete (tabular) and round nodes are +Gaussian. + +<p> +<center> +<IMG SRC="Figures/cg1.gif"> +</center> +<p> + +We can create this model as follows. +<pre> +F = 1; W = 2; E = 3; B = 4; C = 5; D = 6; Min = 7; Mout = 8; L = 9; +n = 9; + +dag = zeros(n); +dag(F,E)=1; +dag(W,[E Min D]) = 1; +dag(E,D)=1; +dag(B,[C D])=1; +dag(D,[L Mout])=1; +dag(Min,Mout)=1; + +% node sizes - all cts nodes are scalar, all discrete nodes are binary +ns = ones(1, n); +dnodes = [F W B]; +cnodes = mysetdiff(1:n, dnodes); +ns(dnodes) = 2; + +bnet = mk_bnet(dag, ns, 'discrete', dnodes); +</pre> +'dnodes' is a list of the discrete nodes; 'cnodes' is the continuous +nodes. 'mysetdiff' is a faster version of the built-in 'setdiff'. +<p> + + +<h3>Specifying the parameters</h3> + +The parameters of the discrete nodes can be specified as follows. +<pre> +bnet.CPD{B} = tabular_CPD(bnet, B, 'CPT', [0.85 0.15]); % 1=stable, 2=unstable +bnet.CPD{F} = tabular_CPD(bnet, F, 'CPT', [0.95 0.05]); % 1=intact, 2=defect +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [2/7 5/7]); % 1=industrial, 2=household +</pre> + +<p> +The parameters of the continuous nodes can be specified as follows. +<pre> +bnet.CPD{E} = gaussian_CPD(bnet, E, 'mean', [-3.9 -0.4 -3.2 -0.5], ... + 'cov', [0.00002 0.0001 0.00002 0.0001]); +bnet.CPD{D} = gaussian_CPD(bnet, D, 'mean', [6.5 6.0 7.5 7.0], ... + 'cov', [0.03 0.04 0.1 0.1], 'weights', [1 1 1 1]); +bnet.CPD{C} = gaussian_CPD(bnet, C, 'mean', [-2 -1], 'cov', [0.1 0.3]); +bnet.CPD{L} = gaussian_CPD(bnet, L, 'mean', 3, 'cov', 0.25, 'weights', -0.5); +bnet.CPD{Min} = gaussian_CPD(bnet, Min, 'mean', [0.5 -0.5], 'cov', [0.01 0.005]); +bnet.CPD{Mout} = gaussian_CPD(bnet, Mout, 'mean', 0, 'cov', 0.002, 'weights', [1 1]); +</pre> + + +<h3><a name="cg_infer">Inference</h3> + +<!--Let us perform inference in the <a href="#cg_model">waste incinerator example</a>.--> +First we compute the unconditional marginals. +<pre> +engine = jtree_inf_engine(bnet); +evidence = cell(1,n); +[engine, ll] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, E); +</pre> +<!--(Of course, we could use <tt>cond_gauss_inf_engine</tt> instead of jtree.)--> +'marg' is a structure that contains the fields 'mu' and 'Sigma', which +contain the mean and (co)variance of the marginal on E. +In this case, they are both scalars. +Let us check they match the published figures (to 2 decimal places). +<!--(We can't expect +more precision than this in general because I have implemented the algorithm of +Lauritzen (1992), which can be numerically unstable.)--> +<pre> +tol = 1e-2; +assert(approxeq(marg.mu, -3.25, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.709, tol)); +</pre> +We can compute the other posteriors similarly. +Now let us add some evidence. +<pre> +evidence = cell(1,n); +evidence{W} = 1; % industrial +evidence{L} = 1.1; +evidence{C} = -0.9; +[engine, ll] = enter_evidence(engine, evidence); +</pre> +Now we find +<pre> +marg = marginal_nodes(engine, E); +assert(approxeq(marg.mu, -3.8983, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.0763, tol)); +</pre> + + +We can also compute the joint probability on a set of nodes. +For example, P(D, Mout | evidence) is a 2D Gaussian: +<pre> +marg = marginal_nodes(engine, [D Mout]) +marg = + domain: [6 8] + mu: [2x1 double] + Sigma: [2x2 double] + T: 1.0000 +</pre> +The mean is +<pre> +marg.mu +ans = + 3.6077 + 4.1077 +</pre> +and the covariance matrix is +<pre> +marg.Sigma +ans = + 0.1062 0.1062 + 0.1062 0.1182 +</pre> +It is easy to visualize this posterior using standard Matlab plotting +functions, e.g., +<pre> +gaussplot2d(marg.mu, marg.Sigma); +</pre> +produces the following picture. + +<p> +<center> +<IMG SRC="Figures/gaussplot.png"> +</center> +<p> + + +The T field indicates that the mixing weight of this Gaussian +component is 1.0. +If the joint contains discrete and continuous variables, the result +will be a mixture of Gaussians, e.g., +<pre> +marg = marginal_nodes(engine, [F E]) + domain: [1 3] + mu: [-3.9000 -0.4003] + Sigma: [1x1x2 double] + T: [0.9995 4.7373e-04] +</pre> +The interpretation is +Sigma(i,j,k) = Cov[ E(i) E(j) | F=k ]. +In this case, E is a scalar, so i=j=1; k specifies the mixture component. +<p> +We saw in the sprinkler network that BNT sets the effective size of +observed discrete nodes to 1, since they only have one legal value. +For continuous nodes, BNT sets their length to 0, +since they have been reduced to a point. +For example, +<pre> +marg = marginal_nodes(engine, [B C]) + domain: [4 5] + mu: [] + Sigma: [] + T: [0.0123 0.9877] +</pre> +It is simple to post-process the output of marginal_nodes. +For example, the file BNT/examples/static/cg1 sets the mu term of +observed nodes to their observed value, and the Sigma term to 0 (since +observed nodes have no variance). + +<p> +Note that the implemented version of the junction tree is numerically +unstable when using CG potentials +(which is why, in the example above, we only required our answers to agree with +the published ones to 2dp.) +This is why you might want to use <tt>stab_cond_gauss_inf_engine</tt>, +implemented by Shan Huang. This is described in + +<ul> +<li> "Stable Local Computation with Conditional Gaussian Distributions", +S. Lauritzen and F. Jensen, Tech Report R-99-2014, +Dept. Math. Sciences, Allborg Univ., 1999. +</ul> + +However, even the numerically stable version +can be computationally intractable if there are many hidden discrete +nodes, because the number of mixture components grows exponentially e.g., in a +<a href="usage_dbn.html#lds">switching linear dynamical system</a>. +In general, one must resort to approximate inference techniques: see +the discussion on <a href="#engines">inference engines</a> below. + + +<h2><a name="hybrid">Other hybrid models</h2> + +When we have C->D arcs, where C is hidden, we need to use +approximate inference. +One approach (not implemented in BNT) is described in +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/hybrid_uai99.ps.gz">A +Variational Approximation for Bayesian Networks with +Discrete and Continuous Latent Variables</a>, +K. Murphy, UAI 99. +</ul> +Of course, one can always use <a href="#sampling">sampling</a> methods +for approximate inference in such models. + + + +<h1><a name="param_learning">Parameter Learning</h1> + +The parameter estimation routines in BNT can be classified into 4 +types, depending on whether the goal is to compute +a full (Bayesian) posterior over the parameters or just a point +estimate (e.g., Maximum Likelihood or Maximum A Posteriori), +and whether all the variables are fully observed or there is missing +data/ hidden variables (partial observability). +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_params</tt></td> + <td><tt>learn_params_em</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>bayes_update_params</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="load_data">Loading data from a file</h2> + +To load numeric data from an ASCII text file called 'dat.txt', where each row is a +case and columns are separated by white-space, such as +<pre> +011979 1626.5 0.0 +021979 1367.0 0.0 +... +</pre> +you can use +<pre> +data = load('dat.txt'); +</pre> +or +<pre> +load dat.txt -ascii +</pre> +In the latter case, the data is stored in a variable called 'dat' (the +filename minus the extension). +Alternatively, suppose the data is stored in a .csv file (has commas +separating the columns, and contains a header line), such as +<pre> +header info goes here +ORD,011979,1626.5,0.0 +DSM,021979,1367.0,0.0 +... +</pre> +You can load this using +<pre> +[a,b,c,d] = textread('dat.txt', '%s %d %f %f', 'delimiter', ',', 'headerlines', 1); +</pre> +If your file is not in either of these formats, you can either use Perl to convert +it to this format, or use the Matlab scanf command. +Type +<tt> +help iofun +</tt> +for more information on Matlab's file functions. +<!-- +<p> +To load data directly from Excel, +you should buy the +<a href="http://www.mathworks.com/products/excellink/">Excel Link</a>. +To load data directly from a relational database, +you should buy the +<a href="http://www.mathworks.com/products/database">Database +toolbox</a>. +--> +<p> +BNT learning routines require data to be stored in a cell array. +data{i,m} is the value of node i in case (example) m, i.e., each +<em>column</em> is a case. +If node i is not observed in case m (missing value), set +data{i,m} = []. +(Not all the learning routines can cope with such missing values, however.) +In the special case that all the nodes are observed and are +scalar-valued (as opposed to vector-valued), the data can be +stored in a matrix (as opposed to a cell-array). +<p> +Suppose, as in the <a href="#mixexp">mixture of experts example</a>, +that we have 3 nodes in the graph: X(1) is the observed input, X(3) is +the observed output, and X(2) is a hidden (gating) node. We can +create the dataset as follows. +<pre> +data = load('dat.txt'); +ncases = size(data, 1); +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); +</pre> +Notice how we transposed the data, to convert rows into columns. +Also, cases{2,m} = [] for all m, since X(2) is always hidden. + + +<h2><a name="mle_complete">Maximum likelihood parameter estimation from complete data</h2> + +As an example, let's generate some data from the sprinkler network, randomize the parameters, +and then try to recover the original model. +First we create some training data using forwards sampling. +<pre> +samples = cell(N, nsamples); +for i=1:nsamples + samples(:,i) = sample_bnet(bnet); +end +</pre> +samples{j,i} contains the value of the j'th node in case i. +sample_bnet returns a cell array because, in general, each node might +be a vector of different length. +In this case, all nodes are discrete (and hence scalars), so we +could have used a regular array instead (which can be quicker): +<pre> +data = cell2num(samples); +</pre +So now data(j,i) = samples{j,i}. +<p> +Now we create a network with random parameters. +(The initial values of bnet2 don't matter in this case, since we can find the +globally optimal MLE independent of where we start.) +<pre> +% Make a tabula rasa +bnet2 = mk_bnet(dag, node_sizes); +seed = 0; +rand('state', seed); +bnet2.CPD{C} = tabular_CPD(bnet2, C); +bnet2.CPD{R} = tabular_CPD(bnet2, R); +bnet2.CPD{S} = tabular_CPD(bnet2, S); +bnet2.CPD{W} = tabular_CPD(bnet2, W); +</pre> +Finally, we find the maximum likelihood estimates of the parameters. +<pre> +bnet3 = learn_params(bnet2, samples); +</pre> +To view the learned parameters, we use a little Matlab hackery. +<pre> +CPT3 = cell(1,N); +for i=1:N + s=struct(bnet3.CPD{i}); % violate object privacy + CPT3{i}=s.CPT; +end +</pre> +Here are the parameters learned for node 4. +<pre> +dispcpt(CPT3{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.2000 0.8000 +1 2 : 0.2273 0.7727 +2 2 : 0.0000 1.0000 +</pre> +So we see that the learned parameters are fairly close to the "true" +ones, which we display below. +<pre> +dispcpt(CPT{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.1000 0.9000 +1 2 : 0.1000 0.9000 +2 2 : 0.0100 0.9900 +</pre> +We can get better results by using a larger training set, or using +informative priors (see <a href="#prior">below</a>). + + + +<h2><a name="prior">Parameter priors</h2> + +Currently, only tabular CPDs can have priors on their parameters. +The conjugate prior for a multinomial is the Dirichlet. +(For binary random variables, the multinomial is the same as the +Bernoulli, and the Dirichlet is the same as the Beta.) +<p> +The Dirichlet has a simple interpretation in terms of pseudo counts. +If we let N_ijk = the num. times X_i=k and Pa_i=j occurs in the +training set, where Pa_i are the parents of X_i, +then the maximum likelihood (ML) estimate is +T_ijk = N_ijk / N_ij (where N_ij = sum_k' N_ijk'), which will be 0 if N_ijk=0. +To prevent us from declaring that (X_i=k, Pa_i=j) is impossible just because this +event was not seen in the training set, +we can pretend we saw value k of X_i, for each value j of Pa_i some number (alpha_ijk) +of times in the past. +The MAP (maximum a posterior) estimate is then +<pre> +T_ijk = (N_ijk + alpha_ijk) / (N_ij + alpha_ij) +</pre> +and is never 0 if all alpha_ijk > 0. +For example, consider the network A->B, where A is binary and B has 3 +values. +A uniform prior for B has the form +<pre> + B=1 B=2 B=3 +A=1 1 1 1 +A=2 1 1 1 +</pre> +which can be created using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'unif'); +</pre> +This prior does not satisfy the likelihood equivalence principle, +which says that <a href="#markov_equiv">Markov equivalent</a> models +should have the same marginal likelihood. +A prior that does satisfy this principle is shown below. +Heckerman (1995) calls this the +BDeu prior (likelihood equivalent uniform Bayesian Dirichlet). +<pre> + B=1 B=2 B=3 +A=1 1/6 1/6 1/6 +A=2 1/6 1/6 1/6 +</pre> +where we put N/(q*r) in each bin; N is the equivalent sample size, +r=|A|, q = |B|. +This can be created as follows +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'BDeu'); +</pre> +Here, 1 is the equivalent sample size, and is the strength of the +prior. +You can change this using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', ... + 'BDeu', 'dirichlet_weight', 10); +</pre> +<!--where counts is an array of pseudo-counts of the same size as the +CPT.--> +<!-- +<p> +When you specify a prior, you should set row i of the CPT to the +normalized version of row i of the pseudo-count matrix, i.e., to the +expected values of the parameters. This will ensure that computing the +marginal likelihood sequentially (see <a +href="#bayes_learn">below</a>) and in batch form gives the same +results. +To do this, proceed as follows. +<pre> +tabular_CPD(bnet, i, 'prior', counts, 'CPT', mk_stochastic(counts)); +</pre> +For a non-informative prior, you can just write +<pre> +tabular_CPD(bnet, i, 'prior', 'unif', 'CPT', 'unif'); +</pre> +--> + + +<h2><a name="bayes_learn">(Sequential) Bayesian parameter updating from complete data</h2> + +If we use conjugate priors and have fully observed data, we can +compute the posterior over the parameters in batch form as follows. +<pre> +cases = sample_bnet(bnet, nsamples); +bnet = bayes_update_params(bnet, cases); +LL = log_marg_lik_complete(bnet, cases); +</pre> +bnet.CPD{i}.prior contains the new Dirichlet pseudocounts, +and bnet.CPD{i}.CPT is set to the mean of the posterior (the +normalized counts). +(Hence if the initial pseudo counts are 0, +<tt>bayes_update_params</tt> and <tt>learn_params</tt> will give the +same result.) + + + + +<p> +We can compute the same result sequentially (on-line) as follows. +<pre> +LL = 0; +for m=1:nsamples + LL = LL + log_marg_lik_complete(bnet, cases(:,m)); + bnet = bayes_update_params(bnet, cases(:,m)); +end +</pre> + +The file <tt>BNT/examples/static/StructLearn/model_select1</tt> has an example of +sequential model selection which uses the same idea. +We generate data from the model A->B +and compute the posterior prob of all 3 dags on 2 nodes: + (1) A B, (2) A <- B , (3) A -> B +Models 2 and 3 are <a href="#markov_equiv">Markov equivalent</a>, and therefore indistinguishable from +observational data alone, so we expect their posteriors to be the same +(assuming a prior which satisfies likelihood equivalence). +If we use random parameters, the "true" model only gets a higher posterior after 2000 trials! +However, if we make B a noisy NOT gate, the true model "wins" after 12 +trials, as shown below (red = model 1, blue/green (superimposed) +represents models 2/3). +<p> +<img src="Figures/model_select.png"> +<p> +The use of marginal likelihood for model selection is discussed in +greater detail in the +section on <a href="structure_learning">structure learning</a>. + + + + +<h2><a name="em">Maximum likelihood parameter estimation with missing values</h2> + +Now we consider learning when some values are not observed. +Let us randomly hide half the values generated from the water +sprinkler example. +<pre> +samples2 = samples; +hide = rand(N, nsamples) > 0.5; +[I,J]=find(hide); +for k=1:length(I) + samples2{I(k), J(k)} = []; +end +</pre> +samples2{i,l} is the value of node i in training case l, or [] if unobserved. +<p> +Now we will compute the MLEs using the EM algorithm. +We need to use an inference algorithm to compute the expected +sufficient statistics in the E step; the M (maximization) step is as +above. +<pre> +engine2 = jtree_inf_engine(bnet2); +max_iter = 10; +[bnet4, LLtrace] = learn_params_em(engine2, samples2, max_iter); +</pre> +LLtrace(i) is the log-likelihood at iteration i. We can plot this as +follows: +<pre> +plot(LLtrace, 'x-') +</pre> +Let's display the results after 10 iterations of EM. +<pre> +celldisp(CPT4) +CPT4{1} = + 0.6616 + 0.3384 +CPT4{2} = + 0.6510 0.3490 + 0.8751 0.1249 +CPT4{3} = + 0.8366 0.1634 + 0.0197 0.9803 +CPT4{4} = +(:,:,1) = + 0.8276 0.0546 + 0.5452 0.1658 +(:,:,2) = + 0.1724 0.9454 + 0.4548 0.8342 +</pre> +We can get improved performance by using one or more of the following +methods: +<ul> +<li> Increasing the size of the training set. +<li> Decreasing the amount of hidden data. +<li> Running EM for longer. +<li> Using informative priors. +<li> Initialising EM from multiple starting points. +</ul> + +Click <a href="#gaussian">here</a> for a discussion of learning +Gaussians, which can cause numerical problems. +<p> +For a more complete example of learning with EM, +see the script BNT/examples/static/learn1.m. + +<h2><a name="tying">Parameter tying</h2> + +In networks with repeated structure (e.g., chains and grids), it is +common to assume that the parameters are the same at every node. This +is called parameter tying, and reduces the amount of data needed for +learning. +<p> +When we have tied parameters, there is no longer a one-to-one +correspondence between nodes and CPDs. +Rather, each CPD species the parameters for a whole equivalence class +of nodes. +It is easiest to see this by example. +Consider the following <a href="usage_dbn.html#hmm">hidden Markov +model (HMM)</a> +<p> +<img src="Figures/hmm3.gif"> +<p> +<!-- +We can create this graph structure, assuming we have T time-slices, +as follows. +(We number the nodes as shown in the figure, but we could equally well +number the hidden nodes 1:T, and the observed nodes T+1:2T.) +<pre> +N = 2*T; +dag = zeros(N); +hnodes = 1:2:2*T; +for i=1:T-1 + dag(hnodes(i), hnodes(i+1))=1; +end +onodes = 2:2:2*T; +for i=1:T + dag(hnodes(i), onodes(i)) = 1; +end +</pre> +<p> +The hidden nodes are always discrete, and have Q possible values each, +but the observed nodes can be discrete or continuous, and have O possible values/length. +<pre> +if cts_obs + dnodes = hnodes; +else + dnodes = 1:N; +end +ns = ones(1,N); +ns(hnodes) = Q; +ns(onodes) = O; +</pre> +--> +When HMMs are used for semi-infinite processes like speech recognition, +we assume the transition matrix +P(H(t+1)|H(t)) is the same for all t; this is called a time-invariant +or homogenous Markov chain. +Hence hidden nodes 2, 3, ..., T +are all in the same equivalence class, say class Hclass. +Similarly, the observation matrix P(O(t)|H(t)) is assumed to be the +same for all t, so the observed nodes are all in the same equivalence +class, say class Oclass. +Finally, the prior term P(H(1)) is in a class all by itself, say class +H1class. +This is illustrated below, where we explicitly represent the +parameters as random variables (dotted nodes). +<p> +<img src="Figures/hmm4_params.gif"> +<p> +In BNT, we cannot represent parameters as random variables (nodes). +Instead, we "hide" the +parameters inside one CPD for each equivalence class, +and then specify that the other CPDs should share these parameters, as +follows. +<pre> +hnodes = 1:2:2*T; +onodes = 2:2:2*T; +H1class = 1; Hclass = 2; Oclass = 3; +eclass = ones(1,N); +eclass(hnodes(2:end)) = Hclass; +eclass(hnodes(1)) = H1class; +eclass(onodes) = Oclass; +% create dag and ns in the usual way +bnet = mk_bnet(dag, ns, 'discrete', dnodes, 'equiv_class', eclass); +</pre> +Finally, we define the parameters for each equivalence class: +<pre> +bnet.CPD{H1class} = tabular_CPD(bnet, hnodes(1)); % prior +bnet.CPD{Hclass} = tabular_CPD(bnet, hnodes(2)); % transition matrix +if cts_obs + bnet.CPD{Oclass} = gaussian_CPD(bnet, onodes(1)); +else + bnet.CPD{Oclass} = tabular_CPD(bnet, onodes(1)); +end +</pre> +In general, if bnet.CPD{e} = xxx_CPD(bnet, j), then j should be a +member of e's equivalence class; that is, it is not always the case +that e == j. You can use bnet.rep_of_eclass(e) to return the +representative of equivalence class e. +BNT will look up the parents of j to determine the size +of the CPT to use. It assumes that this is the same for all members of +the equivalence class. +Click <a href="param_tieing.html">here</a> for +a more complex example of parameter tying. +<p> +Note: +Normally one would define an HMM as a +<a href = "usage_dbn.html">Dynamic Bayes Net</a> +(see the function BNT/examples/dynamic/mk_chmm.m). +However, one can define an HMM as a static BN using the function +BNT/examples/static/Models/mk_hmm_bnet.m. + + + +<h1><a name="structure_learning">Structure learning</h1> + +Update (9/29/03): +Phillipe LeRay is developing some additional structure learning code +on top of BNT. Click +<a href="http://banquiseasi.insa-rouen.fr/projects/bnt-slp/"> +here</a> +for details. + +<p> + +There are two very different approaches to structure learning: +constraint-based and search-and-score. +In the <a href="#constraint">constraint-based approach</a>, +we start with a fully connected graph, and remove edges if certain +conditional independencies are measured in the data. +This has the disadvantage that repeated independence tests lose +statistical power. +<p> +In the more popular search-and-score approach, +we perform a search through the space of possible DAGs, and either +return the best one found (a point estimate), or return a sample of the +models found (an approximation to the Bayesian posterior). +<p> +The number of DAGs as a function of the number of +nodes, G(n), is super-exponential in n, +and is given by the following recurrence +<!--(where R(i)=G(n)):--> +<p> +<center> +<IMG SRC="numDAGsEqn2.png"> +</center> +<p> +The first few values +are shown below. + +<table> +<tr> <th>n</th> <th align=left>G(n)</th> </tr> +<tr> <td>1</td> <td>1</td> </tr> +<tr> <td>2</td> <td>3</td> </tr> +<tr> <td>3</td> <td>25</td> </tr> +<tr> <td>4</td> <td>543</td> </tr> +<tr> <td>5</td> <td>29,281</td> </tr> +<tr> <td>6</td> <td>3,781,503</td> </tr> +<tr> <td>7</td> <td>1.1 x 10^9</td> </tr> +<tr> <td>8</td> <td>7.8 x 10^11</td> </tr> +<tr> <td>9</td> <td>1.2 x 10^15</td> </tr> +<tr> <td>10</td> <td>4.2 x 10^18</td> </tr> +</table> + +Since the number of DAGs is super-exponential in the number of nodes, +we cannot exhaustively search the space, so we either use a local +search algorithm (e.g., greedy hill climbining, perhaps with multiple +restarts) or a global search algorithm (e.g., Markov Chain Monte +Carlo). +<p> +If we know a total ordering on the nodes, +finding the best structure amounts to picking the best set of parents +for each node independently. +This is what the K2 algorithm does. +If the ordering is unknown, we can search over orderings, +which is more efficient than searching over DAGs (Koller and Friedman, 2000). +<p> +In addition to the search procedure, we must specify the scoring +function. There are two popular choices. The Bayesian score integrates +out the parameters, i.e., it is the marginal likelihood of the model. +The BIC (Bayesian Information Criterion) is defined as +log P(D|theta_hat) - 0.5*d*log(N), where D is the data, theta_hat is +the ML estimate of the parameters, d is the number of parameters, and +N is the number of data cases. +The BIC method has the advantage of not requiring a prior. +<p> +BIC can be derived as a large sample +approximation to the marginal likelihood. +(It is also equal to the Minimum Description Length of a model.) +However, in practice, the sample size does not need to be very large +for the approximation to be good. +For example, in the figure below, we plot the ratio between the log marginal likelihood +and the BIC score against data-set size; we see that the ratio rapidly +approaches 1, especially for non-informative priors. +(This plot was generated by the file BNT/examples/static/bic1.m. It +uses the water sprinkler BN with BDeu Dirichlet priors with different +equivalent sample sizes.) + +<p> +<center> +<IMG SRC="Figures/bic.png"> +</center> +<p> + +<p> +As with parameter learning, handling missing data/ hidden variables is +much harder than the fully observed case. +The structure learning routines in BNT can therefore be classified into 4 +types, analogously to the parameter learning case. +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_struct_K2</tt> <br> +<!-- <tt>learn_struct_hill_climb</tt></td> --> + <td><tt>not yet supported</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>learn_struct_mcmc</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="markov_equiv">Markov equivalence</h2> + +If two DAGs encode the same conditional independencies, they are +called Markov equivalent. The set of all DAGs can be paritioned into +Markov equivalence classes. Graphs within the same class can +have +the direction of some of their arcs reversed without changing any of +the CI relationships. +Each class can be represented by a PDAG +(partially directed acyclic graph) called an essential graph or +pattern. This specifies which edges must be oriented in a certain +direction, and which may be reversed. + +<p> +When learning graph structure from observational data, +the best one can hope to do is to identify the model up to Markov +equivalence. To distinguish amongst graphs within the same equivalence +class, one needs interventional data: see the discussion on <a +href="#active">active learning</a> below. + + + +<h2><a name="enumerate">Exhaustive search</h2> + +The brute-force approach to structure learning is to enumerate all +possible DAGs, and score each one. This provides a "gold standard" +with which to compare other algorithms. We can do this as follows. +<pre> +dags = mk_all_dags(N); +score = score_dags(data, ns, dags); +</pre> +where data(i,m) is the value of node i in case m, +and ns(i) is the size of node i. +If the DAGs have a lot of families in common, we can cache the sufficient statistics, +making this potentially more efficient than scoring the DAGs one at a time. +(Caching is not currently implemented, however.) +<p> +By default, we use the Bayesian scoring metric, and assume CPDs are +represented by tables with BDeu(1) priors. +We can override these defaults as follows. +If we want to use uniform priors, we can say +<pre> +params = cell(1,N); +for i=1:N + params{i} = {'prior', 'unif'}; +end +score = score_dags(data, ns, dags, 'params', params); +</pre> +params{i} is a cell-array, containing optional arguments that are +passed to the constructor for CPD i. +<p> +Now suppose we want to use different node types, e.g., +Suppose nodes 1 and 2 are Gaussian, and nodes 3 and 4 softmax (both +these CPDs can support discrete and continuous parents, which is +necessary since all other nodes will be considered as parents). +The Bayesian scoring metric currently only works for tabular CPDs, so +we will use BIC: +<pre> +score = score_dags(data, ns, dags, 'discrete', [3 4], 'params', [], + 'type', {'gaussian', 'gaussian', 'softmax', softmax'}, 'scoring_fn', 'bic') +</pre> +In practice, one can't enumerate all possible DAGs for N > 5, +but one can evaluate any reasonably-sized set of hypotheses in this +way (e.g., nearest neighbors of your current best guess). +Think of this as "computer assisted model refinement" as opposed to de +novo learning. + + +<h2><a name="K2">K2</h2> + +The K2 algorithm (Cooper and Herskovits, 1992) is a greedy search algorithm that works as follows. +Initially each node has no parents. It then adds incrementally that parent whose addition most +increases the score of the resulting structure. When the addition of no single +parent can increase the score, it stops adding parents to the node. +Since we are using a fixed ordering, we do not need to check for +cycles, and can choose the parents for each node independently. +<p> +The original paper used the Bayesian scoring +metric with tabular CPDs and Dirichlet priors. +BNT generalizes this to allow any kind of CPD, and either the Bayesian +scoring metric or BIC, as in the example <a href="#enumerate">above</a>. +In addition, you can specify +an optional upper bound on the number of parents for each node. +The file BNT/examples/static/k2demo1.m gives an example of how to use K2. +We use the water sprinkler network and sample 100 cases from it as before. +Then we see how much data it takes to recover the generating structure: +<pre> +order = [C S R W]; +max_fan_in = 2; +sz = 5:5:100; +for i=1:length(sz) + dag2 = learn_struct_K2(data(:,1:sz(i)), node_sizes, order, 'max_fan_in', max_fan_in); + correct(i) = isequal(dag, dag2); +end +</pre> +Here are the results. +<pre> +correct = + Columns 1 through 12 + 0 0 0 0 0 0 0 1 0 1 1 1 + Columns 13 through 20 + 1 1 1 1 1 1 1 1 +</pre> +So we see it takes about sz(10)=50 cases. (BIC behaves similarly, +showing that the prior doesn't matter too much.) +In general, we cannot hope to recover the "true" generating structure, +only one that is in its <a href="#markov_equiv">Markov equivalence +class</a>. + + +<h2><a name="hill_climb">Hill-climbing</h2> + +Hill-climbing starts at a specific point in space, +considers all nearest neighbors, and moves to the neighbor +that has the highest score; if no neighbors have higher +score than the current point (i.e., we have reached a local maximum), +the algorithm stops. One can then restart in another part of the space. +<p> +A common definition of "neighbor" is all graphs that can be +generated from the current graph by adding, deleting or reversing a +single arc, subject to the acyclicity constraint. +Other neighborhoods are possible: see +<a href="http://research.microsoft.com/~dmax/publications/jmlr02.pdf"> +Optimal Structure Identification with Greedy Search</a>, Max +Chickering, JMLR 2002. + +<!-- +Note: This algorithm is currently (Feb '02) being implemented by Qian +Diao. +--> + + +<h2><a name="mcmc">MCMC</h2> + +We can use a Markov Chain Monte Carlo (MCMC) algorithm called +Metropolis-Hastings (MH) to search the space of all +DAGs. +The standard proposal distribution is to consider moving to all +nearest neighbors in the sense defined <a href="#hill_climb">above</a>. +<p> +The function can be called +as in the following example. +<pre> +[sampled_graphs, accept_ratio] = learn_struct_mcmc(data, ns, 'nsamples', 100, 'burnin', 10); +</pre> +We can convert our set of sampled graphs to a histogram +(empirical posterior over all the DAGs) thus +<pre> +all_dags = mk_all_dags(N); +mcmc_post = mcmc_sample_to_hist(sampled_graphs, all_dags); +</pre> +To see how well this performs, let us compute the exact posterior exhaustively. +<p> +<pre> +score = score_dags(data, ns, all_dags); +post = normalise(exp(score)); % assuming uniform structural prior +</pre> +We plot the results below. +(The data set was 100 samples drawn from a random 4 node bnet; see the +file BNT/examples/static/mcmc1.) +<pre> +subplot(2,1,1) +bar(post) +subplot(2,1,2) +bar(mcmc_post) +</pre> +<img src="Figures/mcmc_post.jpg" width="800" height="500"> +<p> +We can also plot the acceptance ratio versus number of MCMC steps, +as a crude convergence diagnostic. +<pre> +clf +plot(accept_ratio) +</pre> +<img src="Figures/mcmc_accept.jpg" width="800" height="300"> +<p> +Even though the number of samples needed by MCMC is theoretically +polynomial (not exponential) in the dimensionality of the search space, in practice it has been +found that MCMC does not converge in reasonable time for graphs with +more than about 10 nodes. + + + + +<h2><a name="active">Active structure learning</h2> + +As was mentioned <a href="#markov_equiv">above</a>, +one can only learn a DAG up to Markov equivalence, even given infinite data. +If one is interested in learning the structure of a causal network, +one needs interventional data. +(By "intervention" we mean forcing a node to take on a specific value, +thereby effectively severing its incoming arcs.) +<p> +Most of the scoring functions accept an optional argument +that specifies whether a node was observed to have a certain value, or +was forced to have that value: we set clamped(i,m)=1 if node i was +forced in training case m. e.g., see the file +BNT/examples/static/cooper_yoo. +<p> +An interesting question is to decide which interventions to perform +(c.f., design of experiments). For details, see the following tech +report +<ul> +<li> <a href = "../../Papers/alearn.ps.gz"> +Active learning of causal Bayes net structure</a>, Kevin Murphy, March +2001. +</ul> + + +<h2><a name="struct_em">Structural EM</h2> + +Computing the Bayesian score when there is partial observability is +computationally challenging, because the parameter posterior becomes +multimodal (the hidden nodes induce a mixture distribution). +One therefore needs to use approximations such as BIC. +Unfortunately, search algorithms are still expensive, because we need +to run EM at each step to compute the MLE, which is needed to compute +the score of each model. An alternative approach is +to do the local search steps inside of the M step of EM, which is more +efficient since the data has been "filled in" - this is +called the structural EM algorithm (Friedman 1997), and provably +converges to a local maximum of the BIC score. +<p> +Wei Hu has implemented SEM for discrete nodes. +You can download his package from +<a href="../SEM.zip">here</a>. +Please address all questions about this code to +wei.hu@intel.com. +See also <a href="#phl">Phl's implementation of SEM</a>. + +<!-- +<h2><a name="reveal">REVEAL algorithm</h2> + +A simple way to learn the structure of a fully observed, discrete, +factored DBN from a time series is described <a +href="usage_dbn.html#struct_learn">here</a>. +--> + + +<h2><a name="graphdraw">Visualizing the graph</h2> + +Click <a href="graphviz.html">here</a> for more information +on graph visualization. + +<h2><a name = "constraint">Constraint-based methods</h2> + +The IC algorithm (Pearl and Verma, 1991), +and the faster, but otherwise equivalent, PC algorithm (Spirtes, Glymour, and Scheines 1993), +computes many conditional independence tests, +and combines these constraints into a +PDAG to represent the whole +<a href="#markov_equiv">Markov equivalence class</a>. +<p> +IC*/FCI extend IC/PC to handle latent variables: see <a href="#ic_star">below</a>. +(IC stands for inductive causation; PC stands for Peter and Clark, +the first names of Spirtes and Glymour; FCI stands for fast causal +inference. +What we, following Pearl (2000), call IC* was called +IC in the original Pearl and Verma paper.) +For details, see +<ul> +<li> +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Causation, +Prediction, and Search</a>, Spirtes, Glymour and +Scheines (SGS), 2001 (2nd edition), MIT Press. +<li> +<a href="http://bayes.cs.ucla.edu/BOOK-2K/index.html">Causality: Models, Reasoning and Inference</a>, J. Pearl, +2000, Cambridge University Press. +</ul> + +<p> + +The PC algorithm takes as arguments a function f, the number of nodes N, +the maximum fan in K, and additional arguments A which are passed to f. +The function f(X,Y,S,A) returns 1 if X is conditionally independent of Y given S, and 0 +otherwise. +For example, suppose we cheat by +passing in a CI "oracle" which has access to the true DAG; the oracle +tests for d-separation in this DAG, i.e., +f(X,Y,S) calls dsep(X,Y,S,dag). We can to this as follows. +<pre> +pdag = learn_struct_pdag_pc('dsep', N, max_fan_in, dag); +</pre> +pdag(i,j) = -1 if there is definitely an i->j arc, +and pdag(i,j) = 1 if there is either an i->j or and i<-j arc. +<p> +Applied to the sprinkler network, this returns +<pre> +pdag = + 0 1 1 0 + 1 0 0 -1 + 1 0 0 -1 + 0 0 0 0 +</pre> +So as expected, we see that the V-structure at the W node is uniquely identified, +but the other arcs have ambiguous orientation. +<p> +We now give an example from p141 (1st edn) / p103 (2nd end) of the SGS +book. +This example concerns the female orgasm. +We are given a correlation matrix C between 7 measured factors (such +as subjective experiences of coital and masturbatory experiences), +derived from 281 samples, and want to learn a causal model of the +data. We will not discuss the merits of this type of work here, but +merely show how to reproduce the results in the SGS book. +Their program, +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Tetrad</a>, +makes use of the Fisher Z-test for conditional +independence, so we do the same: +<pre> +max_fan_in = 4; +nsamples = 281; +alpha = 0.05; +pdag = learn_struct_pdag_pc('cond_indep_fisher_z', n, max_fan_in, C, nsamples, alpha); +</pre> +In this case, the CI test is +<pre> +f(X,Y,S) = cond_indep_fisher_z(X,Y,S, C,nsamples,alpha) +</pre> +The results match those of Fig 12a of SGS apart from two edge +differences; presumably this is due to rounding error (although it +could be a bug, either in BNT or in Tetrad). +This example can be found in the file BNT/examples/static/pc2.m. + +<p> + +The IC* algorithm (Pearl and Verma, 1991), +and the faster FCI algorithm (Spirtes, Glymour, and Scheines 1993), +are like the IC/PC algorithm, except that they can detect the presence +of latent variables. +See the file <tt>learn_struct_pdag_ic_star</tt> written by Tamar +Kushnir. The output is a matrix P, defined as follows +(see Pearl (2000), p52 for details): +<pre> +% P(i,j) = -1 if there is either a latent variable L such that i <-L->j OR there is a directed edge from i->j. +% P(i,j) = -2 if there is a marked directed i-*>j edge. +% P(i,j) = P(j,i) = 1 if there is and undirected edge i--j +% P(i,j) = P(j,i) = 2 if there is a latent variable L such that i<-L->j. +</pre> + + +<h2><a name="phl">Philippe Leray's structure learning package</h2> + +Philippe Leray has written a +<a href="http://bnt.insa-rouen.fr/ajouts.html"> +structure learning package</a> that uses BNT. + +It currently (Juen 2003) has the following features: +<ul> +<li>PC with Chi2 statistical test +<li> MWST : Maximum weighted Spanning Tree +<li> Hill Climbing +<li> Greedy Search +<li> Structural EM +<li> hist_ic : optimal Histogram based on IC information criterion +<li> cpdag_to_dag +<li> dag_to_cpdag +<li> ... +</ul> + + +</a> + + +<!-- +<h2><a name="read_learning">Further reading on learning</h2> + +I recommend the following tutorials for more details on learning. +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/intel.ps.gz">My short +tutorial</a> on graphical models, which contains an overview of learning. + +<li> +<A HREF="ftp://ftp.research.microsoft.com/pub/tr/TR-95-06.PS"> +A tutorial on learning with Bayesian networks</a>, D. Heckerman, +Microsoft Research Tech Report, 1995. + +<li> <A HREF="http://www-cad.eecs.berkeley.edu/~wray/Mirror/lwgmja"> +Operations for Learning with Graphical Models</a>, +W. L. Buntine, JAIR'94, 159--225. +</ul> +<p> +--> + + + + + +<h1><a name="engines">Inference engines</h1> + +Up until now, we have used the junction tree algorithm for inference. +However, sometimes this is too slow, or not even applicable. +In general, there are many inference algorithms each of which make +different tradeoffs between speed, accuracy, complexity and +generality. Furthermore, there might be many implementations of the +same algorithm; for instance, a general purpose, readable version, +and a highly-optimized, specialized one. +To cope with this variety, we treat each inference algorithm as an +object, which we call an inference engine. + +<p> +An inference engine is an object that contains a bnet and supports the +'enter_evidence' and 'marginal_nodes' methods. The engine constructor +takes the bnet as argument and may do some model-specific processing. +When 'enter_evidence' is called, the engine may do some +evidence-specific processing. Finally, when 'marginal_nodes' is +called, the engine may do some query-specific processing. + +<p> +The amount of work done when each stage is specified -- structure, +parameters, evidence, and query -- depends on the engine. The cost of +work done early in this sequence can be amortized. On the other hand, +one can make better optimizations if one waits until later in the +sequence. +For example, the parameters might imply +conditional indpendencies that are not evident in the graph structure, +but can nevertheless be exploited; the evidence indicates which nodes +are observed and hence can effectively be disconnected from the +graph; and the query might indicate that large parts of the network +are d-separated from the query nodes. (Since it is not the actual +<em>values</em> of the evidence that matters, just which nodes are observed, +many engines allow you to specify which nodes will be observed when they are constructed, +i.e., before calling 'enter_evidence'. Some engines can still cope if +the actual pattern of evidence is different, e.g., if there is missing +data.) +<p> + +Although being maximally lazy (i.e., only doing work when a query is +issued) may seem desirable, +this is not always the most efficient. +For example, +when learning using EM, we need to call marginal_nodes N times, where N is the +number of nodes. <a href="varelim">Variable elimination</a> would end +up repeating a lot of work +each time marginal_nodes is called, making it inefficient for +learning. The junction tree algorithm, by contrast, uses dynamic +programming to avoid this redundant computation --- it calculates all +marginals in two passes during 'enter_evidence', so calling +'marginal_nodes' takes constant time. +<p> +We will discuss some of the inference algorithms implemented in BNT +below, and finish with a <a href="#engine_summary">summary</a> of all +of them. + + + + + + + +<h2><a name="varelim">Variable elimination</h2> + +The variable elimination algorithm, also known as bucket elimination +or peeling, is one of the simplest inference algorithms. +The basic idea is to "push sums inside of products"; this is explained +in more detail +<a +href="http://HTTP.CS.Berkeley.EDU/~murphyk/Bayes/bayes.html#infer">here</a>. +<p> +The principle of distributing sums over products can be generalized +greatly to apply to any commutative semiring. +This forms the basis of many common algorithms, such as Viterbi +decoding and the Fast Fourier Transform. For details, see + +<ul> +<li> R. McEliece and S. M. Aji, 2000. +<!--<a href="http://www.systems.caltech.edu/EE/Faculty/rjm/papers/GDL.ps">--> +<a href="GDL.pdf"> +The Generalized Distributive Law</a>, +IEEE Trans. Inform. Theory, vol. 46, no. 2 (March 2000), +pp. 325--343. + + +<li> +F. R. Kschischang, B. J. Frey and H.-A. Loeliger, 2001. +<a href="http://www.cs.toronto.edu/~frey/papers/fgspa.abs.html"> +Factor graphs and the sum-product algorithm</a> +IEEE Transactions on Information Theory, February, 2001. + +</ul> + +<p> +Choosing an order in which to sum out the variables so as to minimize +computational cost is known to be NP-hard. +The implementation of this algorithm in +<tt>var_elim_inf_engine</tt> makes no attempt to optimize this +ordering (in contrast, say, to <tt>jtree_inf_engine</tt>, which uses a +greedy search procedure to find a good ordering). +<p> +Note: unlike most algorithms, var_elim does all its computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + + + +<h2><a name="global">Global inference methods</h2> + +The simplest inference algorithm of all is to explicitely construct +the joint distribution over all the nodes, and then to marginalize it. +This is implemented in <tt>global_joint_inf_engine</tt>. +Since the size of the joint is exponential in the +number of discrete (hidden) nodes, this is not a very practical algorithm. +It is included merely for pedagogical and debugging purposes. +<p> +Three specialized versions of this algorithm have also been implemented, +corresponding to the cases where all the nodes are discrete (D), all +are Gaussian (G), and some are discrete and some Gaussian (CG). +They are called <tt>enumerative_inf_engine</tt>, +<tt>gaussian_inf_engine</tt>, +and <tt>cond_gauss_inf_engine</tt> respectively. +<p> +Note: unlike most algorithms, these global inference algorithms do all their computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + +<h2><a name="quickscore">Quickscore</h2> + +The junction tree algorithm is quite slow on the <a href="#qmr">QMR</a> network, +since the cliques are so big. +One simple trick we can use is to notice that hidden leaves do not +affect the posteriors on the roots, and hence do not need to be +included in the network. +A second trick is to notice that the negative findings can be +"absorbed" into the prior: +see the file +BNT/examples/static/mk_minimal_qmr_bnet for details. +<p> + +A much more significant speedup is obtained by exploiting special +properties of the noisy-or node, as done by the quickscore +algorithm. For details, see +<ul> +<li> Heckerman, "A tractable inference algorithm for diagnosing multiple diseases", UAI 89. +<li> Rish and Dechter, "On the impact of causal independence", UCI +tech report, 1998. +</ul> + +This has been implemented in BNT as a special-purpose inference +engine, which can be created and used as follows: +<pre> +engine = quickscore_inf_engine(inhibit, leak, prior); +engine = enter_evidence(engine, pos, neg); +m = marginal_nodes(engine, i); +</pre> + + +<h2><a name="belprop">Belief propagation</h2> + +Even using quickscore, exact inference takes time that is exponential +in the number of positive findings. +Hence for large networks we need to resort to approximate inference techniques. +See for example +<ul> +<li> T. Jaakkola and M. Jordan, "Variational probabilistic inference and the +QMR-DT network", JAIR 10, 1999. + +<li> K. Murphy, Y. Weiss and M. Jordan, "Loopy belief propagation for approximate inference: an empirical study", + UAI 99. +</ul> +The latter approximation +entails applying Pearl's belief propagation algorithm to a model even +if it has loops (hence the name loopy belief propagation). +Pearl's algorithm, implemented as <tt>pearl_inf_engine</tt>, gives +exact results when applied to singly-connected graphs +(a.k.a. polytrees, since +the underlying undirected topology is a tree, but a node may have +multiple parents). +To apply this algorithm to a graph with loops, +use <tt>pearl_inf_engine</tt>. +This can use a centralized or distributed message passing protocol. +You can use it as in the following example. +<pre> +engine = pearl_inf_engine(bnet, 'max_iter', 30); +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, i); +</pre> +We found that this algorithm often converges, and when it does, often +is very accurate, but it depends on the precise setting of the +parameter values of the network. +(See the file BNT/examples/static/qmr1 to repeat the experiment for yourself.) +Understanding when and why belief propagation converges/ works +is a topic of ongoing research. +<p> +<tt>pearl_inf_engine</tt> can exploit special structure in noisy-or +and gmux nodes to compute messages efficiently. +<p> +<tt>belprop_inf_engine</tt> is like pearl, but uses potentials to +represent messages. Hence this is slower. +<p> +<tt>belprop_fg_inf_engine</tt> is like belprop, +but is designed for factor graphs. + + + +<h2><a name="sampling">Sampling</h2> + +BNT now (Mar '02) has two sampling (Monte Carlo) inference algorithms: +<ul> +<li> <tt>likelihood_weighting_inf_engine</tt> which does importance +sampling and can handle any node type. +<li> <tt>gibbs_sampling_inf_engine</tt>, written by Bhaskara Marthi. +Currently this can only handle tabular CPDs. +For a much faster and more powerful Gibbs sampling program, see +<a href="http://www.mrc-bsu.cam.ac.uk/bugs">BUGS</a>. +</ul> +Note: To generate samples from a network (which is not the same as inference!), +use <tt>sample_bnet</tt>. + + + +<h2><a name="engine_summary">Summary of inference engines</h2> + + +The inference engines differ in many ways. Here are +some of the major "axes": +<ul> +<li> Works for all topologies or makes restrictions? +<li> Works for all node types or makes restrictions? +<li> Exact or approximate inference? +</ul> + +<p> +In terms of topology, most engines handle any kind of DAG. +<tt>belprop_fg</tt> does approximate inference on factor graphs (FG), which +can be used to represent directed, undirected, and mixed (chain) +graphs. +(In the future, we plan to support exact inference on chain graphs.) +<tt>quickscore</tt> only works on QMR-like models. +<p> +In terms of node types: algorithms that use potentials can handle +discrete (D), Gaussian (G) or conditional Gaussian (CG) models. +Sampling algorithms can essentially handle any kind of node (distribution). +Other algorithms make more restrictive assumptions in exchange for +speed. +<p> +Finally, most algorithms are designed to give the exact answer. +The belief propagation algorithms are exact if applied to trees, and +in some other cases. +Sampling is considered approximate, even though, in the limit of an +infinite number of samples, it gives the exact answer. + +<p> + +Here is a summary of the properties +of all the engines in BNT which work on static networks. +<p> +<table> +<table border units = pixels><tr> +<td align=left width=0>Name +<td align=left width=0>Exact? +<td align=left width=0>Node type? +<td align=left width=0>topology +<tr> +<tr> +<td align=left> belprop +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> belprop_fg +<td align=left> approx +<td align=left> D +<td align=left> factor graph +<tr> +<td align=left> cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> enumerative +<td align=left> exact +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> gaussian +<td align=left> exact +<td align=left> G +<td align=left> DAG +<tr> +<td align=left> gibbs +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> global_joint +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +<tr> +<td align=left> jtree +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +b<tr> +<td align=left> likelihood_weighting +<td align=left> approx +<td align=left> any +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> approx +<td align=left> D,G +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> exact +<td align=left> D,G +<td align=left> polytree +<tr> +<td align=left> quickscore +<td align=left> exact +<td align=left> noisy-or +<td align=left> QMR +<tr> +<td align=left> stab_cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> var_elim +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +</table> + + + +<h1><a name="influence">Influence diagrams/ decision making</h1> + +BNT implements an exact algorithm for solving LIMIDs (limited memory +influence diagrams), described in +<ul> +<li> S. L. Lauritzen and D. Nilsson. +<a href="http://www.math.auc.dk/~steffen/papers/limids.pdf"> +Representing and solving decision problems with limited +information</a> +Management Science, 47, 1238 - 1251. September 2001. +</ul> +LIMIDs explicitely show all information arcs, rather than implicitely +assuming no forgetting. This allows them to model forgetful +controllers. +<p> +See the examples in <tt>BNT/examples/limids</tt> for details. + + + + +<h1>DBNs, HMMs, Kalman filters and all that</h1> + +Click <a href="usage_dbn.html">here</a> for documentation about how to +use BNT for dynamical systems and sequence data. + + +</BODY> diff --git a/sourcecodes/bnt-master/docs/usage_02nov13.html b/sourcecodes/bnt-master/docs/usage_02nov13.html new file mode 100644 index 00000000..a1e24294 --- /dev/null +++ b/sourcecodes/bnt-master/docs/usage_02nov13.html @@ -0,0 +1,3215 @@ +<HEAD> +<TITLE>How to use the Bayes Net Toolbox</TITLE> +</HEAD> + +<BODY BGCOLOR="#FFFFFF"> +<!-- white background is better for the pictures and equations --> + +<h1>How to use the Bayes Net Toolbox</h1> + +This documentation was last updated on 13 November 2002. +<br> +Click <a href="changelog.html">here</a> for a list of changes made to +BNT. +<br> +Click +<a href="http://bnt.insa-rouen.fr/">here</a> +for a French version of this documentation (which might not +be up-to-date). + + +<p> + +<ul> +<li> <a href="#install">Installation</a> +<ul> +<li> <a href="#installM">Installing the Matlab code</a> +<li> <a href="#installC">Installing the C code</a> +<li> <a href="../matlab_tips.html">Useful Matlab tips</a>. +</ul> + +<li> <a href="#basics">Creating your first Bayes net</a> + <ul> + <li> <a href="#basics">Creating a model by hand</a> + <li> <a href="#file">Loading a model from a file</a> + <li> <a href="http://bnt.insa-rouen.fr/ajouts.html">Creating a model using a GUI</a> + </ul> + +<li> <a href="#inference">Inference</a> + <ul> + <li> <a href="#marginal">Computing marginal distributions</a> + <li> <a href="#joint">Computing joint distributions</a> + <li> <a href="#soft">Soft/virtual evidence</a> + <li> <a href="#mpe">Most probable explanation</a> + </ul> + +<li> <a href="#cpd">Conditional Probability Distributions</a> + <ul> + <li> <a href="#tabular">Tabular (multinomial) nodes</a> + <li> <a href="#noisyor">Noisy-or nodes</a> + <li> <a href="#deterministic">Other (noisy) deterministic nodes</a> + <li> <a href="#softmax">Softmax (multinomial logit) nodes</a> + <li> <a href="#mlp">Neural network nodes</a> + <li> <a href="#root">Root nodes</a> + <li> <a href="#gaussian">Gaussian nodes</a> + <li> <a href="#glm">Generalized linear model nodes</a> + <li> <a href="#dtree">Classification/regression tree nodes</a> + <li> <a href="#nongauss">Other continuous distributions</a> + <li> <a href="#cpd_summary">Summary of CPD types</a> + </ul> + +<li> <a href="#examples">Example models</a> + <ul> + <li> <a + href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt"> +Gaussian mixture models</a> + <li> <a href="#pca">PCA, ICA, and all that</a> + <li> <a href="#mixep">Mixtures of experts</a> + <li> <a href="#hme">Hierarchical mixtures of experts</a> + <li> <a href="#qmr">QMR</a> + <li> <a href="#cg_model">Conditional Gaussian models</a> + <li> <a href="#hybrid">Other hybrid models</a> + </ul> + +<li> <a href="#param_learning">Parameter learning</a> + <ul> + <li> <a href="#load_data">Loading data from a file</a> + <li> <a href="#mle_complete">Maximum likelihood parameter estimation from complete data</a> + <li> <a href="#prior">Parameter priors</a> + <li> <a href="#bayes_learn">(Sequential) Bayesian parameter updating from complete data</a> + <li> <a href="#em">Maximum likelihood parameter estimation with missing values (EM)</a> + <li> <a href="#tying">Parameter tying</a> + </ul> + +<li> <a href="#structure_learning">Structure learning</a> + <ul> + <li> <a href="#enumerate">Exhaustive search</a> + <li> <a href="#K2">K2</a> + <li> <a href="#hill_climb">Hill-climbing</a> + <li> <a href="#mcmc">MCMC</a> + <li> <a href="#active">Active learning</a> + <li> <a href="#struct_em">Structural EM</a> + <li> <a href="#graphdraw">Visualizing the learned graph structure</a> + <li> <a href="#constraint">Constraint-based methods</a> + </ul> + + +<li> <a href="#engines">Inference engines</a> + <ul> + <li> <a href="#jtree">Junction tree</a> + <li> <a href="#varelim">Variable elimination</a> + <li> <a href="#global">Global inference methods</a> + <li> <a href="#quickscore">Quickscore</a> + <li> <a href="#belprop">Belief propagation</a> + <li> <a href="#sampling">Sampling (Monte Carlo)</a> + <li> <a href="#engine_summary">Summary of inference engines</a> + </ul> + + +<li> <a href="#influence">Influence diagrams/ decision making</a> + + +<li> <a href="usage_dbn.html">DBNs, HMMs, Kalman filters and all that</a> +</ul> + +</ul> + + + + +<h1><a name="install">Installation</h1> + +<h2><a name="installM">Installing the Matlab code</h2> + +<ul> +<li> <a href="bnt_download.html">Download</a> the BNT.zip file. + +<p> +<li> Unpack the file. In Unix, type +<!--"tar xvf BNT.tar".--> +"unzip FullBNT.zip". +In Windows, use +a program like <a href="http://www.winzip.com">Winzip</a>. This will +create a directory called FullBNT, which contains BNT and other libraries. + +<p> +<li> Read the file <tt>BNT/README</tt> to make sure the date +matches the one on the top of <a href=bnt.html>the BNT home page</a>. +If not, you may need to press 'refresh' on your browser, and download +again, to get the most recent version. + +<p> +<li> <b>Edit the file "BNT/add_BNT_to_path.m"</b> so it contains the correct +pathname. +For example, in Windows, +I download FullBNT.zip into C:\kpmurphy\matlab, and +then comment out the second line (with the % character), and uncomment +the third line, which reads +<pre> +BNT_HOME = 'C:\kpmurphy\matlab\FullBNT'; +</pre> + +<p> +<li> Start up Matlab. + +<p> +<li> Type "ver" at the Matlab prompt (">>"). +<b>You need Matlab version 5.2 or newer to run BNT</b>. +(Versions 5.0 and 5.1 have a memory leak which seems to sometimes +crash BNT.) + +<p> +<li> Move to the BNT directory. +For example, in Windows, I type +<pre> +>> cd C:\kpmurphy\matlab\FullBNT\BNT +</pre> + +<p> +<li> Type "add_BNT_to_path". +This executes the command +<tt>addpath(genpath(BNT_HOME))</tt>, +which adds all directories below FullBNT to the matlab path. + +<p> +<li> Type "test_BNT". + +If all goes well, this will produce a bunch of numbers and maybe some +warning messages (which you can ignore), but no error messages. +(The warnings should only be of the form +"Warning: Maximum number of iterations has been exceeded", and are +produced by Netlab.) + +<p> +<li> <a href="http://groups.yahoo.com/group/BayesNetToolbox/join"> +Join the BNT email list</a> + +</ul> + + +If you are new to Matlab, you might like to check out +<a href="matlab_tips.html">some useful Matlab tips</a>. +For instance, this explains how to create a startup file, which can be +used to set your path variable automatically, so you can avoid having +to type the above commands every time. + + + + +<h2><a name="installC">Installing the C code</h2> + +Some BNT functions also have C implementations. +<b>It is not necessary to install the C code</b>, but it can result in a speedup +of a factor of 5-10. +To install all the C code, +edit installC_BNT.m so it contains the right path, +then type <tt>installC_BNT</tt>. +To uninstall all the C code, +edit uninstallC_BNT.m so it contains the right path, +then type <tt>uninstallC_BNT</tt>. +For an up-to-date list of the files which have C implementations, see +BNT/installC_BNT.m. + +<p> +mex is a script that lets you call C code from Matlab - it does not compile matlab to +C (see mcc below). +If your C/C++ compiler is set up correctly, mex should work out of +the box. +If not, you might need to type +<p> +<tt> mex -setup</tt> +<p> +before calling installC. +<p> +To make mex call gcc on Windows, +you must install <a +href="http://www.mrc-cbu.cam.ac.uk/Imaging/gnumex20.html">gnumex</a>. +You can use the <a href="http://www.mingw.org/">minimalist GNU for +Windows</a> version of gcc, or +the <a href="http://sources.redhat.com/cygwin/">cygwin</a> version. +<p> +In general, typing +'mex foo.c' from inside Matlab creates a file called +'foo.mexglx' or 'foo.dll' (the exact file +extension is system dependent - on Linux it is 'mexglx', on Windows it is '.dll'). +The resulting file will hide the original 'foo.m' (if it existed), i.e., +typing 'foo' at the prompt will call the compiled C version. +To reveal the original matlab version, just delete foo.mexglx (this is +what uninstallC does). +<p> +Sometimes it takes time for Matlab to realize that the file has +changed from matlab to C or vice versa; try typing 'clear all' or +restarting Matlab to refresh it. +To find out which version of a file you are running, type +'which foo'. +<p> +<a href="http://www.mathworks.com/products/compiler">mcc</a>, the +Matlab to C compiler, is a separate product, +and is quite different from mex. It does not yet support +objects/classes, which is why we can't compile all of BNT to C automatically. +Also, hand-written C code is usually much +better than the C code generated by mcc. + + +<p> +Acknowledgements: +Although I wrote some of the C code, most of +the C code (e.g., for jtree and dpot) was written by Wei Hu; +the triangulation C code was written by Ilya Shpitser. + + +<h1><a name="basics">Creating your first Bayes net</h1> + +To define a Bayes net, you must specify the graph structure and then +the parameters. We look at each in turn, using a simple example +(adapted from Russell and +Norvig, "Artificial Intelligence: a Modern Approach", Prentice Hall, +1995, p454). + + +<h2>Graph structure</h2> + + +Consider the following network. + +<p> +<center> +<IMG SRC="Figures/sprinkler.gif"> +</center> +<p> + +<P> +To specify this directed acyclic graph (dag), we create an adjacency matrix: +<PRE> +N = 4; +dag = zeros(N,N); +C = 1; S = 2; R = 3; W = 4; +dag(C,[R S]) = 1; +dag(R,W) = 1; +dag(S,W)=1; +</PRE> +<P> +We have numbered the nodes as follows: +Cloudy = 1, Sprinkler = 2, Rain = 3, WetGrass = 4. +<b>The nodes must always be numbered in topological order, i.e., +ancestors before descendants.</b> +For a more complicated graph, this is a little inconvenient: we will +see how to get around this <a href="usage_dbn.html#bat">below</a>. +<p> +In Matlab 6, you can use logical arrays instead of double arrays, +which are 4 times smaller: +<pre> +dag = false(N,N); +dag(C,[R S]) = true; +... +</pre> +<p> +A preliminary attempt to make a <b>GUI</b> +has been writte by Philippe LeRay and can be downloaded +from <a href="http://bnt.insa-rouen.fr/ajouts.html">here</a>. +<p> +You can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>. + +<h2>Creating the Bayes net shell</h2> + +In addition to specifying the graph structure, +we must specify the size and type of each node. +If a node is discrete, its size is the +number of possible values +each node can take on; if a node is continuous, +it can be a vector, and its size is the length of this vector. +In this case, we will assume all nodes are discrete and binary. +<PRE> +discrete_nodes = 1:N; +node_sizes = 2*ones(1,N); +</pre> +If the nodes were not binary, you could type e.g., +<pre> +node_sizes = [4 2 3 5]; +</pre> +meaning that Cloudy has 4 possible values, +Sprinkler has 2 possible values, etc. +Note that these are cardinal values, not ordinal, i.e., +they are not ordered in any way, like 'low', 'medium', 'high'. +<p> +We are now ready to make the Bayes net: +<pre> +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes); +</PRE> +By default, all nodes are assumed to be discrete, so we can also just +write +<pre> +bnet = mk_bnet(dag, node_sizes); +</PRE> +You may also specify which nodes will be observed. +If you don't know, or if this not fixed in advance, +just use the empty list (the default). +<pre> +onodes = []; +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes, 'observed', onodes); +</PRE> +Note that optional arguments are specified using a name/value syntax. +This is common for many BNT functions. +In general, to find out more about a function (e.g., which optional +arguments it takes), please see its +documentation string by typing +<pre> +help mk_bnet +</pre> +See also other <a href="matlab_tips.html">useful Matlab tips</a>. +<p> +It is possible to associate names with nodes, as follows: +<pre> +bnet = mk_bnet(dag, node_sizes, 'names', {'cloudy','S','R','W'}, 'discrete', 1:4); +</pre> +You can then refer to a node by its name: +<pre> +C = bnet.names{'cloudy'}; % bnet.names is an associative array +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +</pre> + + +<h2><a name="cpt">Parameters</h2> + +A model consists of the graph structure and the parameters. +The parameters are represented by CPD objects (CPD = Conditional +Probability Distribution), which define the probability distribution +of a node given its parents. +(We will use the terms "node" and "random variable" interchangeably.) +The simplest kind of CPD is a table (multi-dimensional array), which +is suitable when all the nodes are discrete-valued. Note that the discrete +values are not assumed to be ordered in any way; that is, they +represent categorical quantities, like male and female, rather than +ordinal quantities, like low, medium and high. +(We will discuss CPDs in more detail <a href="#cpd">below</a>.) +<p> +Tabular CPDs, also called CPTs (conditional probability tables), +are stored as multidimensional arrays, where the dimensions +are arranged in the same order as the nodes, e.g., the CPT for node 4 +(WetGrass) is indexed by Sprinkler (2), Rain (3) and then WetGrass (4) itself. +Hence the child is always the last dimension. +If a node has no parents, its CPT is a column vector representing its +prior. +Note that in Matlab (unlike C), arrays are indexed +from 1, and are layed out in memory such that the first index toggles +fastest, e.g., the CPT for node 4 (WetGrass) is as follows +<P> +<P><IMG ALIGN=BOTTOM SRC="Figures/CPTgrass.gif"><P> +<P> +where we have used the convention that false==1, true==2. +We can create this CPT in Matlab as follows +<PRE> +CPT = zeros(2,2,2); +CPT(1,1,1) = 1.0; +CPT(2,1,1) = 0.1; +... +</PRE> +Here is an easier way: +<PRE> +CPT = reshape([1 0.1 0.1 0.01 0 0.9 0.9 0.99], [2 2 2]); +</PRE> +In fact, we don't need to reshape the array, since the CPD constructor +will do that for us. So we can just write +<pre> +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</pre> +The other nodes are created similarly (using the old syntax for +optional parameters) +<PRE> +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +bnet.CPD{R} = tabular_CPD(bnet, R, [0.8 0.2 0.2 0.8]); +bnet.CPD{S} = tabular_CPD(bnet, S, [0.5 0.9 0.5 0.1]); +bnet.CPD{W} = tabular_CPD(bnet, W, [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</PRE> + + +<h2><a name="rnd_cpt">Random Parameters</h2> + +If we do not specify the CPT, random parameters will be +created, i.e., each "row" of the CPT will be drawn from the uniform distribution. +To ensure repeatable results, use +<pre> +rand('state', seed); +randn('state', seed); +</pre> +To control the degree of randomness (entropy), +you can sample each row of the CPT from a Dirichlet(p,p,...) distribution. +If p << 1, this encourages "deterministic" CPTs (one entry near 1, the rest near 0). +If p = 1, each entry is drawn from U[0,1]. +If p >> 1, the entries will all be near 1/k, where k is the arity of +this node, i.e., each row will be nearly uniform. +You can do this as follows, assuming this node +is number i, and ns is the node_sizes. +<pre> +k = ns(i); +ps = parents(dag, i); +psz = prod(ns(ps)); +CPT = sample_dirichlet(p*ones(1,k), psz); +bnet.CPD{i} = tabular_CPD(bnet, i, 'CPT', CPT); +</pre> + + +<h2><a name="file">Loading a network from a file</h2> + +If you already have a Bayes net represented in the XML-based +<a href="http://www.cs.cmu.edu/afs/cs/user/fgcozman/www/Research/InterchangeFormat/"> +Bayes Net Interchange Format (BNIF)</a> (e.g., downloaded from the +<a +href="http://www.cs.huji.ac.il/labs/compbio/Repository"> +Bayes Net repository</a>), +you can convert it to BNT format using +the +<a href="http://www.digitas.harvard.edu/~ken/bif2bnt/">BIF-BNT Java +program</a> written by Ken Shan. +(This is not necessarily up-to-date.) + + +<h2>Creating a model using a GUI</h2> + +Click <a href="http://bnt.insa-rouen.fr/ajouts.html">here</a>. + + + +<h1><a name="inference">Inference</h1> + +Having created the BN, we can now use it for inference. +There are many different algorithms for doing inference in Bayes nets, +that make different tradeoffs between speed, +complexity, generality, and accuracy. +BNT therefore offers a variety of different inference +"engines". We will discuss these +in more detail <a href="#engines">below</a>. +For now, we will use the junction tree +engine, which is the mother of all exact inference algorithms. +This can be created as follows. +<pre> +engine = jtree_inf_engine(bnet); +</pre> +The other engines have similar constructors, but might take +additional, algorithm-specific parameters. +All engines are used in the same way, once they have been created. +We illustrate this in the following sections. + + +<h2><a name="marginal">Computing marginal distributions</h2> + +Suppose we want to compute the probability that the sprinker was on +given that the grass is wet. +The evidence consists of the fact that W=2. All the other nodes +are hidden (unobserved). We can specify this as follows. +<pre> +evidence = cell(1,N); +evidence{W} = 2; +</pre> +We use a 1D cell array instead of a vector to +cope with the fact that nodes can be vectors of different lengths. +In addition, the value [] can be used +to denote 'no evidence', instead of having to specify the observation +pattern as a separate argument. +(Click <a href="cellarray.html">here</a> for a quick tutorial on cell +arrays in matlab.) +<p> +We are now ready to add the evidence to the engine. +<pre> +[engine, loglik] = enter_evidence(engine, evidence); +</pre> +The behavior of this function is algorithm-specific, and is discussed +in more detail <a href="#engines">below</a>. +In the case of the jtree engine, +enter_evidence implements a two-pass message-passing scheme. +The first return argument contains the modified engine, which +incorporates the evidence. The second return argument contains the +log-likelihood of the evidence. (Not all engines are capable of +computing the log-likelihood.) +<p> +Finally, we can compute p=P(S=2|W=2) as follows. +<PRE> +marg = marginal_nodes(engine, S); +marg.T +ans = + 0.57024 + 0.42976 +p = marg.T(2); +</PRE> +We see that p = 0.4298. +<p> +Now let us add the evidence that it was raining, and see what +difference it makes. +<PRE> +evidence{R} = 2; +[engine, loglik] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, S); +p = marg.T(2); +</PRE> +We find that p = P(S=2|W=2,R=2) = 0.1945, +which is lower than +before, because the rain can ``explain away'' the +fact that the grass is wet. +<p> +You can plot a marginal distribution over a discrete variable +as a barchart using the built 'bar' function: +<pre> +bar(marg.T) +</pre> +This is what it looks like + +<p> +<center> +<IMG SRC="Figures/sprinkler_bar.gif"> +</center> +<p> + +<h2><a name="observed">Observed nodes</h2> + +What happens if we ask for the marginal on an observed node, e.g. P(W|W=2)? +An observed discrete node effectively only has 1 value (the observed + one) --- all other values would result in 0 probability. +For efficiency, BNT treats observed (discrete) nodes as if they were + set to 1, as we see below: +<pre> +evidence = cell(1,N); +evidence{W} = 2; +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, W); +m.T +ans = + 1 +</pre> +This can get a little confusing, since we assigned W=2. +So we can ask BNT to add the evidence back in by passing in an optional argument: +<pre> +m = marginal_nodes(engine, W, 1); +m.T +ans = + 0 + 1 +</pre> +This shows that P(W=1|W=2) = 0 and P(W=2|W=2) = 1. + + + +<h2><a name="joint">Computing joint distributions</h2> + +We can compute the joint probability on a set of nodes as in the +following example. +<pre> +evidence = cell(1,N); +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]); +</pre> +m is a structure. The 'T' field is a multi-dimensional array (in +this case, 3-dimensional) that contains the joint probability +distribution on the specified nodes. +<pre> +>> m.T +ans(:,:,1) = + 0.2900 0.0410 + 0.0210 0.0009 +ans(:,:,2) = + 0 0.3690 + 0.1890 0.0891 +</pre> +We see that P(S=1,R=1,W=2) = 0, since it is impossible for the grass +to be wet if both the rain and sprinkler are off. +<p> +Let us now add some evidence to R. +<pre> +evidence{R} = 2; +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]) +m = + domain: [2 3 4] + T: [2x1x2 double] +>> m.T +m.T +ans(:,:,1) = + 0.0820 + 0.0018 +ans(:,:,2) = + 0.7380 + 0.1782 +</pre> +The joint T(i,j,k) = P(S=i,R=j,W=k|evidence) +should have T(i,1,k) = 0 for all i,k, since R=1 is incompatible +with the evidence that R=2. +Instead of creating large tables with many 0s, BNT sets the effective +size of observed (discrete) nodes to 1, as explained above. +This is why m.T has size 2x1x2. +To get a 2x2x2 table, type +<pre> +m = marginal_nodes(engine, [S R W], 1) +m = + domain: [2 3 4] + T: [2x2x2 double] +>> m.T +m.T +ans(:,:,1) = + 0 0.082 + 0 0.0018 +ans(:,:,2) = + 0 0.738 + 0 0.1782 +</pre> + +<p> +Note: It is not always possible to compute the joint on arbitrary +sets of nodes: it depends on which inference engine you use, as discussed +in more detail <a href="#engines">below</a>. + + +<h2><a name="soft">Soft/virtual evidence</h2> + +Sometimes a node is not observed, but we have some distribution over +its possible values; this is often called "soft" or "virtual" +evidence. +One can use this as follows +<pre> +[engine, loglik] = enter_evidence(engine, evidence, 'soft', soft_evidence); +</pre> +where soft_evidence{i} is either [] (if node i has no soft evidence) +or is a vector representing the probability distribution over i's +possible values. +For example, if we don't know i's exact value, but we know its +likelihood ratio is 60/40, we can write evidence{i} = [] and +soft_evidence{i} = [0.6 0.4]. +<p> +Currently only jtree_inf_engine supports this option. +It assumes that all hidden nodes, and all nodes for +which we have soft evidence, are discrete. +For a longer example, see BNT/examples/static/softev1.m. + + +<h2><a name="mpe">Most probable explanation</h2> + +To compute the most probable explanation (MPE) of the evidence (i.e., +the most probable assignment, or a mode of the joint), use +<pre> +[mpe, ll] = calc_mpe(engine, evidence); +</pre> +mpe{i} is the most likely value of node i. +This calls enter_evidence with the 'maximize' flag set to 1, which +causes the engine to do max-product instead of sum-product. +The resulting max-marginals are then thresholded. +If there is more than one maximum probability assignment, we must take + care to break ties in a consistent manner (thresholding the + max-marginals may give the wrong result). To force this behavior, + type +<pre> +[mpe, ll] = calc_mpe(engine, evidence, 1); +</pre> +Note that computing the MPE is someties called abductive reasoning. + +<p> +You can also use <tt>calc_mpe_bucket</tt> written by Ron Zohar, +that does a forwards max-product pass, and then a backwards traceback +pass, which is how Viterbi is traditionally implemented. + + + +<h1><a name="cpd">Conditional Probability Distributions</h1> + +A Conditional Probability Distributions (CPD) +defines P(X(i) | X(Pa(i))), where X(i) is the i'th node, and X(Pa(i)) +are the parents of node i. There are many ways to represent this +distribution, which depend in part on whether X(i) and X(Pa(i)) are +discrete, continuous, or a combination. +We will discuss various representations below. + + +<h2><a name="tabular">Tabular nodes</h2> + +If the CPD is represented as a table (i.e., if it is a multinomial +distribution), it has a number of parameters that is exponential in +the number of parents. See the example <a href="#cpt">above</a>. + + +<h2><a name="noisyor">Noisy-or nodes</h2> + +A noisy-OR node is like a regular logical OR gate except that +sometimes the effects of parents that are on get inhibited. +Let the prob. that parent i gets inhibited be q(i). +Then a node, C, with 2 parents, A and B, has the following CPD, where +we use F and T to represent off and on (1 and 2 in BNT). +<pre> +A B P(C=off) P(C=on) +--------------------------- +F F 1.0 0.0 +T F q(A) 1-q(A) +F T q(B) 1-q(B) +T T q(A)q(B) q-q(A)q(B) +</pre> +Thus we see that the causes get inhibited independently. +It is common to associate a "leak" node with a noisy-or CPD, which is +like a parent that is always on. This can account for all other unmodelled +causes which might turn the node on. +<p> +The noisy-or distribution is similar to the logistic distribution. +To see this, let the nodes, S(i), have values in {0,1}, and let q(i,j) +be the prob. that j inhibits i. Then +<pre> +Pr(S(i)=1 | parents(S(i))) = 1 - prod_{j} q(i,j)^S(j) +</pre> +Now define w(i,j) = -ln q(i,j) and rho(x) = 1-exp(-x). Then +<pre> +Pr(S(i)=1 | parents(S(i))) = rho(sum_j w(i,j) S(j)) +</pre> +For a sigmoid node, we have +<pre> +Pr(S(i)=1 | parents(S(i))) = sigma(-sum_j w(i,j) S(j)) +</pre> +where sigma(x) = 1/(1+exp(-x)). Hence they differ in the choice of +the activation function (although both are monotonically increasing). +In addition, in the case of a noisy-or, the weights are constrained to be +positive, since they derive from probabilities q(i,j). +In both cases, the number of parameters is <em>linear</em> in the +number of parents, unlike the case of a multinomial distribution, +where the number of parameters is exponential in the number of parents. +We will see an example of noisy-OR nodes <a href="#qmr">below</a>. + + +<h2><a name="deterministic">Other (noisy) deterministic nodes</h2> + +Deterministic CPDs for discrete random variables can be created using +the deterministic_CPD class. It is also possible to 'flip' the output +of the function with some probability, to simulate noise. +The boolean_CPD class is just a special case of a +deterministic CPD, where the parents and child are all binary. +<p> +Both of these classes are just "syntactic sugar" for the tabular_CPD +class. + + + +<h2><a name="softmax">Softmax nodes</h2> + +If we have a discrete node with a continuous parent, +we can define its CPD using a softmax function +(also known as the multinomial logit function). +This acts like a soft thresholding operator, and is defined as follows: +<pre> + exp(w(:,i)'*x + b(i)) +Pr(Q=i | X=x) = ----------------------------- + sum_j exp(w(:,j)'*x + b(j)) + +</pre> +The parameters of a softmax node, w(:,i) and b(i), i=1..|Q|, have the +following interpretation: w(:,i)-w(:,j) is the normal vector to the +decision boundary between classes i and j, +and b(i)-b(j) is its offset (bias). For example, suppose +X is a 2-vector, and Q is binary. Then +<pre> +w = [1 -1; + 0 0]; + +b = [0 0]; +</pre> +means class 1 are points in the 2D plane with positive x coordinate, +and class 2 are points in the 2D plane with negative x coordinate. +If w has large magnitude, the decision boundary is sharp, otherwise it +is soft. +In the special case that Q is binary (0/1), the softmax function reduces to the logistic +(sigmoid) function. +<p> +Fitting a softmax function can be done using the iteratively reweighted +least squares (IRLS) algorithm. +We use the implementation from +<a href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +Note that since +the softmax distribution is not in the exponential family, it does not +have finite sufficient statistics, and hence we must store all the +training data in uncompressed form. +If this takes too much space, one should use online (stochastic) gradient +descent (not implemented in BNT). +<p> +If a softmax node also has discrete parents, +we use a different set of w/b parameters for each combination of +parent values, as in the <a href="#gaussian">conditional linear +Gaussian CPD</a>. +This feature was implemented by Pierpaolo Brutti. +He is currently extending it so that discrete parents can be treated +as if they were continuous, by adding indicator variables to the X +vector. +<p> +We will see an example of softmax nodes <a href="#mixexp">below</a>. + + +<h2><a name="mlp">Neural network nodes</h2> + +Pierpaolo Brutti has implemented the mlp_CPD class, which uses a multi layer perceptron +to implement a mapping from continuous parents to discrete children, +similar to the softmax function. +(If there are also discrete parents, it creates a mixture of MLPs.) +It uses code from <a +href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +This is work in progress. + +<h2><a name="root">Root nodes</h2> + +A root node has no parents and no parameters; it can be used to model +an observed, exogeneous input variable, i.e., one which is "outside" +the model. +This is useful for conditional density models. +We will see an example of root nodes <a href="#mixexp">below</a>. + + +<h2><a name="gaussian">Gaussian nodes</h2> + +We now consider a distribution suitable for the continuous-valued nodes. +Suppose the node is called Y, its continuous parents (if any) are +called X, and its discrete parents (if any) are called Q. +The distribution on Y is defined as follows: +<pre> +- no parents: Y ~ N(mu, Sigma) +- cts parents : Y|X=x ~ N(mu + W x, Sigma) +- discrete parents: Y|Q=i ~ N(mu(:,i), Sigma(:,:,i)) +- cts and discrete parents: Y|X=x,Q=i ~ N(mu(:,i) + W(:,:,i) * x, Sigma(:,:,i)) +</pre> +where N(mu, Sigma) denotes a Normal distribution with mean mu and +covariance Sigma. Let |X|, |Y| and |Q| denote the sizes of X, Y and Q +respectively. +If there are no discrete parents, |Q|=1; if there is +more than one, then |Q| = a vector of the sizes of each discrete parent. +If there are no continuous parents, |X|=0; if there is more than one, +then |X| = the sum of their sizes. +Then mu is a |Y|*|Q| vector, Sigma is a |Y|*|Y|*|Q| positive +semi-definite matrix, and W is a |Y|*|X|*|Q| regression (weight) +matrix. +<p> +We can create a Gaussian node with random parameters as follows. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i); +</pre> +We can specify the value of one or more of the parameters as in the +following example, in which |Y|=2, and |Q|=1. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i, 'mean', [0; 0], 'weights', randn(Y,X), 'cov', eye(Y)); +</pre> +<p> +We will see an example of conditional linear Gaussian nodes <a +href="#cg_model">below</a>. +<p> +<b>When learning Gaussians from data</b>, it is helpful to ensure the +data has a small magnitde +(see e.g., KPMstats/standardize) to prevent numerical problems. +Unless you have a lot of data, it is also a very good idea to use +diagonal instead of full covariance matrices. +(BNT does not currently support spherical covariances, although it +would be easy to add, since KPMstats/clg_Mstep supports this option; +you would just need to modify gaussian_CPD/update_ess to accumulate +weighted inner products.) + + + +<h2><a name="nongauss">Other continuous distributions</h2> + +Currently BNT does not support any CPDs for continuous nodes other +than the Gaussian. +However, you can use a mixture of Gaussians to +approximate other continuous distributions. We will see some an example +of this with the IFA model <a href="#pca">below</a>. + + +<h2><a name="glm">Generalized linear model nodes</h2> + +In the future, we may incorporate some of the functionality of +<a href = +"http://www.sci.usq.edu.au/staff/dunn/glmlab/glmlab.html">glmlab</a> +into BNT. + + +<h2><a name="dtree">Classification/regression tree nodes</h2> + +We plan to add classification and regression trees to define CPDs for +discrete and continuous nodes, respectively. +Trees have many advantages: they are easy to interpret, they can do +feature selection, they can +handle discrete and continuous inputs, they do not make strong +assumptions about the form of the distribution, the number of +parameters can grow in a data-dependent way (i.e., they are +semi-parametric), they can handle missing data, etc. +However, they are not yet implemented. +<!-- +Yimin Zhang is currently (Feb '02) implementing this. +--> + + +<h2><a name="cpd_summary">Summary of CPD types</h2> + +We list all the different types of CPDs supported by BNT. +For each CPD, we specify if the child and parents can be discrete (D) or +continuous (C) (Binary (B) nodes are a special case). +We also specify which methods each class supports. +If a method is inherited, the name of the parent class is mentioned. +If a parent class calls a child method, this is mentioned. +<p> +The <tt>CPD_to_CPT</tt> method converts a CPD to a table; this +requires that the child and all parents are discrete. +The CPT might be exponentially big... +<tt>convert_to_table</tt> evaluates a CPD with evidence, and +represents the the resulting potential as an array. +This requires that the child is discrete, and any continuous parents +are observed. +<tt>convert_to_pot</tt> evaluates a CPD with evidence, and +represents the resulting potential as a dpot, gpot, cgpot or upot, as +requested. (d=discrete, g=Gaussian, cg = conditional Gaussian, u = +utility). + +<p> +When we sample a node, all the parents are observed. +When we compute the (log) probability of a node, all the parents and +the child are observed. +<p> +We also specify if the parameters are learnable. +For learning with EM, we require +the methods <tt>reset_ess</tt>, <tt>update_ess</tt> and +<tt>maximize_params</tt>. +For learning from fully observed data, we require +the method <tt>learn_params</tt>. +By default, all classes inherit this from generic_CPD, which simply +calls <tt>update_ess</tt> N times, once for each data case, followed +by <tt>maximize_params</tt>, i.e., it is like EM, without the E step. +Some classes implement a batch formula, which is quicker. +<p> +Bayesian learning means computing a posterior over the parameters +given fully observed data. +<p> +Pearl means we implement the methods <tt>compute_pi</tt> and +<tt>compute_lambda_msg</tt>, used by +<tt>pearl_inf_engine</tt>, which runs on directed graphs. +<tt>belprop_inf_engine</tt> only needs <tt>convert_to_pot</tt>.H +The pearl methods can exploit special properties of the CPDs for +computing the messages efficiently, whereas belprop does not. +<p> +The only method implemented by generic_CPD is <tt>adjustable_CPD</tt>, +which is not shown, since it is not very interesting. + + +<p> + + +<table> +<table border units = pixels><tr> +<td align=center>Name +<td align=center>Child +<td align=center>Parents +<td align=center>Comments +<td align=center>CPD_to_CPT +<td align=center>conv_to_table +<td align=center>conv_to_pot +<td align=center>sample +<td align=center>prob +<td align=center>learn +<td align=center>Bayes +<td align=center>Pearl + + +<tr> +<!-- Name--><td> +<!-- Child--><td> +<!-- Parents--><td> +<!-- Comments--><td> +<!-- CPD_to_CPT--><td> +<!-- conv_to_table--><td> +<!-- conv_to_pot--><td> +<!-- sample--><td> +<!-- prob--><td> +<!-- learn--><td> +<!-- Bayes--><td> +<!-- Pearl--><td> + +<tr> +<!-- Name--><td>boolean +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>deterministic +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>Discrete +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Calls CPD_to_CPT +<!-- conv_to_pot--><td>Calls conv_to_table +<!-- sample--><td>Calls conv_to_table +<!-- prob--><td>Calls conv_to_table +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>Gaussian +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>gmux +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>multiplexer +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>MLP +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>multi layer perceptron +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>noisy-or +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>root +<!-- Child--><td>C/D +<!-- Parents--><td>none +<!-- Comments--><td>no params +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>softmax +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>generic +<!-- Child--><td>C/D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>N +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>Tabular +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>Y +<!-- Pearl--><td>Y + +</table> + + + +<h1><a name="examples">Example models</h1> + + +<h2>Gaussian mixture models</h2> + +Richard W. DeVaul has made a detailed tutorial on how to fit mixtures +of Gaussians using BNT. Available +<a href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt">here</a>. + + +<h2><a name="pca">PCA, ICA, and all that </h2> + +In Figure (a) below, we show how Factor Analysis can be thought of as a +graphical model. Here, X has an N(0,I) prior, and +Y|X=x ~ N(mu + Wx, Psi), +where Psi is diagonal and W is called the "factor loading matrix". +Since the noise on both X and Y is diagonal, the components of these +vectors are uncorrelated, and hence can be represented as individual +scalar nodes, as we show in (b). +(This is useful if parts of the observations on the Y vector are occasionally missing.) +We usually take k=|X| << |Y|=D, so the model tries to explain +many observations using a low-dimensional subspace. + + +<center> +<table> +<tr> +<td><img src="Figures/fa.gif"> +<td><img src="Figures/fa_scalar.gif"> +<td><img src="Figures/mfa.gif"> +<td><img src="Figures/ifa.gif"> +<tr> +<td align=center> (a) +<td align=center> (b) +<td align=center> (c) +<td align=center> (d) +</table> +</center> + +<p> +We can create this model in BNT as follows. +<pre> +ns = [k D]; +dag = zeros(2,2); +dag(1,2) = 1; +bnet = mk_bnet(dag, ns, 'discrete', []); +bnet.CPD{1} = gaussian_CPD(bnet, 1, 'mean', zeros(k,1), 'cov', eye(k), ... + 'cov_type', 'diag', 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(D,1), 'cov', diag(Psi0), 'weights', W0, ... + 'cov_type', 'diag', 'clamp_mean', 1); +</pre> + +The root node is clamped to the N(0,I) distribution, so that we will +not update these parameters during learning. +The mean of the leaf node is clamped to 0, +since we assume the data has been centered (had its mean subtracted +off); this is just for simplicity. +Finally, the covariance of the leaf node is constrained to be +diagonal. W0 and Psi0 are the initial parameter guesses. + +<p> +We can fit this model (i.e., estimate its parameters in a maximum +likelihood (ML) sense) using EM, as we +explain <a href="#em">below</a>. +Not surprisingly, the ML estimates for mu and Psi turn out to be +identical to the +sample mean and variance, which can be computed directly as +<pre> +mu_ML = mean(data); +Psi_ML = diag(cov(data)); +</pre> +Note that W can only be identified up to a rotation matrix, because of +the spherical symmetry of the source. + +<p> +If we restrict Psi to be spherical, i.e., Psi = sigma*I, +there is a closed-form solution for W as well, +i.e., we do not need to use EM. +In particular, W contains the first |X| eigenvectors of the sample covariance +matrix, with scalings determined by the eigenvalues and sigma. +Classical PCA can be obtained by taking the sigma->0 limit. +For details, see + +<ul> +<li> <a href="ftp://hope.caltech.edu/pub/roweis/Empca/empca.ps"> +"EM algorithms for PCA and SPCA"</a>, Sam Roweis, NIPS 97. +(<a href="ftp://hope.caltech.edu/pub/roweis/Code/empca.tar.gz"> +Matlab software</a>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +By adding a hidden discrete variable, we can create mixtures of FA +models, as shown in (c). +Now we can explain the data using a set of subspaces. +We can create this model in BNT as follows. +<pre> +ns = [M k D]; +dag = zeros(3); +dag(1,3) = 1; +dag(2,3) = 1; +bnet = mk_bnet(dag, ns, 'discrete', 1); +bnet.CPD{1} = tabular_CPD(bnet, 1, Pi0); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(k, 1), 'cov', eye(k), 'cov_type', 'diag', ... + 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{3} = gaussian_CPD(bnet, 3, 'mean', Mu0', 'cov', repmat(diag(Psi0), [1 1 M]), ... + 'weights', W0, 'cov_type', 'diag', 'tied_cov', 1); +</pre> +Notice how the covariance matrix for Y is the same for all values of +Q; that is, the noise level in each sub-space is assumed the same. +However, we allow the offset, mu, to vary. +For details, see +<ul> + +<LI> +<a HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/tr-96-1.ps.gz"> The EM +Algorithm for Mixtures of Factor Analyzers </A>, +Ghahramani, Z. and Hinton, G.E. (1996), +University of Toronto +Technical Report CRG-TR-96-1. +(<A HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/mfa.tar.gz">Matlab software</A>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +I have included Zoubin's specialized MFA code (with his permission) +with the toolbox, so you can check that BNT gives the same results: +see 'BNT/examples/static/mfa1.m'. + +<p> +Independent Factor Analysis (IFA) generalizes FA by allowing a +non-Gaussian prior on each component of X. +(Note that we can approximate a non-Gaussian prior using a mixture of +Gaussians.) +This means that the likelihood function is no longer rotationally +invariant, so we can uniquely identify W and the hidden +sources X. +IFA also allows a non-diagonal Psi (i.e. correlations between the components of Y). +We recover classical Independent Components Analysis (ICA) +in the Psi -> 0 limit, and by assuming that |X|=|Y|, so that the +weight matrix W is square and invertible. +For details, see +<ul> +<li> +<a href="http://www.gatsby.ucl.ac.uk/~hagai/ifa.ps">Independent Factor +Analysis</a>, H. Attias, Neural Computation 11: 803--851, 1998. +</ul> + + + +<h2><a name="mixexp">Mixtures of experts</h2> + +As an example of the use of the softmax function, +we introduce the Mixture of Experts model. +<!-- +We also show +the Hierarchical Mixture of Experts model, where the hierarchy has two +levels. +(This is essentially a probabilistic decision tree of height two.) +--> +As before, +circles denote continuous-valued nodes, +squares denote discrete nodes, clear +means hidden, and shaded means observed. +<p> +<center> +<table> +<tr> +<td><img src="Figures/mixexp.gif"> +<!-- +<td><img src="Figures/hme.gif"> +--> +</table> +</center> +<p> +X is the observed +input, Y is the output, and +the Q nodes are hidden "gating" nodes, which select the appropriate +set of parameters for Y. During training, Y is assumed observed, +but for testing, the goal is to predict Y given X. +Note that this is a <em>conditional</em> density model, so we don't +associate any parameters with X. +Hence X's CPD will be a root CPD, which is a way of modelling +exogenous nodes. +If the output is a continuous-valued quantity, +we assume the "experts" are linear-regression units, +and set Y's CPD to linear-Gaussian. +If the output is discrete, we set Y's CPD to a softmax function. +The Q CPDs will always be softmax functions. + +<p> +As a concrete example, consider the mixture of experts model where X and Y are +scalars, and Q is binary. +This is just piecewise linear regression, where +we have two line segments, i.e., +<P> +<IMG ALIGN=BOTTOM SRC="Eqns/lin_reg_eqn.gif"> +<P> +We can create this model with random parameters as follows. +(This code is bundled in BNT/examples/static/mixexp2.m.) +<PRE> +X = 1; +Q = 2; +Y = 3; +dag = zeros(3,3); +dag(X,[Q Y]) = 1 +dag(Q,Y) = 1; +ns = [1 2 1]; % make X and Y scalars, and have 2 experts +onodes = [1 3]; +bnet = mk_bnet(dag, ns, 'discrete', 2, 'observed', onodes); + +rand('state', 0); +randn('state', 0); +bnet.CPD{1} = root_CPD(bnet, 1); +bnet.CPD{2} = softmax_CPD(bnet, 2); +bnet.CPD{3} = gaussian_CPD(bnet, 3); +</PRE> +Now let us fit this model using <a href="#em">EM</a>. +First we <a href="#load_data">load the data</a> (1000 training cases) and plot them. +<P> +<PRE> +data = load('/examples/static/Misc/mixexp_data.txt', '-ascii'); +plot(data(:,1), data(:,2), '.'); +</PRE> +<p> +<center> +<IMG SRC="Figures/mixexp_data.gif"> +</center> +<p> +This is what the model looks like before training. +(Thanks to Thomas Hofman for writing this plotting routine.) +<p> +<center> +<IMG SRC="Figures/mixexp_before.gif"> +</center> +<p> +Now let's train the model, and plot the final performance. +(We will discuss how to train models in more detail <a href="#param_learning">below</a>.) +<P> +<PRE> +ncases = size(data, 1); % each row of data is a training case +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); % each column of cases is a training case +engine = jtree_inf_engine(bnet); +max_iter = 20; +[bnet2, LLtrace] = learn_params_em(engine, cases, max_iter); +</PRE> +(We specify which nodes will be observed when we create the engine. +Hence BNT knows that the hidden nodes are all discrete. +For complex models, this can lead to a significant speedup.) +Below we show what the model looks like after 16 iterations of EM +(with 100 IRLS iterations per M step), when it converged +using the default convergence tolerance (that the +fractional change in the log-likelihood be less than 1e-3). +Before learning, the log-likelihood was +-322.927442; afterwards, it was -13.728778. +<p> +<center> +<IMG SRC="Figures/mixexp_after.gif"> +</center> +(See BNT/examples/static/mixexp2.m for details of the code.) + + + +<h2><a name="hme">Hierarchical mixtures of experts</h2> + +A hierarchical mixture of experts (HME) extends the mixture of experts +model by having more than one hidden node. A two-level example is shown below, along +with its more traditional representation as a neural network. +This is like a (balanced) probabilistic decision tree of height 2. +<p> +<center> +<IMG SRC="Figures/HMEforMatlab.jpg"> +</center> +<p> +<a href="mailto:pbrutti@stat.cmu.edu">Pierpaolo Brutti</a> +has written an extensive set of routines for HMEs, +which are bundled with BNT: see the examples/static/HME directory. +These routines allow you to choose the number of hidden (gating) +layers, and the form of the experts (softmax or MLP). +See the file hmemenu, which provides a demo. +For example, the figure below shows the decision boundaries learned +for a ternary classification problem, using a 2 level HME with softmax +gates and softmax experts; the training set is on the left, the +testing set on the right. +<p> +<center> +<!--<IMG SRC="Figures/hme_dec_boundary.gif">--> +<IMG SRC="Figures/hme_dec_boundary.png"> +</center> +<p> + + +<p> +For more details, see the following: +<ul> + +<li> <a href="http://www.cs.berkeley.edu/~jordan/papers/hierarchies.ps.Z"> +Hierarchical mixtures of experts and the EM algorithm</a> +M. I. Jordan and R. A. Jacobs. Neural Computation, 6, 181-214, 1994. + +<li> <a href = +"http://www.cs.berkeley.edu/~dmartin/software">David Martin's +matlab code for HME</a> + +<li> <a +href="http://www.cs.berkeley.edu/~jordan/papers/uai.ps.Z">Why the +logistic function? A tutorial discussion on +probabilities and neural networks.</a> M. I. Jordan. MIT Computational +Cognitive Science Report 9503, August 1995. + +<li> "Generalized Linear Models", McCullagh and Nelder, Chapman and +Halll, 1983. + +<li> +"Improved learning algorithms for mixtures of experts in multiclass +classification". +K. Chen, L. Xu, H. Chi. +Neural Networks (1999) 12: 1229-1252. + +<li> <a href="http://www.oigeeza.com/steve/"> +Classification Using Hierarchical Mixtures of Experts</a> +S.R. Waterhouse and A.J. Robinson. +In Proc. IEEE Workshop on Neural Network for Signal Processing IV (1994), pp. 177-186 + +<li> <a href="http://www.idiap.ch/~perry/"> +Localized mixtures of experts</a>, +P. Moerland, 1998. + +<li> "Nonlinear gated experts for time series", +A.S. Weigend and M. Mangeas, 1995. + +</ul> + + +<h2><a name="qmr">QMR</h2> + +Bayes nets originally arose out of an attempt to add probabilities to +expert systems, and this is still the most common use for BNs. +A famous example is +QMR-DT, a decision-theoretic reformulation of the Quick Medical +Reference (QMR) model. +<p> +<center> +<IMG ALIGN=BOTTOM SRC="Figures/qmr.gif"> +</center> +Here, the top layer represents hidden disease nodes, and the bottom +layer represents observed symptom nodes. +The goal is to infer the posterior probability of each disease given +all the symptoms (which can be present, absent or unknown). +Each node in the top layer has a Bernoulli prior (with a low prior +probability that the disease is present). +Since each node in the bottom layer has a high fan-in, we use a +noisy-OR parameterization; each disease has an independent chance of +causing each symptom. +The real QMR-DT model is copyright, but +we can create a random QMR-like model as follows. +<pre> +function bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% MK_QMR_BNET Make a QMR model +% bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% +% G(i,j) = 1 iff there is an arc from disease i to finding j +% inhibit(i,j) = inhibition probability on i->j arc +% leak(j) = inhibition prob. on leak->j arc +% prior(i) = prob. disease i is on + +[Ndiseases Nfindings] = size(inhibit); +N = Ndiseases + Nfindings; +finding_node = Ndiseases+1:N; +ns = 2*ones(1,N); +dag = zeros(N,N); +dag(1:Ndiseases, finding_node) = G; +bnet = mk_bnet(dag, ns, 'observed', finding_node); + +for d=1:Ndiseases + CPT = [1-prior(d) prior(d)]; + bnet.CPD{d} = tabular_CPD(bnet, d, CPT'); +end + +for i=1:Nfindings + fnode = finding_node(i); + ps = parents(G, i); + bnet.CPD{fnode} = noisyor_CPD(bnet, fnode, leak(i), inhibit(ps, i)); +end +</pre> +In the file BNT/examples/static/qmr1, we create a random bipartite +graph G, with 5 diseases and 10 findings, and random parameters. +(In general, to create a random dag, use 'mk_random_dag'.) +We can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>, with the +following results: +<p> +<img src="Figures/qmr.rnd.jpg"> + +<p> +Now let us put some random evidence on all the leaves except the very +first and very last, and compute the disease posteriors. +<pre> +pos = 2:floor(Nfindings/2); +neg = (pos(end)+1):(Nfindings-1); +onodes = myunion(pos, neg); +evidence = cell(1, N); +evidence(findings(pos)) = num2cell(repmat(2, 1, length(pos))); +evidence(findings(neg)) = num2cell(repmat(1, 1, length(neg))); + +engine = jtree_inf_engine(bnet); +[engine, ll] = enter_evidence(engine, evidence); +post = zeros(1, Ndiseases); +for i=diseases(:)' + m = marginal_nodes(engine, i); + post(i) = m.T(2); +end +</pre> +Junction tree can be quite slow on large QMR models. +Fortunately, it is possible to exploit properties of the noisy-OR +function to speed up exact inference using an algorithm called +<a href="#quickscore">quickscore</a>, discussed below. + + + + + +<h2><a name="cg_model">Conditional Gaussian models</h2> + +A conditional Gaussian model is one in which, conditioned on all the discrete +nodes, the distribution over the remaining (continuous) nodes is +multivariate Gaussian. This means we can have arcs from discrete (D) +to continuous (C) nodes, but not vice versa. +(We <em>are</em> allowed C->D arcs if the continuous nodes are observed, +as in the <a href="#mixexp">mixture of experts</a> model, +since this distribution can be represented with a discrete potential.) +<p> +We now give an example of a CG model, from +the paper "Propagation of Probabilities, Means amd +Variances in Mixed Graphical Association Models", Steffen Lauritzen, +JASA 87(420):1098--1108, 1992 (reprinted in the book "Probabilistic Networks and Expert +Systems", R. G. Cowell, A. P. Dawid, S. L. Lauritzen and +D. J. Spiegelhalter, Springer, 1999.) + +<h3>Specifying the graph</h3> + +Consider the model of waste emissions from an incinerator plant shown below. +We follow the standard convention that shaded nodes are observed, +clear nodes are hidden. +We also use the non-standard convention that +square nodes are discrete (tabular) and round nodes are +Gaussian. + +<p> +<center> +<IMG SRC="Figures/cg1.gif"> +</center> +<p> + +We can create this model as follows. +<pre> +F = 1; W = 2; E = 3; B = 4; C = 5; D = 6; Min = 7; Mout = 8; L = 9; +n = 9; + +dag = zeros(n); +dag(F,E)=1; +dag(W,[E Min D]) = 1; +dag(E,D)=1; +dag(B,[C D])=1; +dag(D,[L Mout])=1; +dag(Min,Mout)=1; + +% node sizes - all cts nodes are scalar, all discrete nodes are binary +ns = ones(1, n); +dnodes = [F W B]; +cnodes = mysetdiff(1:n, dnodes); +ns(dnodes) = 2; + +bnet = mk_bnet(dag, ns, 'discrete', dnodes); +</pre> +'dnodes' is a list of the discrete nodes; 'cnodes' is the continuous +nodes. 'mysetdiff' is a faster version of the built-in 'setdiff'. +<p> + + +<h3>Specifying the parameters</h3> + +The parameters of the discrete nodes can be specified as follows. +<pre> +bnet.CPD{B} = tabular_CPD(bnet, B, 'CPT', [0.85 0.15]); % 1=stable, 2=unstable +bnet.CPD{F} = tabular_CPD(bnet, F, 'CPT', [0.95 0.05]); % 1=intact, 2=defect +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [2/7 5/7]); % 1=industrial, 2=household +</pre> + +<p> +The parameters of the continuous nodes can be specified as follows. +<pre> +bnet.CPD{E} = gaussian_CPD(bnet, E, 'mean', [-3.9 -0.4 -3.2 -0.5], ... + 'cov', [0.00002 0.0001 0.00002 0.0001]); +bnet.CPD{D} = gaussian_CPD(bnet, D, 'mean', [6.5 6.0 7.5 7.0], ... + 'cov', [0.03 0.04 0.1 0.1], 'weights', [1 1 1 1]); +bnet.CPD{C} = gaussian_CPD(bnet, C, 'mean', [-2 -1], 'cov', [0.1 0.3]); +bnet.CPD{L} = gaussian_CPD(bnet, L, 'mean', 3, 'cov', 0.25, 'weights', -0.5); +bnet.CPD{Min} = gaussian_CPD(bnet, Min, 'mean', [0.5 -0.5], 'cov', [0.01 0.005]); +bnet.CPD{Mout} = gaussian_CPD(bnet, Mout, 'mean', 0, 'cov', 0.002, 'weights', [1 1]); +</pre> + + +<h3><a name="cg_infer">Inference</h3> + +<!--Let us perform inference in the <a href="#cg_model">waste incinerator example</a>.--> +First we compute the unconditional marginals. +<pre> +engine = jtree_inf_engine(bnet); +evidence = cell(1,n); +[engine, ll] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, E); +</pre> +<!--(Of course, we could use <tt>cond_gauss_inf_engine</tt> instead of jtree.)--> +'marg' is a structure that contains the fields 'mu' and 'Sigma', which +contain the mean and (co)variance of the marginal on E. +In this case, they are both scalars. +Let us check they match the published figures (to 2 decimal places). +<!--(We can't expect +more precision than this in general because I have implemented the algorithm of +Lauritzen (1992), which can be numerically unstable.)--> +<pre> +tol = 1e-2; +assert(approxeq(marg.mu, -3.25, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.709, tol)); +</pre> +We can compute the other posteriors similarly. +Now let us add some evidence. +<pre> +evidence = cell(1,n); +evidence{W} = 1; % industrial +evidence{L} = 1.1; +evidence{C} = -0.9; +[engine, ll] = enter_evidence(engine, evidence); +</pre> +Now we find +<pre> +marg = marginal_nodes(engine, E); +assert(approxeq(marg.mu, -3.8983, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.0763, tol)); +</pre> + + +We can also compute the joint probability on a set of nodes. +For example, P(D, Mout | evidence) is a 2D Gaussian: +<pre> +marg = marginal_nodes(engine, [D Mout]) +marg = + domain: [6 8] + mu: [2x1 double] + Sigma: [2x2 double] + T: 1.0000 +</pre> +The mean is +<pre> +marg.mu +ans = + 3.6077 + 4.1077 +</pre> +and the covariance matrix is +<pre> +marg.Sigma +ans = + 0.1062 0.1062 + 0.1062 0.1182 +</pre> +It is easy to visualize this posterior using standard Matlab plotting +functions, e.g., +<pre> +gaussplot2d(marg.mu, marg.Sigma); +</pre> +produces the following picture. + +<p> +<center> +<IMG SRC="Figures/gaussplot.png"> +</center> +<p> + + +The T field indicates that the mixing weight of this Gaussian +component is 1.0. +If the joint contains discrete and continuous variables, the result +will be a mixture of Gaussians, e.g., +<pre> +marg = marginal_nodes(engine, [F E]) + domain: [1 3] + mu: [-3.9000 -0.4003] + Sigma: [1x1x2 double] + T: [0.9995 4.7373e-04] +</pre> +The interpretation is +Sigma(i,j,k) = Cov[ E(i) E(j) | F=k ]. +In this case, E is a scalar, so i=j=1; k specifies the mixture component. +<p> +We saw in the sprinkler network that BNT sets the effective size of +observed discrete nodes to 1, since they only have one legal value. +For continuous nodes, BNT sets their length to 0, +since they have been reduced to a point. +For example, +<pre> +marg = marginal_nodes(engine, [B C]) + domain: [4 5] + mu: [] + Sigma: [] + T: [0.0123 0.9877] +</pre> +It is simple to post-process the output of marginal_nodes. +For example, the file BNT/examples/static/cg1 sets the mu term of +observed nodes to their observed value, and the Sigma term to 0 (since +observed nodes have no variance). + +<p> +Note that the implemented version of the junction tree is numerically +unstable when using CG potentials +(which is why, in the example above, we only required our answers to agree with +the published ones to 2dp.) +This is why you might want to use <tt>stab_cond_gauss_inf_engine</tt>, +implemented by Shan Huang. This is described in + +<ul> +<li> "Stable Local Computation with Conditional Gaussian Distributions", +S. Lauritzen and F. Jensen, Tech Report R-99-2014, +Dept. Math. Sciences, Allborg Univ., 1999. +</ul> + +However, even the numerically stable version +can be computationally intractable if there are many hidden discrete +nodes, because the number of mixture components grows exponentially e.g., in a +<a href="usage_dbn.html#lds">switching linear dynamical system</a>. +In general, one must resort to approximate inference techniques: see +the discussion on <a href="#engines">inference engines</a> below. + + +<h2><a name="hybrid">Other hybrid models</h2> + +When we have C->D arcs, where C is hidden, we need to use +approximate inference. +One approach (not implemented in BNT) is described in +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/hybrid_uai99.ps.gz">A +Variational Approximation for Bayesian Networks with +Discrete and Continuous Latent Variables</a>, +K. Murphy, UAI 99. +</ul> +Of course, one can always use <a href="#sampling">sampling</a> methods +for approximate inference in such models. + + + +<h1><a name="param_learning">Parameter Learning</h1> + +The parameter estimation routines in BNT can be classified into 4 +types, depending on whether the goal is to compute +a full (Bayesian) posterior over the parameters or just a point +estimate (e.g., Maximum Likelihood or Maximum A Posteriori), +and whether all the variables are fully observed or there is missing +data/ hidden variables (partial observability). +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_params</tt></td> + <td><tt>learn_params_em</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>bayes_update_params</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="load_data">Loading data from a file</h2> + +To load numeric data from an ASCII text file called 'dat.txt', where each row is a +case and columns are separated by white-space, such as +<pre> +011979 1626.5 0.0 +021979 1367.0 0.0 +... +</pre> +you can use +<pre> +data = load('dat.txt'); +</pre> +or +<pre> +load dat.txt -ascii +</pre> +In the latter case, the data is stored in a variable called 'dat' (the +filename minus the extension). +Alternatively, suppose the data is stored in a .csv file (has commas +separating the columns, and contains a header line), such as +<pre> +header info goes here +ORD,011979,1626.5,0.0 +DSM,021979,1367.0,0.0 +... +</pre> +You can load this using +<pre> +[a,b,c,d] = textread('dat.txt', '%s %d %f %f', 'delimiter', ',', 'headerlines', 1); +</pre> +If your file is not in either of these formats, you can either use Perl to convert +it to this format, or use the Matlab scanf command. +Type +<tt> +help iofun +</tt> +for more information on Matlab's file functions. +<!-- +<p> +To load data directly from Excel, +you should buy the +<a href="http://www.mathworks.com/products/excellink/">Excel Link</a>. +To load data directly from a relational database, +you should buy the +<a href="http://www.mathworks.com/products/database">Database +toolbox</a>. +--> +<p> +BNT learning routines require data to be stored in a cell array. +data{i,m} is the value of node i in case (example) m, i.e., each +<em>column</em> is a case. +If node i is not observed in case m (missing value), set +data{i,m} = []. +(Not all the learning routines can cope with such missing values, however.) +In the special case that all the nodes are observed and are +scalar-valued (as opposed to vector-valued), the data can be +stored in a matrix (as opposed to a cell-array). +<p> +Suppose, as in the <a href="#mixexp">mixture of experts example</a>, +that we have 3 nodes in the graph: X(1) is the observed input, X(3) is +the observed output, and X(2) is a hidden (gating) node. We can +create the dataset as follows. +<pre> +data = load('dat.txt'); +ncases = size(data, 1); +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); +</pre> +Notice how we transposed the data, to convert rows into columns. +Also, cases{2,m} = [] for all m, since X(2) is always hidden. + + +<h2><a name="mle_complete">Maximum likelihood parameter estimation from complete data</h2> + +As an example, let's generate some data from the sprinkler network, randomize the parameters, +and then try to recover the original model. +First we create some training data using forwards sampling. +<pre> +samples = cell(N, nsamples); +for i=1:nsamples + samples(:,i) = sample_bnet(bnet); +end +</pre> +samples{j,i} contains the value of the j'th node in case i. +sample_bnet returns a cell array because, in general, each node might +be a vector of different length. +In this case, all nodes are discrete (and hence scalars), so we +could have used a regular array instead (which can be quicker): +<pre> +data = cell2num(samples); +</pre +So now data(j,i) = samples{j,i}. +<p> +Now we create a network with random parameters. +(The initial values of bnet2 don't matter in this case, since we can find the +globally optimal MLE independent of where we start.) +<pre> +% Make a tabula rasa +bnet2 = mk_bnet(dag, node_sizes); +seed = 0; +rand('state', seed); +bnet2.CPD{C} = tabular_CPD(bnet2, C); +bnet2.CPD{R} = tabular_CPD(bnet2, R); +bnet2.CPD{S} = tabular_CPD(bnet2, S); +bnet2.CPD{W} = tabular_CPD(bnet2, W); +</pre> +Finally, we find the maximum likelihood estimates of the parameters. +<pre> +bnet3 = learn_params(bnet2, samples); +</pre> +To view the learned parameters, we use a little Matlab hackery. +<pre> +CPT3 = cell(1,N); +for i=1:N + s=struct(bnet3.CPD{i}); % violate object privacy + CPT3{i}=s.CPT; +end +</pre> +Here are the parameters learned for node 4. +<pre> +dispcpt(CPT3{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.2000 0.8000 +1 2 : 0.2273 0.7727 +2 2 : 0.0000 1.0000 +</pre> +So we see that the learned parameters are fairly close to the "true" +ones, which we display below. +<pre> +dispcpt(CPT{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.1000 0.9000 +1 2 : 0.1000 0.9000 +2 2 : 0.0100 0.9900 +</pre> +We can get better results by using a larger training set, or using +informative priors (see <a href="#prior">below</a>). + + + +<h2><a name="prior">Parameter priors</h2> + +Currently, only tabular CPDs can have priors on their parameters. +The conjugate prior for a multinomial is the Dirichlet. +(For binary random variables, the multinomial is the same as the +Bernoulli, and the Dirichlet is the same as the Beta.) +<p> +The Dirichlet has a simple interpretation in terms of pseudo counts. +If we let N_ijk = the num. times X_i=k and Pa_i=j occurs in the +training set, where Pa_i are the parents of X_i, +then the maximum likelihood (ML) estimate is +T_ijk = N_ijk / N_ij (where N_ij = sum_k' N_ijk'), which will be 0 if N_ijk=0. +To prevent us from declaring that (X_i=k, Pa_i=j) is impossible just because this +event was not seen in the training set, +we can pretend we saw value k of X_i, for each value j of Pa_i some number (alpha_ijk) +of times in the past. +The MAP (maximum a posterior) estimate is then +<pre> +T_ijk = (N_ijk + alpha_ijk) / (N_ij + alpha_ij) +</pre> +and is never 0 if all alpha_ijk > 0. +For example, consider the network A->B, where A is binary and B has 3 +values. +A uniform prior for B has the form +<pre> + B=1 B=2 B=3 +A=1 1 1 1 +A=2 1 1 1 +</pre> +which can be created using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'unif'); +</pre> +This prior does not satisfy the likelihood equivalence principle, +which says that <a href="#markov_equiv">Markov equivalent</a> models +should have the same marginal likelihood. +A prior that does satisfy this principle is shown below. +Heckerman (1995) calls this the +BDeu prior (likelihood equivalent uniform Bayesian Dirichlet). +<pre> + B=1 B=2 B=3 +A=1 1/6 1/6 1/6 +A=2 1/6 1/6 1/6 +</pre> +where we put N/(q*r) in each bin; N is the equivalent sample size, +r=|A|, q = |B|. +This can be created as follows +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'BDeu'); +</pre> +Here, 1 is the equivalent sample size, and is the strength of the +prior. +You can change this using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', ... + 'BDeu', 'dirichlet_weight', 10); +</pre> +<!--where counts is an array of pseudo-counts of the same size as the +CPT.--> +<!-- +<p> +When you specify a prior, you should set row i of the CPT to the +normalized version of row i of the pseudo-count matrix, i.e., to the +expected values of the parameters. This will ensure that computing the +marginal likelihood sequentially (see <a +href="#bayes_learn">below</a>) and in batch form gives the same +results. +To do this, proceed as follows. +<pre> +tabular_CPD(bnet, i, 'prior', counts, 'CPT', mk_stochastic(counts)); +</pre> +For a non-informative prior, you can just write +<pre> +tabular_CPD(bnet, i, 'prior', 'unif', 'CPT', 'unif'); +</pre> +--> + + +<h2><a name="bayes_learn">(Sequential) Bayesian parameter updating from complete data</h2> + +If we use conjugate priors and have fully observed data, we can +compute the posterior over the parameters in batch form as follows. +<pre> +cases = sample_bnet(bnet, nsamples); +bnet = bayes_update_params(bnet, cases); +LL = log_marg_lik_complete(bnet, cases); +</pre> +bnet.CPD{i}.prior contains the new Dirichlet pseudocounts, +and bnet.CPD{i}.CPT is set to the mean of the posterior (the +normalized counts). +(Hence if the initial pseudo counts are 0, +<tt>bayes_update_params</tt> and <tt>learn_params</tt> will give the +same result.) + + + + +<p> +We can compute the same result sequentially (on-line) as follows. +<pre> +LL = 0; +for m=1:nsamples + LL = LL + log_marg_lik_complete(bnet, cases(:,m)); + bnet = bayes_update_params(bnet, cases(:,m)); +end +</pre> + +The file <tt>BNT/examples/static/StructLearn/model_select1</tt> has an example of +sequential model selection which uses the same idea. +We generate data from the model A->B +and compute the posterior prob of all 3 dags on 2 nodes: + (1) A B, (2) A <- B , (3) A -> B +Models 2 and 3 are <a href="#markov_equiv">Markov equivalent</a>, and therefore indistinguishable from +observational data alone, so we expect their posteriors to be the same +(assuming a prior which satisfies likelihood equivalence). +If we use random parameters, the "true" model only gets a higher posterior after 2000 trials! +However, if we make B a noisy NOT gate, the true model "wins" after 12 +trials, as shown below (red = model 1, blue/green (superimposed) +represents models 2/3). +<p> +<img src="Figures/model_select.png"> +<p> +The use of marginal likelihood for model selection is discussed in +greater detail in the +section on <a href="structure_learning">structure learning</a>. + + + + +<h2><a name="em">Maximum likelihood parameter estimation with missing values</h2> + +Now we consider learning when some values are not observed. +Let us randomly hide half the values generated from the water +sprinkler example. +<pre> +samples2 = samples; +hide = rand(N, nsamples) > 0.5; +[I,J]=find(hide); +for k=1:length(I) + samples2{I(k), J(k)} = []; +end +</pre> +samples2{i,l} is the value of node i in training case l, or [] if unobserved. +<p> +Now we will compute the MLEs using the EM algorithm. +We need to use an inference algorithm to compute the expected +sufficient statistics in the E step; the M (maximization) step is as +above. +<pre> +engine2 = jtree_inf_engine(bnet2); +max_iter = 10; +[bnet4, LLtrace] = learn_params_em(engine2, samples2, max_iter); +</pre> +LLtrace(i) is the log-likelihood at iteration i. We can plot this as +follows: +<pre> +plot(LLtrace, 'x-') +</pre> +Let's display the results after 10 iterations of EM. +<pre> +celldisp(CPT4) +CPT4{1} = + 0.6616 + 0.3384 +CPT4{2} = + 0.6510 0.3490 + 0.8751 0.1249 +CPT4{3} = + 0.8366 0.1634 + 0.0197 0.9803 +CPT4{4} = +(:,:,1) = + 0.8276 0.0546 + 0.5452 0.1658 +(:,:,2) = + 0.1724 0.9454 + 0.4548 0.8342 +</pre> +We can get improved performance by using one or more of the following +methods: +<ul> +<li> Increasing the size of the training set. +<li> Decreasing the amount of hidden data. +<li> Running EM for longer. +<li> Using informative priors. +<li> Initialising EM from multiple starting points. +</ul> + +Click <a href="#gaussian">here</a> for a discussion of learning +Gaussians, which can cause numerical problems. + +<h2><a name="tying">Parameter tying</h2> + +In networks with repeated structure (e.g., chains and grids), it is +common to assume that the parameters are the same at every node. This +is called parameter tying, and reduces the amount of data needed for +learning. +<p> +When we have tied parameters, there is no longer a one-to-one +correspondence between nodes and CPDs. +Rather, each CPD species the parameters for a whole equivalence class +of nodes. +It is easiest to see this by example. +Consider the following <a href="usage_dbn.html#hmm">hidden Markov +model (HMM)</a> +<p> +<img src="Figures/hmm3.gif"> +<p> +<!-- +We can create this graph structure, assuming we have T time-slices, +as follows. +(We number the nodes as shown in the figure, but we could equally well +number the hidden nodes 1:T, and the observed nodes T+1:2T.) +<pre> +N = 2*T; +dag = zeros(N); +hnodes = 1:2:2*T; +for i=1:T-1 + dag(hnodes(i), hnodes(i+1))=1; +end +onodes = 2:2:2*T; +for i=1:T + dag(hnodes(i), onodes(i)) = 1; +end +</pre> +<p> +The hidden nodes are always discrete, and have Q possible values each, +but the observed nodes can be discrete or continuous, and have O possible values/length. +<pre> +if cts_obs + dnodes = hnodes; +else + dnodes = 1:N; +end +ns = ones(1,N); +ns(hnodes) = Q; +ns(onodes) = O; +</pre> +--> +When HMMs are used for semi-infinite processes like speech recognition, +we assume the transition matrix +P(H(t+1)|H(t)) is the same for all t; this is called a time-invariant +or homogenous Markov chain. +Hence hidden nodes 2, 3, ..., T +are all in the same equivalence class, say class Hclass. +Similarly, the observation matrix P(O(t)|H(t)) is assumed to be the +same for all t, so the observed nodes are all in the same equivalence +class, say class Oclass. +Finally, the prior term P(H(1)) is in a class all by itself, say class +H1class. +This is illustrated below, where we explicitly represent the +parameters as random variables (dotted nodes). +<p> +<img src="Figures/hmm4_params.gif"> +<p> +In BNT, we cannot represent parameters as random variables (nodes). +Instead, we "hide" the +parameters inside one CPD for each equivalence class, +and then specify that the other CPDs should share these parameters, as +follows. +<pre> +hnodes = 1:2:2*T; +onodes = 2:2:2*T; +H1class = 1; Hclass = 2; Oclass = 3; +eclass = ones(1,N); +eclass(hnodes(2:end)) = Hclass; +eclass(hnodes(1)) = H1class; +eclass(onodes) = Oclass; +% create dag and ns in the usual way +bnet = mk_bnet(dag, ns, 'discrete', dnodes, 'equiv_class', eclass); +</pre> +Finally, we define the parameters for each equivalence class: +<pre> +bnet.CPD{H1class} = tabular_CPD(bnet, hnodes(1)); % prior +bnet.CPD{Hclass} = tabular_CPD(bnet, hnodes(2)); % transition matrix +if cts_obs + bnet.CPD{Oclass} = gaussian_CPD(bnet, onodes(1)); +else + bnet.CPD{Oclass} = tabular_CPD(bnet, onodes(1)); +end +</pre> +In general, if bnet.CPD{e} = xxx_CPD(bnet, j), then j should be a +member of e's equivalence class; that is, it is not always the case +that e == j. You can use bnet.rep_of_eclass(e) to return the +representative of equivalence class e. +BNT will look up the parents of j to determine the size +of the CPT to use. It assumes that this is the same for all members of +the equivalence class. +Click <a href="http://www.cs.berkeley.edu/~murphyk/Bayes/param_tieing.html">here</a> for +a more complex example of parameter tying. +<p> +Note: +Normally one would define an HMM as a +<a href = "usage_dbn.html">Dynamic Bayes Net</a> +(see the function BNT/examples/dynamic/mk_chmm.m). +However, one can define an HMM as a static BN using the function +BNT/examples/static/Models/mk_hmm_bnet.m. + + + +<h1><a name="structure_learning">Structure learning</h1> + +Update (9/29/03): +Phillipe LeRay is developing some additional structure learning code +on top of BNT. Click <a +href="http://bnt.insa-rouen.fr/ajouts.html">here</a> +for details. + +<p> + +There are two very different approaches to structure learning: +constraint-based and search-and-score. +In the <a href="#constraint">constraint-based approach</a>, +we start with a fully connected graph, and remove edges if certain +conditional independencies are measured in the data. +This has the disadvantage that repeated independence tests lose +statistical power. +<p> +In the more popular search-and-score approach, +we perform a search through the space of possible DAGs, and either +return the best one found (a point estimate), or return a sample of the +models found (an approximation to the Bayesian posterior). +<p> +Unfortunately, the number of DAGs as a function of the number of +nodes, G(n), is super-exponential in n. +A closed form formula for G(n) is not known, but the first few values +are shown below (from Cooper, 1999). + +<table> +<tr> <th>n</th> <th align=left>G(n)</th> </tr> +<tr> <td>1</td> <td>1</td> </tr> +<tr> <td>2</td> <td>3</td> </tr> +<tr> <td>3</td> <td>25</td> </tr> +<tr> <td>4</td> <td>543</td> </tr> +<tr> <td>5</td> <td>29,281</td> </tr> +<tr> <td>6</td> <td>3,781,503</td> </tr> +<tr> <td>7</td> <td>1.1 x 10^9</td> </tr> +<tr> <td>8</td> <td>7.8 x 10^11</td> </tr> +<tr> <td>9</td> <td>1.2 x 10^15</td> </tr> +<tr> <td>10</td> <td>4.2 x 10^18</td> </tr> +</table> + +Since the number of DAGs is super-exponential in the number of nodes, +we cannot exhaustively search the space, so we either use a local +search algorithm (e.g., greedy hill climbining, perhaps with multiple +restarts) or a global search algorithm (e.g., Markov Chain Monte +Carlo). +<p> +If we know a total ordering on the nodes, +finding the best structure amounts to picking the best set of parents +for each node independently. +This is what the K2 algorithm does. +If the ordering is unknown, we can search over orderings, +which is more efficient than searching over DAGs (Koller and Friedman, 2000). +<p> +In addition to the search procedure, we must specify the scoring +function. There are two popular choices. The Bayesian score integrates +out the parameters, i.e., it is the marginal likelihood of the model. +The BIC (Bayesian Information Criterion) is defined as +log P(D|theta_hat) - 0.5*d*log(N), where D is the data, theta_hat is +the ML estimate of the parameters, d is the number of parameters, and +N is the number of data cases. +The BIC method has the advantage of not requiring a prior. +<p> +BIC can be derived as a large sample +approximation to the marginal likelihood. +(It is also equal to the Minimum Description Length of a model.) +However, in practice, the sample size does not need to be very large +for the approximation to be good. +For example, in the figure below, we plot the ratio between the log marginal likelihood +and the BIC score against data-set size; we see that the ratio rapidly +approaches 1, especially for non-informative priors. +(This plot was generated by the file BNT/examples/static/bic1.m. It +uses the water sprinkler BN with BDeu Dirichlet priors with different +equivalent sample sizes.) + +<p> +<center> +<IMG SRC="Figures/bic.png"> +</center> +<p> + +<p> +As with parameter learning, handling missing data/ hidden variables is +much harder than the fully observed case. +The structure learning routines in BNT can therefore be classified into 4 +types, analogously to the parameter learning case. +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_struct_K2</tt> <br> +<!-- <tt>learn_struct_hill_climb</tt></td> --> + <td><tt>not yet supported</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>learn_struct_mcmc</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="markov_equiv">Markov equivalence</h2> + +If two DAGs encode the same conditional independencies, they are +called Markov equivalent. The set of all DAGs can be paritioned into +Markov equivalence classes. Graphs within the same class can +have +the direction of some of their arcs reversed without changing any of +the CI relationships. +Each class can be represented by a PDAG +(partially directed acyclic graph) called an essential graph or +pattern. This specifies which edges must be oriented in a certain +direction, and which may be reversed. + +<p> +When learning graph structure from observational data, +the best one can hope to do is to identify the model up to Markov +equivalence. To distinguish amongst graphs within the same equivalence +class, one needs interventional data: see the discussion on <a +href="#active">active learning</a> below. + + + +<h2><a name="enumerate">Exhaustive search</h2> + +The brute-force approach to structure learning is to enumerate all +possible DAGs, and score each one. This provides a "gold standard" +with which to compare other algorithms. We can do this as follows. +<pre> +dags = mk_all_dags(N); +score = score_dags(data, ns, dags); +</pre> +where data(i,m) is the value of node i in case m, +and ns(i) is the size of node i. +If the DAGs have a lot of families in common, we can cache the sufficient statistics, +making this potentially more efficient than scoring the DAGs one at a time. +(Caching is not currently implemented, however.) +<p> +By default, we use the Bayesian scoring metric, and assume CPDs are +represented by tables with BDeu(1) priors. +We can override these defaults as follows. +If we want to use uniform priors, we can say +<pre> +params = cell(1,N); +for i=1:N + params{i} = {'prior', 'unif'}; +end +score = score_dags(data, ns, dags, 'params', params); +</pre> +params{i} is a cell-array, containing optional arguments that are +passed to the constructor for CPD i. +<p> +Now suppose we want to use different node types, e.g., +Suppose nodes 1 and 2 are Gaussian, and nodes 3 and 4 softmax (both +these CPDs can support discrete and continuous parents, which is +necessary since all other nodes will be considered as parents). +The Bayesian scoring metric currently only works for tabular CPDs, so +we will use BIC: +<pre> +score = score_dags(data, ns, dags, 'discrete', [3 4], 'params', [], + 'type', {'gaussian', 'gaussian', 'softmax', softmax'}, 'scoring_fn', 'bic') +</pre> +In practice, one can't enumerate all possible DAGs for N > 5, +but one can evaluate any reasonably-sized set of hypotheses in this +way (e.g., nearest neighbors of your current best guess). +Think of this as "computer assisted model refinement" as opposed to de +novo learning. + + +<h2><a name="K2">K2</h2> + +The K2 algorithm (Cooper and Herskovits, 1992) is a greedy search algorithm that works as follows. +Initially each node has no parents. It then adds incrementally that parent whose addition most +increases the score of the resulting structure. When the addition of no single +parent can increase the score, it stops adding parents to the node. +Since we are using a fixed ordering, we do not need to check for +cycles, and can choose the parents for each node independently. +<p> +The original paper used the Bayesian scoring +metric with tabular CPDs and Dirichlet priors. +BNT generalizes this to allow any kind of CPD, and either the Bayesian +scoring metric or BIC, as in the example <a href="#enumerate">above</a>. +In addition, you can specify +an optional upper bound on the number of parents for each node. +The file BNT/examples/static/k2demo1.m gives an example of how to use K2. +We use the water sprinkler network and sample 100 cases from it as before. +Then we see how much data it takes to recover the generating structure: +<pre> +order = [C S R W]; +max_fan_in = 2; +sz = 5:5:100; +for i=1:length(sz) + dag2 = learn_struct_K2(data(:,1:sz(i)), node_sizes, order, 'max_fan_in', max_fan_in); + correct(i) = isequal(dag, dag2); +end +</pre> +Here are the results. +<pre> +correct = + Columns 1 through 12 + 0 0 0 0 0 0 0 1 0 1 1 1 + Columns 13 through 20 + 1 1 1 1 1 1 1 1 +</pre> +So we see it takes about sz(10)=50 cases. (BIC behaves similarly, +showing that the prior doesn't matter too much.) +In general, we cannot hope to recover the "true" generating structure, +only one that is in its <a href="#markov_equiv">Markov equivalence +class</a>. + + +<h2><a name="hill_climb">Hill-climbing</h2> + +Hill-climbing starts at a specific point in space, +considers all nearest neighbors, and moves to the neighbor +that has the highest score; if no neighbors have higher +score than the current point (i.e., we have reached a local maximum), +the algorithm stops. One can then restart in another part of the space. +<p> +A common definition of "neighbor" is all graphs that can be +generated from the current graph by adding, deleting or reversing a +single arc, subject to the acyclicity constraint. +Other neighborhoods are possible: see +<a href="http://research.microsoft.com/~dmax/publications/jmlr02.pdf"> +Optimal Structure Identification with Greedy Search</a>, Max +Chickering, JMLR 2002. + +<!-- +Note: This algorithm is currently (Feb '02) being implemented by Qian +Diao. +--> + + +<h2><a name="mcmc">MCMC</h2> + +We can use a Markov Chain Monte Carlo (MCMC) algorithm called +Metropolis-Hastings (MH) to search the space of all +DAGs. +The standard proposal distribution is to consider moving to all +nearest neighbors in the sense defined <a href="#hill_climb">above</a>. +<p> +The function can be called +as in the following example. +<pre> +[sampled_graphs, accept_ratio] = learn_struct_mcmc(data, ns, 'nsamples', 100, 'burnin', 10); +</pre> +We can convert our set of sampled graphs to a histogram +(empirical posterior over all the DAGs) thus +<pre> +all_dags = mk_all_dags(N); +mcmc_post = mcmc_sample_to_hist(sampled_graphs, all_dags); +</pre> +To see how well this performs, let us compute the exact posterior exhaustively. +<p> +<pre> +score = score_dags(data, ns, all_dags); +post = normalise(exp(score)); % assuming uniform structural prior +</pre> +We plot the results below. +(The data set was 100 samples drawn from a random 4 node bnet; see the +file BNT/examples/static/mcmc1.) +<pre> +subplot(2,1,1) +bar(post) +subplot(2,1,2) +bar(mcmc_post) +</pre> +<img src="Figures/mcmc_post.jpg" width="800" height="500"> +<p> +We can also plot the acceptance ratio versus number of MCMC steps, +as a crude convergence diagnostic. +<pre> +clf +plot(accept_ratio) +</pre> +<img src="Figures/mcmc_accept.jpg" width="800" height="300"> +<p> +Even though the number of samples needed by MCMC is theoretically +polynomial (not exponential) in the dimensionality of the search space, in practice it has been +found that MCMC does not converge in reasonable time for graphs with +more than about 10 nodes. + + + + +<h2><a name="active">Active structure learning</h2> + +As was mentioned <a href="#markov_equiv">above</a>, +one can only learn a DAG up to Markov equivalence, even given infinite data. +If one is interested in learning the structure of a causal network, +one needs interventional data. +(By "intervention" we mean forcing a node to take on a specific value, +thereby effectively severing its incoming arcs.) +<p> +Most of the scoring functions accept an optional argument +that specifies whether a node was observed to have a certain value, or +was forced to have that value: we set clamped(i,m)=1 if node i was +forced in training case m. e.g., see the file +BNT/examples/static/cooper_yoo. +<p> +An interesting question is to decide which interventions to perform +(c.f., design of experiments). For details, see the following tech +report +<ul> +<li> <a href = "../../Papers/alearn.ps.gz"> +Active learning of causal Bayes net structure</a>, Kevin Murphy, March +2001. +</ul> + + +<h2><a name="struct_em">Structural EM</h2> + +Computing the Bayesian score when there is partial observability is +computationally challenging, because the parameter posterior becomes +multimodal (the hidden nodes induce a mixture distribution). +One therefore needs to use approximations such as BIC. +Unfortunately, search algorithms are still expensive, because we need +to run EM at each step to compute the MLE, which is needed to compute +the score of each model. An alternative approach is +to do the local search steps inside of the M step of EM, which is more +efficient since the data has been "filled in" - this is +called the structural EM algorithm (Friedman 1997), and provably +converges to a local maximum of the BIC score. +<p> +Wei Hu has implemented SEM for discrete nodes. +You can download his package from +<a href="../SEM.zip">here</a>. +Please address all questions about this code to +wei.hu@intel.com. +See also <a href="#phl">Phl's implementation of SEM</a>. + +<!-- +<h2><a name="reveal">REVEAL algorithm</h2> + +A simple way to learn the structure of a fully observed, discrete, +factored DBN from a time series is described <a +href="usage_dbn.html#struct_learn">here</a>. +--> + + +<h2><a name="graphdraw">Visualizing the graph</h2> + +You can visualize an arbitrary graph (such as one learned using the +structure learning routines) with Matlab code contributed by +<a href="http://www.mbfys.kun.nl/~cemgil/matlab/layout.html">Ali +Taylan Cemgil</a> +from the University of Nijmegen. +For static BNs, call it as follows: +<pre> +draw_graph(bnet.dag); +</pre> +For example, this is the output produced on a +<a href="#qmr">random QMR-like model</a>: +<p> +<img src="Figures/qmr.rnd.jpg"> +<p> +If you install the excellent <a +href="http://www.research.att.com/sw/tools/graphviz">graphhviz</a>, an +open-source graph visualization package from AT&T, +you can create a much better visualization as follows +<pre> +graph_to_dot(bnet.dag) +</pre> +This works by converting the adjacency matrix to a file suitable +for input to graphviz (using the dot format), +then converting the output of graphviz to postscript, and displaying the results using +ghostview. +You can do each of these steps separately for more control, as shown +below. +<pre> +graph_to_dot(bnet.dag, 'filename', 'foo.dot'); +dot -Tps foo.dot -o foo.ps +ghostview foo.ps & +</pre> + +<h2><a name = "constraint">Constraint-based methods</h2> + +The IC algorithm (Pearl and Verma, 1991), +and the faster, but otherwise equivalent, PC algorithm (Spirtes, Glymour, and Scheines 1993), +computes many conditional independence tests, +and combines these constraints into a +PDAG to represent the whole +<a href="#markov_equiv">Markov equivalence class</a>. +<p> +IC*/FCI extend IC/PC to handle latent variables: see <a href="#ic_star">below</a>. +(IC stands for inductive causation; PC stands for Peter and Clark, +the first names of Spirtes and Glymour; FCI stands for fast causal +inference. +What we, following Pearl (2000), call IC* was called +IC in the original Pearl and Verma paper.) +For details, see +<ul> +<li> +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Causation, +Prediction, and Search</a>, Spirtes, Glymour and +Scheines (SGS), 2001 (2nd edition), MIT Press. +<li> +<a href="http://bayes.cs.ucla.edu/BOOK-2K/index.html">Causality: Models, Reasoning and Inference</a>, J. Pearl, +2000, Cambridge University Press. +</ul> + +<p> + +The PC algorithm takes as arguments a function f, the number of nodes N, +the maximum fan in K, and additional arguments A which are passed to f. +The function f(X,Y,S,A) returns 1 if X is conditionally independent of Y given S, and 0 +otherwise. +For example, suppose we cheat by +passing in a CI "oracle" which has access to the true DAG; the oracle +tests for d-separation in this DAG, i.e., +f(X,Y,S) calls dsep(X,Y,S,dag). We can to this as follows. +<pre> +pdag = learn_struct_pdag_pc('dsep', N, max_fan_in, dag); +</pre> +pdag(i,j) = -1 if there is definitely an i->j arc, +and pdag(i,j) = 1 if there is either an i->j or and i<-j arc. +<p> +Applied to the sprinkler network, this returns +<pre> +pdag = + 0 1 1 0 + 1 0 0 -1 + 1 0 0 -1 + 0 0 0 0 +</pre> +So as expected, we see that the V-structure at the W node is uniquely identified, +but the other arcs have ambiguous orientation. +<p> +We now give an example from p141 (1st edn) / p103 (2nd end) of the SGS +book. +This example concerns the female orgasm. +We are given a correlation matrix C between 7 measured factors (such +as subjective experiences of coital and masturbatory experiences), +derived from 281 samples, and want to learn a causal model of the +data. We will not discuss the merits of this type of work here, but +merely show how to reproduce the results in the SGS book. +Their program, +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Tetrad</a>, +makes use of the Fisher Z-test for conditional +independence, so we do the same: +<pre> +max_fan_in = 4; +nsamples = 281; +alpha = 0.05; +pdag = learn_struct_pdag_pc('cond_indep_fisher_z', n, max_fan_in, C, nsamples, alpha); +</pre> +In this case, the CI test is +<pre> +f(X,Y,S) = cond_indep_fisher_z(X,Y,S, C,nsamples,alpha) +</pre> +The results match those of Fig 12a of SGS apart from two edge +differences; presumably this is due to rounding error (although it +could be a bug, either in BNT or in Tetrad). +This example can be found in the file BNT/examples/static/pc2.m. + +<p> + +The IC* algorithm (Pearl and Verma, 1991), +and the faster FCI algorithm (Spirtes, Glymour, and Scheines 1993), +are like the IC/PC algorithm, except that they can detect the presence +of latent variables. +See the file <tt>learn_struct_pdag_ic_star</tt> written by Tamar +Kushnir. The output is a matrix P, defined as follows +(see Pearl (2000), p52 for details): +<pre> +% P(i,j) = -1 if there is either a latent variable L such that i <-L->j OR there is a directed edge from i->j. +% P(i,j) = -2 if there is a marked directed i-*>j edge. +% P(i,j) = P(j,i) = 1 if there is and undirected edge i--j +% P(i,j) = P(j,i) = 2 if there is a latent variable L such that i<-L->j. +</pre> + + +<h2><a name="phl">Philippe Leray's structure learning package</h2> + +Philippe Leray has written a +<a href="http://bnt.insa-rouen.fr/ajouts.html"> +structure learning package</a> that uses BNT. + +It currently (Juen 2003) has the following features: +<ul> +<li>PC with Chi2 statistical test +<li> MWST : Maximum weighted Spanning Tree +<li> Hill Climbing +<li> Greedy Search +<li> Structural EM +<li> hist_ic : optimal Histogram based on IC information criterion +<li> cpdag_to_dag +<li> dag_to_cpdag +<li> ... +</ul> + + +</a> + + +<!-- +<h2><a name="read_learning">Further reading on learning</h2> + +I recommend the following tutorials for more details on learning. +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/intel.ps.gz">My short +tutorial</a> on graphical models, which contains an overview of learning. + +<li> +<A HREF="ftp://ftp.research.microsoft.com/pub/tr/TR-95-06.PS"> +A tutorial on learning with Bayesian networks</a>, D. Heckerman, +Microsoft Research Tech Report, 1995. + +<li> <A HREF="http://www-cad.eecs.berkeley.edu/~wray/Mirror/lwgmja"> +Operations for Learning with Graphical Models</a>, +W. L. Buntine, JAIR'94, 159--225. +</ul> +<p> +--> + + + + + +<h1><a name="engines">Inference engines</h1> + +Up until now, we have used the junction tree algorithm for inference. +However, sometimes this is too slow, or not even applicable. +In general, there are many inference algorithms each of which make +different tradeoffs between speed, accuracy, complexity and +generality. Furthermore, there might be many implementations of the +same algorithm; for instance, a general purpose, readable version, +and a highly-optimized, specialized one. +To cope with this variety, we treat each inference algorithm as an +object, which we call an inference engine. + +<p> +An inference engine is an object that contains a bnet and supports the +'enter_evidence' and 'marginal_nodes' methods. The engine constructor +takes the bnet as argument and may do some model-specific processing. +When 'enter_evidence' is called, the engine may do some +evidence-specific processing. Finally, when 'marginal_nodes' is +called, the engine may do some query-specific processing. + +<p> +The amount of work done when each stage is specified -- structure, +parameters, evidence, and query -- depends on the engine. The cost of +work done early in this sequence can be amortized. On the other hand, +one can make better optimizations if one waits until later in the +sequence. +For example, the parameters might imply +conditional indpendencies that are not evident in the graph structure, +but can nevertheless be exploited; the evidence indicates which nodes +are observed and hence can effectively be disconnected from the +graph; and the query might indicate that large parts of the network +are d-separated from the query nodes. (Since it is not the actual +<em>values</em> of the evidence that matters, just which nodes are observed, +many engines allow you to specify which nodes will be observed when they are constructed, +i.e., before calling 'enter_evidence'. Some engines can still cope if +the actual pattern of evidence is different, e.g., if there is missing +data.) +<p> + +Although being maximally lazy (i.e., only doing work when a query is +issued) may seem desirable, +this is not always the most efficient. +For example, +when learning using EM, we need to call marginal_nodes N times, where N is the +number of nodes. <a href="varelim">Variable elimination</a> would end +up repeating a lot of work +each time marginal_nodes is called, making it inefficient for +learning. The junction tree algorithm, by contrast, uses dynamic +programming to avoid this redundant computation --- it calculates all +marginals in two passes during 'enter_evidence', so calling +'marginal_nodes' takes constant time. +<p> +We will discuss some of the inference algorithms implemented in BNT +below, and finish with a <a href="#engine_summary">summary</a> of all +of them. + + + + + + + +<h2><a name="varelim">Variable elimination</h2> + +The variable elimination algorithm, also known as bucket elimination +or peeling, is one of the simplest inference algorithms. +The basic idea is to "push sums inside of products"; this is explained +in more detail +<a +href="http://HTTP.CS.Berkeley.EDU/~murphyk/Bayes/bayes.html#infer">here</a>. +<p> +The principle of distributing sums over products can be generalized +greatly to apply to any commutative semiring. +This forms the basis of many common algorithms, such as Viterbi +decoding and the Fast Fourier Transform. For details, see + +<ul> +<li> R. McEliece and S. M. Aji, 2000. +<!--<a href="http://www.systems.caltech.edu/EE/Faculty/rjm/papers/GDL.ps">--> +<a href="GDL.pdf"> +The Generalized Distributive Law</a>, +IEEE Trans. Inform. Theory, vol. 46, no. 2 (March 2000), +pp. 325--343. + + +<li> +F. R. Kschischang, B. J. Frey and H.-A. Loeliger, 2001. +<a href="http://www.cs.toronto.edu/~frey/papers/fgspa.abs.html"> +Factor graphs and the sum-product algorithm</a> +IEEE Transactions on Information Theory, February, 2001. + +</ul> + +<p> +Choosing an order in which to sum out the variables so as to minimize +computational cost is known to be NP-hard. +The implementation of this algorithm in +<tt>var_elim_inf_engine</tt> makes no attempt to optimize this +ordering (in contrast, say, to <tt>jtree_inf_engine</tt>, which uses a +greedy search procedure to find a good ordering). +<p> +Note: unlike most algorithms, var_elim does all its computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + + + +<h2><a name="global">Global inference methods</h2> + +The simplest inference algorithm of all is to explicitely construct +the joint distribution over all the nodes, and then to marginalize it. +This is implemented in <tt>global_joint_inf_engine</tt>. +Since the size of the joint is exponential in the +number of discrete (hidden) nodes, this is not a very practical algorithm. +It is included merely for pedagogical and debugging purposes. +<p> +Three specialized versions of this algorithm have also been implemented, +corresponding to the cases where all the nodes are discrete (D), all +are Gaussian (G), and some are discrete and some Gaussian (CG). +They are called <tt>enumerative_inf_engine</tt>, +<tt>gaussian_inf_engine</tt>, +and <tt>cond_gauss_inf_engine</tt> respectively. +<p> +Note: unlike most algorithms, these global inference algorithms do all their computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + +<h2><a name="quickscore">Quickscore</h2> + +The junction tree algorithm is quite slow on the <a href="#qmr">QMR</a> network, +since the cliques are so big. +One simple trick we can use is to notice that hidden leaves do not +affect the posteriors on the roots, and hence do not need to be +included in the network. +A second trick is to notice that the negative findings can be +"absorbed" into the prior: +see the file +BNT/examples/static/mk_minimal_qmr_bnet for details. +<p> + +A much more significant speedup is obtained by exploiting special +properties of the noisy-or node, as done by the quickscore +algorithm. For details, see +<ul> +<li> Heckerman, "A tractable inference algorithm for diagnosing multiple diseases", UAI 89. +<li> Rish and Dechter, "On the impact of causal independence", UCI +tech report, 1998. +</ul> + +This has been implemented in BNT as a special-purpose inference +engine, which can be created and used as follows: +<pre> +engine = quickscore_inf_engine(inhibit, leak, prior); +engine = enter_evidence(engine, pos, neg); +m = marginal_nodes(engine, i); +</pre> + + +<h2><a name="belprop">Belief propagation</h2> + +Even using quickscore, exact inference takes time that is exponential +in the number of positive findings. +Hence for large networks we need to resort to approximate inference techniques. +See for example +<ul> +<li> T. Jaakkola and M. Jordan, "Variational probabilistic inference and the +QMR-DT network", JAIR 10, 1999. + +<li> K. Murphy, Y. Weiss and M. Jordan, "Loopy belief propagation for approximate inference: an empirical study", + UAI 99. +</ul> +The latter approximation +entails applying Pearl's belief propagation algorithm to a model even +if it has loops (hence the name loopy belief propagation). +Pearl's algorithm, implemented as <tt>pearl_inf_engine</tt>, gives +exact results when applied to singly-connected graphs +(a.k.a. polytrees, since +the underlying undirected topology is a tree, but a node may have +multiple parents). +To apply this algorithm to a graph with loops, +use <tt>pearl_inf_engine</tt>. +This can use a centralized or distributed message passing protocol. +You can use it as in the following example. +<pre> +engine = pearl_inf_engine(bnet, 'max_iter', 30); +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, i); +</pre> +We found that this algorithm often converges, and when it does, often +is very accurate, but it depends on the precise setting of the +parameter values of the network. +(See the file BNT/examples/static/qmr1 to repeat the experiment for yourself.) +Understanding when and why belief propagation converges/ works +is a topic of ongoing research. +<p> +<tt>pearl_inf_engine</tt> can exploit special structure in noisy-or +and gmux nodes to compute messages efficiently. +<p> +<tt>belprop_inf_engine</tt> is like pearl, but uses potentials to +represent messages. Hence this is slower. +<p> +<tt>belprop_fg_inf_engine</tt> is like belprop, +but is designed for factor graphs. + + + +<h2><a name="sampling">Sampling</h2> + +BNT now (Mar '02) has two sampling (Monte Carlo) inference algorithms: +<ul> +<li> <tt>likelihood_weighting_inf_engine</tt> which does importance +sampling and can handle any node type. +<li> <tt>gibbs_sampling_inf_engine</tt>, written by Bhaskara Marthi. +Currently this can only handle tabular CPDs. +For a much faster and more powerful Gibbs sampling program, see +<a href="http://www.mrc-bsu.cam.ac.uk/bugs">BUGS</a>. +</ul> +Note: To generate samples from a network (which is not the same as inference!), +use <tt>sample_bnet</tt>. + + + +<h2><a name="engine_summary">Summary of inference engines</h2> + + +The inference engines differ in many ways. Here are +some of the major "axes": +<ul> +<li> Works for all topologies or makes restrictions? +<li> Works for all node types or makes restrictions? +<li> Exact or approximate inference? +</ul> + +<p> +In terms of topology, most engines handle any kind of DAG. +<tt>belprop_fg</tt> does approximate inference on factor graphs (FG), which +can be used to represent directed, undirected, and mixed (chain) +graphs. +(In the future, we plan to support exact inference on chain graphs.) +<tt>quickscore</tt> only works on QMR-like models. +<p> +In terms of node types: algorithms that use potentials can handle +discrete (D), Gaussian (G) or conditional Gaussian (CG) models. +Sampling algorithms can essentially handle any kind of node (distribution). +Other algorithms make more restrictive assumptions in exchange for +speed. +<p> +Finally, most algorithms are designed to give the exact answer. +The belief propagation algorithms are exact if applied to trees, and +in some other cases. +Sampling is considered approximate, even though, in the limit of an +infinite number of samples, it gives the exact answer. + +<p> + +Here is a summary of the properties +of all the engines in BNT which work on static networks. +<p> +<table> +<table border units = pixels><tr> +<td align=left width=0>Name +<td align=left width=0>Exact? +<td align=left width=0>Node type? +<td align=left width=0>topology +<tr> +<tr> +<td align=left> belprop +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> belprop_fg +<td align=left> approx +<td align=left> D +<td align=left> factor graph +<tr> +<td align=left> cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> enumerative +<td align=left> exact +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> gaussian +<td align=left> exact +<td align=left> G +<td align=left> DAG +<tr> +<td align=left> gibbs +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> global_joint +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +<tr> +<td align=left> jtree +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +b<tr> +<td align=left> likelihood_weighting +<td align=left> approx +<td align=left> any +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> approx +<td align=left> D,G +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> exact +<td align=left> D,G +<td align=left> polytree +<tr> +<td align=left> quickscore +<td align=left> exact +<td align=left> noisy-or +<td align=left> QMR +<tr> +<td align=left> stab_cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> var_elim +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +</table> + + + +<h1><a name="influence">Influence diagrams/ decision making</h1> + +BNT implements an exact algorithm for solving LIMIDs (limited memory +influence diagrams), described in +<ul> +<li> S. L. Lauritzen and D. Nilsson. +<a href="http://www.math.auc.dk/~steffen/papers/limids.pdf"> +Representing and solving decision problems with limited +information</a> +Management Science, 47, 1238 - 1251. September 2001. +</ul> +LIMIDs explicitely show all information arcs, rather than implicitely +assuming no forgetting. This allows them to model forgetful +controllers. +<p> +See the examples in <tt>BNT/examples/limids</tt> for details. + + + + +<h1>DBNs, HMMs, Kalman filters and all that</h1> + +Click <a href="usage_dbn.html">here</a> for documentation about how to +use BNT for dynamical systems and sequence data. + + +</BODY> diff --git a/sourcecodes/bnt-master/docs/usage_cropped.html b/sourcecodes/bnt-master/docs/usage_cropped.html new file mode 100644 index 00000000..11bb4e12 --- /dev/null +++ b/sourcecodes/bnt-master/docs/usage_cropped.html @@ -0,0 +1,3232 @@ +<HEAD> +<TITLE>How to use the Bayes Net Toolbox</TITLE> +</HEAD> + +<BODY BGCOLOR="#FFFFFF"> +<!-- white background is better for the pictures and equations --> + +<h1>How to use the Bayes Net Toolbox</h1> + +This documentation was last updated on 13 November 2002. +<br> +Click <a href="changelog.html">here</a> for a list of changes made to +BNT. +<br> +Click +<a href="http://bnt.insa-rouen.fr/">here</a> +for a French version of this documentation (which might not +be up-to-date). + + +<p> + +<ul> +<li> <a href="#install">Installation</a> +<ul> +<li> <a href="#installM">Installing the Matlab code</a> +<li> <a href="#installC">Installing the C code</a> +<li> <a href="../matlab_tips.html">Useful Matlab tips</a>. +</ul> + +<li> <a href="#basics">Creating your first Bayes net</a> + <ul> + <li> <a href="#basics">Creating a model by hand</a> + <li> <a href="#file">Loading a model from a file</a> + <li> <a href="http://bnt.insa-rouen.fr/ajouts.html">Creating a model using a GUI</a> + </ul> + +<li> <a href="#inference">Inference</a> + <ul> + <li> <a href="#marginal">Computing marginal distributions</a> + <li> <a href="#joint">Computing joint distributions</a> + <li> <a href="#soft">Soft/virtual evidence</a> + <li> <a href="#mpe">Most probable explanation</a> + </ul> + +<li> <a href="#cpd">Conditional Probability Distributions</a> + <ul> + <li> <a href="#tabular">Tabular (multinomial) nodes</a> + <li> <a href="#noisyor">Noisy-or nodes</a> + <li> <a href="#deterministic">Other (noisy) deterministic nodes</a> + <li> <a href="#softmax">Softmax (multinomial logit) nodes</a> + <li> <a href="#mlp">Neural network nodes</a> + <li> <a href="#root">Root nodes</a> + <li> <a href="#gaussian">Gaussian nodes</a> + <li> <a href="#glm">Generalized linear model nodes</a> + <li> <a href="#dtree">Classification/regression tree nodes</a> + <li> <a href="#nongauss">Other continuous distributions</a> + <li> <a href="#cpd_summary">Summary of CPD types</a> + </ul> + +<li> <a href="#examples">Example models</a> + <ul> + <li> <a + href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt"> +Gaussian mixture models</a> + <li> <a href="#pca">PCA, ICA, and all that</a> + <li> <a href="#mixep">Mixtures of experts</a> + <li> <a href="#hme">Hierarchical mixtures of experts</a> + <li> <a href="#qmr">QMR</a> + <li> <a href="#cg_model">Conditional Gaussian models</a> + <li> <a href="#hybrid">Other hybrid models</a> + </ul> + +<li> <a href="#param_learning">Parameter learning</a> + <ul> + <li> <a href="#load_data">Loading data from a file</a> + <li> <a href="#mle_complete">Maximum likelihood parameter estimation from complete data</a> + <li> <a href="#prior">Parameter priors</a> + <li> <a href="#bayes_learn">(Sequential) Bayesian parameter updating from complete data</a> + <li> <a href="#em">Maximum likelihood parameter estimation with missing values (EM)</a> + <li> <a href="#tying">Parameter tying</a> + </ul> + +<li> <a href="#structure_learning">Structure learning</a> + <ul> + <li> <a href="#enumerate">Exhaustive search</a> + <li> <a href="#K2">K2</a> + <li> <a href="#hill_climb">Hill-climbing</a> + <li> <a href="#mcmc">MCMC</a> + <li> <a href="#active">Active learning</a> + <li> <a href="#struct_em">Structural EM</a> + <li> <a href="#graphdraw">Visualizing the learned graph structure</a> + <li> <a href="#constraint">Constraint-based methods</a> + </ul> + + +<li> <a href="#engines">Inference engines</a> + <ul> + <li> <a href="#jtree">Junction tree</a> + <li> <a href="#varelim">Variable elimination</a> + <li> <a href="#global">Global inference methods</a> + <li> <a href="#quickscore">Quickscore</a> + <li> <a href="#belprop">Belief propagation</a> + <li> <a href="#sampling">Sampling (Monte Carlo)</a> + <li> <a href="#engine_summary">Summary of inference engines</a> + </ul> + + +<li> <a href="#influence">Influence diagrams/ decision making</a> + + +<li> <a href="usage_dbn.html">DBNs, HMMs, Kalman filters and all that</a> +</ul> + +</ul> + + + + +<h1><a name="install">Installation</h1> + +<h2><a name="installM">Installing the Matlab code</h2> + +<ul> +<li> <a href="bnt_download.html">Download</a> the BNT.zip file. + +<p> +<li> Unpack the file. In Unix, type +<!--"tar xvf BNT.tar".--> +"unzip FullBNT.zip". +In Windows, use +a program like <a href="http://www.winzip.com">Winzip</a>. This will +create a directory called FullBNT, which contains BNT and other libraries. + +<p> +<li> Read the file <tt>BNT/README</tt> to make sure the date +matches the one on the top of <a href=bnt.html>the BNT home page</a>. +If not, you may need to press 'refresh' on your browser, and download +again, to get the most recent version. + +<p> +<li> <b>Edit the file "BNT/add_BNT_to_path.m"</b> so it contains the correct +pathname. +For example, in Windows, +I download FullBNT.zip into C:\kpmurphy\matlab, and +then comment out the second line (with the % character), and uncomment +the third line, which reads +<pre> +BNT_HOME = 'C:\kpmurphy\matlab\FullBNT'; +</pre> + +<p> +<li> Start up Matlab. + +<p> +<li> Type "ver" at the Matlab prompt (">>"). +<b>You need Matlab version 5.2 or newer to run BNT</b>. +(Versions 5.0 and 5.1 have a memory leak which seems to sometimes +crash BNT.) + +<p> +<li> Move to the BNT directory. +For example, in Windows, I type +<pre> +>> cd C:\kpmurphy\matlab\FullBNT\BNT +</pre> + +<p> +<li> Type "add_BNT_to_path". +This executes the command +<tt>addpath(genpath(BNT_HOME))</tt>, +which adds all directories below FullBNT to the matlab path. + +<p> +<li> Type "test_BNT". +<b>If all goes well, this will just produce a bunch of numbers and pictures.</b> +It may produce some +warning messages (which you can ignore), but should not produce any error messages. +(The warnings should only be of the form +"Warning: Maximum number of iterations has been exceeded", and are +produced by Netlab.) + +<p> +<li> <a href="http://groups.yahoo.com/group/BayesNetToolbox/join"> +Join the BNT email list</a> + +</ul> + + +If you are new to Matlab, you might like to check out +<a href="matlab_tips.html">some useful Matlab tips</a>. +For instance, this explains how to create a startup file, which can be +used to set your path variable automatically, so you can avoid having +to type the above commands every time. + + + + +<h2><a name="installC">Installing the C code</h2> + +Some BNT functions also have C implementations. +<b>It is not necessary to install the C code</b>, but it can result in a speedup +of a factor of 2-5 of certain simple functions. +To install all the C code, +edit installC_BNT.m so it contains the right path, +then type <tt>installC_BNT</tt>. +To uninstall all the C code, +edit uninstallC_BNT.m so it contains the right path, +then type <tt>uninstallC_BNT</tt>. +For an up-to-date list of the files which have C implementations, see +BNT/installC_BNT.m. + +<p> +mex is a script that lets you call C code from Matlab - it does not compile matlab to +C (see mcc below). +If your C/C++ compiler is set up correctly, mex should work out of +the box. (Matlab 6 now ships with its own minimal C compiler.) +If not, you might need to type +<p> +<tt> mex -setup</tt> +<p> +before calling installC. +<p> +To make mex call gcc on Windows, +you must install <a +href="http://www.mrc-cbu.cam.ac.uk/Imaging/gnumex20.html">gnumex</a>. +You can use the <a href="http://www.mingw.org/">minimalist GNU for +Windows</a> version of gcc, or +the <a href="http://sources.redhat.com/cygwin/">cygwin</a> version. +<p> +In general, typing +'mex foo.c' from inside Matlab creates a file called +'foo.mexglx' or 'foo.dll' (the exact file +extension is system dependent - on Linux it is 'mexglx', on Windows it is '.dll'). +The resulting file will hide the original 'foo.m' (if it existed), i.e., +typing 'foo' at the prompt will call the compiled C version. +To reveal the original matlab version, just delete foo.mexglx (this is +what uninstallC does). +<p> +Sometimes it takes time for Matlab to realize that the file has +changed from matlab to C or vice versa; try typing 'clear all' or +restarting Matlab to refresh it. +To find out which version of a file you are running, type +'which foo'. +<p> +<a href="http://www.mathworks.com/products/compiler">mcc</a>, the +Matlab to C compiler, is a separate product, +and is quite different from mex. It does not yet support +objects/classes, which is why we can't compile all of BNT to C automatically. +Also, hand-written C code is usually much +better than the C code generated by mcc. + + +<p> +Acknowledgements: +Although I wrote some of the C code, most of +the C code (e.g., for jtree and dpot) was written by Wei Hu; +the triangulation C code was written by Ilya Shpitser. + + +<h1><a name="basics">Creating your first Bayes net</h1> + +To define a Bayes net, you must specify the graph structure and then +the parameters. We look at each in turn, using a simple example +(adapted from Russell and +Norvig, "Artificial Intelligence: a Modern Approach", Prentice Hall, +1995, p454). + + +<h2>Graph structure</h2> + + +Consider the following network. + +<p> +<center> +<IMG SRC="Figures/sprinkler.gif"> +</center> +<p> + +<P> +To specify this directed acyclic graph (dag), we create an adjacency matrix: +<PRE> +N = 4; +dag = zeros(N,N); +C = 1; S = 2; R = 3; W = 4; +dag(C,[R S]) = 1; +dag(R,W) = 1; +dag(S,W)=1; +</PRE> +<P> +We have numbered the nodes as follows: +Cloudy = 1, Sprinkler = 2, Rain = 3, WetGrass = 4. +<b>The nodes must always be numbered in topological order, i.e., +ancestors before descendants.</b> +For a more complicated graph, this is a little inconvenient: we will +see how to get around this <a href="usage_dbn.html#bat">below</a>. +<p> +<!-- +In Matlab 6, you can use logical arrays instead of double arrays, +which are 4 times smaller: +<pre> +dag = false(N,N); +dag(C,[R S]) = true; +... +</pre> +-- +<p> +A preliminary attempt to make a <b>GUI</b> +has been writte by Philippe LeRay and can be downloaded +from <a href="http://bnt.insa-rouen.fr/ajouts.html">here</a>. +<p> +You can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>. + +<h2>Creating the Bayes net shell</h2> + +In addition to specifying the graph structure, +we must specify the size and type of each node. +If a node is discrete, its size is the +number of possible values +each node can take on; if a node is continuous, +it can be a vector, and its size is the length of this vector. +In this case, we will assume all nodes are discrete and binary. +<PRE> +discrete_nodes = 1:N; +node_sizes = 2*ones(1,N); +</pre> +If the nodes were not binary, you could type e.g., +<pre> +node_sizes = [4 2 3 5]; +</pre> +meaning that Cloudy has 4 possible values, +Sprinkler has 2 possible values, etc. +Note that these are cardinal values, not ordinal, i.e., +they are not ordered in any way, like 'low', 'medium', 'high'. +<p> +We are now ready to make the Bayes net: +<pre> +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes); +</PRE> +By default, all nodes are assumed to be discrete, so we can also just +write +<pre> +bnet = mk_bnet(dag, node_sizes); +</PRE> +You may also specify which nodes will be observed. +If you don't know, or if this not fixed in advance, +just use the empty list (the default). +<pre> +onodes = []; +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes, 'observed', onodes); +</PRE> +Note that optional arguments are specified using a name/value syntax. +This is common for many BNT functions. +In general, to find out more about a function (e.g., which optional +arguments it takes), please see its +documentation string by typing +<pre> +help mk_bnet +</pre> +See also other <a href="matlab_tips.html">useful Matlab tips</a>. +<p> +It is possible to associate names with nodes, as follows: +<pre> +bnet = mk_bnet(dag, node_sizes, 'names', {'cloudy','S','R','W'}, 'discrete', 1:4); +</pre> +You can then refer to a node by its name: +<pre> +C = bnet.names{'cloudy'}; % bnet.names is an associative array +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +</pre> + + +<h2><a name="cpt">Parameters</h2> + +A model consists of the graph structure and the parameters. +The parameters are represented by CPD objects (CPD = Conditional +Probability Distribution), which define the probability distribution +of a node given its parents. +(We will use the terms "node" and "random variable" interchangeably.) +The simplest kind of CPD is a table (multi-dimensional array), which +is suitable when all the nodes are discrete-valued. Note that the discrete +values are not assumed to be ordered in any way; that is, they +represent categorical quantities, like male and female, rather than +ordinal quantities, like low, medium and high. +(We will discuss CPDs in more detail <a href="#cpd">below</a>.) +<p> +Tabular CPDs, also called CPTs (conditional probability tables), +are stored as multidimensional arrays, where the dimensions +are arranged in the same order as the nodes, e.g., the CPT for node 4 +(WetGrass) is indexed by Sprinkler (2), Rain (3) and then WetGrass (4) itself. +Hence the child is always the last dimension. +If a node has no parents, its CPT is a column vector representing its +prior. +Note that in Matlab (unlike C), arrays are indexed +from 1, and are layed out in memory such that the first index toggles +fastest, e.g., the CPT for node 4 (WetGrass) is as follows +<P> +<P><IMG ALIGN=BOTTOM SRC="Figures/CPTgrass.gif"><P> +<P> +where we have used the convention that false==1, true==2. +We can create this CPT in Matlab as follows +<PRE> +CPT = zeros(2,2,2); +CPT(1,1,1) = 1.0; +CPT(2,1,1) = 0.1; +... +</PRE> +Here is an easier way: +<PRE> +CPT = reshape([1 0.1 0.1 0.01 0 0.9 0.9 0.99], [2 2 2]); +</PRE> +In fact, we don't need to reshape the array, since the CPD constructor +will do that for us. So we can just write +<pre> +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</pre> +The other nodes are created similarly (using the old syntax for +optional parameters) +<PRE> +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +bnet.CPD{R} = tabular_CPD(bnet, R, [0.8 0.2 0.2 0.8]); +bnet.CPD{S} = tabular_CPD(bnet, S, [0.5 0.9 0.5 0.1]); +bnet.CPD{W} = tabular_CPD(bnet, W, [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</PRE> + + +<h2><a name="rnd_cpt">Random Parameters</h2> + +If we do not specify the CPT, random parameters will be +created, i.e., each "row" of the CPT will be drawn from the uniform distribution. +To ensure repeatable results, use +<pre> +rand('state', seed); +randn('state', seed); +</pre> +To control the degree of randomness (entropy), +you can sample each row of the CPT from a Dirichlet(p,p,...) distribution. +If p << 1, this encourages "deterministic" CPTs (one entry near 1, the rest near 0). +If p = 1, each entry is drawn from U[0,1]. +If p >> 1, the entries will all be near 1/k, where k is the arity of +this node, i.e., each row will be nearly uniform. +You can do this as follows, assuming this node +is number i, and ns is the node_sizes. +<pre> +k = ns(i); +ps = parents(dag, i); +psz = prod(ns(ps)); +CPT = sample_dirichlet(p*ones(1,k), psz); +bnet.CPD{i} = tabular_CPD(bnet, i, 'CPT', CPT); +</pre> + + +<h2><a name="file">Loading a network from a file</h2> + +If you already have a Bayes net represented in the XML-based +<a href="http://www.cs.cmu.edu/afs/cs/user/fgcozman/www/Research/InterchangeFormat/"> +Bayes Net Interchange Format (BNIF)</a> (e.g., downloaded from the +<a +href="http://www.cs.huji.ac.il/labs/compbio/Repository"> +Bayes Net repository</a>), +you can convert it to BNT format using +the +<a href="http://www.digitas.harvard.edu/~ken/bif2bnt/">BIF-BNT Java +program</a> written by Ken Shan. +(This is not necessarily up-to-date.) + + +<h2>Creating a model using a GUI</h2> + +Click <a href="http://bnt.insa-rouen.fr/ajouts.html">here</a>. + + + +<h1><a name="inference">Inference</h1> + +Having created the BN, we can now use it for inference. +There are many different algorithms for doing inference in Bayes nets, +that make different tradeoffs between speed, +complexity, generality, and accuracy. +BNT therefore offers a variety of different inference +"engines". We will discuss these +in more detail <a href="#engines">below</a>. +For now, we will use the junction tree +engine, which is the mother of all exact inference algorithms. +This can be created as follows. +<pre> +engine = jtree_inf_engine(bnet); +</pre> +The other engines have similar constructors, but might take +additional, algorithm-specific parameters. +All engines are used in the same way, once they have been created. +We illustrate this in the following sections. + + +<h2><a name="marginal">Computing marginal distributions</h2> + +Suppose we want to compute the probability that the sprinker was on +given that the grass is wet. +The evidence consists of the fact that W=2. All the other nodes +are hidden (unobserved). We can specify this as follows. +<pre> +evidence = cell(1,N); +evidence{W} = 2; +</pre> +We use a 1D cell array instead of a vector to +cope with the fact that nodes can be vectors of different lengths. +In addition, the value [] can be used +to denote 'no evidence', instead of having to specify the observation +pattern as a separate argument. +(Click <a href="cellarray.html">here</a> for a quick tutorial on cell +arrays in matlab.) +<p> +We are now ready to add the evidence to the engine. +<pre> +[engine, loglik] = enter_evidence(engine, evidence); +</pre> +The behavior of this function is algorithm-specific, and is discussed +in more detail <a href="#engines">below</a>. +In the case of the jtree engine, +enter_evidence implements a two-pass message-passing scheme. +The first return argument contains the modified engine, which +incorporates the evidence. The second return argument contains the +log-likelihood of the evidence. (Not all engines are capable of +computing the log-likelihood.) +<p> +Finally, we can compute p=P(S=2|W=2) as follows. +<PRE> +marg = marginal_nodes(engine, S); +marg.T +ans = + 0.57024 + 0.42976 +p = marg.T(2); +</PRE> +We see that p = 0.4298. +<p> +Now let us add the evidence that it was raining, and see what +difference it makes. +<PRE> +evidence{R} = 2; +[engine, loglik] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, S); +p = marg.T(2); +</PRE> +We find that p = P(S=2|W=2,R=2) = 0.1945, +which is lower than +before, because the rain can ``explain away'' the +fact that the grass is wet. +<p> +You can plot a marginal distribution over a discrete variable +as a barchart using the built 'bar' function: +<pre> +bar(marg.T) +</pre> +This is what it looks like + +<p> +<center> +<IMG SRC="Figures/sprinkler_bar.gif"> +</center> +<p> + +<h2><a name="observed">Observed nodes</h2> + +What happens if we ask for the marginal on an observed node, e.g. P(W|W=2)? +An observed discrete node effectively only has 1 value (the observed + one) --- all other values would result in 0 probability. +For efficiency, BNT treats observed (discrete) nodes as if they were + set to 1, as we see below: +<pre> +evidence = cell(1,N); +evidence{W} = 2; +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, W); +m.T +ans = + 1 +</pre> +This can get a little confusing, since we assigned W=2. +So we can ask BNT to add the evidence back in by passing in an optional argument: +<pre> +m = marginal_nodes(engine, W, 1); +m.T +ans = + 0 + 1 +</pre> +This shows that P(W=1|W=2) = 0 and P(W=2|W=2) = 1. + + + +<h2><a name="joint">Computing joint distributions</h2> + +We can compute the joint probability on a set of nodes as in the +following example. +<pre> +evidence = cell(1,N); +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]); +</pre> +m is a structure. The 'T' field is a multi-dimensional array (in +this case, 3-dimensional) that contains the joint probability +distribution on the specified nodes. +<pre> +>> m.T +ans(:,:,1) = + 0.2900 0.0410 + 0.0210 0.0009 +ans(:,:,2) = + 0 0.3690 + 0.1890 0.0891 +</pre> +We see that P(S=1,R=1,W=2) = 0, since it is impossible for the grass +to be wet if both the rain and sprinkler are off. +<p> +Let us now add some evidence to R. +<pre> +evidence{R} = 2; +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]) +m = + domain: [2 3 4] + T: [2x1x2 double] +>> m.T +m.T +ans(:,:,1) = + 0.0820 + 0.0018 +ans(:,:,2) = + 0.7380 + 0.1782 +</pre> +The joint T(i,j,k) = P(S=i,R=j,W=k|evidence) +should have T(i,1,k) = 0 for all i,k, since R=1 is incompatible +with the evidence that R=2. +Instead of creating large tables with many 0s, BNT sets the effective +size of observed (discrete) nodes to 1, as explained above. +This is why m.T has size 2x1x2. +To get a 2x2x2 table, type +<pre> +m = marginal_nodes(engine, [S R W], 1) +m = + domain: [2 3 4] + T: [2x2x2 double] +>> m.T +m.T +ans(:,:,1) = + 0 0.082 + 0 0.0018 +ans(:,:,2) = + 0 0.738 + 0 0.1782 +</pre> + +<p> +Note: It is not always possible to compute the joint on arbitrary +sets of nodes: it depends on which inference engine you use, as discussed +in more detail <a href="#engines">below</a>. + + +<h2><a name="soft">Soft/virtual evidence</h2> + +Sometimes a node is not observed, but we have some distribution over +its possible values; this is often called "soft" or "virtual" +evidence. +One can use this as follows +<pre> +[engine, loglik] = enter_evidence(engine, evidence, 'soft', soft_evidence); +</pre> +where soft_evidence{i} is either [] (if node i has no soft evidence) +or is a vector representing the probability distribution over i's +possible values. +For example, if we don't know i's exact value, but we know its +likelihood ratio is 60/40, we can write evidence{i} = [] and +soft_evidence{i} = [0.6 0.4]. +<p> +Currently only jtree_inf_engine supports this option. +It assumes that all hidden nodes, and all nodes for +which we have soft evidence, are discrete. +For a longer example, see BNT/examples/static/softev1.m. + + +<h2><a name="mpe">Most probable explanation</h2> + +To compute the most probable explanation (MPE) of the evidence (i.e., +the most probable assignment, or a mode of the joint), use +<pre> +[mpe, ll] = calc_mpe(engine, evidence); +</pre> +mpe{i} is the most likely value of node i. +This calls enter_evidence with the 'maximize' flag set to 1, which +causes the engine to do max-product instead of sum-product. +The resulting max-marginals are then thresholded. +If there is more than one maximum probability assignment, we must take + care to break ties in a consistent manner (thresholding the + max-marginals may give the wrong result). To force this behavior, + type +<pre> +[mpe, ll] = calc_mpe(engine, evidence, 1); +</pre> +Note that computing the MPE is someties called abductive reasoning. + +<p> +You can also use <tt>calc_mpe_bucket</tt> written by Ron Zohar, +that does a forwards max-product pass, and then a backwards traceback +pass, which is how Viterbi is traditionally implemented. + + + +<h1><a name="cpd">Conditional Probability Distributions</h1> + +A Conditional Probability Distributions (CPD) +defines P(X(i) | X(Pa(i))), where X(i) is the i'th node, and X(Pa(i)) +are the parents of node i. There are many ways to represent this +distribution, which depend in part on whether X(i) and X(Pa(i)) are +discrete, continuous, or a combination. +We will discuss various representations below. + + +<h2><a name="tabular">Tabular nodes</h2> + +If the CPD is represented as a table (i.e., if it is a multinomial +distribution), it has a number of parameters that is exponential in +the number of parents. See the example <a href="#cpt">above</a>. + + +<h2><a name="noisyor">Noisy-or nodes</h2> + +A noisy-OR node is like a regular logical OR gate except that +sometimes the effects of parents that are on get inhibited. +Let the prob. that parent i gets inhibited be q(i). +Then a node, C, with 2 parents, A and B, has the following CPD, where +we use F and T to represent off and on (1 and 2 in BNT). +<pre> +A B P(C=off) P(C=on) +--------------------------- +F F 1.0 0.0 +T F q(A) 1-q(A) +F T q(B) 1-q(B) +T T q(A)q(B) q-q(A)q(B) +</pre> +Thus we see that the causes get inhibited independently. +It is common to associate a "leak" node with a noisy-or CPD, which is +like a parent that is always on. This can account for all other unmodelled +causes which might turn the node on. +<p> +The noisy-or distribution is similar to the logistic distribution. +To see this, let the nodes, S(i), have values in {0,1}, and let q(i,j) +be the prob. that j inhibits i. Then +<pre> +Pr(S(i)=1 | parents(S(i))) = 1 - prod_{j} q(i,j)^S(j) +</pre> +Now define w(i,j) = -ln q(i,j) and rho(x) = 1-exp(-x). Then +<pre> +Pr(S(i)=1 | parents(S(i))) = rho(sum_j w(i,j) S(j)) +</pre> +For a sigmoid node, we have +<pre> +Pr(S(i)=1 | parents(S(i))) = sigma(-sum_j w(i,j) S(j)) +</pre> +where sigma(x) = 1/(1+exp(-x)). Hence they differ in the choice of +the activation function (although both are monotonically increasing). +In addition, in the case of a noisy-or, the weights are constrained to be +positive, since they derive from probabilities q(i,j). +In both cases, the number of parameters is <em>linear</em> in the +number of parents, unlike the case of a multinomial distribution, +where the number of parameters is exponential in the number of parents. +We will see an example of noisy-OR nodes <a href="#qmr">below</a>. + + +<h2><a name="deterministic">Other (noisy) deterministic nodes</h2> + +Deterministic CPDs for discrete random variables can be created using +the deterministic_CPD class. It is also possible to 'flip' the output +of the function with some probability, to simulate noise. +The boolean_CPD class is just a special case of a +deterministic CPD, where the parents and child are all binary. +<p> +Both of these classes are just "syntactic sugar" for the tabular_CPD +class. + + + +<h2><a name="softmax">Softmax nodes</h2> + +If we have a discrete node with a continuous parent, +we can define its CPD using a softmax function +(also known as the multinomial logit function). +This acts like a soft thresholding operator, and is defined as follows: +<pre> + exp(w(:,i)'*x + b(i)) +Pr(Q=i | X=x) = ----------------------------- + sum_j exp(w(:,j)'*x + b(j)) + +</pre> +The parameters of a softmax node, w(:,i) and b(i), i=1..|Q|, have the +following interpretation: w(:,i)-w(:,j) is the normal vector to the +decision boundary between classes i and j, +and b(i)-b(j) is its offset (bias). For example, suppose +X is a 2-vector, and Q is binary. Then +<pre> +w = [1 -1; + 0 0]; + +b = [0 0]; +</pre> +means class 1 are points in the 2D plane with positive x coordinate, +and class 2 are points in the 2D plane with negative x coordinate. +If w has large magnitude, the decision boundary is sharp, otherwise it +is soft. +In the special case that Q is binary (0/1), the softmax function reduces to the logistic +(sigmoid) function. +<p> +Fitting a softmax function can be done using the iteratively reweighted +least squares (IRLS) algorithm. +We use the implementation from +<a href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +Note that since +the softmax distribution is not in the exponential family, it does not +have finite sufficient statistics, and hence we must store all the +training data in uncompressed form. +If this takes too much space, one should use online (stochastic) gradient +descent (not implemented in BNT). +<p> +If a softmax node also has discrete parents, +we use a different set of w/b parameters for each combination of +parent values, as in the <a href="#gaussian">conditional linear +Gaussian CPD</a>. +This feature was implemented by Pierpaolo Brutti. +He is currently extending it so that discrete parents can be treated +as if they were continuous, by adding indicator variables to the X +vector. +<p> +We will see an example of softmax nodes <a href="#mixexp">below</a>. + + +<h2><a name="mlp">Neural network nodes</h2> + +Pierpaolo Brutti has implemented the mlp_CPD class, which uses a multi layer perceptron +to implement a mapping from continuous parents to discrete children, +similar to the softmax function. +(If there are also discrete parents, it creates a mixture of MLPs.) +It uses code from <a +href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +This is work in progress. + +<h2><a name="root">Root nodes</h2> + +A root node has no parents and no parameters; it can be used to model +an observed, exogeneous input variable, i.e., one which is "outside" +the model. +This is useful for conditional density models. +We will see an example of root nodes <a href="#mixexp">below</a>. + + +<h2><a name="gaussian">Gaussian nodes</h2> + +We now consider a distribution suitable for the continuous-valued nodes. +Suppose the node is called Y, its continuous parents (if any) are +called X, and its discrete parents (if any) are called Q. +The distribution on Y is defined as follows: +<pre> +- no parents: Y ~ N(mu, Sigma) +- cts parents : Y|X=x ~ N(mu + W x, Sigma) +- discrete parents: Y|Q=i ~ N(mu(:,i), Sigma(:,:,i)) +- cts and discrete parents: Y|X=x,Q=i ~ N(mu(:,i) + W(:,:,i) * x, Sigma(:,:,i)) +</pre> +where N(mu, Sigma) denotes a Normal distribution with mean mu and +covariance Sigma. Let |X|, |Y| and |Q| denote the sizes of X, Y and Q +respectively. +If there are no discrete parents, |Q|=1; if there is +more than one, then |Q| = a vector of the sizes of each discrete parent. +If there are no continuous parents, |X|=0; if there is more than one, +then |X| = the sum of their sizes. +Then mu is a |Y|*|Q| vector, Sigma is a |Y|*|Y|*|Q| positive +semi-definite matrix, and W is a |Y|*|X|*|Q| regression (weight) +matrix. +<p> +We can create a Gaussian node with random parameters as follows. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i); +</pre> +We can specify the value of one or more of the parameters as in the +following example, in which |Y|=2, and |Q|=1. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i, 'mean', [0; 0], 'weights', randn(Y,X), 'cov', eye(Y)); +</pre> +<p> +We will see an example of conditional linear Gaussian nodes <a +href="#cg_model">below</a>. +<p> +When fitting a Gaussian using EM, you might encounter some numerical +problems. See <a +href="http://www.ai.mit.edu/~murphyk/Software/HMM/hmm.html#loglikpos">here</a> +for a discussion of this in the context of HMMs. +(Note: BNT does not currently support spherical covariances, although it +would be easy to add, since KPMstats/clg_Mstep supports this option; +you would just need to modify gaussian_CPD/update_ess to accumulate +weighted inner products.) + + +<h2><a name="nongauss">Other continuous distributions</h2> + +Currently BNT does not support any CPDs for continuous nodes other +than the Gaussian. +However, you can use a mixture of Gaussians to +approximate other continuous distributions. We will see some an example +of this with the IFA model <a href="#pca">below</a>. + + +<h2><a name="glm">Generalized linear model nodes</h2> + +In the future, we may incorporate some of the functionality of +<a href = +"http://www.sci.usq.edu.au/staff/dunn/glmlab/glmlab.html">glmlab</a> +into BNT. + + +<h2><a name="dtree">Classification/regression tree nodes</h2> + +We plan to add classification and regression trees to define CPDs for +discrete and continuous nodes, respectively. +Trees have many advantages: they are easy to interpret, they can do +feature selection, they can +handle discrete and continuous inputs, they do not make strong +assumptions about the form of the distribution, the number of +parameters can grow in a data-dependent way (i.e., they are +semi-parametric), they can handle missing data, etc. +However, they are not yet implemented. +<!-- +Yimin Zhang is currently (Feb '02) implementing this. +--> + + +<h2><a name="cpd_summary">Summary of CPD types</h2> + +We list all the different types of CPDs supported by BNT. +For each CPD, we specify if the child and parents can be discrete (D) or +continuous (C) (Binary (B) nodes are a special case). +We also specify which methods each class supports. +If a method is inherited, the name of the parent class is mentioned. +If a parent class calls a child method, this is mentioned. +<p> +The <tt>CPD_to_CPT</tt> method converts a CPD to a table; this +requires that the child and all parents are discrete. +The CPT might be exponentially big... +<tt>convert_to_table</tt> evaluates a CPD with evidence, and +represents the the resulting potential as an array. +This requires that the child is discrete, and any continuous parents +are observed. +<tt>convert_to_pot</tt> evaluates a CPD with evidence, and +represents the resulting potential as a dpot, gpot, cgpot or upot, as +requested. (d=discrete, g=Gaussian, cg = conditional Gaussian, u = +utility). + +<p> +When we sample a node, all the parents are observed. +When we compute the (log) probability of a node, all the parents and +the child are observed. +<p> +We also specify if the parameters are learnable. +For learning with EM, we require +the methods <tt>reset_ess</tt>, <tt>update_ess</tt> and +<tt>maximize_params</tt>. +For learning from fully observed data, we require +the method <tt>learn_params</tt>. +By default, all classes inherit this from generic_CPD, which simply +calls <tt>update_ess</tt> N times, once for each data case, followed +by <tt>maximize_params</tt>, i.e., it is like EM, without the E step. +Some classes implement a batch formula, which is quicker. +<p> +Bayesian learning means computing a posterior over the parameters +given fully observed data. +<p> +Pearl means we implement the methods <tt>compute_pi</tt> and +<tt>compute_lambda_msg</tt>, used by +<tt>pearl_inf_engine</tt>, which runs on directed graphs. +<tt>belprop_inf_engine</tt> only needs <tt>convert_to_pot</tt>.H +The pearl methods can exploit special properties of the CPDs for +computing the messages efficiently, whereas belprop does not. +<p> +The only method implemented by generic_CPD is <tt>adjustable_CPD</tt>, +which is not shown, since it is not very interesting. + + +<p> + + +<table> +<table border units = pixels><tr> +<td align=center>Name +<td align=center>Child +<td align=center>Parents +<td align=center>Comments +<td align=center>CPD_to_CPT +<td align=center>conv_to_table +<td align=center>conv_to_pot +<td align=center>sample +<td align=center>prob +<td align=center>learn +<td align=center>Bayes +<td align=center>Pearl + + +<tr> +<!-- Name--><td> +<!-- Child--><td> +<!-- Parents--><td> +<!-- Comments--><td> +<!-- CPD_to_CPT--><td> +<!-- conv_to_table--><td> +<!-- conv_to_pot--><td> +<!-- sample--><td> +<!-- prob--><td> +<!-- learn--><td> +<!-- Bayes--><td> +<!-- Pearl--><td> + +<tr> +<!-- Name--><td>boolean +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>deterministic +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>Discrete +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Calls CPD_to_CPT +<!-- conv_to_pot--><td>Calls conv_to_table +<!-- sample--><td>Calls conv_to_table +<!-- prob--><td>Calls conv_to_table +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>Gaussian +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>gmux +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>multiplexer +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>MLP +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>multi layer perceptron +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>noisy-or +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>root +<!-- Child--><td>C/D +<!-- Parents--><td>none +<!-- Comments--><td>no params +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>softmax +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>generic +<!-- Child--><td>C/D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>N +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>Tabular +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>Y +<!-- Pearl--><td>Y + +</table> + + + +<h1><a name="examples">Example models</h1> + + +<h2>Gaussian mixture models</h2> + +Richard W. DeVaul has made a detailed tutorial on how to fit mixtures +of Gaussians using BNT. Available +<a href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt">here</a>. + + +<h2><a name="pca">PCA, ICA, and all that </h2> + +In Figure (a) below, we show how Factor Analysis can be thought of as a +graphical model. Here, X has an N(0,I) prior, and +Y|X=x ~ N(mu + Wx, Psi), +where Psi is diagonal and W is called the "factor loading matrix". +Since the noise on both X and Y is diagonal, the components of these +vectors are uncorrelated, and hence can be represented as individual +scalar nodes, as we show in (b). +(This is useful if parts of the observations on the Y vector are occasionally missing.) +We usually take k=|X| << |Y|=D, so the model tries to explain +many observations using a low-dimensional subspace. + + +<center> +<table> +<tr> +<td><img src="Figures/fa.gif"> +<td><img src="Figures/fa_scalar.gif"> +<td><img src="Figures/mfa.gif"> +<td><img src="Figures/ifa.gif"> +<tr> +<td align=center> (a) +<td align=center> (b) +<td align=center> (c) +<td align=center> (d) +</table> +</center> + +<p> +We can create this model in BNT as follows. +<pre> +ns = [k D]; +dag = zeros(2,2); +dag(1,2) = 1; +bnet = mk_bnet(dag, ns, 'discrete', []); +bnet.CPD{1} = gaussian_CPD(bnet, 1, 'mean', zeros(k,1), 'cov', eye(k), ... + 'cov_type', 'diag', 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(D,1), 'cov', diag(Psi0), 'weights', W0, ... + 'cov_type', 'diag', 'clamp_mean', 1); +</pre> + +The root node is clamped to the N(0,I) distribution, so that we will +not update these parameters during learning. +The mean of the leaf node is clamped to 0, +since we assume the data has been centered (had its mean subtracted +off); this is just for simplicity. +Finally, the covariance of the leaf node is constrained to be +diagonal. W0 and Psi0 are the initial parameter guesses. + +<p> +We can fit this model (i.e., estimate its parameters in a maximum +likelihood (ML) sense) using EM, as we +explain <a href="#em">below</a>. +Not surprisingly, the ML estimates for mu and Psi turn out to be +identical to the +sample mean and variance, which can be computed directly as +<pre> +mu_ML = mean(data); +Psi_ML = diag(cov(data)); +</pre> +Note that W can only be identified up to a rotation matrix, because of +the spherical symmetry of the source. + +<p> +If we restrict Psi to be spherical, i.e., Psi = sigma*I, +there is a closed-form solution for W as well, +i.e., we do not need to use EM. +In particular, W contains the first |X| eigenvectors of the sample covariance +matrix, with scalings determined by the eigenvalues and sigma. +Classical PCA can be obtained by taking the sigma->0 limit. +For details, see + +<ul> +<li> <a href="ftp://hope.caltech.edu/pub/roweis/Empca/empca.ps"> +"EM algorithms for PCA and SPCA"</a>, Sam Roweis, NIPS 97. +(<a href="ftp://hope.caltech.edu/pub/roweis/Code/empca.tar.gz"> +Matlab software</a>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +By adding a hidden discrete variable, we can create mixtures of FA +models, as shown in (c). +Now we can explain the data using a set of subspaces. +We can create this model in BNT as follows. +<pre> +ns = [M k D]; +dag = zeros(3); +dag(1,3) = 1; +dag(2,3) = 1; +bnet = mk_bnet(dag, ns, 'discrete', 1); +bnet.CPD{1} = tabular_CPD(bnet, 1, Pi0); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(k, 1), 'cov', eye(k), 'cov_type', 'diag', ... + 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{3} = gaussian_CPD(bnet, 3, 'mean', Mu0', 'cov', repmat(diag(Psi0), [1 1 M]), ... + 'weights', W0, 'cov_type', 'diag', 'tied_cov', 1); +</pre> +Notice how the covariance matrix for Y is the same for all values of +Q; that is, the noise level in each sub-space is assumed the same. +However, we allow the offset, mu, to vary. +For details, see +<ul> + +<LI> +<a HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/tr-96-1.ps.gz"> The EM +Algorithm for Mixtures of Factor Analyzers </A>, +Ghahramani, Z. and Hinton, G.E. (1996), +University of Toronto +Technical Report CRG-TR-96-1. +(<A HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/mfa.tar.gz">Matlab software</A>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +I have included Zoubin's specialized MFA code (with his permission) +with the toolbox, so you can check that BNT gives the same results: +see 'BNT/examples/static/mfa1.m'. + +<p> +Independent Factor Analysis (IFA) generalizes FA by allowing a +non-Gaussian prior on each component of X. +(Note that we can approximate a non-Gaussian prior using a mixture of +Gaussians.) +This means that the likelihood function is no longer rotationally +invariant, so we can uniquely identify W and the hidden +sources X. +IFA also allows a non-diagonal Psi (i.e. correlations between the components of Y). +We recover classical Independent Components Analysis (ICA) +in the Psi -> 0 limit, and by assuming that |X|=|Y|, so that the +weight matrix W is square and invertible. +For details, see +<ul> +<li> +<a href="http://www.gatsby.ucl.ac.uk/~hagai/ifa.ps">Independent Factor +Analysis</a>, H. Attias, Neural Computation 11: 803--851, 1998. +</ul> + + + +<h2><a name="mixexp">Mixtures of experts</h2> + +As an example of the use of the softmax function, +we introduce the Mixture of Experts model. +<!-- +We also show +the Hierarchical Mixture of Experts model, where the hierarchy has two +levels. +(This is essentially a probabilistic decision tree of height two.) +--> +As before, +circles denote continuous-valued nodes, +squares denote discrete nodes, clear +means hidden, and shaded means observed. +<p> +<center> +<table> +<tr> +<td><img src="Figures/mixexp.gif"> +<!-- +<td><img src="Figures/hme.gif"> +--> +</table> +</center> +<p> +X is the observed +input, Y is the output, and +the Q nodes are hidden "gating" nodes, which select the appropriate +set of parameters for Y. During training, Y is assumed observed, +but for testing, the goal is to predict Y given X. +Note that this is a <em>conditional</em> density model, so we don't +associate any parameters with X. +Hence X's CPD will be a root CPD, which is a way of modelling +exogenous nodes. +If the output is a continuous-valued quantity, +we assume the "experts" are linear-regression units, +and set Y's CPD to linear-Gaussian. +If the output is discrete, we set Y's CPD to a softmax function. +The Q CPDs will always be softmax functions. + +<p> +As a concrete example, consider the mixture of experts model where X and Y are +scalars, and Q is binary. +This is just piecewise linear regression, where +we have two line segments, i.e., +<P> +<IMG ALIGN=BOTTOM SRC="Eqns/lin_reg_eqn.gif"> +<P> +We can create this model with random parameters as follows. +(This code is bundled in BNT/examples/static/mixexp2.m.) +<PRE> +X = 1; +Q = 2; +Y = 3; +dag = zeros(3,3); +dag(X,[Q Y]) = 1 +dag(Q,Y) = 1; +ns = [1 2 1]; % make X and Y scalars, and have 2 experts +onodes = [1 3]; +bnet = mk_bnet(dag, ns, 'discrete', 2, 'observed', onodes); + +rand('state', 0); +randn('state', 0); +bnet.CPD{1} = root_CPD(bnet, 1); +bnet.CPD{2} = softmax_CPD(bnet, 2); +bnet.CPD{3} = gaussian_CPD(bnet, 3); +</PRE> +Now let us fit this model using <a href="#em">EM</a>. +First we <a href="#load_data">load the data</a> (1000 training cases) and plot them. +<P> +<PRE> +data = load('/examples/static/Misc/mixexp_data.txt', '-ascii'); +plot(data(:,1), data(:,2), '.'); +</PRE> +<p> +<center> +<IMG SRC="Figures/mixexp_data.gif"> +</center> +<p> +This is what the model looks like before training. +(Thanks to Thomas Hofman for writing this plotting routine.) +<p> +<center> +<IMG SRC="Figures/mixexp_before.gif"> +</center> +<p> +Now let's train the model, and plot the final performance. +(We will discuss how to train models in more detail <a href="#param_learning">below</a>.) +<P> +<PRE> +ncases = size(data, 1); % each row of data is a training case +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); % each column of cases is a training case +engine = jtree_inf_engine(bnet); +max_iter = 20; +[bnet2, LLtrace] = learn_params_em(engine, cases, max_iter); +</PRE> +(We specify which nodes will be observed when we create the engine. +Hence BNT knows that the hidden nodes are all discrete. +For complex models, this can lead to a significant speedup.) +Below we show what the model looks like after 16 iterations of EM +(with 100 IRLS iterations per M step), when it converged +using the default convergence tolerance (that the +fractional change in the log-likelihood be less than 1e-3). +Before learning, the log-likelihood was +-322.927442; afterwards, it was -13.728778. +<p> +<center> +<IMG SRC="Figures/mixexp_after.gif"> +</center> +(See BNT/examples/static/mixexp2.m for details of the code.) + + + +<h2><a name="hme">Hierarchical mixtures of experts</h2> + +A hierarchical mixture of experts (HME) extends the mixture of experts +model by having more than one hidden node. A two-level example is shown below, along +with its more traditional representation as a neural network. +This is like a (balanced) probabilistic decision tree of height 2. +<p> +<center> +<IMG SRC="Figures/HMEforMatlab.jpg"> +</center> +<p> +<a href="mailto:pbrutti@stat.cmu.edu">Pierpaolo Brutti</a> +has written an extensive set of routines for HMEs, +which are bundled with BNT: see the examples/static/HME directory. +These routines allow you to choose the number of hidden (gating) +layers, and the form of the experts (softmax or MLP). +See the file hmemenu, which provides a demo. +For example, the figure below shows the decision boundaries learned +for a ternary classification problem, using a 2 level HME with softmax +gates and softmax experts; the training set is on the left, the +testing set on the right. +<p> +<center> +<!--<IMG SRC="Figures/hme_dec_boundary.gif">--> +<IMG SRC="Figures/hme_dec_boundary.png"> +</center> +<p> + + +<p> +For more details, see the following: +<ul> + +<li> <a href="http://www.cs.berkeley.edu/~jordan/papers/hierarchies.ps.Z"> +Hierarchical mixtures of experts and the EM algorithm</a> +M. I. Jordan and R. A. Jacobs. Neural Computation, 6, 181-214, 1994. + +<li> <a href = +"http://www.cs.berkeley.edu/~dmartin/software">David Martin's +matlab code for HME</a> + +<li> <a +href="http://www.cs.berkeley.edu/~jordan/papers/uai.ps.Z">Why the +logistic function? A tutorial discussion on +probabilities and neural networks.</a> M. I. Jordan. MIT Computational +Cognitive Science Report 9503, August 1995. + +<li> "Generalized Linear Models", McCullagh and Nelder, Chapman and +Halll, 1983. + +<li> +"Improved learning algorithms for mixtures of experts in multiclass +classification". +K. Chen, L. Xu, H. Chi. +Neural Networks (1999) 12: 1229-1252. + +<li> <a href="http://www.oigeeza.com/steve/"> +Classification Using Hierarchical Mixtures of Experts</a> +S.R. Waterhouse and A.J. Robinson. +In Proc. IEEE Workshop on Neural Network for Signal Processing IV (1994), pp. 177-186 + +<li> <a href="http://www.idiap.ch/~perry/"> +Localized mixtures of experts</a>, +P. Moerland, 1998. + +<li> "Nonlinear gated experts for time series", +A.S. Weigend and M. Mangeas, 1995. + +</ul> + + +<h2><a name="qmr">QMR</h2> + +Bayes nets originally arose out of an attempt to add probabilities to +expert systems, and this is still the most common use for BNs. +A famous example is +QMR-DT, a decision-theoretic reformulation of the Quick Medical +Reference (QMR) model. +<p> +<center> +<IMG ALIGN=BOTTOM SRC="Figures/qmr.gif"> +</center> +Here, the top layer represents hidden disease nodes, and the bottom +layer represents observed symptom nodes. +The goal is to infer the posterior probability of each disease given +all the symptoms (which can be present, absent or unknown). +Each node in the top layer has a Bernoulli prior (with a low prior +probability that the disease is present). +Since each node in the bottom layer has a high fan-in, we use a +noisy-OR parameterization; each disease has an independent chance of +causing each symptom. +The real QMR-DT model is copyright, but +we can create a random QMR-like model as follows. +<pre> +function bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% MK_QMR_BNET Make a QMR model +% bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% +% G(i,j) = 1 iff there is an arc from disease i to finding j +% inhibit(i,j) = inhibition probability on i->j arc +% leak(j) = inhibition prob. on leak->j arc +% prior(i) = prob. disease i is on + +[Ndiseases Nfindings] = size(inhibit); +N = Ndiseases + Nfindings; +finding_node = Ndiseases+1:N; +ns = 2*ones(1,N); +dag = zeros(N,N); +dag(1:Ndiseases, finding_node) = G; +bnet = mk_bnet(dag, ns, 'observed', finding_node); + +for d=1:Ndiseases + CPT = [1-prior(d) prior(d)]; + bnet.CPD{d} = tabular_CPD(bnet, d, CPT'); +end + +for i=1:Nfindings + fnode = finding_node(i); + ps = parents(G, i); + bnet.CPD{fnode} = noisyor_CPD(bnet, fnode, leak(i), inhibit(ps, i)); +end +</pre> +In the file BNT/examples/static/qmr1, we create a random bipartite +graph G, with 5 diseases and 10 findings, and random parameters. +(In general, to create a random dag, use 'mk_random_dag'.) +We can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>, with the +following results: +<p> +<img src="Figures/qmr.rnd.jpg"> + +<p> +Now let us put some random evidence on all the leaves except the very +first and very last, and compute the disease posteriors. +<pre> +pos = 2:floor(Nfindings/2); +neg = (pos(end)+1):(Nfindings-1); +onodes = myunion(pos, neg); +evidence = cell(1, N); +evidence(findings(pos)) = num2cell(repmat(2, 1, length(pos))); +evidence(findings(neg)) = num2cell(repmat(1, 1, length(neg))); + +engine = jtree_inf_engine(bnet); +[engine, ll] = enter_evidence(engine, evidence); +post = zeros(1, Ndiseases); +for i=diseases(:)' + m = marginal_nodes(engine, i); + post(i) = m.T(2); +end +</pre> +Junction tree can be quite slow on large QMR models. +Fortunately, it is possible to exploit properties of the noisy-OR +function to speed up exact inference using an algorithm called +<a href="#quickscore">quickscore</a>, discussed below. + + + + + +<h2><a name="cg_model">Conditional Gaussian models</h2> + +A conditional Gaussian model is one in which, conditioned on all the discrete +nodes, the distribution over the remaining (continuous) nodes is +multivariate Gaussian. This means we can have arcs from discrete (D) +to continuous (C) nodes, but not vice versa. +(We <em>are</em> allowed C->D arcs if the continuous nodes are observed, +as in the <a href="#mixexp">mixture of experts</a> model, +since this distribution can be represented with a discrete potential.) +<p> +We now give an example of a CG model, from +the paper "Propagation of Probabilities, Means amd +Variances in Mixed Graphical Association Models", Steffen Lauritzen, +JASA 87(420):1098--1108, 1992 (reprinted in the book "Probabilistic Networks and Expert +Systems", R. G. Cowell, A. P. Dawid, S. L. Lauritzen and +D. J. Spiegelhalter, Springer, 1999.) + +<h3>Specifying the graph</h3> + +Consider the model of waste emissions from an incinerator plant shown below. +We follow the standard convention that shaded nodes are observed, +clear nodes are hidden. +We also use the non-standard convention that +square nodes are discrete (tabular) and round nodes are +Gaussian. + +<p> +<center> +<IMG SRC="Figures/cg1.gif"> +</center> +<p> + +We can create this model as follows. +<pre> +F = 1; W = 2; E = 3; B = 4; C = 5; D = 6; Min = 7; Mout = 8; L = 9; +n = 9; + +dag = zeros(n); +dag(F,E)=1; +dag(W,[E Min D]) = 1; +dag(E,D)=1; +dag(B,[C D])=1; +dag(D,[L Mout])=1; +dag(Min,Mout)=1; + +% node sizes - all cts nodes are scalar, all discrete nodes are binary +ns = ones(1, n); +dnodes = [F W B]; +cnodes = mysetdiff(1:n, dnodes); +ns(dnodes) = 2; + +bnet = mk_bnet(dag, ns, 'discrete', dnodes); +</pre> +'dnodes' is a list of the discrete nodes; 'cnodes' is the continuous +nodes. 'mysetdiff' is a faster version of the built-in 'setdiff'. +<p> + + +<h3>Specifying the parameters</h3> + +The parameters of the discrete nodes can be specified as follows. +<pre> +bnet.CPD{B} = tabular_CPD(bnet, B, 'CPT', [0.85 0.15]); % 1=stable, 2=unstable +bnet.CPD{F} = tabular_CPD(bnet, F, 'CPT', [0.95 0.05]); % 1=intact, 2=defect +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [2/7 5/7]); % 1=industrial, 2=household +</pre> + +<p> +The parameters of the continuous nodes can be specified as follows. +<pre> +bnet.CPD{E} = gaussian_CPD(bnet, E, 'mean', [-3.9 -0.4 -3.2 -0.5], ... + 'cov', [0.00002 0.0001 0.00002 0.0001]); +bnet.CPD{D} = gaussian_CPD(bnet, D, 'mean', [6.5 6.0 7.5 7.0], ... + 'cov', [0.03 0.04 0.1 0.1], 'weights', [1 1 1 1]); +bnet.CPD{C} = gaussian_CPD(bnet, C, 'mean', [-2 -1], 'cov', [0.1 0.3]); +bnet.CPD{L} = gaussian_CPD(bnet, L, 'mean', 3, 'cov', 0.25, 'weights', -0.5); +bnet.CPD{Min} = gaussian_CPD(bnet, Min, 'mean', [0.5 -0.5], 'cov', [0.01 0.005]); +bnet.CPD{Mout} = gaussian_CPD(bnet, Mout, 'mean', 0, 'cov', 0.002, 'weights', [1 1]); +</pre> + + +<h3><a name="cg_infer">Inference</h3> + +<!--Let us perform inference in the <a href="#cg_model">waste incinerator example</a>.--> +First we compute the unconditional marginals. +<pre> +engine = jtree_inf_engine(bnet); +evidence = cell(1,n); +[engine, ll] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, E); +</pre> +<!--(Of course, we could use <tt>cond_gauss_inf_engine</tt> instead of jtree.)--> +'marg' is a structure that contains the fields 'mu' and 'Sigma', which +contain the mean and (co)variance of the marginal on E. +In this case, they are both scalars. +Let us check they match the published figures (to 2 decimal places). +<!--(We can't expect +more precision than this in general because I have implemented the algorithm of +Lauritzen (1992), which can be numerically unstable.)--> +<pre> +tol = 1e-2; +assert(approxeq(marg.mu, -3.25, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.709, tol)); +</pre> +We can compute the other posteriors similarly. +Now let us add some evidence. +<pre> +evidence = cell(1,n); +evidence{W} = 1; % industrial +evidence{L} = 1.1; +evidence{C} = -0.9; +[engine, ll] = enter_evidence(engine, evidence); +</pre> +Now we find +<pre> +marg = marginal_nodes(engine, E); +assert(approxeq(marg.mu, -3.8983, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.0763, tol)); +</pre> + + +We can also compute the joint probability on a set of nodes. +For example, P(D, Mout | evidence) is a 2D Gaussian: +<pre> +marg = marginal_nodes(engine, [D Mout]) +marg = + domain: [6 8] + mu: [2x1 double] + Sigma: [2x2 double] + T: 1.0000 +</pre> +The mean is +<pre> +marg.mu +ans = + 3.6077 + 4.1077 +</pre> +and the covariance matrix is +<pre> +marg.Sigma +ans = + 0.1062 0.1062 + 0.1062 0.1182 +</pre> +It is easy to visualize this posterior using standard Matlab plotting +functions, e.g., +<pre> +gaussplot2d(marg.mu, marg.Sigma); +</pre> +produces the following picture. + +<p> +<center> +<IMG SRC="Figures/gaussplot.png"> +</center> +<p> + + +The T field indicates that the mixing weight of this Gaussian +component is 1.0. +If the joint contains discrete and continuous variables, the result +will be a mixture of Gaussians, e.g., +<pre> +marg = marginal_nodes(engine, [F E]) + domain: [1 3] + mu: [-3.9000 -0.4003] + Sigma: [1x1x2 double] + T: [0.9995 4.7373e-04] +</pre> +The interpretation is +Sigma(i,j,k) = Cov[ E(i) E(j) | F=k ]. +In this case, E is a scalar, so i=j=1; k specifies the mixture component. +<p> +We saw in the sprinkler network that BNT sets the effective size of +observed discrete nodes to 1, since they only have one legal value. +For continuous nodes, BNT sets their length to 0, +since they have been reduced to a point. +For example, +<pre> +marg = marginal_nodes(engine, [B C]) + domain: [4 5] + mu: [] + Sigma: [] + T: [0.0123 0.9877] +</pre> +It is simple to post-process the output of marginal_nodes. +For example, the file BNT/examples/static/cg1 sets the mu term of +observed nodes to their observed value, and the Sigma term to 0 (since +observed nodes have no variance). + +<p> +Note that the implemented version of the junction tree is numerically +unstable when using CG potentials +(which is why, in the example above, we only required our answers to agree with +the published ones to 2dp.) +This is why you might want to use <tt>stab_cond_gauss_inf_engine</tt>, +implemented by Shan Huang. This is described in + +<ul> +<li> "Stable Local Computation with Conditional Gaussian Distributions", +S. Lauritzen and F. Jensen, Tech Report R-99-2014, +Dept. Math. Sciences, Allborg Univ., 1999. +</ul> + +However, even the numerically stable version +can be computationally intractable if there are many hidden discrete +nodes, because the number of mixture components grows exponentially e.g., in a +<a href="usage_dbn.html#lds">switching linear dynamical system</a>. +In general, one must resort to approximate inference techniques: see +the discussion on <a href="#engines">inference engines</a> below. + + +<h2><a name="hybrid">Other hybrid models</h2> + +When we have C->D arcs, where C is hidden, we need to use +approximate inference. +One approach (not implemented in BNT) is described in +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/hybrid_uai99.ps.gz">A +Variational Approximation for Bayesian Networks with +Discrete and Continuous Latent Variables</a>, +K. Murphy, UAI 99. +</ul> +Of course, one can always use <a href="#sampling">sampling</a> methods +for approximate inference in such models. + + + +<h1><a name="param_learning">Parameter Learning</h1> + +The parameter estimation routines in BNT can be classified into 4 +types, depending on whether the goal is to compute +a full (Bayesian) posterior over the parameters or just a point +estimate (e.g., Maximum Likelihood or Maximum A Posteriori), +and whether all the variables are fully observed or there is missing +data/ hidden variables (partial observability). +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_params</tt></td> + <td><tt>learn_params_em</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>bayes_update_params</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="load_data">Loading data from a file</h2> + +To load numeric data from an ASCII text file called 'dat.txt', where each row is a +case and columns are separated by white-space, such as +<pre> +011979 1626.5 0.0 +021979 1367.0 0.0 +... +</pre> +you can use +<pre> +data = load('dat.txt'); +</pre> +or +<pre> +load dat.txt -ascii +</pre> +In the latter case, the data is stored in a variable called 'dat' (the +filename minus the extension). +Alternatively, suppose the data is stored in a .csv file (has commas +separating the columns, and contains a header line), such as +<pre> +header info goes here +ORD,011979,1626.5,0.0 +DSM,021979,1367.0,0.0 +... +</pre> +You can load this using +<pre> +[a,b,c,d] = textread('dat.txt', '%s %d %f %f', 'delimiter', ',', 'headerlines', 1); +</pre> +If your file is not in either of these formats, you can either use Perl to convert +it to this format, or use the Matlab scanf command. +Type +<tt> +help iofun +</tt> +for more information on Matlab's file functions. +<!-- +<p> +To load data directly from Excel, +you should buy the +<a href="http://www.mathworks.com/products/excellink/">Excel Link</a>. +To load data directly from a relational database, +you should buy the +<a href="http://www.mathworks.com/products/database">Database +toolbox</a>. +--> +<p> +BNT learning routines require data to be stored in a cell array. +data{i,m} is the value of node i in case (example) m, i.e., each +<em>column</em> is a case. +If node i is not observed in case m (missing value), set +data{i,m} = []. +(Not all the learning routines can cope with such missing values, however.) +In the special case that all the nodes are observed and are +scalar-valued (as opposed to vector-valued), the data can be +stored in a matrix (as opposed to a cell-array). +<p> +Suppose, as in the <a href="#mixexp">mixture of experts example</a>, +that we have 3 nodes in the graph: X(1) is the observed input, X(3) is +the observed output, and X(2) is a hidden (gating) node. We can +create the dataset as follows. +<pre> +data = load('dat.txt'); +ncases = size(data, 1); +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); +</pre> +Notice how we transposed the data, to convert rows into columns. +Also, cases{2,m} = [] for all m, since X(2) is always hidden. + + +<h2><a name="mle_complete">Maximum likelihood parameter estimation from complete data</h2> + +As an example, let's generate some data from the sprinkler network, randomize the parameters, +and then try to recover the original model. +First we create some training data using forwards sampling. +<pre> +samples = cell(N, nsamples); +for i=1:nsamples + samples(:,i) = sample_bnet(bnet); +end +</pre> +samples{j,i} contains the value of the j'th node in case i. +sample_bnet returns a cell array because, in general, each node might +be a vector of different length. +In this case, all nodes are discrete (and hence scalars), so we +could have used a regular array instead (which can be quicker): +<pre> +data = cell2num(samples); +</pre +So now data(j,i) = samples{j,i}. +<p> +Now we create a network with random parameters. +(The initial values of bnet2 don't matter in this case, since we can find the +globally optimal MLE independent of where we start.) +<pre> +% Make a tabula rasa +bnet2 = mk_bnet(dag, node_sizes); +seed = 0; +rand('state', seed); +bnet2.CPD{C} = tabular_CPD(bnet2, C); +bnet2.CPD{R} = tabular_CPD(bnet2, R); +bnet2.CPD{S} = tabular_CPD(bnet2, S); +bnet2.CPD{W} = tabular_CPD(bnet2, W); +</pre> +Finally, we find the maximum likelihood estimates of the parameters. +<pre> +bnet3 = learn_params(bnet2, samples); +</pre> +To view the learned parameters, we use a little Matlab hackery. +<pre> +CPT3 = cell(1,N); +for i=1:N + s=struct(bnet3.CPD{i}); % violate object privacy + CPT3{i}=s.CPT; +end +</pre> +Here are the parameters learned for node 4. +<pre> +dispcpt(CPT3{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.2000 0.8000 +1 2 : 0.2273 0.7727 +2 2 : 0.0000 1.0000 +</pre> +So we see that the learned parameters are fairly close to the "true" +ones, which we display below. +<pre> +dispcpt(CPT{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.1000 0.9000 +1 2 : 0.1000 0.9000 +2 2 : 0.0100 0.9900 +</pre> +We can get better results by using a larger training set, or using +informative priors (see <a href="#prior">below</a>). + + + +<h2><a name="prior">Parameter priors</h2> + +Currently, only tabular CPDs can have priors on their parameters. +The conjugate prior for a multinomial is the Dirichlet. +(For binary random variables, the multinomial is the same as the +Bernoulli, and the Dirichlet is the same as the Beta.) +<p> +The Dirichlet has a simple interpretation in terms of pseudo counts. +If we let N_ijk = the num. times X_i=k and Pa_i=j occurs in the +training set, where Pa_i are the parents of X_i, +then the maximum likelihood (ML) estimate is +T_ijk = N_ijk / N_ij (where N_ij = sum_k' N_ijk'), which will be 0 if N_ijk=0. +To prevent us from declaring that (X_i=k, Pa_i=j) is impossible just because this +event was not seen in the training set, +we can pretend we saw value k of X_i, for each value j of Pa_i some number (alpha_ijk) +of times in the past. +The MAP (maximum a posterior) estimate is then +<pre> +T_ijk = (N_ijk + alpha_ijk) / (N_ij + alpha_ij) +</pre> +and is never 0 if all alpha_ijk > 0. +For example, consider the network A->B, where A is binary and B has 3 +values. +A uniform prior for B has the form +<pre> + B=1 B=2 B=3 +A=1 1 1 1 +A=2 1 1 1 +</pre> +which can be created using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'unif'); +</pre> +This prior does not satisfy the likelihood equivalence principle, +which says that <a href="#markov_equiv">Markov equivalent</a> models +should have the same marginal likelihood. +A prior that does satisfy this principle is shown below. +Heckerman (1995) calls this the +BDeu prior (likelihood equivalent uniform Bayesian Dirichlet). +<pre> + B=1 B=2 B=3 +A=1 1/6 1/6 1/6 +A=2 1/6 1/6 1/6 +</pre> +where we put N/(q*r) in each bin; N is the equivalent sample size, +r=|A|, q = |B|. +This can be created as follows +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'BDeu'); +</pre> +Here, 1 is the equivalent sample size, and is the strength of the +prior. +You can change this using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', ... + 'BDeu', 'dirichlet_weight', 10); +</pre> +<!--where counts is an array of pseudo-counts of the same size as the +CPT.--> +<!-- +<p> +When you specify a prior, you should set row i of the CPT to the +normalized version of row i of the pseudo-count matrix, i.e., to the +expected values of the parameters. This will ensure that computing the +marginal likelihood sequentially (see <a +href="#bayes_learn">below</a>) and in batch form gives the same +results. +To do this, proceed as follows. +<pre> +tabular_CPD(bnet, i, 'prior', counts, 'CPT', mk_stochastic(counts)); +</pre> +For a non-informative prior, you can just write +<pre> +tabular_CPD(bnet, i, 'prior', 'unif', 'CPT', 'unif'); +</pre> +--> + + +<h2><a name="bayes_learn">(Sequential) Bayesian parameter updating from complete data</h2> + +If we use conjugate priors and have fully observed data, we can +compute the posterior over the parameters in batch form as follows. +<pre> +cases = sample_bnet(bnet, nsamples); +bnet = bayes_update_params(bnet, cases); +LL = log_marg_lik_complete(bnet, cases); +</pre> +bnet.CPD{i}.prior contains the new Dirichlet pseudocounts, +and bnet.CPD{i}.CPT is set to the mean of the posterior (the +normalized counts). +(Hence if the initial pseudo counts are 0, +<tt>bayes_update_params</tt> and <tt>learn_params</tt> will give the +same result.) + + + + +<p> +We can compute the same result sequentially (on-line) as follows. +<pre> +LL = 0; +for m=1:nsamples + LL = LL + log_marg_lik_complete(bnet, cases(:,m)); + bnet = bayes_update_params(bnet, cases(:,m)); +end +</pre> + +The file <tt>BNT/examples/static/StructLearn/model_select1</tt> has an example of +sequential model selection which uses the same idea. +We generate data from the model A->B +and compute the posterior prob of all 3 dags on 2 nodes: + (1) A B, (2) A <- B , (3) A -> B +Models 2 and 3 are <a href="#markov_equiv">Markov equivalent</a>, and therefore indistinguishable from +observational data alone, so we expect their posteriors to be the same +(assuming a prior which satisfies likelihood equivalence). +If we use random parameters, the "true" model only gets a higher posterior after 2000 trials! +However, if we make B a noisy NOT gate, the true model "wins" after 12 +trials, as shown below (red = model 1, blue/green (superimposed) +represents models 2/3). +<p> +<img src="Figures/model_select.png"> +<p> +The use of marginal likelihood for model selection is discussed in +greater detail in the +section on <a href="structure_learning">structure learning</a>. + + + + +<h2><a name="em">Maximum likelihood parameter estimation with missing values</h2> + +Now we consider learning when some values are not observed. +Let us randomly hide half the values generated from the water +sprinkler example. +<pre> +samples2 = samples; +hide = rand(N, nsamples) > 0.5; +[I,J]=find(hide); +for k=1:length(I) + samples2{I(k), J(k)} = []; +end +</pre> +samples2{i,l} is the value of node i in training case l, or [] if unobserved. +<p> +Now we will compute the MLEs using the EM algorithm. +We need to use an inference algorithm to compute the expected +sufficient statistics in the E step; the M (maximization) step is as +above. +<pre> +engine2 = jtree_inf_engine(bnet2); +max_iter = 10; +[bnet4, LLtrace] = learn_params_em(engine2, samples2, max_iter); +</pre> +LLtrace(i) is the log-likelihood at iteration i. We can plot this as +follows: +<pre> +plot(LLtrace, 'x-') +</pre> +Let's display the results after 10 iterations of EM. +<pre> +celldisp(CPT4) +CPT4{1} = + 0.6616 + 0.3384 +CPT4{2} = + 0.6510 0.3490 + 0.8751 0.1249 +CPT4{3} = + 0.8366 0.1634 + 0.0197 0.9803 +CPT4{4} = +(:,:,1) = + 0.8276 0.0546 + 0.5452 0.1658 +(:,:,2) = + 0.1724 0.9454 + 0.4548 0.8342 +</pre> +We can get improved performance by using one or more of the following +methods: +<ul> +<li> Increasing the size of the training set. +<li> Decreasing the amount of hidden data. +<li> Running EM for longer. +<li> Using informative priors. +<li> Initialising EM from multiple starting points. +</ul> + +<h2><a name="gaussianNumerical">Numerical problems when fitting +Gaussians using EM</h2> + +When fitting a <a href="#gaussian">Gaussian CPD</a> using EM, you +might encounter some numerical +problems. See <a +href="http://www.ai.mit.edu/~murphyk/Software/HMM/hmm.html#loglikpos">here</a> +for a discussion of this in the context of HMMs. +<!-- +(Note: BNT does not currently support spherical covariances, although it +would be easy to add, since KPMstats/clg_Mstep supports this option; +you would just need to modify gaussian_CPD/update_ess to accumulate +weighted inner products.) +--> + + +<h2><a name="tying">Parameter tying</h2> + +In networks with repeated structure (e.g., chains and grids), it is +common to assume that the parameters are the same at every node. This +is called parameter tying, and reduces the amount of data needed for +learning. +<p> +When we have tied parameters, there is no longer a one-to-one +correspondence between nodes and CPDs. +Rather, each CPD species the parameters for a whole equivalence class +of nodes. +It is easiest to see this by example. +Consider the following <a href="usage_dbn.html#hmm">hidden Markov +model (HMM)</a> +<p> +<img src="Figures/hmm3.gif"> +<p> +<!-- +We can create this graph structure, assuming we have T time-slices, +as follows. +(We number the nodes as shown in the figure, but we could equally well +number the hidden nodes 1:T, and the observed nodes T+1:2T.) +<pre> +N = 2*T; +dag = zeros(N); +hnodes = 1:2:2*T; +for i=1:T-1 + dag(hnodes(i), hnodes(i+1))=1; +end +onodes = 2:2:2*T; +for i=1:T + dag(hnodes(i), onodes(i)) = 1; +end +</pre> +<p> +The hidden nodes are always discrete, and have Q possible values each, +but the observed nodes can be discrete or continuous, and have O possible values/length. +<pre> +if cts_obs + dnodes = hnodes; +else + dnodes = 1:N; +end +ns = ones(1,N); +ns(hnodes) = Q; +ns(onodes) = O; +</pre> +--> +When HMMs are used for semi-infinite processes like speech recognition, +we assume the transition matrix +P(H(t+1)|H(t)) is the same for all t; this is called a time-invariant +or homogenous Markov chain. +Hence hidden nodes 2, 3, ..., T +are all in the same equivalence class, say class Hclass. +Similarly, the observation matrix P(O(t)|H(t)) is assumed to be the +same for all t, so the observed nodes are all in the same equivalence +class, say class Oclass. +Finally, the prior term P(H(1)) is in a class all by itself, say class +H1class. +This is illustrated below, where we explicitly represent the +parameters as random variables (dotted nodes). +<p> +<img src="Figures/hmm4_params.gif"> +<p> +In BNT, we cannot represent parameters as random variables (nodes). +Instead, we "hide" the +parameters inside one CPD for each equivalence class, +and then specify that the other CPDs should share these parameters, as +follows. +<pre> +hnodes = 1:2:2*T; +onodes = 2:2:2*T; +H1class = 1; Hclass = 2; Oclass = 3; +eclass = ones(1,N); +eclass(hnodes(2:end)) = Hclass; +eclass(hnodes(1)) = H1class; +eclass(onodes) = Oclass; +% create dag and ns in the usual way +bnet = mk_bnet(dag, ns, 'discrete', dnodes, 'equiv_class', eclass); +</pre> +Finally, we define the parameters for each equivalence class: +<pre> +bnet.CPD{H1class} = tabular_CPD(bnet, hnodes(1)); % prior +bnet.CPD{Hclass} = tabular_CPD(bnet, hnodes(2)); % transition matrix +if cts_obs + bnet.CPD{Oclass} = gaussian_CPD(bnet, onodes(1)); +else + bnet.CPD{Oclass} = tabular_CPD(bnet, onodes(1)); +end +</pre> +In general, if bnet.CPD{e} = xxx_CPD(bnet, j), then j should be a +member of e's equivalence class; that is, it is not always the case +that e == j. You can use bnet.rep_of_eclass(e) to return the +representative of equivalence class e. +BNT will look up the parents of j to determine the size +of the CPT to use. It assumes that this is the same for all members of +the equivalence class. +Click <a href="http://www.cs.berkeley.edu/~murphyk/Bayes/param_tieing.html">here</a> for +a more complex example of parameter tying. +<p> +Note: +Normally one would define an HMM as a +<a href = "usage_dbn.html">Dynamic Bayes Net</a> +(see the function BNT/examples/dynamic/mk_chmm.m). +However, one can define an HMM as a static BN using the function +BNT/examples/static/Models/mk_hmm_bnet.m. + + + +<h1><a name="structure_learning">Structure learning</h1> + +Update (9/29/03): +Phillipe LeRay is developing some additional structure learning code +on top of BNT. Click <a +href="http://bnt.insa-rouen.fr/ajouts.html">here</a> +for details. + +<p> + +There are two very different approaches to structure learning: +constraint-based and search-and-score. +In the <a href="#constraint">constraint-based approach</a>, +we start with a fully connected graph, and remove edges if certain +conditional independencies are measured in the data. +This has the disadvantage that repeated independence tests lose +statistical power. +<p> +In the more popular search-and-score approach, +we perform a search through the space of possible DAGs, and either +return the best one found (a point estimate), or return a sample of the +models found (an approximation to the Bayesian posterior). +<p> +Unfortunately, the number of DAGs as a function of the number of +nodes, G(n), is super-exponential in n. +A closed form formula for G(n) is not known, but the first few values +are shown below (from Cooper, 1999). + +<table> +<tr> <th>n</th> <th align=left>G(n)</th> </tr> +<tr> <td>1</td> <td>1</td> </tr> +<tr> <td>2</td> <td>3</td> </tr> +<tr> <td>3</td> <td>25</td> </tr> +<tr> <td>4</td> <td>543</td> </tr> +<tr> <td>5</td> <td>29,281</td> </tr> +<tr> <td>6</td> <td>3,781,503</td> </tr> +<tr> <td>7</td> <td>1.1 x 10^9</td> </tr> +<tr> <td>8</td> <td>7.8 x 10^11</td> </tr> +<tr> <td>9</td> <td>1.2 x 10^15</td> </tr> +<tr> <td>10</td> <td>4.2 x 10^18</td> </tr> +</table> + +Since the number of DAGs is super-exponential in the number of nodes, +we cannot exhaustively search the space, so we either use a local +search algorithm (e.g., greedy hill climbining, perhaps with multiple +restarts) or a global search algorithm (e.g., Markov Chain Monte +Carlo). +<p> +If we know a total ordering on the nodes, +finding the best structure amounts to picking the best set of parents +for each node independently. +This is what the K2 algorithm does. +If the ordering is unknown, we can search over orderings, +which is more efficient than searching over DAGs (Koller and Friedman, 2000). +<p> +In addition to the search procedure, we must specify the scoring +function. There are two popular choices. The Bayesian score integrates +out the parameters, i.e., it is the marginal likelihood of the model. +The BIC (Bayesian Information Criterion) is defined as +log P(D|theta_hat) - 0.5*d*log(N), where D is the data, theta_hat is +the ML estimate of the parameters, d is the number of parameters, and +N is the number of data cases. +The BIC method has the advantage of not requiring a prior. +<p> +BIC can be derived as a large sample +approximation to the marginal likelihood. +(It is also equal to the Minimum Description Length of a model.) +However, in practice, the sample size does not need to be very large +for the approximation to be good. +For example, in the figure below, we plot the ratio between the log marginal likelihood +and the BIC score against data-set size; we see that the ratio rapidly +approaches 1, especially for non-informative priors. +(This plot was generated by the file BNT/examples/static/bic1.m. It +uses the water sprinkler BN with BDeu Dirichlet priors with different +equivalent sample sizes.) + +<p> +<center> +<IMG SRC="Figures/bic.png"> +</center> +<p> + +<p> +As with parameter learning, handling missing data/ hidden variables is +much harder than the fully observed case. +The structure learning routines in BNT can therefore be classified into 4 +types, analogously to the parameter learning case. +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_struct_K2</tt> <br> +<!-- <tt>learn_struct_hill_climb</tt></td> --> + <td><tt>not yet supported</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>learn_struct_mcmc</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="markov_equiv">Markov equivalence</h2> + +If two DAGs encode the same conditional independencies, they are +called Markov equivalent. The set of all DAGs can be paritioned into +Markov equivalence classes. Graphs within the same class can +have +the direction of some of their arcs reversed without changing any of +the CI relationships. +Each class can be represented by a PDAG +(partially directed acyclic graph) called an essential graph or +pattern. This specifies which edges must be oriented in a certain +direction, and which may be reversed. + +<p> +When learning graph structure from observational data, +the best one can hope to do is to identify the model up to Markov +equivalence. To distinguish amongst graphs within the same equivalence +class, one needs interventional data: see the discussion on <a +href="#active">active learning</a> below. + + + +<h2><a name="enumerate">Exhaustive search</h2> + +The brute-force approach to structure learning is to enumerate all +possible DAGs, and score each one. This provides a "gold standard" +with which to compare other algorithms. We can do this as follows. +<pre> +dags = mk_all_dags(N); +score = score_dags(data, ns, dags); +</pre> +where data(i,m) is the value of node i in case m, +and ns(i) is the size of node i. +If the DAGs have a lot of families in common, we can cache the sufficient statistics, +making this potentially more efficient than scoring the DAGs one at a time. +(Caching is not currently implemented, however.) +<p> +By default, we use the Bayesian scoring metric, and assume CPDs are +represented by tables with BDeu(1) priors. +We can override these defaults as follows. +If we want to use uniform priors, we can say +<pre> +params = cell(1,N); +for i=1:N + params{i} = {'prior', 'unif'}; +end +score = score_dags(data, ns, dags, 'params', params); +</pre> +params{i} is a cell-array, containing optional arguments that are +passed to the constructor for CPD i. +<p> +Now suppose we want to use different node types, e.g., +Suppose nodes 1 and 2 are Gaussian, and nodes 3 and 4 softmax (both +these CPDs can support discrete and continuous parents, which is +necessary since all other nodes will be considered as parents). +The Bayesian scoring metric currently only works for tabular CPDs, so +we will use BIC: +<pre> +score = score_dags(data, ns, dags, 'discrete', [3 4], 'params', [], + 'type', {'gaussian', 'gaussian', 'softmax', softmax'}, 'scoring_fn', 'bic') +</pre> +In practice, one can't enumerate all possible DAGs for N > 5, +but one can evaluate any reasonably-sized set of hypotheses in this +way (e.g., nearest neighbors of your current best guess). +Think of this as "computer assisted model refinement" as opposed to de +novo learning. + + +<h2><a name="K2">K2</h2> + +The K2 algorithm (Cooper and Herskovits, 1992) is a greedy search algorithm that works as follows. +Initially each node has no parents. It then adds incrementally that parent whose addition most +increases the score of the resulting structure. When the addition of no single +parent can increase the score, it stops adding parents to the node. +Since we are using a fixed ordering, we do not need to check for +cycles, and can choose the parents for each node independently. +<p> +The original paper used the Bayesian scoring +metric with tabular CPDs and Dirichlet priors. +BNT generalizes this to allow any kind of CPD, and either the Bayesian +scoring metric or BIC, as in the example <a href="#enumerate">above</a>. +In addition, you can specify +an optional upper bound on the number of parents for each node. +The file BNT/examples/static/k2demo1.m gives an example of how to use K2. +We use the water sprinkler network and sample 100 cases from it as before. +Then we see how much data it takes to recover the generating structure: +<pre> +order = [C S R W]; +max_fan_in = 2; +sz = 5:5:100; +for i=1:length(sz) + dag2 = learn_struct_K2(data(:,1:sz(i)), node_sizes, order, 'max_fan_in', max_fan_in); + correct(i) = isequal(dag, dag2); +end +</pre> +Here are the results. +<pre> +correct = + Columns 1 through 12 + 0 0 0 0 0 0 0 1 0 1 1 1 + Columns 13 through 20 + 1 1 1 1 1 1 1 1 +</pre> +So we see it takes about sz(10)=50 cases. (BIC behaves similarly, +showing that the prior doesn't matter too much.) +In general, we cannot hope to recover the "true" generating structure, +only one that is in its <a href="#markov_equiv">Markov equivalence +class</a>. + + +<h2><a name="hill_climb">Hill-climbing</h2> + +Hill-climbing starts at a specific point in space, +considers all nearest neighbors, and moves to the neighbor +that has the highest score; if no neighbors have higher +score than the current point (i.e., we have reached a local maximum), +the algorithm stops. One can then restart in another part of the space. +<p> +A common definition of "neighbor" is all graphs that can be +generated from the current graph by adding, deleting or reversing a +single arc, subject to the acyclicity constraint. +Other neighborhoods are possible: see +<a href="http://research.microsoft.com/~dmax/publications/jmlr02.pdf"> +Optimal Structure Identification with Greedy Search</a>, Max +Chickering, JMLR 2002. + +<!-- +Note: This algorithm is currently (Feb '02) being implemented by Qian +Diao. +--> + + +<h2><a name="mcmc">MCMC</h2> + +We can use a Markov Chain Monte Carlo (MCMC) algorithm called +Metropolis-Hastings (MH) to search the space of all +DAGs. +The standard proposal distribution is to consider moving to all +nearest neighbors in the sense defined <a href="#hill_climb">above</a>. +<p> +The function can be called +as in the following example. +<pre> +[sampled_graphs, accept_ratio] = learn_struct_mcmc(data, ns, 'nsamples', 100, 'burnin', 10); +</pre> +We can convert our set of sampled graphs to a histogram +(empirical posterior over all the DAGs) thus +<pre> +all_dags = mk_all_dags(N); +mcmc_post = mcmc_sample_to_hist(sampled_graphs, all_dags); +</pre> +To see how well this performs, let us compute the exact posterior exhaustively. +<p> +<pre> +score = score_dags(data, ns, all_dags); +post = normalise(exp(score)); % assuming uniform structural prior +</pre> +We plot the results below. +(The data set was 100 samples drawn from a random 4 node bnet; see the +file BNT/examples/static/mcmc1.) +<pre> +subplot(2,1,1) +bar(post) +subplot(2,1,2) +bar(mcmc_post) +</pre> +<img src="Figures/mcmc_post.jpg" width="800" height="500"> +<p> +We can also plot the acceptance ratio versus number of MCMC steps, +as a crude convergence diagnostic. +<pre> +clf +plot(accept_ratio) +</pre> +<img src="Figures/mcmc_accept.jpg" width="800" height="300"> +<p> +Better convergence diagnostics can be found +<a href="<a href="http://www.lce.hut.fi/~ave/code/mcmcdiag/">here</a> +<p> +Even though the number of samples needed by MCMC is theoretically +polynomial (not exponential) in the dimensionality of the search space, in practice it has been +found that MCMC does not converge in reasonable time for graphs with +more than about 10 nodes. +<p> +<a href="http://www.bioss.sari.ac.uk/~dirk/software/DBmcmc/"> +Dirk Husmeier has extended MCMC model selection to DBNs</a>. + + +<h2><a name="active">Active structure learning</h2> + +As was mentioned <a href="#markov_equiv">above</a>, +one can only learn a DAG up to Markov equivalence, even given infinite data. +If one is interested in learning the structure of a causal network, +one needs interventional data. +(By "intervention" we mean forcing a node to take on a specific value, +thereby effectively severing its incoming arcs.) +<p> +Most of the scoring functions accept an optional argument +that specifies whether a node was observed to have a certain value, or +was forced to have that value: we set clamped(i,m)=1 if node i was +forced in training case m. e.g., see the file +BNT/examples/static/cooper_yoo. +<p> +An interesting question is to decide which interventions to perform +(c.f., design of experiments). For details, see the following tech +report +<ul> +<li> <a href = "../../Papers/alearn.ps.gz"> +Active learning of causal Bayes net structure</a>, Kevin Murphy, March +2001. +</ul> + + +<h2><a name="struct_em">Structural EM</h2> + +Computing the Bayesian score when there is partial observability is +computationally challenging, because the parameter posterior becomes +multimodal (the hidden nodes induce a mixture distribution). +One therefore needs to use approximations such as BIC. +Unfortunately, search algorithms are still expensive, because we need +to run EM at each step to compute the MLE, which is needed to compute +the score of each model. An alternative approach is +to do the local search steps inside of the M step of EM, which is more +efficient since the data has been "filled in" - this is +called the structural EM algorithm (Friedman 1997), and provably +converges to a local maximum of the BIC score. +<p> +Wei Hu has implemented SEM for discrete nodes. +You can download his package from +<a href="../SEM.zip">here</a>. +Please address all questions about this code to +wei.hu@intel.com. +See also <a href="#phl">Phl's implementation of SEM</a>. + +<!-- +<h2><a name="reveal">REVEAL algorithm</h2> + +A simple way to learn the structure of a fully observed, discrete, +factored DBN from a time series is described <a +href="usage_dbn.html#struct_learn">here</a>. +--> + + +<h2><a name="graphdraw">Visualizing the graph</h2> + +You can visualize an arbitrary graph (such as one learned using the +structure learning routines) with Matlab code contributed by +<a href="http://www.mbfys.kun.nl/~cemgil/matlab/layout.html">Ali +Taylan Cemgil</a> +from the University of Nijmegen. +For static BNs, call it as follows: +<pre> +draw_graph(bnet.dag); +</pre> +For example, this is the output produced on a +<a href="#qmr">random QMR-like model</a>: +<p> +<img src="Figures/qmr.rnd.jpg"> +<p> +If you install the excellent <a +href="http://www.research.att.com/sw/tools/graphviz">graphhviz</a>, an +open-source graph visualization package from AT&T, +you can create a much better visualization as follows +<pre> +graph_to_dot(bnet.dag) +</pre> +This works by converting the adjacency matrix to a file suitable +for input to graphviz (using the dot format), +then converting the output of graphviz to postscript, and displaying the results using +ghostview. +You can do each of these steps separately for more control, as shown +below. +<pre> +graph_to_dot(bnet.dag, 'filename', 'foo.dot'); +dot -Tps foo.dot -o foo.ps +ghostview foo.ps & +</pre> + +<h2><a name = "constraint">Constraint-based methods</h2> + +The IC algorithm (Pearl and Verma, 1991), +and the faster, but otherwise equivalent, PC algorithm (Spirtes, Glymour, and Scheines 1993), +computes many conditional independence tests, +and combines these constraints into a +PDAG to represent the whole +<a href="#markov_equiv">Markov equivalence class</a>. +<p> +IC*/FCI extend IC/PC to handle latent variables: see <a href="#ic_star">below</a>. +(IC stands for inductive causation; PC stands for Peter and Clark, +the first names of Spirtes and Glymour; FCI stands for fast causal +inference. +What we, following Pearl (2000), call IC* was called +IC in the original Pearl and Verma paper.) +For details, see +<ul> +<li> +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Causation, +Prediction, and Search</a>, Spirtes, Glymour and +Scheines (SGS), 2001 (2nd edition), MIT Press. +<li> +<a href="http://bayes.cs.ucla.edu/BOOK-2K/index.html">Causality: Models, Reasoning and Inference</a>, J. Pearl, +2000, Cambridge University Press. +</ul> + +<p> + +The PC algorithm takes as arguments a function f, the number of nodes N, +the maximum fan in K, and additional arguments A which are passed to f. +The function f(X,Y,S,A) returns 1 if X is conditionally independent of Y given S, and 0 +otherwise. +For example, suppose we cheat by +passing in a CI "oracle" which has access to the true DAG; the oracle +tests for d-separation in this DAG, i.e., +f(X,Y,S) calls dsep(X,Y,S,dag). We can to this as follows. +<pre> +pdag = learn_struct_pdag_pc('dsep', N, max_fan_in, dag); +</pre> +pdag(i,j) = -1 if there is definitely an i->j arc, +and pdag(i,j) = 1 if there is either an i->j or and i<-j arc. +<p> +Applied to the sprinkler network, this returns +<pre> +pdag = + 0 1 1 0 + 1 0 0 -1 + 1 0 0 -1 + 0 0 0 0 +</pre> +So as expected, we see that the V-structure at the W node is uniquely identified, +but the other arcs have ambiguous orientation. +<p> +We now give an example from p141 (1st edn) / p103 (2nd end) of the SGS +book. +This example concerns the female orgasm. +We are given a correlation matrix C between 7 measured factors (such +as subjective experiences of coital and masturbatory experiences), +derived from 281 samples, and want to learn a causal model of the +data. We will not discuss the merits of this type of work here, but +merely show how to reproduce the results in the SGS book. +Their program, +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Tetrad</a>, +makes use of the Fisher Z-test for conditional +independence, so we do the same: +<pre> +max_fan_in = 4; +nsamples = 281; +alpha = 0.05; +pdag = learn_struct_pdag_pc('cond_indep_fisher_z', n, max_fan_in, C, nsamples, alpha); +</pre> +In this case, the CI test is +<pre> +f(X,Y,S) = cond_indep_fisher_z(X,Y,S, C,nsamples,alpha) +</pre> +The results match those of Fig 12a of SGS apart from two edge +differences; presumably this is due to rounding error (although it +could be a bug, either in BNT or in Tetrad). +This example can be found in the file BNT/examples/static/pc2.m. + +<p> + +The IC* algorithm (Pearl and Verma, 1991), +and the faster FCI algorithm (Spirtes, Glymour, and Scheines 1993), +are like the IC/PC algorithm, except that they can detect the presence +of latent variables. +See the file <tt>learn_struct_pdag_ic_star</tt> written by Tamar +Kushnir. The output is a matrix P, defined as follows +(see Pearl (2000), p52 for details): +<pre> +% P(i,j) = -1 if there is either a latent variable L such that i <-L->j OR there is a directed edge from i->j. +% P(i,j) = -2 if there is a marked directed i-*>j edge. +% P(i,j) = P(j,i) = 1 if there is and undirected edge i--j +% P(i,j) = P(j,i) = 2 if there is a latent variable L such that i<-L->j. +</pre> + + +<h2><a name="phl">Philippe Leray's structure learning package</h2> + +Philippe Leray has written a +<a href="http://bnt.insa-rouen.fr/ajouts.html"> +structure learning package</a> that uses BNT. + +It currently (Juen 2003) has the following features: +<ul> +<li>PC with Chi2 statistical test +<li> MWST : Maximum weighted Spanning Tree +<li> Hill Climbing +<li> Greedy Search +<li> Structural EM +<li> hist_ic : optimal Histogram based on IC information criterion +<li> cpdag_to_dag +<li> dag_to_cpdag +<li> ... +</ul> + + +</a> + + +<!-- +<h2><a name="read_learning">Further reading on learning</h2> + +I recommend the following tutorials for more details on learning. +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/intel.ps.gz">My short +tutorial</a> on graphical models, which contains an overview of learning. + +<li> +<A HREF="ftp://ftp.research.microsoft.com/pub/tr/TR-95-06.PS"> +A tutorial on learning with Bayesian networks</a>, D. Heckerman, +Microsoft Research Tech Report, 1995. + +<li> <A HREF="http://www-cad.eecs.berkeley.edu/~wray/Mirror/lwgmja"> +Operations for Learning with Graphical Models</a>, +W. L. Buntine, JAIR'94, 159--225. +</ul> +<p> +--> + + + + + +<h1><a name="engines">Inference engines</h1> + +Up until now, we have used the junction tree algorithm for inference. +However, sometimes this is too slow, or not even applicable. +In general, there are many inference algorithms each of which make +different tradeoffs between speed, accuracy, complexity and +generality. Furthermore, there might be many implementations of the +same algorithm; for instance, a general purpose, readable version, +and a highly-optimized, specialized one. +To cope with this variety, we treat each inference algorithm as an +object, which we call an inference engine. + +<p> +An inference engine is an object that contains a bnet and supports the +'enter_evidence' and 'marginal_nodes' methods. The engine constructor +takes the bnet as argument and may do some model-specific processing. +When 'enter_evidence' is called, the engine may do some +evidence-specific processing. Finally, when 'marginal_nodes' is +called, the engine may do some query-specific processing. + +<p> +The amount of work done when each stage is specified -- structure, +parameters, evidence, and query -- depends on the engine. The cost of +work done early in this sequence can be amortized. On the other hand, +one can make better optimizations if one waits until later in the +sequence. +For example, the parameters might imply +conditional indpendencies that are not evident in the graph structure, +but can nevertheless be exploited; the evidence indicates which nodes +are observed and hence can effectively be disconnected from the +graph; and the query might indicate that large parts of the network +are d-separated from the query nodes. (Since it is not the actual +<em>values</em> of the evidence that matters, just which nodes are observed, +many engines allow you to specify which nodes will be observed when they are constructed, +i.e., before calling 'enter_evidence'. Some engines can still cope if +the actual pattern of evidence is different, e.g., if there is missing +data.) +<p> + +Although being maximally lazy (i.e., only doing work when a query is +issued) may seem desirable, +this is not always the most efficient. +For example, +when learning using EM, we need to call marginal_nodes N times, where N is the +number of nodes. <a href="varelim">Variable elimination</a> would end +up repeating a lot of work +each time marginal_nodes is called, making it inefficient for +learning. The junction tree algorithm, by contrast, uses dynamic +programming to avoid this redundant computation --- it calculates all +marginals in two passes during 'enter_evidence', so calling +'marginal_nodes' takes constant time. +<p> +We will discuss some of the inference algorithms implemented in BNT +below, and finish with a <a href="#engine_summary">summary</a> of all +of them. + + + + + + + +<h2><a name="varelim">Variable elimination</h2> + +The variable elimination algorithm, also known as bucket elimination +or peeling, is one of the simplest inference algorithms. +The basic idea is to "push sums inside of products"; this is explained +in more detail +<a +href="http://HTTP.CS.Berkeley.EDU/~murphyk/Bayes/bayes.html#infer">here</a>. +<p> +The principle of distributing sums over products can be generalized +greatly to apply to any commutative semiring. +This forms the basis of many common algorithms, such as Viterbi +decoding and the Fast Fourier Transform. For details, see + +<ul> +<li> R. McEliece and S. M. Aji, 2000. +<!--<a href="http://www.systems.caltech.edu/EE/Faculty/rjm/papers/GDL.ps">--> +<a href="GDL.pdf"> +The Generalized Distributive Law</a>, +IEEE Trans. Inform. Theory, vol. 46, no. 2 (March 2000), +pp. 325--343. + + +<li> +F. R. Kschischang, B. J. Frey and H.-A. Loeliger, 2001. +<a href="http://www.cs.toronto.edu/~frey/papers/fgspa.abs.html"> +Factor graphs and the sum-product algorithm</a> +IEEE Transactions on Information Theory, February, 2001. + +</ul> + +<p> +Choosing an order in which to sum out the variables so as to minimize +computational cost is known to be NP-hard. +The implementation of this algorithm in +<tt>var_elim_inf_engine</tt> makes no attempt to optimize this +ordering (in contrast, say, to <tt>jtree_inf_engine</tt>, which uses a +greedy search procedure to find a good ordering). +<p> +Note: unlike most algorithms, var_elim does all its computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + + + +<h2><a name="global">Global inference methods</h2> + +The simplest inference algorithm of all is to explicitely construct +the joint distribution over all the nodes, and then to marginalize it. +This is implemented in <tt>global_joint_inf_engine</tt>. +Since the size of the joint is exponential in the +number of discrete (hidden) nodes, this is not a very practical algorithm. +It is included merely for pedagogical and debugging purposes. +<p> +Three specialized versions of this algorithm have also been implemented, +corresponding to the cases where all the nodes are discrete (D), all +are Gaussian (G), and some are discrete and some Gaussian (CG). +They are called <tt>enumerative_inf_engine</tt>, +<tt>gaussian_inf_engine</tt>, +and <tt>cond_gauss_inf_engine</tt> respectively. +<p> +Note: unlike most algorithms, these global inference algorithms do all their computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + +<h2><a name="quickscore">Quickscore</h2> + +The junction tree algorithm is quite slow on the <a href="#qmr">QMR</a> network, +since the cliques are so big. +One simple trick we can use is to notice that hidden leaves do not +affect the posteriors on the roots, and hence do not need to be +included in the network. +A second trick is to notice that the negative findings can be +"absorbed" into the prior: +see the file +BNT/examples/static/mk_minimal_qmr_bnet for details. +<p> + +A much more significant speedup is obtained by exploiting special +properties of the noisy-or node, as done by the quickscore +algorithm. For details, see +<ul> +<li> Heckerman, "A tractable inference algorithm for diagnosing multiple diseases", UAI 89. +<li> Rish and Dechter, "On the impact of causal independence", UCI +tech report, 1998. +</ul> + +This has been implemented in BNT as a special-purpose inference +engine, which can be created and used as follows: +<pre> +engine = quickscore_inf_engine(inhibit, leak, prior); +engine = enter_evidence(engine, pos, neg); +m = marginal_nodes(engine, i); +</pre> + + +<h2><a name="belprop">Belief propagation</h2> + +Even using quickscore, exact inference takes time that is exponential +in the number of positive findings. +Hence for large networks we need to resort to approximate inference techniques. +See for example +<ul> +<li> T. Jaakkola and M. Jordan, "Variational probabilistic inference and the +QMR-DT network", JAIR 10, 1999. + +<li> K. Murphy, Y. Weiss and M. Jordan, "Loopy belief propagation for approximate inference: an empirical study", + UAI 99. +</ul> +The latter approximation +entails applying Pearl's belief propagation algorithm to a model even +if it has loops (hence the name loopy belief propagation). +Pearl's algorithm, implemented as <tt>pearl_inf_engine</tt>, gives +exact results when applied to singly-connected graphs +(a.k.a. polytrees, since +the underlying undirected topology is a tree, but a node may have +multiple parents). +To apply this algorithm to a graph with loops, +use <tt>pearl_inf_engine</tt>. +This can use a centralized or distributed message passing protocol. +You can use it as in the following example. +<pre> +engine = pearl_inf_engine(bnet, 'max_iter', 30); +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, i); +</pre> +We found that this algorithm often converges, and when it does, often +is very accurate, but it depends on the precise setting of the +parameter values of the network. +(See the file BNT/examples/static/qmr1 to repeat the experiment for yourself.) +Understanding when and why belief propagation converges/ works +is a topic of ongoing research. +<p> +<tt>pearl_inf_engine</tt> can exploit special structure in noisy-or +and gmux nodes to compute messages efficiently. +<p> +<tt>belprop_inf_engine</tt> is like pearl, but uses potentials to +represent messages. Hence this is slower. +<p> +<tt>belprop_fg_inf_engine</tt> is like belprop, +but is designed for factor graphs. + + + +<h2><a name="sampling">Sampling</h2> + +BNT now (Mar '02) has two sampling (Monte Carlo) inference algorithms: +<ul> +<li> <tt>likelihood_weighting_inf_engine</tt> which does importance +sampling and can handle any node type. +<li> <tt>gibbs_sampling_inf_engine</tt>, written by Bhaskara Marthi. +Currently this can only handle tabular CPDs. +For a much faster and more powerful Gibbs sampling program, see +<a href="http://www.mrc-bsu.cam.ac.uk/bugs">BUGS</a>. +</ul> +Note: To generate samples from a network (which is not the same as inference!), +use <tt>sample_bnet</tt>. + + + +<h2><a name="engine_summary">Summary of inference engines</h2> + + +The inference engines differ in many ways. Here are +some of the major "axes": +<ul> +<li> Works for all topologies or makes restrictions? +<li> Works for all node types or makes restrictions? +<li> Exact or approximate inference? +</ul> + +<p> +In terms of topology, most engines handle any kind of DAG. +<tt>belprop_fg</tt> does approximate inference on factor graphs (FG), which +can be used to represent directed, undirected, and mixed (chain) +graphs. +(In the future, we plan to support exact inference on chain graphs.) +<tt>quickscore</tt> only works on QMR-like models. +<p> +In terms of node types: algorithms that use potentials can handle +discrete (D), Gaussian (G) or conditional Gaussian (CG) models. +Sampling algorithms can essentially handle any kind of node (distribution). +Other algorithms make more restrictive assumptions in exchange for +speed. +<p> +Finally, most algorithms are designed to give the exact answer. +The belief propagation algorithms are exact if applied to trees, and +in some other cases. +Sampling is considered approximate, even though, in the limit of an +infinite number of samples, it gives the exact answer. + +<p> + +Here is a summary of the properties +of all the engines in BNT which work on static networks. +<p> +<table> +<table border units = pixels><tr> +<td align=left width=0>Name +<td align=left width=0>Exact? +<td align=left width=0>Node type? +<td align=left width=0>topology +<tr> +<tr> +<td align=left> belprop +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> belprop_fg +<td align=left> approx +<td align=left> D +<td align=left> factor graph +<tr> +<td align=left> cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> enumerative +<td align=left> exact +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> gaussian +<td align=left> exact +<td align=left> G +<td align=left> DAG +<tr> +<td align=left> gibbs +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> global_joint +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +<tr> +<td align=left> jtree +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +b<tr> +<td align=left> likelihood_weighting +<td align=left> approx +<td align=left> any +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> approx +<td align=left> D,G +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> exact +<td align=left> D,G +<td align=left> polytree +<tr> +<td align=left> quickscore +<td align=left> exact +<td align=left> noisy-or +<td align=left> QMR +<tr> +<td align=left> stab_cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> var_elim +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +</table> + + + +<h1><a name="influence">Influence diagrams/ decision making</h1> + +BNT implements an exact algorithm for solving LIMIDs (limited memory +influence diagrams), described in +<ul> +<li> S. L. Lauritzen and D. Nilsson. +<a href="http://www.math.auc.dk/~steffen/papers/limids.pdf"> +Representing and solving decision problems with limited +information</a> +Management Science, 47, 1238 - 1251. September 2001. +</ul> +LIMIDs explicitely show all information arcs, rather than implicitely +assuming no forgetting. This allows them to model forgetful +controllers. +<p> +See the examples in <tt>BNT/examples/limids</tt> for details. + + + + +<h1>DBNs, HMMs, Kalman filters and all that</h1> + +Click <a href="usage_dbn.html">here</a> for documentation about how to +use BNT for dynamical systems and sequence data. + + +</BODY> diff --git a/sourcecodes/bnt-master/docs/usage_dbn.html b/sourcecodes/bnt-master/docs/usage_dbn.html new file mode 100644 index 00000000..7582a0cb --- /dev/null +++ b/sourcecodes/bnt-master/docs/usage_dbn.html @@ -0,0 +1,719 @@ +<HEAD> +<TITLE>How to use BNT for DBNs</TITLE> +</HEAD> + +<BODY BGCOLOR="#FFFFFF"> +<!-- white background is better for the pictures and equations --> + +Documentation last updated on 7 June 2004 + +<h1>How to use BNT for DBNs</h1> + +<p> +<ul> +<li> <a href="#spec">Model specification</a> +<ul> +<li> <a href="#hmm">HMMs</a> +<li> <a href="#lds">Kalman filters</a> +<li> <a href="#chmm">Coupled HMMs</a> +<li> <a href="#water">Water network</a> +<li> <a href="#bat">BAT network</a> +</ul> + +<li> <a href="#inf">Inference</a> +<ul> +<li> <a href="#discrete">Discrete hidden nodes</a> +<li> <a href="#cts">Continuous hidden nodes</a> +</ul> + +<li> <a href="#learn">Learning</a> +<ul> +<li> <a href="#param_learn">Parameter learning</a> +<li> <a href="#struct_learn">Structure learning</a> +</ul> + +</ul> + +Note: +you are recommended to read an introduction +to DBNs first, such as +<a href="http://www.ai.mit.edu/~murphyk/Papers/dbnchapter.pdf"> +this book chapter</a>. +<br> +You may also want to consider using +<a href=http://ssli.ee.washington.edu/~bilmes/gmtk/>GMTk</a>, which is +an excellent C++ package for DBNs. + + +<h1><a name="spec">Model specification</h1> + + +<!--<h1><a name="dbn_intro">Dynamic Bayesian Networks (DBNs)</h1>--> + +Dynamic Bayesian Networks (DBNs) are directed graphical models of stochastic +processes. +They generalise <a href="#hmm">hidden Markov models (HMMs)</a> +and <a href="#lds">linear dynamical systems (LDSs)</a> +by representing the hidden (and observed) state in terms of state +variables, which can have complex interdependencies. +The graphical structure provides an easy way to specify these +conditional independencies, and hence to provide a compact +parameterization of the model. +<p> +Note that "temporal Bayesian network" would be a better name than +"dynamic Bayesian network", since +it is assumed that the model structure does not change, but +the term DBN has become entrenched. +We also normally assume that the parameters do not +change, i.e., the model is time-invariant. +However, we can always add extra +hidden nodes to represent the current "regime", thereby creating +mixtures of models to capture periodic non-stationarities. +<p> +There are some cases where the size of the state space can change over +time, e.g., tracking a variable, but unknown, number of objects. +In this case, we need to change the model structure over time. +BNT does not support this. +<!-- +, but see the following paper for a +discussion of some of the issues: +<ul> +<li> <a href="ftp://ftp.cs.monash.edu.au/pub/annn/smc.ps"> +Dynamic belief networks for discrete monitoring</a>, +A. E. Nicholson and J. M. Brady. +IEEE Systems, Man and Cybernetics, 24(11):1593-1610, 1994. +</ul> +--> + + +<h2><a name="hmm">Hidden Markov Models (HMMs)</h2> + +The simplest kind of DBN is a Hidden Markov Model (HMM), which has +one discrete hidden node and one discrete or continuous +observed node per slice. We illustrate this below. +As before, circles denote continuous nodes, squares denote +discrete nodes, clear means hidden, shaded means observed. +<!-- +(The observed nodes can be +discrete or continuous; the crucial thing about an HMM is that the +hidden nodes are discrete, so the system can model arbitrary dynamics +-- providing, of course, that the hidden state space is large enough.) +--> +<p> +<img src="Figures/hmm3.gif"> +<p> +We have "unrolled" the model for three "time slices" -- the structure and parameters are +assumed to repeat as the model is unrolled further. +Hence to specify a DBN, we need to +define the intra-slice topology (within a slice), +the inter-slice topology (between two slices), +as well as the parameters for the first two slices. +(Such a two-slice temporal Bayes net is often called a 2TBN.) +<p> +We can specify the topology as follows. +<PRE> +intra = zeros(2); +intra(1,2) = 1; % node 1 in slice t connects to node 2 in slice t + +inter = zeros(2); +inter(1,1) = 1; % node 1 in slice t-1 connects to node 1 in slice t +</pre> +We can specify the parameters as follows, +where for simplicity we assume the observed node is discrete. +<pre> +Q = 2; % num hidden states +O = 2; % num observable symbols + +ns = [Q O]; +dnodes = 1:2; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes); +for i=1:4 + bnet.CPD{i} = tabular_CPD(bnet, i); +end +</pre> +<p> +We assume the distributions P(X(t) | X(t-1)) and +P(Y(t) | X(t)) are independent of t for t > 1. +Hence the CPD for nodes 5, 7, ... is the same as for node 3, so we say they +are in the same equivalence class, with node 3 being the "representative" +for this class. In other words, we have tied the parameters for nodes +3, 5, 7, ... +Similarly, nodes 4, 6, 8, ... are tied. +Note, however, that (the parameters for) nodes 1 and 2 are not tied to +subsequent slices. +<p> +Above we assumed the observation model P(Y(t) | X(t)) is independent of t for t>1, but +it is conventional to assume this is true for all t. +So we would like to put nodes 2, 4, 6, ... all in the same class. +We can do this by explicitely defining the equivalence classes, as +follows (see <a href="usage.html#tying">here</a> for more details on +parameter tying). +<p> +We define eclass1(i) to be the equivalence class that node i in slice +1 belongs to. +Similarly, we define eclass2(i) to be the equivalence class that node i in slice +2, 3, ..., belongs to. +For an HMM, we have +<pre> +eclass1 = [1 2]; +eclass2 = [3 2]; +eclass = [eclass1 eclass2]; +</pre> +This ties the observation model across slices, +since e.g., eclass(4) = eclass(2) = 2. +<p> +By default, +eclass1 = 1:ss, and eclass2 = (1:ss)+ss, where ss = slice size = the +number of nodes per slice. +<!--This will tie nodes in slices 3, 4, ... to the the nodes in slice 2, +but none of the nodes in slice 2 to any in slice 1.--> +But by using the above tieing pattern, +we now only have 3 CPDs to specify, instead of 4: +<pre> +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'eclass1', eclass1, 'eclass2', eclass2); +prior0 = normalise(rand(Q,1)); +transmat0 = mk_stochastic(rand(Q,Q)); +obsmat0 = mk_stochastic(rand(Q,O)); +bnet.CPD{1} = tabular_CPD(bnet, 1, prior0); +bnet.CPD{2} = tabular_CPD(bnet, 2, obsmat0); +bnet.CPD{3} = tabular_CPD(bnet, 3, transmat0); +</pre> +We discuss how to do <a href="#inf">inference</a> and <a href="#learn">learning</a> on this model +below. +(See also +my <a href="../HMM/hmm.html">HMM toolbox</a>, which is included with BNT.) + +<p> +Some common variants on HMMs are shown below. +BNT can handle all of these. +<p> +<center> +<table> +<tr> +<td><img src="Figures/hmm_gauss.gif"> +<td><img src="Figures/hmm_mixgauss.gif" +<td><img src="Figures/hmm_ar.gif"> +<tr> +<td><img src="Figures/hmm_factorial.gif"> +<td><img src="Figures/hmm_coupled.gif" +<td><img src="Figures/hmm_io.gif"> +<tr> +</table> +</center> + + + +<h2><a name="lds">Linear Dynamical Systems (LDSs) and Kalman filters</h2> + +A Linear Dynamical System (LDS) has the same topology as an HMM, but +all the nodes are assumed to have linear-Gaussian distributions, i.e., +<pre> + x(t+1) = A*x(t) + w(t), w ~ N(0, Q), x(0) ~ N(init_x, init_V) + y(t) = C*x(t) + v(t), v ~ N(0, R) +</pre> +Some simple variants are shown below. +<p> +<center> +<table> +<tr> +<td><img src="Figures/ar1.gif"> +<td><img src="Figures/sar.gif"> +<td><img src="Figures/kf.gif"> +<td><img src="Figures/skf.gif"> +</table> +</center> +<p> + +We can create a regular LDS in BNT as follows. +<pre> + +intra = zeros(2); +intra(1,2) = 1; +inter = zeros(2); +inter(1,1) = 1; +n = 2; + +X = 2; % size of hidden state +Y = 2; % size of observable state + +ns = [X Y]; +dnodes = []; +onodes = [2]; +eclass1 = [1 2]; +eclass2 = [3 2]; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'eclass1', eclass1, 'eclass2', eclass2); + +x0 = rand(X,1); +V0 = eye(X); % must be positive semi definite! +C0 = rand(Y,X); +R0 = eye(Y); +A0 = rand(X,X); +Q0 = eye(X); + +bnet.CPD{1} = gaussian_CPD(bnet, 1, 'mean', x0, 'cov', V0, 'cov_prior_weight', 0); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(Y,1), 'cov', R0, 'weights', C0, ... + 'clamp_mean', 1, 'cov_prior_weight', 0); +bnet.CPD{3} = gaussian_CPD(bnet, 3, 'mean', zeros(X,1), 'cov', Q0, 'weights', A0, ... + 'clamp_mean', 1, 'cov_prior_weight', 0); +</pre> +We discuss how to do <a href="#inf">inference</a> and <a href="#learn">learning</a> on this model +below. +(See also +my <a href="../Kalman/kalman.html">Kalman filter toolbox</a>, which is included with BNT.) +<p> + + +<h2><a name="chmm">Coupled HMMs</h2> + +Here is an example of a coupled HMM with N=5 chains, unrolled for T=3 +slices. Each hidden discrete node has a private observed Gaussian +child. +<p> +<img src="Figures/chmm5.gif"> +<p> +We can make this using the function +<pre> +Q = 2; % binary hidden nodes +discrete_obs = 0; % cts observed nodes +Y = 1; % scalar observed nodes +bnet = mk_chmm(N, Q, Y, discrete_obs); +</pre> + +<!--We will use this model <a href="#pred">below</a> to illustrate online prediction.--> + + + +<h2><a name="water">Water network</h2> + +Consider the following model +of a water purification plant, developed +by Finn V. Jensen, Uffe Kjærulff, Kristian G. Olesen, and Jan +Pedersen. +<!-- +The clear nodes represent the hidden state of the system in +factored form, and the shaded nodes represent the observations in +factored form. +--> +<!-- +(Click <a +href="http://www-nt.cs.berkeley.edu/home/nir/public_html/Repository/water.htm">here</a> +for more details on this model. +Following Boyen and Koller, we have added discrete evidence nodes.) +--> +<!-- +We have "unrolled" the model for three "time slices" -- the structure and parameters are +assumed to repeat as the model is unrolled further. +Hence to specify a DBN, we need to +define the intra-slice topology (within a slice), +the inter-slice topology (between two slices), +as well as the parameters for the first two slices. +(Such a two-slice temporal Bayes net is often called a 2TBN.) +--> +<p> +<center> +<IMG SRC="Figures/water3_75.gif"> +</center> +We now show how to specify this model in BNT. +<pre> +ss = 12; % slice size +intra = zeros(ss); +intra(1,9) = 1; +intra(3,10) = 1; +intra(4,11) = 1; +intra(8,12) = 1; + +inter = zeros(ss); +inter(1, [1 3]) = 1; % node 1 in slice 1 connects to nodes 1 and 3 in slice 2 +inter(2, [2 3 7]) = 1; +inter(3, [3 4 5]) = 1; +inter(4, [3 4 6]) = 1; +inter(5, [3 5 6]) = 1; +inter(6, [4 5 6]) = 1; +inter(7, [7 8]) = 1; +inter(8, [6 7 8]) = 1; + +onodes = 9:12; % observed +dnodes = 1:ss; % discrete +ns = 2*ones(1,ss); % binary nodes +eclass1 = 1:12; +eclass2 = [13:20 9:12]; +eclass = [eclass1 eclass2]; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'eclass1', eclass1, 'eclass2', eclass2); +for e=1:max(eclass) + bnet.CPD{e} = tabular_CPD(bnet, e); +end +</pre> +We have tied the observation parameters across all slices. +Click <a href="param_tieing.html">here</a> for a more complex example +of parameter tieing. + +<!-- +Let X(i,t) denote the i'th hidden node in slice t, +and Y(i,y) denote the i'th observed node in slice t. +We also use the notation Nj to refer to the j'th node in the +unrolled network, e.g., N25 = X(1,3), N33 = Y(1,3). +<p> +We assume the distributions P(X(i,t) | X(i,t-1)) and +P(Y(i,t) | X(i,t)) are independent of t for t > 1 and for all i. +Hence the CPD for N25, N37, ... is the same as for N13, so we say they +are in the same equivalence class, with N13 being the "representative" +for this class. In other words, we have tied the parameters for nodes +N13, N25, N37, ... +Note, however, that the parameters for the nodes in the first slice +are not tied, so each equivalence class for nodes 1..12 contains a +single node. +<p> +Above we assumed P(Y(i,t) | X(i,t)) is independent of t for t>1, but +it is conventional to assume this is true for all t. +So we would like to put N9, N21, N33, ... all in the same class, and +similarly for the other observed nodes. +We can do this by explicitely defining the equivalence classes, as +follows. +<p> +We define eclass1(i) to be the equivalence class that node i in slice +1 belongs to. +Similarly, we define eclass2(i) to be the equivalence class that node i in slice +2, 3, ..., belongs to. +For the water model, we have +<pre> +</pre> +This ties the observation model across slices, +since e.g., eclass(9) = eclass(21) = 9, so Y(1,1) and Y(1,2) belong to the +same class. +<p> +By default, +eclass1 = 1:ss, and eclass2 = (1:ss)+ss, where ss = slice size = the +number of nodes per slice. +This will tie nodes in slices 3, 4, ... to the the nodes in slice 2, +but none of the nodes in slice 2 to any in slice 1. +By using the above tieing pattern, +we now only have 20 CPDs to specify, instead of 24: +<pre> +bnet = mk_dbn(intra, inter, ns, dnodes, eclass1, eclass2); +for e=1:max(eclass) + bnet.CPD{e} = tabular_CPD(bnet, e); +end +</pre> +--> + + + +<h2><a name="bat">BATnet</h2> + +As an example of a more complicated DBN, consider the following +example, +which is a model of a car's high level state, as might be used by +an automated car. +(The model is from Forbes, Huang, Kanazawa and Russell, "The BATmobile: Towards a +Bayesian Automated Taxi", IJCAI 95. The figure is from +Boyen and Koller, "Tractable Inference for Complex Stochastic +Processes", UAI98. +For simplicity, we only show the observed nodes for slice 2.) +<p> +<center> +<IMG SRC="Figures/batnet.gif"> +</center> +<p> +Since this topology is so complicated, +it is useful to be able to refer to the nodes by name, instead of +number. +<pre> +names = {'LeftClr', 'RightClr', 'LatAct', ... 'Bclr', 'BYdotDiff'}; +ss = length(names); +</pre> +We can specify the intra-slice topology using a cell array as follows, +where each row specifies a connection between two named nodes: +<pre> +intrac = {... + 'LeftClr', 'LeftClrSens'; + 'RightClr', 'RightClrSens'; + ... + 'BYdotDiff', 'BcloseFast'}; +</pre> +Finally, we can convert this cell array to an adjacency matrix using +the following function: +<pre> +[intra, names] = mk_adj_mat(intrac, names, 1); +</pre> +This function also permutes the names so that they are in topological +order. +Given this ordering of the names, we can make the inter-slice +connectivity matrix as follows: +<pre> +interc = {... + 'LeftClr', 'LeftClr'; + 'LeftClr', 'LatAct'; + ... + 'FBStatus', 'LatAct'}; + +inter = mk_adj_mat(interc, names, 0); +</pre> + +To refer to a node, we must know its number, which can be computed as +in the following example: +<pre> +obs = {'LeftClrSens', 'RightClrSens', 'TurnSignalSens', 'XdotSens', 'YdotSens', 'FYdotDiffSens', ... + 'FclrSens', 'BXdotSens', 'BclrSens', 'BYdotDiffSens'}; +for i=1:length(obs) + onodes(i) = strmatch(obs{i}, names); +end +onodes = sort(onodes); +</pre> +(We sort the onodes since most BNT routines assume that set-valued +arguments are in sorted order.) +We can now make the DBN: +<pre> +dnodes = 1:ss; +ns = 2*ones(1,ss); % binary nodes +bnet = mk_dbn(intra, inter, ns, 'iscrete', dnodes); +</pre> +To specify the parameters, we must know the order of the parents. +See the function BNT/general/mk_named_CPT for a way to do this in the +case of tabular nodes. For simplicity, we just generate random +parameters: +<pre> +for i=1:2*ss + bnet.CPD{i} = tabular_CPD(bnet, i); +end +</pre> +A complete version of this example is available in BNT/examples/dynamic/bat1.m. + + + + +<h1><a name="inf">Inference</h1> + + +The general inference problem for DBNs is to compute +P(X(i,t0) | Y(:, t1:t2)), where X(i,t) represents the i'th hidden +variable at time t and Y(:,t1:t2) represents all the evidence +between times t1 and t2. +There are several special cases of interest, illustrated below. +The arrow indicates t0: it is X(t0) that we are trying to estimate. +The shaded region denotes t1:t2, the available data. +<p> + +<img src="Figures/filter.gif"> + +<p> +BNT can currently only handle offline smoothing. +(The HMM engine handles filtering and, to a limited extent, prediction.) +The usage is similar to static +inference engines, except now the evidence is a 2D cell array of +size ss*T, where ss is the number of nodes per slice (ss = slice sizee) and T is the +number of slices. +Also, 'marginal_nodes' takes two arguments, the nodes and the time-slice. +For example, to compute P(X(i,t) | y(:,1:T)), we proceed as follows +(where onodes are the indices of the observedd nodes in each slice, +which correspond to y): +<pre> +ev = sample_dbn(bnet, T); +evidence = cell(ss,T); +evidence(onodes,:) = ev(onodes, :); % all cells besides onodes are empty +[engine, ll] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, i, t); +</pre> + + +<h2><a name="discrete">Discrete hidden nodes</h2> + +If all the hidden nodes are discrete, +we can use the junction tree algorithm to perform inference. +The simplest approach, +<tt>jtree_unrolled_dbn_inf_engine</tt>, +unrolls the DBN into a static network and applies jtree; however, for +long sequences, this +can be very slow and can result in numerical underflow. +A better approach is to apply the jtree algorithm to pairs of +neighboring slices at a time; this is implemented in +<tt>jtree_dbn_inf_engine</tt>. + +<p> +A DBN can be converted to an HMM if all the hidden nodes are discrete. +In this case, you can use +<tt>hmm_inf_engine</tt>. This is faster than jtree for small models +because the constant factors of the algorithm are lower, but can be +exponentially slower for models with many variables +(e.g., > 6 binary hidden nodes). + +<p> +The use of both +<tt>jtree_dbn_inf_engine</tt> +and +<tt>hmm_inf_engine</tt> +is deprecated. +A better approach is to construct a smoother engine out of lower-level +engines, which implement forward/backward operators. +You can create these engines as follows. +<pre> +engine = smoother_engine(hmm_2TBN_inf_engine(bnet)); +or +engine = smoother_engine(jtree_2TBN_inf_engine(bnet)); +</pre> +You then call them in the usual way: +<pre> +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, nodes, t); +</pre> +Note: you must declare the observed nodes in the bnet before using +hmm_2TBN_inf_engine. + + +<p> +Unfortunately, when all the hiddden nodes are discrete, +exact inference takes O(2^n) time, where n is the number of hidden +nodes per slice, +even if the model is sparse. +The basic reason for this is that two nodes become correlated, even if +there is no direct connection between them in the 2TBN, +by virtue of sharing common ancestors in the past. +Hence we need to use approximations. +<p> +A popular approximate inference algorithm for discrete DBNs, known as BK, is described in +<ul> +<li> +<A HREF="http://robotics.Stanford.EDU/~xb/uai98/index.html"> +Tractable inference for complex stochastic processes </A>, +Boyen and Koller, UAI 1998 +<li> +<A HREF="http://robotics.Stanford.EDU/~xb/nips98/index.html"> +Approximate learning of dynamic models</a>, Boyen and Koller, NIPS +1998. +</ul> +This approximates the belief state with a product of +marginals on a specified set of clusters. For example, +in the water network, we might use the following clusters: +<pre> +engine = bk_inf_engine(bnet, { [1 2], [3 4 5 6], [7 8] }); +</pre> +This engine can now be used just like the jtree engine. +Two special cases of the BK algorithm are supported: 'ff' (fully +factored) means each node has its own cluster, and 'exact' means there +is 1 cluster that contains the whole slice. These can be created as +follows: +<pre> +engine = bk_inf_engine(bnet, 'ff'); +engine = bk_inf_engine(bnet, 'exact'); +</pre> +For pedagogical purposes, an implementation of BK-FF that uses an HMM +instead of junction tree is available at +<tt>bk_ff_hmm_inf_engine</tt>. + + + +<h2><a name="cts">Continuous hidden nodes</h2> + +If all the hidden nodes are linear-Gaussian, <em>and</em> the observed nodes are +linear-Gaussian, +the model is a <a href="http://www.cs.berkeley.edu/~murphyk/Bayes/kalman.html"> +linear dynamical system</a> (LDS). +A DBN can be converted to an LDS if all the hidden nodes are linear-Gaussian +and if they are all persistent. In this case, you can use +<tt>kalman_inf_engine</tt>. +For more general linear-gaussian models, you can use +<tt>jtree_dbn_inf_engine</tt> or <tt>jtree_unrolled_dbn_inf_engine</tt>. + +<p> +For nonlinear systems with Gaussian noise, the unscented Kalman filter (UKF), +due to Julier and Uhlmann, is far superior to the well-known extended Kalman +filter (EKF), both in theory and practice. +<!-- +See +<A HREF="http://phoebe.robots.ox.ac.uk/default.html">"A General Method for +Approximating Nonlinear Transformations of +Probability Distributions"</A>. +(If the above link is down, +try <a href="http://www.ece.ogi.edu/~ericwan/pubs.html">Eric Wan's</a> +page, who has done a lot of work on the UKF.) +<p> +--> +The key idea of the UKF is that it is easier to estimate a Gaussian distribution +from a set of points than to approximate an arbitrary non-linear +function. +We start with points that are plus/minus sigma away from the mean along +each dimension, and then pipe them through the nonlinearity, and +then fit a Gaussian to the transformed points. +(No need to compute Jacobians, unlike the EKF!) + +<p> +For systems with non-Gaussian noise, I recommend +<a href="http://www.cs.berkeley.edu/~jfgf/smc/">Particle +filtering</a> (PF), which is a popular sequential Monte Carlo technique. + +<p> +The EKF can be used as a proposal distribution for a PF. +This method is better than either one alone. +See <a href="http://www.cs.berkeley.edu/~jfgf/upf.ps.gz">The Unscented Particle Filter</a>, +by R van der Merwe, A Doucet, JFG de Freitas and E Wan, May 2000. +<a href="http://www.cs.berkeley.edu/~jfgf/software.html">Matlab +software</a> for the UPF is also available. +<p> +Note: none of this software is part of BNT. + + + +<h1><a name="learn">Learning</h1> + +Learning in DBNs can be done online or offline. +Currently only offline learning is implemented in BNT. + + +<h2><a name="param_learn">Parameter learning</h2> + +Offline parameter learning is very similar to learning in static networks, +except now the training data is a cell-array of 2D cell-arrays. +For example, +cases{l}{i,t} is the value of node i in slice t in sequence l, or [] +if unobserved. +Each sequence can be a different length, and may have missing values +in arbitrary locations. +Here is a typical code fragment for using EM. +<pre> +ncases = 2; +cases = cell(1, ncases); +for i=1:ncases + ev = sample_dbn(bnet, T); + cases{i} = cell(ss,T); + cases{i}(onodes,:) = ev(onodes, :); +end +[bnet2, LLtrace] = learn_params_dbn_em(engine, cases, 'max_iter', 10); +</pre> +If the observed node is vector-valued and stored in an OxT array, you +need to assign each vector to a single cell, as in the following +example. +<pre> +data = [xpos(:)'; ypos(:)']; +ncases = 1; +cases = cell(1, ncases); +onodes = bnet.observed; +for i=1:ncases + cases{i} = cell(ss,T); + cases{i}(onodes,:) = num2cell(data(:,1:T), 1); +end +</pre> +<p> +For a complete code listing of how to do EM in a simple DBN, click +<a href="dbn_hmm_demo.m">here</a>. + +<h2><a name="struct_learn">Structure learning</h2> + +There is currently only one structure learning algorithm for DBNs. +This assumes all nodes are tabular and observed, and that there are +no intra-slice connections. Hence we can find the optimal set of +parents for each node separately, without worrying about directed +cycles or node orderings. +The function is called as follows +<pre> +inter = learn_struct_dbn_reveal(cases, ns, max_fan_in, penalty) +</pre> +A full example is given in BNT/examples/dynamic/reveal1.m. +Setting the penalty term to 0 gives the maximum likelihood model; this +is equivalent to maximizing the mutual information between parents and +child (in the bioinformatics community, this is known as the REVEAL +algorithm). A non-zero penalty invokes the BIC criterion, which +lessens the chance of overfitting. +<p> +<a href="http://www.bioss.sari.ac.uk/~dirk/software/DBmcmc/"> +Dirk Husmeier has extended MCMC model selection to DBNs</a>. + +</BODY> diff --git a/sourcecodes/bnt-master/docs/usage_dbn_02nov13.html b/sourcecodes/bnt-master/docs/usage_dbn_02nov13.html new file mode 100644 index 00000000..a1cd8f3a --- /dev/null +++ b/sourcecodes/bnt-master/docs/usage_dbn_02nov13.html @@ -0,0 +1,715 @@ +<HEAD> +<TITLE>How to use BNT for DBNs</TITLE> +</HEAD> + +<BODY BGCOLOR="#FFFFFF"> +<!-- white background is better for the pictures and equations --> + +Documentation last updated on 13 November 2002 + +<h1>How to use BNT for DBNs</h1> + +<p> +<ul> +<li> <a href="#spec">Model specification</a> +<ul> +<li> <a href="#hmm">HMMs</a> +<li> <a href="#lds">Kalman filters</a> +<li> <a href="#chmm">Coupled HMMs</a> +<li> <a href="#water">Water network</a> +<li> <a href="#bat">BAT network</a> +</ul> + +<li> <a href="#inf">Inference</a> +<ul> +<li> <a href="#discrete">Discrete hidden nodes</a> +<li> <a href="#cts">Continuous hidden nodes</a> +</ul> + +<li> <a href="#learn">Learning</a> +<ul> +<li> <a href="#param_learn">Parameter learning</a> +<li> <a href="#struct_learn">Structure learning</a> +</ul> + +</ul> + +Note: +you are recommended to read an introduction +to DBNs first, such as +<a href="http://www.ai.mit.edu/~murphyk/Papers/dbnchapter.pdf"> +this book chapter</a>. + + +<h1><a name="spec">Model specification</h1> + + +<!--<h1><a name="dbn_intro">Dynamic Bayesian Networks (DBNs)</h1>--> + +Dynamic Bayesian Networks (DBNs) are directed graphical models of stochastic +processes. +They generalise <a href="#hmm">hidden Markov models (HMMs)</a> +and <a href="#lds">linear dynamical systems (LDSs)</a> +by representing the hidden (and observed) state in terms of state +variables, which can have complex interdependencies. +The graphical structure provides an easy way to specify these +conditional independencies, and hence to provide a compact +parameterization of the model. +<p> +Note that "temporal Bayesian network" would be a better name than +"dynamic Bayesian network", since +it is assumed that the model structure does not change, but +the term DBN has become entrenched. +We also normally assume that the parameters do not +change, i.e., the model is time-invariant. +However, we can always add extra +hidden nodes to represent the current "regime", thereby creating +mixtures of models to capture periodic non-stationarities. +<p> +There are some cases where the size of the state space can change over +time, e.g., tracking a variable, but unknown, number of objects. +In this case, we need to change the model structure over time. +BNT does not support this. +<!-- +, but see the following paper for a +discussion of some of the issues: +<ul> +<li> <a href="ftp://ftp.cs.monash.edu.au/pub/annn/smc.ps"> +Dynamic belief networks for discrete monitoring</a>, +A. E. Nicholson and J. M. Brady. +IEEE Systems, Man and Cybernetics, 24(11):1593-1610, 1994. +</ul> +--> + + +<h2><a name="hmm">Hidden Markov Models (HMMs)</h2> + +The simplest kind of DBN is a Hidden Markov Model (HMM), which has +one discrete hidden node and one discrete or continuous +observed node per slice. We illustrate this below. +As before, circles denote continuous nodes, squares denote +discrete nodes, clear means hidden, shaded means observed. +<!-- +(The observed nodes can be +discrete or continuous; the crucial thing about an HMM is that the +hidden nodes are discrete, so the system can model arbitrary dynamics +-- providing, of course, that the hidden state space is large enough.) +--> +<p> +<img src="Figures/hmm3.gif"> +<p> +We have "unrolled" the model for three "time slices" -- the structure and parameters are +assumed to repeat as the model is unrolled further. +Hence to specify a DBN, we need to +define the intra-slice topology (within a slice), +the inter-slice topology (between two slices), +as well as the parameters for the first two slices. +(Such a two-slice temporal Bayes net is often called a 2TBN.) +<p> +We can specify the topology as follows. +<PRE> +intra = zeros(2); +intra(1,2) = 1; % node 1 in slice t connects to node 2 in slice t + +inter = zeros(2); +inter(1,1) = 1; % node 1 in slice t-1 connects to node 1 in slice t +</pre> +We can specify the parameters as follows, +where for simplicity we assume the observed node is discrete. +<pre> +Q = 2; % num hidden states +O = 2; % num observable symbols + +ns = [Q O]; +dnodes = 1:2; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes); +for i=1:4 + bnet.CPD{i} = tabular_CPD(bnet, i); +end +</pre> +<p> +We assume the distributions P(X(t) | X(t-1)) and +P(Y(t) | X(t)) are independent of t for t > 1. +Hence the CPD for nodes 5, 7, ... is the same as for node 3, so we say they +are in the same equivalence class, with node 3 being the "representative" +for this class. In other words, we have tied the parameters for nodes +3, 5, 7, ... +Similarly, nodes 4, 6, 8, ... are tied. +Note, however, that (the parameters for) nodes 1 and 2 are not tied to +subsequent slices. +<p> +Above we assumed the observation model P(Y(t) | X(t)) is independent of t for t>1, but +it is conventional to assume this is true for all t. +So we would like to put nodes 2, 4, 6, ... all in the same class. +We can do this by explicitely defining the equivalence classes, as +follows (see <a href="usage.html#tying">here</a> for more details on +parameter tying). +<p> +We define eclass1(i) to be the equivalence class that node i in slice +1 belongs to. +Similarly, we define eclass2(i) to be the equivalence class that node i in slice +2, 3, ..., belongs to. +For an HMM, we have +<pre> +eclass1 = [1 2]; +eclass2 = [3 2]; +eclass = [eclass1 eclass2]; +</pre> +This ties the observation model across slices, +since e.g., eclass(4) = eclass(2) = 2. +<p> +By default, +eclass1 = 1:ss, and eclass2 = (1:ss)+ss, where ss = slice size = the +number of nodes per slice. +<!--This will tie nodes in slices 3, 4, ... to the the nodes in slice 2, +but none of the nodes in slice 2 to any in slice 1.--> +But by using the above tieing pattern, +we now only have 3 CPDs to specify, instead of 4: +<pre> +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'eclass1', eclass1, 'eclass2', eclass2); +prior0 = normalise(rand(Q,1)); +transmat0 = mk_stochastic(rand(Q,Q)); +obsmat0 = mk_stochastic(rand(Q,O)); +bnet.CPD{1} = tabular_CPD(bnet, 1, prior0); +bnet.CPD{2} = tabular_CPD(bnet, 2, obsmat0); +bnet.CPD{3} = tabular_CPD(bnet, 3, transmat0); +</pre> +We discuss how to do <a href="#inf">inference</a> and <a href="#learn">learning</a> on this model +below. +(See also +my <a href="../HMM/hmm.html">HMM toolbox</a>, which is included with BNT.) + +<p> +Some common variants on HMMs are shown below. +BNT can handle all of these. +<p> +<center> +<table> +<tr> +<td><img src="Figures/hmm_gauss.gif"> +<td><img src="Figures/hmm_mixgauss.gif" +<td><img src="Figures/hmm_ar.gif"> +<tr> +<td><img src="Figures/hmm_factorial.gif"> +<td><img src="Figures/hmm_coupled.gif" +<td><img src="Figures/hmm_io.gif"> +<tr> +</table> +</center> + + + +<h2><a name="lds">Linear Dynamical Systems (LDSs) and Kalman filters</h2> + +A Linear Dynamical System (LDS) has the same topology as an HMM, but +all the nodes are assumed to have linear-Gaussian distributions, i.e., +<pre> + x(t+1) = A*x(t) + w(t), w ~ N(0, Q), x(0) ~ N(init_x, init_V) + y(t) = C*x(t) + v(t), v ~ N(0, R) +</pre> +Some simple variants are shown below. +<p> +<center> +<table> +<tr> +<td><img src="Figures/ar1.gif"> +<td><img src="Figures/sar.gif"> +<td><img src="Figures/kf.gif"> +<td><img src="Figures/skf.gif"> +</table> +</center> +<p> + +We can create a regular LDS in BNT as follows. +<pre> + +intra = zeros(2); +intra(1,2) = 1; +inter = zeros(2); +inter(1,1) = 1; +n = 2; + +X = 2; % size of hidden state +Y = 2; % size of observable state + +ns = [X Y]; +dnodes = []; +onodes = [2]; +eclass1 = [1 2]; +eclass2 = [3 2]; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'eclass1', eclass1, 'eclass2', eclass2); + +x0 = rand(X,1); +V0 = eye(X); % must be positive semi definite! +C0 = rand(Y,X); +R0 = eye(Y); +A0 = rand(X,X); +Q0 = eye(X); + +bnet.CPD{1} = gaussian_CPD(bnet, 1, 'mean', x0, 'cov', V0, 'cov_prior_weight', 0); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(Y,1), 'cov', R0, 'weights', C0, ... + 'clamp_mean', 1, 'cov_prior_weight', 0); +bnet.CPD{3} = gaussian_CPD(bnet, 3, 'mean', zeros(X,1), 'cov', Q0, 'weights', A0, ... + 'clamp_mean', 1, 'cov_prior_weight', 0); +</pre> +We discuss how to do <a href="#inf">inference</a> and <a href="#learn">learning</a> on this model +below. +(See also +my <a href="../Kalman/kalman.html">Kalman filter toolbox</a>, which is included with BNT.) +<p> + + +<h2><a name="chmm">Coupled HMMs</h2> + +Here is an example of a coupled HMM with N=5 chains, unrolled for T=3 +slices. Each hidden discrete node has a private observed Gaussian +child. +<p> +<img src="Figures/chmm5.gif"> +<p> +We can make this using the function +<pre> +Q = 2; % binary hidden nodes +discrete_obs = 0; % cts observed nodes +Y = 1; % scalar observed nodes +bnet = mk_chmm(N, Q, Y, discrete_obs); +</pre> + +<!--We will use this model <a href="#pred">below</a> to illustrate online prediction.--> + + + +<h2><a name="water">Water network</h2> + +Consider the following model +of a water purification plant, developed +by Finn V. Jensen, Uffe Kjærulff, Kristian G. Olesen, and Jan +Pedersen. +<!-- +The clear nodes represent the hidden state of the system in +factored form, and the shaded nodes represent the observations in +factored form. +--> +<!-- +(Click <a +href="http://www-nt.cs.berkeley.edu/home/nir/public_html/Repository/water.htm">here</a> +for more details on this model. +Following Boyen and Koller, we have added discrete evidence nodes.) +--> +<!-- +We have "unrolled" the model for three "time slices" -- the structure and parameters are +assumed to repeat as the model is unrolled further. +Hence to specify a DBN, we need to +define the intra-slice topology (within a slice), +the inter-slice topology (between two slices), +as well as the parameters for the first two slices. +(Such a two-slice temporal Bayes net is often called a 2TBN.) +--> +<p> +<center> +<IMG SRC="Figures/water3_75.gif"> +</center> +We now show how to specify this model in BNT. +<pre> +ss = 12; % slice size +intra = zeros(ss); +intra(1,9) = 1; +intra(3,10) = 1; +intra(4,11) = 1; +intra(8,12) = 1; + +inter = zeros(ss); +inter(1, [1 3]) = 1; % node 1 in slice 1 connects to nodes 1 and 3 in slice 2 +inter(2, [2 3 7]) = 1; +inter(3, [3 4 5]) = 1; +inter(4, [3 4 6]) = 1; +inter(5, [3 5 6]) = 1; +inter(6, [4 5 6]) = 1; +inter(7, [7 8]) = 1; +inter(8, [6 7 8]) = 1; + +onodes = 9:12; % observed +dnodes = 1:ss; % discrete +ns = 2*ones(1,ss); % binary nodes +eclass1 = 1:12; +eclass2 = [13:20 9:12]; +eclass = [eclass1 eclass2]; +bnet = mk_dbn(intra, inter, ns, 'discrete', dnodes, 'eclass1', eclass1, 'eclass2', eclass2); +for e=1:max(eclass) + bnet.CPD{e} = tabular_CPD(bnet, e); +end +</pre> +We have tied the observation parameters across all slices. +Click <a href="param_tieing.html">here</a> for a more complex example +of parameter tieing. + +<!-- +Let X(i,t) denote the i'th hidden node in slice t, +and Y(i,y) denote the i'th observed node in slice t. +We also use the notation Nj to refer to the j'th node in the +unrolled network, e.g., N25 = X(1,3), N33 = Y(1,3). +<p> +We assume the distributions P(X(i,t) | X(i,t-1)) and +P(Y(i,t) | X(i,t)) are independent of t for t > 1 and for all i. +Hence the CPD for N25, N37, ... is the same as for N13, so we say they +are in the same equivalence class, with N13 being the "representative" +for this class. In other words, we have tied the parameters for nodes +N13, N25, N37, ... +Note, however, that the parameters for the nodes in the first slice +are not tied, so each equivalence class for nodes 1..12 contains a +single node. +<p> +Above we assumed P(Y(i,t) | X(i,t)) is independent of t for t>1, but +it is conventional to assume this is true for all t. +So we would like to put N9, N21, N33, ... all in the same class, and +similarly for the other observed nodes. +We can do this by explicitely defining the equivalence classes, as +follows. +<p> +We define eclass1(i) to be the equivalence class that node i in slice +1 belongs to. +Similarly, we define eclass2(i) to be the equivalence class that node i in slice +2, 3, ..., belongs to. +For the water model, we have +<pre> +</pre> +This ties the observation model across slices, +since e.g., eclass(9) = eclass(21) = 9, so Y(1,1) and Y(1,2) belong to the +same class. +<p> +By default, +eclass1 = 1:ss, and eclass2 = (1:ss)+ss, where ss = slice size = the +number of nodes per slice. +This will tie nodes in slices 3, 4, ... to the the nodes in slice 2, +but none of the nodes in slice 2 to any in slice 1. +By using the above tieing pattern, +we now only have 20 CPDs to specify, instead of 24: +<pre> +bnet = mk_dbn(intra, inter, ns, dnodes, eclass1, eclass2); +for e=1:max(eclass) + bnet.CPD{e} = tabular_CPD(bnet, e); +end +</pre> +--> + + + +<h2><a name="bat">BATnet</h2> + +As an example of a more complicated DBN, consider the following +example, +which is a model of a car's high level state, as might be used by +an automated car. +(The model is from Forbes, Huang, Kanazawa and Russell, "The BATmobile: Towards a +Bayesian Automated Taxi", IJCAI 95. The figure is from +Boyen and Koller, "Tractable Inference for Complex Stochastic +Processes", UAI98. +For simplicity, we only show the observed nodes for slice 2.) +<p> +<center> +<IMG SRC="Figures/batnet.gif"> +</center> +<p> +Since this topology is so complicated, +it is useful to be able to refer to the nodes by name, instead of +number. +<pre> +names = {'LeftClr', 'RightClr', 'LatAct', ... 'Bclr', 'BYdotDiff'}; +ss = length(names); +</pre> +We can specify the intra-slice topology using a cell array as follows, +where each row specifies a connection between two named nodes: +<pre> +intrac = {... + 'LeftClr', 'LeftClrSens'; + 'RightClr', 'RightClrSens'; + ... + 'BYdotDiff', 'BcloseFast'}; +</pre> +Finally, we can convert this cell array to an adjacency matrix using +the following function: +<pre> +[intra, names] = mk_adj_mat(intrac, names, 1); +</pre> +This function also permutes the names so that they are in topological +order. +Given this ordering of the names, we can make the inter-slice +connectivity matrix as follows: +<pre> +interc = {... + 'LeftClr', 'LeftClr'; + 'LeftClr', 'LatAct'; + ... + 'FBStatus', 'LatAct'}; + +inter = mk_adj_mat(interc, names, 0); +</pre> + +To refer to a node, we must know its number, which can be computed as +in the following example: +<pre> +obs = {'LeftClrSens', 'RightClrSens', 'TurnSignalSens', 'XdotSens', 'YdotSens', 'FYdotDiffSens', ... + 'FclrSens', 'BXdotSens', 'BclrSens', 'BYdotDiffSens'}; +for i=1:length(obs) + onodes(i) = stringmatch(obs{i}, names); +end +onodes = sort(onodes); +</pre> +(We sort the onodes since most BNT routines assume that set-valued +arguments are in sorted order.) +We can now make the DBN: +<pre> +dnodes = 1:ss; +ns = 2*ones(1,ss); % binary nodes +bnet = mk_dbn(intra, inter, ns, 'iscrete', dnodes); +</pre> +To specify the parameters, we must know the order of the parents. +See the function BNT/general/mk_named_CPT for a way to do this in the +case of tabular nodes. For simplicity, we just generate random +parameters: +<pre> +for i=1:2*ss + bnet.CPD{i} = tabular_CPD(bnet, i); +end +</pre> +A complete version of this example is available in BNT/examples/dynamic/bat1.m. + + + + +<h1><a name="inf">Inference</h1> + + +The general inference problem for DBNs is to compute +P(X(i,t0) | Y(:, t1:t2)), where X(i,t) represents the i'th hidden +variable at time t and Y(:,t1:t2) represents all the evidence +between times t1 and t2. +There are several special cases of interest, illustrated below. +The arrow indicates t0: it is X(t0) that we are trying to estimate. +The shaded region denotes t1:t2, the available data. +<p> + +<img src="Figures/filter.gif"> + +<p> +BNT can currently only handle offline smoothing. +(The HMM engine handles filtering and, to a limited extent, prediction.) +The usage is similar to static +inference engines, except now the evidence is a 2D cell array of +size ss*T, where ss is the number of nodes per slice (ss = slice sizee) and T is the +number of slices. +Also, 'marginal_nodes' takes two arguments, the nodes and the time-slice. +For example, to compute P(X(i,t) | y(:,1:T)), we proceed as follows +(where onodes are the indices of the observedd nodes in each slice, +which correspond to y): +<pre> +ev = sample_dbn(bnet, T); +evidence = cell(ss,T); +evidence(onodes,:) = ev(onodes, :); % all cells besides onodes are empty +[engine, ll] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, i, t); +</pre> + + +<h2><a name="discrete">Discrete hidden nodes</h2> + +If all the hidden nodes are discrete, +we can use the junction tree algorithm to perform inference. +The simplest approach, +<tt>jtree_unrolled_dbn_inf_engine</tt>, +unrolls the DBN into a static network and applies jtree; however, for +long sequences, this +can be very slow and can result in numerical underflow. +A better approach is to apply the jtree algorithm to pairs of +neighboring slices at a time; this is implemented in +<tt>jtree_dbn_inf_engine</tt>. + +<p> +A DBN can be converted to an HMM if all the hidden nodes are discrete. +In this case, you can use +<tt>hmm_inf_engine</tt>. This is faster than jtree for small models +because the constant factors of the algorithm are lower, but can be +exponentially slower for models with many variables +(e.g., > 6 binary hidden nodes). + +<p> +The use of both +<tt>jtree_dbn_inf_engine</tt> +and +<tt>hmm_inf_engine</tt> +is deprecated. +A better approach is to construct a smoother engine out of lower-level +engines, which implement forward/backward operators. +You can create these engines as follows. +<pre> +engine = smoother_engine(hmm_2TBN_inf_engine(bnet)); +or +engine = smoother_engine(jtree_2TBN_inf_engine(bnet)); +</pre> +You then call them in the usual way: +<pre> +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, nodes, t); +</pre> +Note: you must declare the observed nodes in the bnet before using +hmm_2TBN_inf_engine. + + +<p> +Unfortunately, when all the hiddden nodes are discrete, +exact inference takes O(2^n) time, where n is the number of hidden +nodes per slice, +even if the model is sparse. +The basic reason for this is that two nodes become correlated, even if +there is no direct connection between them in the 2TBN, +by virtue of sharing common ancestors in the past. +Hence we need to use approximations. +<p> +A popular approximate inference algorithm for discrete DBNs, known as BK, is described in +<ul> +<li> +<A HREF="http://robotics.Stanford.EDU/~xb/uai98/index.html"> +Tractable inference for complex stochastic processes </A>, +Boyen and Koller, UAI 1998 +<li> +<A HREF="http://robotics.Stanford.EDU/~xb/nips98/index.html"> +Approximate learning of dynamic models</a>, Boyen and Koller, NIPS +1998. +</ul> +This approximates the belief state with a product of +marginals on a specified set of clusters. For example, +in the water network, we might use the following clusters: +<pre> +engine = bk_inf_engine(bnet, { [1 2], [3 4 5 6], [7 8] }); +</pre> +This engine can now be used just like the jtree engine. +Two special cases of the BK algorithm are supported: 'ff' (fully +factored) means each node has its own cluster, and 'exact' means there +is 1 cluster that contains the whole slice. These can be created as +follows: +<pre> +engine = bk_inf_engine(bnet, 'ff'); +engine = bk_inf_engine(bnet, 'exact'); +</pre> +For pedagogical purposes, an implementation of BK-FF that uses an HMM +instead of junction tree is available at +<tt>bk_ff_hmm_inf_engine</tt>. + + + +<h2><a name="cts">Continuous hidden nodes</h2> + +If all the hidden nodes are linear-Gaussian, <em>and</em> the observed nodes are +linear-Gaussian, +the model is a <a href="http://www.cs.berkeley.edu/~murphyk/Bayes/kalman.html"> +linear dynamical system</a> (LDS). +A DBN can be converted to an LDS if all the hidden nodes are linear-Gaussian +and if they are all persistent. In this case, you can use +<tt>kalman_inf_engine</tt>. +For more general linear-gaussian models, you can use +<tt>jtree_dbn_inf_engine</tt> or <tt>jtree_unrolled_dbn_inf_engine</tt>. + +<p> +For nonlinear systems with Gaussian noise, the unscented Kalman filter (UKF), +due to Julier and Uhlmann, is far superior to the well-known extended Kalman +filter (EKF), both in theory and practice. +<!-- +See +<A HREF="http://phoebe.robots.ox.ac.uk/default.html">"A General Method for +Approximating Nonlinear Transformations of +Probability Distributions"</A>. +(If the above link is down, +try <a href="http://www.ece.ogi.edu/~ericwan/pubs.html">Eric Wan's</a> +page, who has done a lot of work on the UKF.) +<p> +--> +The key idea of the UKF is that it is easier to estimate a Gaussian distribution +from a set of points than to approximate an arbitrary non-linear +function. +We start with points that are plus/minus sigma away from the mean along +each dimension, and then pipe them through the nonlinearity, and +then fit a Gaussian to the transformed points. +(No need to compute Jacobians, unlike the EKF!) + +<p> +For systems with non-Gaussian noise, I recommend +<a href="http://www.cs.berkeley.edu/~jfgf/smc/">Particle +filtering</a> (PF), which is a popular sequential Monte Carlo technique. + +<p> +The EKF can be used as a proposal distribution for a PF. +This method is better than either one alone. +See <a href="http://www.cs.berkeley.edu/~jfgf/upf.ps.gz">The Unscented Particle Filter</a>, +by R van der Merwe, A Doucet, JFG de Freitas and E Wan, May 2000. +<a href="http://www.cs.berkeley.edu/~jfgf/software.html">Matlab +software</a> for the UPF is also available. +<p> +Note: none of this software is part of BNT. + + + +<h1><a name="learn">Learning</h1> + +Learning in DBNs can be done online or offline. +Currently only offline learning is implemented in BNT. + + +<h2><a name="param_learn">Parameter learning</h2> + +Offline parameter learning is very similar to learning in static networks, +except now the training data is a cell-array of 2D cell-arrays. +For example, +cases{l}{i,t} is the value of node i in slice t in sequence l, or [] +if unobserved. +Each sequence can be a different length, and may have missing values +in arbitrary locations. +Here is a typical code fragment for using EM. +<pre> +ncases = 2; +cases = cell(1, ncases); +for i=1:ncases + ev = sample_dbn(bnet, T); + cases{i} = cell(ss,T); + cases{i}(onodes,:) = ev(onodes, :); +end +[bnet2, LLtrace] = learn_params_dbn_em(engine, cases, 'max_iter', 10); +</pre> +If the observed node is vector-valued and stored in an OxT array, you +need to assign each vector to a single cell, as in the following +example. +<pre> +data = [xpos(:)'; ypos(:)']; +ncases = 1; +cases = cell(1, ncases); +onodes = bnet.observed; +for i=1:ncases + cases{i} = cell(ss,T); + cases{i}(onodes,:) = num2cell(data(:,1:T), 1); +end +</pre> +<p> +For a complete code listing of how to do EM in a simple DBN, click +<a href="dbn_hmm_demo.m">here</a>. + +<h2><a name="struct_learn">Structure learning</h2> + +There is currently only one structure learning algorithm for DBNs. +This assumes all nodes are tabular and observed, and that there are +no intra-slice connections. Hence we can find the optimal set of +parents for each node separately, without worrying about directed +cycles or node orderings. +The function is called as follows +<pre> +inter = learn_struct_dbn_reveal(cases, ns, max_fan_in, penalty) +</pre> +A full example is given in BNT/examples/dynamic/reveal1.m. +Setting the penalty term to 0 gives the maximum likelihood model; this +is equivalent to maximizing the mutual information between parents and +child (in the bioinformatics community, this is known as the REVEAL +algorithm). A non-zero penalty invokes the BIC criterion, which +lessens the chance of overfitting. +<p> +<a href="http://www.bioss.sari.ac.uk/~dirk/software/DBmcmc/"> +Dirk Husmeier has extended MCMC model selection to DBNs</a>. + +</BODY> diff --git a/sourcecodes/bnt-master/docs/usage_sf.html b/sourcecodes/bnt-master/docs/usage_sf.html new file mode 100644 index 00000000..7fc3d2c5 --- /dev/null +++ b/sourcecodes/bnt-master/docs/usage_sf.html @@ -0,0 +1,3242 @@ +<HEAD> +<TITLE>How to use the Bayes Net Toolbox</TITLE> +</HEAD> + +<BODY BGCOLOR="#FFFFFF"> +<!-- white background is better for the pictures and equations --> + +<h1>How to use the Bayes Net Toolbox</h1> + +This documentation was last updated on 7 June 2004. +<br> +Click <a href="changelog.html">here</a> for a list of changes made to +BNT. +<br> +Click +<a href="http://bnt.insa-rouen.fr/">here</a> +for a French version of this documentation (which might not +be up-to-date). +<br> +Update 23 May 2005: +Philippe LeRay has written +a +<a href="http://banquiseasi.insa-rouen.fr/projects/bnt-editor/"> +BNT GUI</a> +and +<a href="http://banquiseasi.insa-rouen.fr/projects/bnt-slp/"> +BNT Structure Learning Package</a>. + +<p> + +<ul> +<li> <a href="#install">Installation</a> +<ul> +<li> <a href="#install">Installing the Matlab code</a> +<li> <a href="#installC">Installing the C code</a> +<li> <a href="../matlab_tips.html">Useful Matlab tips</a>. +</ul> + +<li> <a href="#basics">Creating your first Bayes net</a> + <ul> + <li> <a href="#basics">Creating a model by hand</a> + <li> <a href="#file">Loading a model from a file</a> + <li> <a href="http://bnt.insa-rouen.fr/ajouts.html">Creating a model using a GUI</a> + </ul> + +<li> <a href="#inference">Inference</a> + <ul> + <li> <a href="#marginal">Computing marginal distributions</a> + <li> <a href="#joint">Computing joint distributions</a> + <li> <a href="#soft">Soft/virtual evidence</a> + <li> <a href="#mpe">Most probable explanation</a> + </ul> + +<li> <a href="#cpd">Conditional Probability Distributions</a> + <ul> + <li> <a href="#tabular">Tabular (multinomial) nodes</a> + <li> <a href="#noisyor">Noisy-or nodes</a> + <li> <a href="#deterministic">Other (noisy) deterministic nodes</a> + <li> <a href="#softmax">Softmax (multinomial logit) nodes</a> + <li> <a href="#mlp">Neural network nodes</a> + <li> <a href="#root">Root nodes</a> + <li> <a href="#gaussian">Gaussian nodes</a> + <li> <a href="#glm">Generalized linear model nodes</a> + <li> <a href="#dtree">Classification/regression tree nodes</a> + <li> <a href="#nongauss">Other continuous distributions</a> + <li> <a href="#cpd_summary">Summary of CPD types</a> + </ul> + +<li> <a href="#examples">Example models</a> + <ul> + <li> <a + href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt"> +Gaussian mixture models</a> + <li> <a href="#pca">PCA, ICA, and all that</a> + <li> <a href="#mixep">Mixtures of experts</a> + <li> <a href="#hme">Hierarchical mixtures of experts</a> + <li> <a href="#qmr">QMR</a> + <li> <a href="#cg_model">Conditional Gaussian models</a> + <li> <a href="#hybrid">Other hybrid models</a> + </ul> + +<li> <a href="#param_learning">Parameter learning</a> + <ul> + <li> <a href="#load_data">Loading data from a file</a> + <li> <a href="#mle_complete">Maximum likelihood parameter estimation from complete data</a> + <li> <a href="#prior">Parameter priors</a> + <li> <a href="#bayes_learn">(Sequential) Bayesian parameter updating from complete data</a> + <li> <a href="#em">Maximum likelihood parameter estimation with missing values (EM)</a> + <li> <a href="#tying">Parameter tying</a> + </ul> + +<li> <a href="#structure_learning">Structure learning</a> + <ul> + <li> <a href="#enumerate">Exhaustive search</a> + <li> <a href="#K2">K2</a> + <li> <a href="#hill_climb">Hill-climbing</a> + <li> <a href="#mcmc">MCMC</a> + <li> <a href="#active">Active learning</a> + <li> <a href="#struct_em">Structural EM</a> + <li> <a href="#graphdraw">Visualizing the learned graph structure</a> + <li> <a href="#constraint">Constraint-based methods</a> + </ul> + + +<li> <a href="#engines">Inference engines</a> + <ul> + <li> <a href="#jtree">Junction tree</a> + <li> <a href="#varelim">Variable elimination</a> + <li> <a href="#global">Global inference methods</a> + <li> <a href="#quickscore">Quickscore</a> + <li> <a href="#belprop">Belief propagation</a> + <li> <a href="#sampling">Sampling (Monte Carlo)</a> + <li> <a href="#engine_summary">Summary of inference engines</a> + </ul> + + +<li> <a href="#influence">Influence diagrams/ decision making</a> + + +<li> <a href="usage_dbn.html">DBNs, HMMs, Kalman filters and all that</a> +</ul> + +</ul> + + + + +<h1><a name="install">Installation</h1> + +<h2><a name="installM">Installing the Matlab code</h2> + +<ul> +<li> <a href="bnt_download.html">Download</a> the FullBNT.zip file. + +<p> +<li> Unpack the file. In Unix, type +<!--"tar xvf BNT.tar".--> +"unzip FullBNT.zip". +In Windows, use +a program like <a href="http://www.winzip.com">Winzip</a>. This will +create a directory called FullBNT, which contains BNT and other libraries. +(Files ending in ~ or # are emacs backup files, and can be ignored.) + +<p> +<li> Read the file <tt>BNT/README.txt</tt> to make sure the date +matches the one on the top of <a href=bnt.html>the BNT home page</a>. +If not, you may need to press 'refresh' on your browser, and download +again, to get the most recent version. + +<p> +<li> <b>Edit the file "FullBNT/BNT/add_BNT_to_path.m"</b> so it contains the correct +pathname. +For example, in Windows, +I download FullBNT.zip into C:\kmurphy\matlab, and +then ensure the second lines reads +<pre> +BNT_HOME = 'C:\kmurphy\matlab\FullBNT'; +</pre> + +<p> +<li> Start up Matlab. + +<p> +<li> Type "ver" at the Matlab prompt (">>"). +<b>You need Matlab version 5.2 or newer to run BNT</b>. +(Versions 5.0 and 5.1 have a memory leak which seems to sometimes +crash BNT.) +<b>BNT will not run on Octave</b>. + +<p> +<li> Move to the BNT directory. +For example, in Windows, I type +<pre> +>> cd C:\kpmurphy\matlab\FullBNT\BNT +</pre> + +<p> +<li> Type "add_BNT_to_path". +This executes the command +<tt>addpath(genpath(BNT_HOME))</tt>, +which adds all directories below FullBNT to the matlab path. + +<p> +<li> Type "test_BNT". + +If all goes well, this will produce a bunch of numbers and maybe some +warning messages (which you can ignore), but no error messages. +(The warnings should only be of the form +"Warning: Maximum number of iterations has been exceeded", and are +produced by Netlab.) + +<p> +<li> Problems? Did you remember to +<b>Edit the file "FullBNT/BNT/add_BNT_to_path.m"</b> so it contains +the right path?? + +<p> +<li> <a href="http://groups.yahoo.com/group/BayesNetToolbox/join"> +Join the BNT email list</a> + +<p> +<li>Read +<a href="../matlab_tips.html">some useful Matlab tips</a>. +<!-- +For instance, this explains how to create a startup file, which can be +used to set your path variable automatically, so you can avoid having +to type the above commands every time. +--> + +</ul> + + + +<h2><a name="installC">Installing the C code</h2> + +Some BNT functions also have C implementations. +<b>It is not necessary to install the C code</b>, but it can result in a speedup +of a factor of 2-5. +To install all the C code, +edit installC_BNT.m so it contains the right path, +then type <tt>installC_BNT</tt>. +(Ignore warnings of the form 'invalid white space character in directive'.) +To uninstall all the C code, +edit uninstallC_BNT.m so it contains the right path, +then type <tt>uninstallC_BNT</tt>. +For an up-to-date list of the files which have C implementations, see +BNT/installC_BNT.m. + +<p> +mex is a script that lets you call C code from Matlab - it does not compile matlab to +C (see mcc below). +If your C/C++ compiler is set up correctly, mex should work out of +the box. +If not, you might need to type +<p> +<tt> mex -setup</tt> +<p> +before calling installC. +<p> +To make mex call gcc on Windows, +you must install <a +href="http://www.mrc-cbu.cam.ac.uk/Imaging/gnumex20.html">gnumex</a>. +You can use the <a href="http://www.mingw.org/">minimalist GNU for +Windows</a> version of gcc, or +the <a href="http://sources.redhat.com/cygwin/">cygwin</a> version. +<p> +In general, typing +'mex foo.c' from inside Matlab creates a file called +'foo.mexglx' or 'foo.dll' (the exact file +extension is system dependent - on Linux it is 'mexglx', on Windows it is '.dll'). +The resulting file will hide the original 'foo.m' (if it existed), i.e., +typing 'foo' at the prompt will call the compiled C version. +To reveal the original matlab version, just delete foo.mexglx (this is +what uninstallC does). +<p> +Sometimes it takes time for Matlab to realize that the file has +changed from matlab to C or vice versa; try typing 'clear all' or +restarting Matlab to refresh it. +To find out which version of a file you are running, type +'which foo'. +<p> +<a href="http://www.mathworks.com/products/compiler">mcc</a>, the +Matlab to C compiler, is a separate product, +and is quite different from mex. It does not yet support +objects/classes, which is why we can't compile all of BNT to C automatically. +Also, hand-written C code is usually much +better than the C code generated by mcc. + + +<p> +Acknowledgements: +Most of the C code (e.g., for jtree and dpot) was written by Wei Hu; +the triangulation C code was written by Ilya Shpitser; +the Gibbs sampling C code (for discrete nodes) was written by Bhaskara +Marthi. + + + +<h1><a name="basics">Creating your first Bayes net</h1> + +To define a Bayes net, you must specify the graph structure and then +the parameters. We look at each in turn, using a simple example +(adapted from Russell and +Norvig, "Artificial Intelligence: a Modern Approach", Prentice Hall, +1995, p454). + + +<h2>Graph structure</h2> + + +Consider the following network. + +<p> +<center> +<IMG SRC="Figures/sprinkler.gif"> +</center> +<p> + +<P> +To specify this directed acyclic graph (dag), we create an adjacency matrix: +<PRE> +N = 4; +dag = zeros(N,N); +C = 1; S = 2; R = 3; W = 4; +dag(C,[R S]) = 1; +dag(R,W) = 1; +dag(S,W)=1; +</PRE> +<P> +We have numbered the nodes as follows: +Cloudy = 1, Sprinkler = 2, Rain = 3, WetGrass = 4. +<b>The nodes must always be numbered in topological order, i.e., +ancestors before descendants.</b> +For a more complicated graph, this is a little inconvenient: we will +see how to get around this <a href="usage_dbn.html#bat">below</a>. +<p> +In Matlab 6, you can use logical arrays instead of double arrays, +which are 4 times smaller: +<pre> +dag = false(N,N); +dag(C,[R S]) = true; +... +</pre> +However, <b>some graph functions (eg acyclic) do not work on +logical arrays</b>! +<p> +A preliminary attempt to make a <b>GUI</b> +has been writte by Philippe LeRay and can be downloaded +from <a href="http://bnt.insa-rouen.fr/ajouts.html">here</a>. +<p> +You can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>. + +<h2>Creating the Bayes net shell</h2> + +In addition to specifying the graph structure, +we must specify the size and type of each node. +If a node is discrete, its size is the +number of possible values +each node can take on; if a node is continuous, +it can be a vector, and its size is the length of this vector. +In this case, we will assume all nodes are discrete and binary. +<PRE> +discrete_nodes = 1:N; +node_sizes = 2*ones(1,N); +</pre> +If the nodes were not binary, you could type e.g., +<pre> +node_sizes = [4 2 3 5]; +</pre> +meaning that Cloudy has 4 possible values, +Sprinkler has 2 possible values, etc. +Note that these are cardinal values, not ordinal, i.e., +they are not ordered in any way, like 'low', 'medium', 'high'. +<p> +We are now ready to make the Bayes net: +<pre> +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes); +</PRE> +By default, all nodes are assumed to be discrete, so we can also just +write +<pre> +bnet = mk_bnet(dag, node_sizes); +</PRE> +You may also specify which nodes will be observed. +If you don't know, or if this not fixed in advance, +just use the empty list (the default). +<pre> +onodes = []; +bnet = mk_bnet(dag, node_sizes, 'discrete', discrete_nodes, 'observed', onodes); +</PRE> +Note that optional arguments are specified using a name/value syntax. +This is common for many BNT functions. +In general, to find out more about a function (e.g., which optional +arguments it takes), please see its +documentation string by typing +<pre> +help mk_bnet +</pre> +See also other <a href="matlab_tips.html">useful Matlab tips</a>. +<p> +It is possible to associate names with nodes, as follows: +<pre> +bnet = mk_bnet(dag, node_sizes, 'names', {'cloudy','S','R','W'}, 'discrete', 1:4); +</pre> +You can then refer to a node by its name: +<pre> +C = bnet.names('cloudy'); % bnet.names is an associative array +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +</pre> +This feature uses my own associative array class. + + +<h2><a name="cpt">Parameters</h2> + +A model consists of the graph structure and the parameters. +The parameters are represented by CPD objects (CPD = Conditional +Probability Distribution), which define the probability distribution +of a node given its parents. +(We will use the terms "node" and "random variable" interchangeably.) +The simplest kind of CPD is a table (multi-dimensional array), which +is suitable when all the nodes are discrete-valued. Note that the discrete +values are not assumed to be ordered in any way; that is, they +represent categorical quantities, like male and female, rather than +ordinal quantities, like low, medium and high. +(We will discuss CPDs in more detail <a href="#cpd">below</a>.) +<p> +Tabular CPDs, also called CPTs (conditional probability tables), +are stored as multidimensional arrays, where the dimensions +are arranged in the same order as the nodes, e.g., the CPT for node 4 +(WetGrass) is indexed by Sprinkler (2), Rain (3) and then WetGrass (4) itself. +Hence the child is always the last dimension. +If a node has no parents, its CPT is a column vector representing its +prior. +Note that in Matlab (unlike C), arrays are indexed +from 1, and are layed out in memory such that the first index toggles +fastest, e.g., the CPT for node 4 (WetGrass) is as follows +<P> +<P><IMG ALIGN=BOTTOM SRC="Figures/CPTgrass.gif"><P> +<P> +where we have used the convention that false==1, true==2. +We can create this CPT in Matlab as follows +<PRE> +CPT = zeros(2,2,2); +CPT(1,1,1) = 1.0; +CPT(2,1,1) = 0.1; +... +</PRE> +Here is an easier way: +<PRE> +CPT = reshape([1 0.1 0.1 0.01 0 0.9 0.9 0.99], [2 2 2]); +</PRE> +In fact, we don't need to reshape the array, since the CPD constructor +will do that for us. So we can just write +<pre> +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</pre> +The other nodes are created similarly (using the old syntax for +optional parameters) +<PRE> +bnet.CPD{C} = tabular_CPD(bnet, C, [0.5 0.5]); +bnet.CPD{R} = tabular_CPD(bnet, R, [0.8 0.2 0.2 0.8]); +bnet.CPD{S} = tabular_CPD(bnet, S, [0.5 0.9 0.5 0.1]); +bnet.CPD{W} = tabular_CPD(bnet, W, [1 0.1 0.1 0.01 0 0.9 0.9 0.99]); +</PRE> + + +<h2><a name="rnd_cpt">Random Parameters</h2> + +If we do not specify the CPT, random parameters will be +created, i.e., each "row" of the CPT will be drawn from the uniform distribution. +To ensure repeatable results, use +<pre> +rand('state', seed); +randn('state', seed); +</pre> +To control the degree of randomness (entropy), +you can sample each row of the CPT from a Dirichlet(p,p,...) distribution. +If p << 1, this encourages "deterministic" CPTs (one entry near 1, the rest near 0). +If p = 1, each entry is drawn from U[0,1]. +If p >> 1, the entries will all be near 1/k, where k is the arity of +this node, i.e., each row will be nearly uniform. +You can do this as follows, assuming this node +is number i, and ns is the node_sizes. +<pre> +k = ns(i); +ps = parents(dag, i); +psz = prod(ns(ps)); +CPT = sample_dirichlet(p*ones(1,k), psz); +bnet.CPD{i} = tabular_CPD(bnet, i, 'CPT', CPT); +</pre> + + +<h2><a name="file">Loading a network from a file</h2> + +If you already have a Bayes net represented in the XML-based +<a href="http://www.cs.cmu.edu/afs/cs/user/fgcozman/www/Research/InterchangeFormat/"> +Bayes Net Interchange Format (BNIF)</a> (e.g., downloaded from the +<a +href="http://www.cs.huji.ac.il/labs/compbio/Repository"> +Bayes Net repository</a>), +you can convert it to BNT format using +the +<a href="http://www.digitas.harvard.edu/~ken/bif2bnt/">BIF-BNT Java +program</a> written by Ken Shan. +(This is not necessarily up-to-date.) +<p> +<b>It is currently not possible to save/load a BNT matlab object to +file</b>, but this is easily fixed if you modify all the constructors +for all the classes (see matlab documentation). + +<h2>Creating a model using a GUI</h2> + +Click <a href="http://bnt.insa-rouen.fr/ajouts.html">here</a>. + + + +<h1><a name="inference">Inference</h1> + +Having created the BN, we can now use it for inference. +There are many different algorithms for doing inference in Bayes nets, +that make different tradeoffs between speed, +complexity, generality, and accuracy. +BNT therefore offers a variety of different inference +"engines". We will discuss these +in more detail <a href="#engines">below</a>. +For now, we will use the junction tree +engine, which is the mother of all exact inference algorithms. +This can be created as follows. +<pre> +engine = jtree_inf_engine(bnet); +</pre> +The other engines have similar constructors, but might take +additional, algorithm-specific parameters. +All engines are used in the same way, once they have been created. +We illustrate this in the following sections. + + +<h2><a name="marginal">Computing marginal distributions</h2> + +Suppose we want to compute the probability that the sprinker was on +given that the grass is wet. +The evidence consists of the fact that W=2. All the other nodes +are hidden (unobserved). We can specify this as follows. +<pre> +evidence = cell(1,N); +evidence{W} = 2; +</pre> +We use a 1D cell array instead of a vector to +cope with the fact that nodes can be vectors of different lengths. +In addition, the value [] can be used +to denote 'no evidence', instead of having to specify the observation +pattern as a separate argument. +(Click <a href="cellarray.html">here</a> for a quick tutorial on cell +arrays in matlab.) +<p> +We are now ready to add the evidence to the engine. +<pre> +[engine, loglik] = enter_evidence(engine, evidence); +</pre> +The behavior of this function is algorithm-specific, and is discussed +in more detail <a href="#engines">below</a>. +In the case of the jtree engine, +enter_evidence implements a two-pass message-passing scheme. +The first return argument contains the modified engine, which +incorporates the evidence. The second return argument contains the +log-likelihood of the evidence. (Not all engines are capable of +computing the log-likelihood.) +<p> +Finally, we can compute p=P(S=2|W=2) as follows. +<PRE> +marg = marginal_nodes(engine, S); +marg.T +ans = + 0.57024 + 0.42976 +p = marg.T(2); +</PRE> +We see that p = 0.4298. +<p> +Now let us add the evidence that it was raining, and see what +difference it makes. +<PRE> +evidence{R} = 2; +[engine, loglik] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, S); +p = marg.T(2); +</PRE> +We find that p = P(S=2|W=2,R=2) = 0.1945, +which is lower than +before, because the rain can ``explain away'' the +fact that the grass is wet. +<p> +You can plot a marginal distribution over a discrete variable +as a barchart using the built 'bar' function: +<pre> +bar(marg.T) +</pre> +This is what it looks like + +<p> +<center> +<IMG SRC="Figures/sprinkler_bar.gif"> +</center> +<p> + +<h2><a name="observed">Observed nodes</h2> + +What happens if we ask for the marginal on an observed node, e.g. P(W|W=2)? +An observed discrete node effectively only has 1 value (the observed + one) --- all other values would result in 0 probability. +For efficiency, BNT treats observed (discrete) nodes as if they were + set to 1, as we see below: +<pre> +evidence = cell(1,N); +evidence{W} = 2; +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, W); +m.T +ans = + 1 +</pre> +This can get a little confusing, since we assigned W=2. +So we can ask BNT to add the evidence back in by passing in an optional argument: +<pre> +m = marginal_nodes(engine, W, 1); +m.T +ans = + 0 + 1 +</pre> +This shows that P(W=1|W=2) = 0 and P(W=2|W=2) = 1. + + + +<h2><a name="joint">Computing joint distributions</h2> + +We can compute the joint probability on a set of nodes as in the +following example. +<pre> +evidence = cell(1,N); +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]); +</pre> +m is a structure. The 'T' field is a multi-dimensional array (in +this case, 3-dimensional) that contains the joint probability +distribution on the specified nodes. +<pre> +>> m.T +ans(:,:,1) = + 0.2900 0.0410 + 0.0210 0.0009 +ans(:,:,2) = + 0 0.3690 + 0.1890 0.0891 +</pre> +We see that P(S=1,R=1,W=2) = 0, since it is impossible for the grass +to be wet if both the rain and sprinkler are off. +<p> +Let us now add some evidence to R. +<pre> +evidence{R} = 2; +[engine, ll] = enter_evidence(engine, evidence); +m = marginal_nodes(engine, [S R W]) +m = + domain: [2 3 4] + T: [2x1x2 double] +>> m.T +m.T +ans(:,:,1) = + 0.0820 + 0.0018 +ans(:,:,2) = + 0.7380 + 0.1782 +</pre> +The joint T(i,j,k) = P(S=i,R=j,W=k|evidence) +should have T(i,1,k) = 0 for all i,k, since R=1 is incompatible +with the evidence that R=2. +Instead of creating large tables with many 0s, BNT sets the effective +size of observed (discrete) nodes to 1, as explained above. +This is why m.T has size 2x1x2. +To get a 2x2x2 table, type +<pre> +m = marginal_nodes(engine, [S R W], 1) +m = + domain: [2 3 4] + T: [2x2x2 double] +>> m.T +m.T +ans(:,:,1) = + 0 0.082 + 0 0.0018 +ans(:,:,2) = + 0 0.738 + 0 0.1782 +</pre> + +<p> +Note: It is not always possible to compute the joint on arbitrary +sets of nodes: it depends on which inference engine you use, as discussed +in more detail <a href="#engines">below</a>. + + +<h2><a name="soft">Soft/virtual evidence</h2> + +Sometimes a node is not observed, but we have some distribution over +its possible values; this is often called "soft" or "virtual" +evidence. +One can use this as follows +<pre> +[engine, loglik] = enter_evidence(engine, evidence, 'soft', soft_evidence); +</pre> +where soft_evidence{i} is either [] (if node i has no soft evidence) +or is a vector representing the probability distribution over i's +possible values. +For example, if we don't know i's exact value, but we know its +likelihood ratio is 60/40, we can write evidence{i} = [] and +soft_evidence{i} = [0.6 0.4]. +<p> +Currently only jtree_inf_engine supports this option. +It assumes that all hidden nodes, and all nodes for +which we have soft evidence, are discrete. +For a longer example, see BNT/examples/static/softev1.m. + + +<h2><a name="mpe">Most probable explanation</h2> + +To compute the most probable explanation (MPE) of the evidence (i.e., +the most probable assignment, or a mode of the joint), use +<pre> +[mpe, ll] = calc_mpe(engine, evidence); +</pre> +mpe{i} is the most likely value of node i. +This calls enter_evidence with the 'maximize' flag set to 1, which +causes the engine to do max-product instead of sum-product. +The resulting max-marginals are then thresholded. +If there is more than one maximum probability assignment, we must take + care to break ties in a consistent manner (thresholding the + max-marginals may give the wrong result). To force this behavior, + type +<pre> +[mpe, ll] = calc_mpe(engine, evidence, 1); +</pre> +Note that computing the MPE is someties called abductive reasoning. + +<p> +You can also use <tt>calc_mpe_bucket</tt> written by Ron Zohar, +that does a forwards max-product pass, and then a backwards traceback +pass, which is how Viterbi is traditionally implemented. + + + +<h1><a name="cpd">Conditional Probability Distributions</h1> + +A Conditional Probability Distributions (CPD) +defines P(X(i) | X(Pa(i))), where X(i) is the i'th node, and X(Pa(i)) +are the parents of node i. There are many ways to represent this +distribution, which depend in part on whether X(i) and X(Pa(i)) are +discrete, continuous, or a combination. +We will discuss various representations below. + + +<h2><a name="tabular">Tabular nodes</h2> + +If the CPD is represented as a table (i.e., if it is a multinomial +distribution), it has a number of parameters that is exponential in +the number of parents. See the example <a href="#cpt">above</a>. + + +<h2><a name="noisyor">Noisy-or nodes</h2> + +A noisy-OR node is like a regular logical OR gate except that +sometimes the effects of parents that are on get inhibited. +Let the prob. that parent i gets inhibited be q(i). +Then a node, C, with 2 parents, A and B, has the following CPD, where +we use F and T to represent off and on (1 and 2 in BNT). +<pre> +A B P(C=off) P(C=on) +--------------------------- +F F 1.0 0.0 +T F q(A) 1-q(A) +F T q(B) 1-q(B) +T T q(A)q(B) q-q(A)q(B) +</pre> +Thus we see that the causes get inhibited independently. +It is common to associate a "leak" node with a noisy-or CPD, which is +like a parent that is always on. This can account for all other unmodelled +causes which might turn the node on. +<p> +The noisy-or distribution is similar to the logistic distribution. +To see this, let the nodes, S(i), have values in {0,1}, and let q(i,j) +be the prob. that j inhibits i. Then +<pre> +Pr(S(i)=1 | parents(S(i))) = 1 - prod_{j} q(i,j)^S(j) +</pre> +Now define w(i,j) = -ln q(i,j) and rho(x) = 1-exp(-x). Then +<pre> +Pr(S(i)=1 | parents(S(i))) = rho(sum_j w(i,j) S(j)) +</pre> +For a sigmoid node, we have +<pre> +Pr(S(i)=1 | parents(S(i))) = sigma(-sum_j w(i,j) S(j)) +</pre> +where sigma(x) = 1/(1+exp(-x)). Hence they differ in the choice of +the activation function (although both are monotonically increasing). +In addition, in the case of a noisy-or, the weights are constrained to be +positive, since they derive from probabilities q(i,j). +In both cases, the number of parameters is <em>linear</em> in the +number of parents, unlike the case of a multinomial distribution, +where the number of parameters is exponential in the number of parents. +We will see an example of noisy-OR nodes <a href="#qmr">below</a>. + + +<h2><a name="deterministic">Other (noisy) deterministic nodes</h2> + +Deterministic CPDs for discrete random variables can be created using +the deterministic_CPD class. It is also possible to 'flip' the output +of the function with some probability, to simulate noise. +The boolean_CPD class is just a special case of a +deterministic CPD, where the parents and child are all binary. +<p> +Both of these classes are just "syntactic sugar" for the tabular_CPD +class. + + + +<h2><a name="softmax">Softmax nodes</h2> + +If we have a discrete node with a continuous parent, +we can define its CPD using a softmax function +(also known as the multinomial logit function). +This acts like a soft thresholding operator, and is defined as follows: +<pre> + exp(w(:,i)'*x + b(i)) +Pr(Q=i | X=x) = ----------------------------- + sum_j exp(w(:,j)'*x + b(j)) + +</pre> +The parameters of a softmax node, w(:,i) and b(i), i=1..|Q|, have the +following interpretation: w(:,i)-w(:,j) is the normal vector to the +decision boundary between classes i and j, +and b(i)-b(j) is its offset (bias). For example, suppose +X is a 2-vector, and Q is binary. Then +<pre> +w = [1 -1; + 0 0]; + +b = [0 0]; +</pre> +means class 1 are points in the 2D plane with positive x coordinate, +and class 2 are points in the 2D plane with negative x coordinate. +If w has large magnitude, the decision boundary is sharp, otherwise it +is soft. +In the special case that Q is binary (0/1), the softmax function reduces to the logistic +(sigmoid) function. +<p> +Fitting a softmax function can be done using the iteratively reweighted +least squares (IRLS) algorithm. +We use the implementation from +<a href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +Note that since +the softmax distribution is not in the exponential family, it does not +have finite sufficient statistics, and hence we must store all the +training data in uncompressed form. +If this takes too much space, one should use online (stochastic) gradient +descent (not implemented in BNT). +<p> +If a softmax node also has discrete parents, +we use a different set of w/b parameters for each combination of +parent values, as in the <a href="#gaussian">conditional linear +Gaussian CPD</a>. +This feature was implemented by Pierpaolo Brutti. +He is currently extending it so that discrete parents can be treated +as if they were continuous, by adding indicator variables to the X +vector. +<p> +We will see an example of softmax nodes <a href="#mixexp">below</a>. + + +<h2><a name="mlp">Neural network nodes</h2> + +Pierpaolo Brutti has implemented the mlp_CPD class, which uses a multi layer perceptron +to implement a mapping from continuous parents to discrete children, +similar to the softmax function. +(If there are also discrete parents, it creates a mixture of MLPs.) +It uses code from <a +href="http://www.ncrg.aston.ac.uk/netlab/">Netlab</a>. +This is work in progress. + +<h2><a name="root">Root nodes</h2> + +A root node has no parents and no parameters; it can be used to model +an observed, exogeneous input variable, i.e., one which is "outside" +the model. +This is useful for conditional density models. +We will see an example of root nodes <a href="#mixexp">below</a>. + + +<h2><a name="gaussian">Gaussian nodes</h2> + +We now consider a distribution suitable for the continuous-valued nodes. +Suppose the node is called Y, its continuous parents (if any) are +called X, and its discrete parents (if any) are called Q. +The distribution on Y is defined as follows: +<pre> +- no parents: Y ~ N(mu, Sigma) +- cts parents : Y|X=x ~ N(mu + W x, Sigma) +- discrete parents: Y|Q=i ~ N(mu(:,i), Sigma(:,:,i)) +- cts and discrete parents: Y|X=x,Q=i ~ N(mu(:,i) + W(:,:,i) * x, Sigma(:,:,i)) +</pre> +where N(mu, Sigma) denotes a Normal distribution with mean mu and +covariance Sigma. Let |X|, |Y| and |Q| denote the sizes of X, Y and Q +respectively. +If there are no discrete parents, |Q|=1; if there is +more than one, then |Q| = a vector of the sizes of each discrete parent. +If there are no continuous parents, |X|=0; if there is more than one, +then |X| = the sum of their sizes. +Then mu is a |Y|*|Q| vector, Sigma is a |Y|*|Y|*|Q| positive +semi-definite matrix, and W is a |Y|*|X|*|Q| regression (weight) +matrix. +<p> +We can create a Gaussian node with random parameters as follows. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i); +</pre> +We can specify the value of one or more of the parameters as in the +following example, in which |Y|=2, and |Q|=1. +<pre> +bnet.CPD{i} = gaussian_CPD(bnet, i, 'mean', [0; 0], 'weights', randn(Y,X), 'cov', eye(Y)); +</pre> +<p> +We will see an example of conditional linear Gaussian nodes <a +href="#cg_model">below</a>. +<p> +<b>When learning Gaussians from data</b>, it is helpful to ensure the +data has a small magnitde +(see e.g., KPMstats/standardize) to prevent numerical problems. +Unless you have a lot of data, it is also a very good idea to use +diagonal instead of full covariance matrices. +(BNT does not currently support spherical covariances, although it +would be easy to add, since KPMstats/clg_Mstep supports this option; +you would just need to modify gaussian_CPD/update_ess to accumulate +weighted inner products.) + + + +<h2><a name="nongauss">Other continuous distributions</h2> + +Currently BNT does not support any CPDs for continuous nodes other +than the Gaussian. +However, you can use a mixture of Gaussians to +approximate other continuous distributions. We will see some an example +of this with the IFA model <a href="#pca">below</a>. + + +<h2><a name="glm">Generalized linear model nodes</h2> + +In the future, we may incorporate some of the functionality of +<a href = +"http://www.sci.usq.edu.au/staff/dunn/glmlab/glmlab.html">glmlab</a> +into BNT. + + +<h2><a name="dtree">Classification/regression tree nodes</h2> + +We plan to add classification and regression trees to define CPDs for +discrete and continuous nodes, respectively. +Trees have many advantages: they are easy to interpret, they can do +feature selection, they can +handle discrete and continuous inputs, they do not make strong +assumptions about the form of the distribution, the number of +parameters can grow in a data-dependent way (i.e., they are +semi-parametric), they can handle missing data, etc. +However, they are not yet implemented. +<!-- +Yimin Zhang is currently (Feb '02) implementing this. +--> + + +<h2><a name="cpd_summary">Summary of CPD types</h2> + +We list all the different types of CPDs supported by BNT. +For each CPD, we specify if the child and parents can be discrete (D) or +continuous (C) (Binary (B) nodes are a special case). +We also specify which methods each class supports. +If a method is inherited, the name of the parent class is mentioned. +If a parent class calls a child method, this is mentioned. +<p> +The <tt>CPD_to_CPT</tt> method converts a CPD to a table; this +requires that the child and all parents are discrete. +The CPT might be exponentially big... +<tt>convert_to_table</tt> evaluates a CPD with evidence, and +represents the the resulting potential as an array. +This requires that the child is discrete, and any continuous parents +are observed. +<tt>convert_to_pot</tt> evaluates a CPD with evidence, and +represents the resulting potential as a dpot, gpot, cgpot or upot, as +requested. (d=discrete, g=Gaussian, cg = conditional Gaussian, u = +utility). + +<p> +When we sample a node, all the parents are observed. +When we compute the (log) probability of a node, all the parents and +the child are observed. +<p> +We also specify if the parameters are learnable. +For learning with EM, we require +the methods <tt>reset_ess</tt>, <tt>update_ess</tt> and +<tt>maximize_params</tt>. +For learning from fully observed data, we require +the method <tt>learn_params</tt>. +By default, all classes inherit this from generic_CPD, which simply +calls <tt>update_ess</tt> N times, once for each data case, followed +by <tt>maximize_params</tt>, i.e., it is like EM, without the E step. +Some classes implement a batch formula, which is quicker. +<p> +Bayesian learning means computing a posterior over the parameters +given fully observed data. +<p> +Pearl means we implement the methods <tt>compute_pi</tt> and +<tt>compute_lambda_msg</tt>, used by +<tt>pearl_inf_engine</tt>, which runs on directed graphs. +<tt>belprop_inf_engine</tt> only needs <tt>convert_to_pot</tt>.H +The pearl methods can exploit special properties of the CPDs for +computing the messages efficiently, whereas belprop does not. +<p> +The only method implemented by generic_CPD is <tt>adjustable_CPD</tt>, +which is not shown, since it is not very interesting. + + +<p> + + +<table> +<table border units = pixels><tr> +<td align=center>Name +<td align=center>Child +<td align=center>Parents +<td align=center>Comments +<td align=center>CPD_to_CPT +<td align=center>conv_to_table +<td align=center>conv_to_pot +<td align=center>sample +<td align=center>prob +<td align=center>learn +<td align=center>Bayes +<td align=center>Pearl + + +<tr> +<!-- Name--><td> +<!-- Child--><td> +<!-- Parents--><td> +<!-- Comments--><td> +<!-- CPD_to_CPT--><td> +<!-- conv_to_table--><td> +<!-- conv_to_pot--><td> +<!-- sample--><td> +<!-- prob--><td> +<!-- learn--><td> +<!-- Bayes--><td> +<!-- Pearl--><td> + +<tr> +<!-- Name--><td>boolean +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>deterministic +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>Syntactic sugar for tabular +<!-- CPD_to_CPT--><td>- +<!-- conv_to_table--><td>- +<!-- conv_to_pot--><td>- +<!-- sample--><td>- +<!-- prob--><td>- +<!-- learn--><td>- +<!-- Bayes--><td>- +<!-- Pearl--><td>- + +<tr> +<!-- Name--><td>Discrete +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Calls CPD_to_CPT +<!-- conv_to_pot--><td>Calls conv_to_table +<!-- sample--><td>Calls conv_to_table +<!-- prob--><td>Calls conv_to_table +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>Gaussian +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + +<tr> +<!-- Name--><td>gmux +<!-- Child--><td>C +<!-- Parents--><td>C/D +<!-- Comments--><td>multiplexer +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>MLP +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>multi layer perceptron +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>noisy-or +<!-- Child--><td>B +<!-- Parents--><td>B +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>Y + + +<tr> +<!-- Name--><td>root +<!-- Child--><td>C/D +<!-- Parents--><td>none +<!-- Comments--><td>no params +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>Y +<!-- sample--><td>Y +<!-- prob--><td>Y +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>softmax +<!-- Child--><td>D +<!-- Parents--><td>C/D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>Y +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>generic +<!-- Child--><td>C/D +<!-- Parents--><td>C/D +<!-- Comments--><td>Virtual class +<!-- CPD_to_CPT--><td>N +<!-- conv_to_table--><td>N +<!-- conv_to_pot--><td>N +<!-- sample--><td>N +<!-- prob--><td>N +<!-- learn--><td>N +<!-- Bayes--><td>N +<!-- Pearl--><td>N + + +<tr> +<!-- Name--><td>Tabular +<!-- Child--><td>D +<!-- Parents--><td>D +<!-- Comments--><td>- +<!-- CPD_to_CPT--><td>Y +<!-- conv_to_table--><td>Inherits from discrete +<!-- conv_to_pot--><td>Inherits from discrete +<!-- sample--><td>Inherits from discrete +<!-- prob--><td>Inherits from discrete +<!-- learn--><td>Y +<!-- Bayes--><td>Y +<!-- Pearl--><td>Y + +</table> + + + +<h1><a name="examples">Example models</h1> + + +<h2>Gaussian mixture models</h2> + +Richard W. DeVaul has made a detailed tutorial on how to fit mixtures +of Gaussians using BNT. Available +<a href="http://www.media.mit.edu/wearables/mithril/BNT/mixtureBNT.txt">here</a>. + + +<h2><a name="pca">PCA, ICA, and all that </h2> + +In Figure (a) below, we show how Factor Analysis can be thought of as a +graphical model. Here, X has an N(0,I) prior, and +Y|X=x ~ N(mu + Wx, Psi), +where Psi is diagonal and W is called the "factor loading matrix". +Since the noise on both X and Y is diagonal, the components of these +vectors are uncorrelated, and hence can be represented as individual +scalar nodes, as we show in (b). +(This is useful if parts of the observations on the Y vector are occasionally missing.) +We usually take k=|X| << |Y|=D, so the model tries to explain +many observations using a low-dimensional subspace. + + +<center> +<table> +<tr> +<td><img src="Figures/fa.gif"> +<td><img src="Figures/fa_scalar.gif"> +<td><img src="Figures/mfa.gif"> +<td><img src="Figures/ifa.gif"> +<tr> +<td align=center> (a) +<td align=center> (b) +<td align=center> (c) +<td align=center> (d) +</table> +</center> + +<p> +We can create this model in BNT as follows. +<pre> +ns = [k D]; +dag = zeros(2,2); +dag(1,2) = 1; +bnet = mk_bnet(dag, ns, 'discrete', []); +bnet.CPD{1} = gaussian_CPD(bnet, 1, 'mean', zeros(k,1), 'cov', eye(k), ... + 'cov_type', 'diag', 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(D,1), 'cov', diag(Psi0), 'weights', W0, ... + 'cov_type', 'diag', 'clamp_mean', 1); +</pre> + +The root node is clamped to the N(0,I) distribution, so that we will +not update these parameters during learning. +The mean of the leaf node is clamped to 0, +since we assume the data has been centered (had its mean subtracted +off); this is just for simplicity. +Finally, the covariance of the leaf node is constrained to be +diagonal. W0 and Psi0 are the initial parameter guesses. + +<p> +We can fit this model (i.e., estimate its parameters in a maximum +likelihood (ML) sense) using EM, as we +explain <a href="#em">below</a>. +Not surprisingly, the ML estimates for mu and Psi turn out to be +identical to the +sample mean and variance, which can be computed directly as +<pre> +mu_ML = mean(data); +Psi_ML = diag(cov(data)); +</pre> +Note that W can only be identified up to a rotation matrix, because of +the spherical symmetry of the source. + +<p> +If we restrict Psi to be spherical, i.e., Psi = sigma*I, +there is a closed-form solution for W as well, +i.e., we do not need to use EM. +In particular, W contains the first |X| eigenvectors of the sample covariance +matrix, with scalings determined by the eigenvalues and sigma. +Classical PCA can be obtained by taking the sigma->0 limit. +For details, see + +<ul> +<li> <a href="ftp://hope.caltech.edu/pub/roweis/Empca/empca.ps"> +"EM algorithms for PCA and SPCA"</a>, Sam Roweis, NIPS 97. +(<a href="ftp://hope.caltech.edu/pub/roweis/Code/empca.tar.gz"> +Matlab software</a>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +By adding a hidden discrete variable, we can create mixtures of FA +models, as shown in (c). +Now we can explain the data using a set of subspaces. +We can create this model in BNT as follows. +<pre> +ns = [M k D]; +dag = zeros(3); +dag(1,3) = 1; +dag(2,3) = 1; +bnet = mk_bnet(dag, ns, 'discrete', 1); +bnet.CPD{1} = tabular_CPD(bnet, 1, Pi0); +bnet.CPD{2} = gaussian_CPD(bnet, 2, 'mean', zeros(k, 1), 'cov', eye(k), 'cov_type', 'diag', ... + 'clamp_mean', 1, 'clamp_cov', 1); +bnet.CPD{3} = gaussian_CPD(bnet, 3, 'mean', Mu0', 'cov', repmat(diag(Psi0), [1 1 M]), ... + 'weights', W0, 'cov_type', 'diag', 'tied_cov', 1); +</pre> +Notice how the covariance matrix for Y is the same for all values of +Q; that is, the noise level in each sub-space is assumed the same. +However, we allow the offset, mu, to vary. +For details, see +<ul> + +<LI> +<a HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/tr-96-1.ps.gz"> The EM +Algorithm for Mixtures of Factor Analyzers </A>, +Ghahramani, Z. and Hinton, G.E. (1996), +University of Toronto +Technical Report CRG-TR-96-1. +(<A HREF="ftp://ftp.cs.toronto.edu/pub/zoubin/mfa.tar.gz">Matlab software</A>) + +<p> +<li> +<a +href=http://neural-server.aston.ac.uk/cgi-bin/tr_avail.pl?trnumber=NCRG/97/003> +"Mixtures of probabilistic principal component analyzers"</a>, +Tipping and Bishop, Neural Computation 11(2):443--482, 1999. +</ul> + +<p> +I have included Zoubin's specialized MFA code (with his permission) +with the toolbox, so you can check that BNT gives the same results: +see 'BNT/examples/static/mfa1.m'. + +<p> +Independent Factor Analysis (IFA) generalizes FA by allowing a +non-Gaussian prior on each component of X. +(Note that we can approximate a non-Gaussian prior using a mixture of +Gaussians.) +This means that the likelihood function is no longer rotationally +invariant, so we can uniquely identify W and the hidden +sources X. +IFA also allows a non-diagonal Psi (i.e. correlations between the components of Y). +We recover classical Independent Components Analysis (ICA) +in the Psi -> 0 limit, and by assuming that |X|=|Y|, so that the +weight matrix W is square and invertible. +For details, see +<ul> +<li> +<a href="http://www.gatsby.ucl.ac.uk/~hagai/ifa.ps">Independent Factor +Analysis</a>, H. Attias, Neural Computation 11: 803--851, 1998. +</ul> + + + +<h2><a name="mixexp">Mixtures of experts</h2> + +As an example of the use of the softmax function, +we introduce the Mixture of Experts model. +<!-- +We also show +the Hierarchical Mixture of Experts model, where the hierarchy has two +levels. +(This is essentially a probabilistic decision tree of height two.) +--> +As before, +circles denote continuous-valued nodes, +squares denote discrete nodes, clear +means hidden, and shaded means observed. +<p> +<center> +<table> +<tr> +<td><img src="Figures/mixexp.gif"> +<!-- +<td><img src="Figures/hme.gif"> +--> +</table> +</center> +<p> +X is the observed +input, Y is the output, and +the Q nodes are hidden "gating" nodes, which select the appropriate +set of parameters for Y. During training, Y is assumed observed, +but for testing, the goal is to predict Y given X. +Note that this is a <em>conditional</em> density model, so we don't +associate any parameters with X. +Hence X's CPD will be a root CPD, which is a way of modelling +exogenous nodes. +If the output is a continuous-valued quantity, +we assume the "experts" are linear-regression units, +and set Y's CPD to linear-Gaussian. +If the output is discrete, we set Y's CPD to a softmax function. +The Q CPDs will always be softmax functions. + +<p> +As a concrete example, consider the mixture of experts model where X and Y are +scalars, and Q is binary. +This is just piecewise linear regression, where +we have two line segments, i.e., +<P> +<IMG ALIGN=BOTTOM SRC="Eqns/lin_reg_eqn.gif"> +<P> +We can create this model with random parameters as follows. +(This code is bundled in BNT/examples/static/mixexp2.m.) +<PRE> +X = 1; +Q = 2; +Y = 3; +dag = zeros(3,3); +dag(X,[Q Y]) = 1 +dag(Q,Y) = 1; +ns = [1 2 1]; % make X and Y scalars, and have 2 experts +onodes = [1 3]; +bnet = mk_bnet(dag, ns, 'discrete', 2, 'observed', onodes); + +rand('state', 0); +randn('state', 0); +bnet.CPD{1} = root_CPD(bnet, 1); +bnet.CPD{2} = softmax_CPD(bnet, 2); +bnet.CPD{3} = gaussian_CPD(bnet, 3); +</PRE> +Now let us fit this model using <a href="#em">EM</a>. +First we <a href="#load_data">load the data</a> (1000 training cases) and plot them. +<P> +<PRE> +data = load('/examples/static/Misc/mixexp_data.txt', '-ascii'); +plot(data(:,1), data(:,2), '.'); +</PRE> +<p> +<center> +<IMG SRC="Figures/mixexp_data.gif"> +</center> +<p> +This is what the model looks like before training. +(Thanks to Thomas Hofman for writing this plotting routine.) +<p> +<center> +<IMG SRC="Figures/mixexp_before.gif"> +</center> +<p> +Now let's train the model, and plot the final performance. +(We will discuss how to train models in more detail <a href="#param_learning">below</a>.) +<P> +<PRE> +ncases = size(data, 1); % each row of data is a training case +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); % each column of cases is a training case +engine = jtree_inf_engine(bnet); +max_iter = 20; +[bnet2, LLtrace] = learn_params_em(engine, cases, max_iter); +</PRE> +(We specify which nodes will be observed when we create the engine. +Hence BNT knows that the hidden nodes are all discrete. +For complex models, this can lead to a significant speedup.) +Below we show what the model looks like after 16 iterations of EM +(with 100 IRLS iterations per M step), when it converged +using the default convergence tolerance (that the +fractional change in the log-likelihood be less than 1e-3). +Before learning, the log-likelihood was +-322.927442; afterwards, it was -13.728778. +<p> +<center> +<IMG SRC="Figures/mixexp_after.gif"> +</center> +(See BNT/examples/static/mixexp2.m for details of the code.) + + + +<h2><a name="hme">Hierarchical mixtures of experts</h2> + +A hierarchical mixture of experts (HME) extends the mixture of experts +model by having more than one hidden node. A two-level example is shown below, along +with its more traditional representation as a neural network. +This is like a (balanced) probabilistic decision tree of height 2. +<p> +<center> +<IMG SRC="Figures/HMEforMatlab.jpg"> +</center> +<p> +<a href="mailto:pbrutti@stat.cmu.edu">Pierpaolo Brutti</a> +has written an extensive set of routines for HMEs, +which are bundled with BNT: see the examples/static/HME directory. +These routines allow you to choose the number of hidden (gating) +layers, and the form of the experts (softmax or MLP). +See the file hmemenu, which provides a demo. +For example, the figure below shows the decision boundaries learned +for a ternary classification problem, using a 2 level HME with softmax +gates and softmax experts; the training set is on the left, the +testing set on the right. +<p> +<center> +<!--<IMG SRC="Figures/hme_dec_boundary.gif">--> +<IMG SRC="Figures/hme_dec_boundary.png"> +</center> +<p> + + +<p> +For more details, see the following: +<ul> + +<li> <a href="http://www.cs.berkeley.edu/~jordan/papers/hierarchies.ps.Z"> +Hierarchical mixtures of experts and the EM algorithm</a> +M. I. Jordan and R. A. Jacobs. Neural Computation, 6, 181-214, 1994. + +<li> <a href = +"http://www.cs.berkeley.edu/~dmartin/software">David Martin's +matlab code for HME</a> + +<li> <a +href="http://www.cs.berkeley.edu/~jordan/papers/uai.ps.Z">Why the +logistic function? A tutorial discussion on +probabilities and neural networks.</a> M. I. Jordan. MIT Computational +Cognitive Science Report 9503, August 1995. + +<li> "Generalized Linear Models", McCullagh and Nelder, Chapman and +Halll, 1983. + +<li> +"Improved learning algorithms for mixtures of experts in multiclass +classification". +K. Chen, L. Xu, H. Chi. +Neural Networks (1999) 12: 1229-1252. + +<li> <a href="http://www.oigeeza.com/steve/"> +Classification Using Hierarchical Mixtures of Experts</a> +S.R. Waterhouse and A.J. Robinson. +In Proc. IEEE Workshop on Neural Network for Signal Processing IV (1994), pp. 177-186 + +<li> <a href="http://www.idiap.ch/~perry/"> +Localized mixtures of experts</a>, +P. Moerland, 1998. + +<li> "Nonlinear gated experts for time series", +A.S. Weigend and M. Mangeas, 1995. + +</ul> + + +<h2><a name="qmr">QMR</h2> + +Bayes nets originally arose out of an attempt to add probabilities to +expert systems, and this is still the most common use for BNs. +A famous example is +QMR-DT, a decision-theoretic reformulation of the Quick Medical +Reference (QMR) model. +<p> +<center> +<IMG ALIGN=BOTTOM SRC="Figures/qmr.gif"> +</center> +Here, the top layer represents hidden disease nodes, and the bottom +layer represents observed symptom nodes. +The goal is to infer the posterior probability of each disease given +all the symptoms (which can be present, absent or unknown). +Each node in the top layer has a Bernoulli prior (with a low prior +probability that the disease is present). +Since each node in the bottom layer has a high fan-in, we use a +noisy-OR parameterization; each disease has an independent chance of +causing each symptom. +The real QMR-DT model is copyright, but +we can create a random QMR-like model as follows. +<pre> +function bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% MK_QMR_BNET Make a QMR model +% bnet = mk_qmr_bnet(G, inhibit, leak, prior) +% +% G(i,j) = 1 iff there is an arc from disease i to finding j +% inhibit(i,j) = inhibition probability on i->j arc +% leak(j) = inhibition prob. on leak->j arc +% prior(i) = prob. disease i is on + +[Ndiseases Nfindings] = size(inhibit); +N = Ndiseases + Nfindings; +finding_node = Ndiseases+1:N; +ns = 2*ones(1,N); +dag = zeros(N,N); +dag(1:Ndiseases, finding_node) = G; +bnet = mk_bnet(dag, ns, 'observed', finding_node); + +for d=1:Ndiseases + CPT = [1-prior(d) prior(d)]; + bnet.CPD{d} = tabular_CPD(bnet, d, CPT'); +end + +for i=1:Nfindings + fnode = finding_node(i); + ps = parents(G, i); + bnet.CPD{fnode} = noisyor_CPD(bnet, fnode, leak(i), inhibit(ps, i)); +end +</pre> +In the file BNT/examples/static/qmr1, we create a random bipartite +graph G, with 5 diseases and 10 findings, and random parameters. +(In general, to create a random dag, use 'mk_random_dag'.) +We can visualize the resulting graph structure using +the methods discussed <a href="#graphdraw">below</a>, with the +following results: +<p> +<img src="Figures/qmr.rnd.jpg"> + +<p> +Now let us put some random evidence on all the leaves except the very +first and very last, and compute the disease posteriors. +<pre> +pos = 2:floor(Nfindings/2); +neg = (pos(end)+1):(Nfindings-1); +onodes = myunion(pos, neg); +evidence = cell(1, N); +evidence(findings(pos)) = num2cell(repmat(2, 1, length(pos))); +evidence(findings(neg)) = num2cell(repmat(1, 1, length(neg))); + +engine = jtree_inf_engine(bnet); +[engine, ll] = enter_evidence(engine, evidence); +post = zeros(1, Ndiseases); +for i=diseases(:)' + m = marginal_nodes(engine, i); + post(i) = m.T(2); +end +</pre> +Junction tree can be quite slow on large QMR models. +Fortunately, it is possible to exploit properties of the noisy-OR +function to speed up exact inference using an algorithm called +<a href="#quickscore">quickscore</a>, discussed below. + + + + + +<h2><a name="cg_model">Conditional Gaussian models</h2> + +A conditional Gaussian model is one in which, conditioned on all the discrete +nodes, the distribution over the remaining (continuous) nodes is +multivariate Gaussian. This means we can have arcs from discrete (D) +to continuous (C) nodes, but not vice versa. +(We <em>are</em> allowed C->D arcs if the continuous nodes are observed, +as in the <a href="#mixexp">mixture of experts</a> model, +since this distribution can be represented with a discrete potential.) +<p> +We now give an example of a CG model, from +the paper "Propagation of Probabilities, Means amd +Variances in Mixed Graphical Association Models", Steffen Lauritzen, +JASA 87(420):1098--1108, 1992 (reprinted in the book "Probabilistic Networks and Expert +Systems", R. G. Cowell, A. P. Dawid, S. L. Lauritzen and +D. J. Spiegelhalter, Springer, 1999.) + +<h3>Specifying the graph</h3> + +Consider the model of waste emissions from an incinerator plant shown below. +We follow the standard convention that shaded nodes are observed, +clear nodes are hidden. +We also use the non-standard convention that +square nodes are discrete (tabular) and round nodes are +Gaussian. + +<p> +<center> +<IMG SRC="Figures/cg1.gif"> +</center> +<p> + +We can create this model as follows. +<pre> +F = 1; W = 2; E = 3; B = 4; C = 5; D = 6; Min = 7; Mout = 8; L = 9; +n = 9; + +dag = zeros(n); +dag(F,E)=1; +dag(W,[E Min D]) = 1; +dag(E,D)=1; +dag(B,[C D])=1; +dag(D,[L Mout])=1; +dag(Min,Mout)=1; + +% node sizes - all cts nodes are scalar, all discrete nodes are binary +ns = ones(1, n); +dnodes = [F W B]; +cnodes = mysetdiff(1:n, dnodes); +ns(dnodes) = 2; + +bnet = mk_bnet(dag, ns, 'discrete', dnodes); +</pre> +'dnodes' is a list of the discrete nodes; 'cnodes' is the continuous +nodes. 'mysetdiff' is a faster version of the built-in 'setdiff'. +<p> + + +<h3>Specifying the parameters</h3> + +The parameters of the discrete nodes can be specified as follows. +<pre> +bnet.CPD{B} = tabular_CPD(bnet, B, 'CPT', [0.85 0.15]); % 1=stable, 2=unstable +bnet.CPD{F} = tabular_CPD(bnet, F, 'CPT', [0.95 0.05]); % 1=intact, 2=defect +bnet.CPD{W} = tabular_CPD(bnet, W, 'CPT', [2/7 5/7]); % 1=industrial, 2=household +</pre> + +<p> +The parameters of the continuous nodes can be specified as follows. +<pre> +bnet.CPD{E} = gaussian_CPD(bnet, E, 'mean', [-3.9 -0.4 -3.2 -0.5], ... + 'cov', [0.00002 0.0001 0.00002 0.0001]); +bnet.CPD{D} = gaussian_CPD(bnet, D, 'mean', [6.5 6.0 7.5 7.0], ... + 'cov', [0.03 0.04 0.1 0.1], 'weights', [1 1 1 1]); +bnet.CPD{C} = gaussian_CPD(bnet, C, 'mean', [-2 -1], 'cov', [0.1 0.3]); +bnet.CPD{L} = gaussian_CPD(bnet, L, 'mean', 3, 'cov', 0.25, 'weights', -0.5); +bnet.CPD{Min} = gaussian_CPD(bnet, Min, 'mean', [0.5 -0.5], 'cov', [0.01 0.005]); +bnet.CPD{Mout} = gaussian_CPD(bnet, Mout, 'mean', 0, 'cov', 0.002, 'weights', [1 1]); +</pre> + + +<h3><a name="cg_infer">Inference</h3> + +<!--Let us perform inference in the <a href="#cg_model">waste incinerator example</a>.--> +First we compute the unconditional marginals. +<pre> +engine = jtree_inf_engine(bnet); +evidence = cell(1,n); +[engine, ll] = enter_evidence(engine, evidence); +marg = marginal_nodes(engine, E); +</pre> +<!--(Of course, we could use <tt>cond_gauss_inf_engine</tt> instead of jtree.)--> +'marg' is a structure that contains the fields 'mu' and 'Sigma', which +contain the mean and (co)variance of the marginal on E. +In this case, they are both scalars. +Let us check they match the published figures (to 2 decimal places). +<!--(We can't expect +more precision than this in general because I have implemented the algorithm of +Lauritzen (1992), which can be numerically unstable.)--> +<pre> +tol = 1e-2; +assert(approxeq(marg.mu, -3.25, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.709, tol)); +</pre> +We can compute the other posteriors similarly. +Now let us add some evidence. +<pre> +evidence = cell(1,n); +evidence{W} = 1; % industrial +evidence{L} = 1.1; +evidence{C} = -0.9; +[engine, ll] = enter_evidence(engine, evidence); +</pre> +Now we find +<pre> +marg = marginal_nodes(engine, E); +assert(approxeq(marg.mu, -3.8983, tol)); +assert(approxeq(sqrt(marg.Sigma), 0.0763, tol)); +</pre> + + +We can also compute the joint probability on a set of nodes. +For example, P(D, Mout | evidence) is a 2D Gaussian: +<pre> +marg = marginal_nodes(engine, [D Mout]) +marg = + domain: [6 8] + mu: [2x1 double] + Sigma: [2x2 double] + T: 1.0000 +</pre> +The mean is +<pre> +marg.mu +ans = + 3.6077 + 4.1077 +</pre> +and the covariance matrix is +<pre> +marg.Sigma +ans = + 0.1062 0.1062 + 0.1062 0.1182 +</pre> +It is easy to visualize this posterior using standard Matlab plotting +functions, e.g., +<pre> +gaussplot2d(marg.mu, marg.Sigma); +</pre> +produces the following picture. + +<p> +<center> +<IMG SRC="Figures/gaussplot.png"> +</center> +<p> + + +The T field indicates that the mixing weight of this Gaussian +component is 1.0. +If the joint contains discrete and continuous variables, the result +will be a mixture of Gaussians, e.g., +<pre> +marg = marginal_nodes(engine, [F E]) + domain: [1 3] + mu: [-3.9000 -0.4003] + Sigma: [1x1x2 double] + T: [0.9995 4.7373e-04] +</pre> +The interpretation is +Sigma(i,j,k) = Cov[ E(i) E(j) | F=k ]. +In this case, E is a scalar, so i=j=1; k specifies the mixture component. +<p> +We saw in the sprinkler network that BNT sets the effective size of +observed discrete nodes to 1, since they only have one legal value. +For continuous nodes, BNT sets their length to 0, +since they have been reduced to a point. +For example, +<pre> +marg = marginal_nodes(engine, [B C]) + domain: [4 5] + mu: [] + Sigma: [] + T: [0.0123 0.9877] +</pre> +It is simple to post-process the output of marginal_nodes. +For example, the file BNT/examples/static/cg1 sets the mu term of +observed nodes to their observed value, and the Sigma term to 0 (since +observed nodes have no variance). + +<p> +Note that the implemented version of the junction tree is numerically +unstable when using CG potentials +(which is why, in the example above, we only required our answers to agree with +the published ones to 2dp.) +This is why you might want to use <tt>stab_cond_gauss_inf_engine</tt>, +implemented by Shan Huang. This is described in + +<ul> +<li> "Stable Local Computation with Conditional Gaussian Distributions", +S. Lauritzen and F. Jensen, Tech Report R-99-2014, +Dept. Math. Sciences, Allborg Univ., 1999. +</ul> + +However, even the numerically stable version +can be computationally intractable if there are many hidden discrete +nodes, because the number of mixture components grows exponentially e.g., in a +<a href="usage_dbn.html#lds">switching linear dynamical system</a>. +In general, one must resort to approximate inference techniques: see +the discussion on <a href="#engines">inference engines</a> below. + + +<h2><a name="hybrid">Other hybrid models</h2> + +When we have C->D arcs, where C is hidden, we need to use +approximate inference. +One approach (not implemented in BNT) is described in +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/hybrid_uai99.ps.gz">A +Variational Approximation for Bayesian Networks with +Discrete and Continuous Latent Variables</a>, +K. Murphy, UAI 99. +</ul> +Of course, one can always use <a href="#sampling">sampling</a> methods +for approximate inference in such models. + + + +<h1><a name="param_learning">Parameter Learning</h1> + +The parameter estimation routines in BNT can be classified into 4 +types, depending on whether the goal is to compute +a full (Bayesian) posterior over the parameters or just a point +estimate (e.g., Maximum Likelihood or Maximum A Posteriori), +and whether all the variables are fully observed or there is missing +data/ hidden variables (partial observability). +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_params</tt></td> + <td><tt>learn_params_em</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>bayes_update_params</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="load_data">Loading data from a file</h2> + +To load numeric data from an ASCII text file called 'dat.txt', where each row is a +case and columns are separated by white-space, such as +<pre> +011979 1626.5 0.0 +021979 1367.0 0.0 +... +</pre> +you can use +<pre> +data = load('dat.txt'); +</pre> +or +<pre> +load dat.txt -ascii +</pre> +In the latter case, the data is stored in a variable called 'dat' (the +filename minus the extension). +Alternatively, suppose the data is stored in a .csv file (has commas +separating the columns, and contains a header line), such as +<pre> +header info goes here +ORD,011979,1626.5,0.0 +DSM,021979,1367.0,0.0 +... +</pre> +You can load this using +<pre> +[a,b,c,d] = textread('dat.txt', '%s %d %f %f', 'delimiter', ',', 'headerlines', 1); +</pre> +If your file is not in either of these formats, you can either use Perl to convert +it to this format, or use the Matlab scanf command. +Type +<tt> +help iofun +</tt> +for more information on Matlab's file functions. +<!-- +<p> +To load data directly from Excel, +you should buy the +<a href="http://www.mathworks.com/products/excellink/">Excel Link</a>. +To load data directly from a relational database, +you should buy the +<a href="http://www.mathworks.com/products/database">Database +toolbox</a>. +--> +<p> +BNT learning routines require data to be stored in a cell array. +data{i,m} is the value of node i in case (example) m, i.e., each +<em>column</em> is a case. +If node i is not observed in case m (missing value), set +data{i,m} = []. +(Not all the learning routines can cope with such missing values, however.) +In the special case that all the nodes are observed and are +scalar-valued (as opposed to vector-valued), the data can be +stored in a matrix (as opposed to a cell-array). +<p> +Suppose, as in the <a href="#mixexp">mixture of experts example</a>, +that we have 3 nodes in the graph: X(1) is the observed input, X(3) is +the observed output, and X(2) is a hidden (gating) node. We can +create the dataset as follows. +<pre> +data = load('dat.txt'); +ncases = size(data, 1); +cases = cell(3, ncases); +cases([1 3], :) = num2cell(data'); +</pre> +Notice how we transposed the data, to convert rows into columns. +Also, cases{2,m} = [] for all m, since X(2) is always hidden. + + +<h2><a name="mle_complete">Maximum likelihood parameter estimation from complete data</h2> + +As an example, let's generate some data from the sprinkler network, randomize the parameters, +and then try to recover the original model. +First we create some training data using forwards sampling. +<pre> +samples = cell(N, nsamples); +for i=1:nsamples + samples(:,i) = sample_bnet(bnet); +end +</pre> +samples{j,i} contains the value of the j'th node in case i. +sample_bnet returns a cell array because, in general, each node might +be a vector of different length. +In this case, all nodes are discrete (and hence scalars), so we +could have used a regular array instead (which can be quicker): +<pre> +data = cell2num(samples); +</pre +So now data(j,i) = samples{j,i}. +<p> +Now we create a network with random parameters. +(The initial values of bnet2 don't matter in this case, since we can find the +globally optimal MLE independent of where we start.) +<pre> +% Make a tabula rasa +bnet2 = mk_bnet(dag, node_sizes); +seed = 0; +rand('state', seed); +bnet2.CPD{C} = tabular_CPD(bnet2, C); +bnet2.CPD{R} = tabular_CPD(bnet2, R); +bnet2.CPD{S} = tabular_CPD(bnet2, S); +bnet2.CPD{W} = tabular_CPD(bnet2, W); +</pre> +Finally, we find the maximum likelihood estimates of the parameters. +<pre> +bnet3 = learn_params(bnet2, samples); +</pre> +To view the learned parameters, we use a little Matlab hackery. +<pre> +CPT3 = cell(1,N); +for i=1:N + s=struct(bnet3.CPD{i}); % violate object privacy + CPT3{i}=s.CPT; +end +</pre> +Here are the parameters learned for node 4. +<pre> +dispcpt(CPT3{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.2000 0.8000 +1 2 : 0.2273 0.7727 +2 2 : 0.0000 1.0000 +</pre> +So we see that the learned parameters are fairly close to the "true" +ones, which we display below. +<pre> +dispcpt(CPT{4}) +1 1 : 1.0000 0.0000 +2 1 : 0.1000 0.9000 +1 2 : 0.1000 0.9000 +2 2 : 0.0100 0.9900 +</pre> +We can get better results by using a larger training set, or using +informative priors (see <a href="#prior">below</a>). + + + +<h2><a name="prior">Parameter priors</h2> + +Currently, only tabular CPDs can have priors on their parameters. +The conjugate prior for a multinomial is the Dirichlet. +(For binary random variables, the multinomial is the same as the +Bernoulli, and the Dirichlet is the same as the Beta.) +<p> +The Dirichlet has a simple interpretation in terms of pseudo counts. +If we let N_ijk = the num. times X_i=k and Pa_i=j occurs in the +training set, where Pa_i are the parents of X_i, +then the maximum likelihood (ML) estimate is +T_ijk = N_ijk / N_ij (where N_ij = sum_k' N_ijk'), which will be 0 if N_ijk=0. +To prevent us from declaring that (X_i=k, Pa_i=j) is impossible just because this +event was not seen in the training set, +we can pretend we saw value k of X_i, for each value j of Pa_i some number (alpha_ijk) +of times in the past. +The MAP (maximum a posterior) estimate is then +<pre> +T_ijk = (N_ijk + alpha_ijk) / (N_ij + alpha_ij) +</pre> +and is never 0 if all alpha_ijk > 0. +For example, consider the network A->B, where A is binary and B has 3 +values. +A uniform prior for B has the form +<pre> + B=1 B=2 B=3 +A=1 1 1 1 +A=2 1 1 1 +</pre> +which can be created using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'unif'); +</pre> +This prior does not satisfy the likelihood equivalence principle, +which says that <a href="#markov_equiv">Markov equivalent</a> models +should have the same marginal likelihood. +A prior that does satisfy this principle is shown below. +Heckerman (1995) calls this the +BDeu prior (likelihood equivalent uniform Bayesian Dirichlet). +<pre> + B=1 B=2 B=3 +A=1 1/6 1/6 1/6 +A=2 1/6 1/6 1/6 +</pre> +where we put N/(q*r) in each bin; N is the equivalent sample size, +r=|A|, q = |B|. +This can be created as follows +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', 'BDeu'); +</pre> +Here, 1 is the equivalent sample size, and is the strength of the +prior. +You can change this using +<pre> +tabular_CPD(bnet, i, 'prior_type', 'dirichlet', 'dirichlet_type', ... + 'BDeu', 'dirichlet_weight', 10); +</pre> +<!--where counts is an array of pseudo-counts of the same size as the +CPT.--> +<!-- +<p> +When you specify a prior, you should set row i of the CPT to the +normalized version of row i of the pseudo-count matrix, i.e., to the +expected values of the parameters. This will ensure that computing the +marginal likelihood sequentially (see <a +href="#bayes_learn">below</a>) and in batch form gives the same +results. +To do this, proceed as follows. +<pre> +tabular_CPD(bnet, i, 'prior', counts, 'CPT', mk_stochastic(counts)); +</pre> +For a non-informative prior, you can just write +<pre> +tabular_CPD(bnet, i, 'prior', 'unif', 'CPT', 'unif'); +</pre> +--> + + +<h2><a name="bayes_learn">(Sequential) Bayesian parameter updating from complete data</h2> + +If we use conjugate priors and have fully observed data, we can +compute the posterior over the parameters in batch form as follows. +<pre> +cases = sample_bnet(bnet, nsamples); +bnet = bayes_update_params(bnet, cases); +LL = log_marg_lik_complete(bnet, cases); +</pre> +bnet.CPD{i}.prior contains the new Dirichlet pseudocounts, +and bnet.CPD{i}.CPT is set to the mean of the posterior (the +normalized counts). +(Hence if the initial pseudo counts are 0, +<tt>bayes_update_params</tt> and <tt>learn_params</tt> will give the +same result.) + + + + +<p> +We can compute the same result sequentially (on-line) as follows. +<pre> +LL = 0; +for m=1:nsamples + LL = LL + log_marg_lik_complete(bnet, cases(:,m)); + bnet = bayes_update_params(bnet, cases(:,m)); +end +</pre> + +The file <tt>BNT/examples/static/StructLearn/model_select1</tt> has an example of +sequential model selection which uses the same idea. +We generate data from the model A->B +and compute the posterior prob of all 3 dags on 2 nodes: + (1) A B, (2) A <- B , (3) A -> B +Models 2 and 3 are <a href="#markov_equiv">Markov equivalent</a>, and therefore indistinguishable from +observational data alone, so we expect their posteriors to be the same +(assuming a prior which satisfies likelihood equivalence). +If we use random parameters, the "true" model only gets a higher posterior after 2000 trials! +However, if we make B a noisy NOT gate, the true model "wins" after 12 +trials, as shown below (red = model 1, blue/green (superimposed) +represents models 2/3). +<p> +<img src="Figures/model_select.png"> +<p> +The use of marginal likelihood for model selection is discussed in +greater detail in the +section on <a href="structure_learning">structure learning</a>. + + + + +<h2><a name="em">Maximum likelihood parameter estimation with missing values</h2> + +Now we consider learning when some values are not observed. +Let us randomly hide half the values generated from the water +sprinkler example. +<pre> +samples2 = samples; +hide = rand(N, nsamples) > 0.5; +[I,J]=find(hide); +for k=1:length(I) + samples2{I(k), J(k)} = []; +end +</pre> +samples2{i,l} is the value of node i in training case l, or [] if unobserved. +<p> +Now we will compute the MLEs using the EM algorithm. +We need to use an inference algorithm to compute the expected +sufficient statistics in the E step; the M (maximization) step is as +above. +<pre> +engine2 = jtree_inf_engine(bnet2); +max_iter = 10; +[bnet4, LLtrace] = learn_params_em(engine2, samples2, max_iter); +</pre> +LLtrace(i) is the log-likelihood at iteration i. We can plot this as +follows: +<pre> +plot(LLtrace, 'x-') +</pre> +Let's display the results after 10 iterations of EM. +<pre> +celldisp(CPT4) +CPT4{1} = + 0.6616 + 0.3384 +CPT4{2} = + 0.6510 0.3490 + 0.8751 0.1249 +CPT4{3} = + 0.8366 0.1634 + 0.0197 0.9803 +CPT4{4} = +(:,:,1) = + 0.8276 0.0546 + 0.5452 0.1658 +(:,:,2) = + 0.1724 0.9454 + 0.4548 0.8342 +</pre> +We can get improved performance by using one or more of the following +methods: +<ul> +<li> Increasing the size of the training set. +<li> Decreasing the amount of hidden data. +<li> Running EM for longer. +<li> Using informative priors. +<li> Initialising EM from multiple starting points. +</ul> + +Click <a href="#gaussian">here</a> for a discussion of learning +Gaussians, which can cause numerical problems. +<p> +For a more complete example of learning with EM, +see the script BNT/examples/static/learn1.m. + +<h2><a name="tying">Parameter tying</h2> + +In networks with repeated structure (e.g., chains and grids), it is +common to assume that the parameters are the same at every node. This +is called parameter tying, and reduces the amount of data needed for +learning. +<p> +When we have tied parameters, there is no longer a one-to-one +correspondence between nodes and CPDs. +Rather, each CPD species the parameters for a whole equivalence class +of nodes. +It is easiest to see this by example. +Consider the following <a href="usage_dbn.html#hmm">hidden Markov +model (HMM)</a> +<p> +<img src="Figures/hmm3.gif"> +<p> +<!-- +We can create this graph structure, assuming we have T time-slices, +as follows. +(We number the nodes as shown in the figure, but we could equally well +number the hidden nodes 1:T, and the observed nodes T+1:2T.) +<pre> +N = 2*T; +dag = zeros(N); +hnodes = 1:2:2*T; +for i=1:T-1 + dag(hnodes(i), hnodes(i+1))=1; +end +onodes = 2:2:2*T; +for i=1:T + dag(hnodes(i), onodes(i)) = 1; +end +</pre> +<p> +The hidden nodes are always discrete, and have Q possible values each, +but the observed nodes can be discrete or continuous, and have O possible values/length. +<pre> +if cts_obs + dnodes = hnodes; +else + dnodes = 1:N; +end +ns = ones(1,N); +ns(hnodes) = Q; +ns(onodes) = O; +</pre> +--> +When HMMs are used for semi-infinite processes like speech recognition, +we assume the transition matrix +P(H(t+1)|H(t)) is the same for all t; this is called a time-invariant +or homogenous Markov chain. +Hence hidden nodes 2, 3, ..., T +are all in the same equivalence class, say class Hclass. +Similarly, the observation matrix P(O(t)|H(t)) is assumed to be the +same for all t, so the observed nodes are all in the same equivalence +class, say class Oclass. +Finally, the prior term P(H(1)) is in a class all by itself, say class +H1class. +This is illustrated below, where we explicitly represent the +parameters as random variables (dotted nodes). +<p> +<img src="Figures/hmm4_params.gif"> +<p> +In BNT, we cannot represent parameters as random variables (nodes). +Instead, we "hide" the +parameters inside one CPD for each equivalence class, +and then specify that the other CPDs should share these parameters, as +follows. +<pre> +hnodes = 1:2:2*T; +onodes = 2:2:2*T; +H1class = 1; Hclass = 2; Oclass = 3; +eclass = ones(1,N); +eclass(hnodes(2:end)) = Hclass; +eclass(hnodes(1)) = H1class; +eclass(onodes) = Oclass; +% create dag and ns in the usual way +bnet = mk_bnet(dag, ns, 'discrete', dnodes, 'equiv_class', eclass); +</pre> +Finally, we define the parameters for each equivalence class: +<pre> +bnet.CPD{H1class} = tabular_CPD(bnet, hnodes(1)); % prior +bnet.CPD{Hclass} = tabular_CPD(bnet, hnodes(2)); % transition matrix +if cts_obs + bnet.CPD{Oclass} = gaussian_CPD(bnet, onodes(1)); +else + bnet.CPD{Oclass} = tabular_CPD(bnet, onodes(1)); +end +</pre> +In general, if bnet.CPD{e} = xxx_CPD(bnet, j), then j should be a +member of e's equivalence class; that is, it is not always the case +that e == j. You can use bnet.rep_of_eclass(e) to return the +representative of equivalence class e. +BNT will look up the parents of j to determine the size +of the CPT to use. It assumes that this is the same for all members of +the equivalence class. +Click <a href="param_tieing.html">here</a> for +a more complex example of parameter tying. +<p> +Note: +Normally one would define an HMM as a +<a href = "usage_dbn.html">Dynamic Bayes Net</a> +(see the function BNT/examples/dynamic/mk_chmm.m). +However, one can define an HMM as a static BN using the function +BNT/examples/static/Models/mk_hmm_bnet.m. + + + +<h1><a name="structure_learning">Structure learning</h1> + +Update (9/29/03): +Phillipe LeRay is developing some additional structure learning code +on top of BNT. Click <a +href="http://bnt.insa-rouen.fr/ajouts.html">here</a> +for details. + +<p> + +There are two very different approaches to structure learning: +constraint-based and search-and-score. +In the <a href="#constraint">constraint-based approach</a>, +we start with a fully connected graph, and remove edges if certain +conditional independencies are measured in the data. +This has the disadvantage that repeated independence tests lose +statistical power. +<p> +In the more popular search-and-score approach, +we perform a search through the space of possible DAGs, and either +return the best one found (a point estimate), or return a sample of the +models found (an approximation to the Bayesian posterior). +<p> +Unfortunately, the number of DAGs as a function of the number of +nodes, G(n), is super-exponential in n. +A closed form formula for G(n) is not known, but the first few values +are shown below (from Cooper, 1999). + +<table> +<tr> <th>n</th> <th align=left>G(n)</th> </tr> +<tr> <td>1</td> <td>1</td> </tr> +<tr> <td>2</td> <td>3</td> </tr> +<tr> <td>3</td> <td>25</td> </tr> +<tr> <td>4</td> <td>543</td> </tr> +<tr> <td>5</td> <td>29,281</td> </tr> +<tr> <td>6</td> <td>3,781,503</td> </tr> +<tr> <td>7</td> <td>1.1 x 10^9</td> </tr> +<tr> <td>8</td> <td>7.8 x 10^11</td> </tr> +<tr> <td>9</td> <td>1.2 x 10^15</td> </tr> +<tr> <td>10</td> <td>4.2 x 10^18</td> </tr> +</table> + +Since the number of DAGs is super-exponential in the number of nodes, +we cannot exhaustively search the space, so we either use a local +search algorithm (e.g., greedy hill climbining, perhaps with multiple +restarts) or a global search algorithm (e.g., Markov Chain Monte +Carlo). +<p> +If we know a total ordering on the nodes, +finding the best structure amounts to picking the best set of parents +for each node independently. +This is what the K2 algorithm does. +If the ordering is unknown, we can search over orderings, +which is more efficient than searching over DAGs (Koller and Friedman, 2000). +<p> +In addition to the search procedure, we must specify the scoring +function. There are two popular choices. The Bayesian score integrates +out the parameters, i.e., it is the marginal likelihood of the model. +The BIC (Bayesian Information Criterion) is defined as +log P(D|theta_hat) - 0.5*d*log(N), where D is the data, theta_hat is +the ML estimate of the parameters, d is the number of parameters, and +N is the number of data cases. +The BIC method has the advantage of not requiring a prior. +<p> +BIC can be derived as a large sample +approximation to the marginal likelihood. +(It is also equal to the Minimum Description Length of a model.) +However, in practice, the sample size does not need to be very large +for the approximation to be good. +For example, in the figure below, we plot the ratio between the log marginal likelihood +and the BIC score against data-set size; we see that the ratio rapidly +approaches 1, especially for non-informative priors. +(This plot was generated by the file BNT/examples/static/bic1.m. It +uses the water sprinkler BN with BDeu Dirichlet priors with different +equivalent sample sizes.) + +<p> +<center> +<IMG SRC="Figures/bic.png"> +</center> +<p> + +<p> +As with parameter learning, handling missing data/ hidden variables is +much harder than the fully observed case. +The structure learning routines in BNT can therefore be classified into 4 +types, analogously to the parameter learning case. +<p> + +<TABLE BORDER> +<tr> + <TH></TH> + <th>Full obs</th> + <th>Partial obs</th> +</tr> +<tr> + <th>Point</th> + <td><tt>learn_struct_K2</tt> <br> +<!-- <tt>learn_struct_hill_climb</tt></td> --> + <td><tt>not yet supported</tt></td> +</tr> +<tr> + <th>Bayes</th> + <td><tt>learn_struct_mcmc</tt></td> + <td>not yet supported</td> +</tr> +</table> + + +<h2><a name="markov_equiv">Markov equivalence</h2> + +If two DAGs encode the same conditional independencies, they are +called Markov equivalent. The set of all DAGs can be paritioned into +Markov equivalence classes. Graphs within the same class can +have +the direction of some of their arcs reversed without changing any of +the CI relationships. +Each class can be represented by a PDAG +(partially directed acyclic graph) called an essential graph or +pattern. This specifies which edges must be oriented in a certain +direction, and which may be reversed. + +<p> +When learning graph structure from observational data, +the best one can hope to do is to identify the model up to Markov +equivalence. To distinguish amongst graphs within the same equivalence +class, one needs interventional data: see the discussion on <a +href="#active">active learning</a> below. + + + +<h2><a name="enumerate">Exhaustive search</h2> + +The brute-force approach to structure learning is to enumerate all +possible DAGs, and score each one. This provides a "gold standard" +with which to compare other algorithms. We can do this as follows. +<pre> +dags = mk_all_dags(N); +score = score_dags(data, ns, dags); +</pre> +where data(i,m) is the value of node i in case m, +and ns(i) is the size of node i. +If the DAGs have a lot of families in common, we can cache the sufficient statistics, +making this potentially more efficient than scoring the DAGs one at a time. +(Caching is not currently implemented, however.) +<p> +By default, we use the Bayesian scoring metric, and assume CPDs are +represented by tables with BDeu(1) priors. +We can override these defaults as follows. +If we want to use uniform priors, we can say +<pre> +params = cell(1,N); +for i=1:N + params{i} = {'prior', 'unif'}; +end +score = score_dags(data, ns, dags, 'params', params); +</pre> +params{i} is a cell-array, containing optional arguments that are +passed to the constructor for CPD i. +<p> +Now suppose we want to use different node types, e.g., +Suppose nodes 1 and 2 are Gaussian, and nodes 3 and 4 softmax (both +these CPDs can support discrete and continuous parents, which is +necessary since all other nodes will be considered as parents). +The Bayesian scoring metric currently only works for tabular CPDs, so +we will use BIC: +<pre> +score = score_dags(data, ns, dags, 'discrete', [3 4], 'params', [], + 'type', {'gaussian', 'gaussian', 'softmax', softmax'}, 'scoring_fn', 'bic') +</pre> +In practice, one can't enumerate all possible DAGs for N > 5, +but one can evaluate any reasonably-sized set of hypotheses in this +way (e.g., nearest neighbors of your current best guess). +Think of this as "computer assisted model refinement" as opposed to de +novo learning. + + +<h2><a name="K2">K2</h2> + +The K2 algorithm (Cooper and Herskovits, 1992) is a greedy search algorithm that works as follows. +Initially each node has no parents. It then adds incrementally that parent whose addition most +increases the score of the resulting structure. When the addition of no single +parent can increase the score, it stops adding parents to the node. +Since we are using a fixed ordering, we do not need to check for +cycles, and can choose the parents for each node independently. +<p> +The original paper used the Bayesian scoring +metric with tabular CPDs and Dirichlet priors. +BNT generalizes this to allow any kind of CPD, and either the Bayesian +scoring metric or BIC, as in the example <a href="#enumerate">above</a>. +In addition, you can specify +an optional upper bound on the number of parents for each node. +The file BNT/examples/static/k2demo1.m gives an example of how to use K2. +We use the water sprinkler network and sample 100 cases from it as before. +Then we see how much data it takes to recover the generating structure: +<pre> +order = [C S R W]; +max_fan_in = 2; +sz = 5:5:100; +for i=1:length(sz) + dag2 = learn_struct_K2(data(:,1:sz(i)), node_sizes, order, 'max_fan_in', max_fan_in); + correct(i) = isequal(dag, dag2); +end +</pre> +Here are the results. +<pre> +correct = + Columns 1 through 12 + 0 0 0 0 0 0 0 1 0 1 1 1 + Columns 13 through 20 + 1 1 1 1 1 1 1 1 +</pre> +So we see it takes about sz(10)=50 cases. (BIC behaves similarly, +showing that the prior doesn't matter too much.) +In general, we cannot hope to recover the "true" generating structure, +only one that is in its <a href="#markov_equiv">Markov equivalence +class</a>. + + +<h2><a name="hill_climb">Hill-climbing</h2> + +Hill-climbing starts at a specific point in space, +considers all nearest neighbors, and moves to the neighbor +that has the highest score; if no neighbors have higher +score than the current point (i.e., we have reached a local maximum), +the algorithm stops. One can then restart in another part of the space. +<p> +A common definition of "neighbor" is all graphs that can be +generated from the current graph by adding, deleting or reversing a +single arc, subject to the acyclicity constraint. +Other neighborhoods are possible: see +<a href="http://research.microsoft.com/~dmax/publications/jmlr02.pdf"> +Optimal Structure Identification with Greedy Search</a>, Max +Chickering, JMLR 2002. + +<!-- +Note: This algorithm is currently (Feb '02) being implemented by Qian +Diao. +--> + + +<h2><a name="mcmc">MCMC</h2> + +We can use a Markov Chain Monte Carlo (MCMC) algorithm called +Metropolis-Hastings (MH) to search the space of all +DAGs. +The standard proposal distribution is to consider moving to all +nearest neighbors in the sense defined <a href="#hill_climb">above</a>. +<p> +The function can be called +as in the following example. +<pre> +[sampled_graphs, accept_ratio] = learn_struct_mcmc(data, ns, 'nsamples', 100, 'burnin', 10); +</pre> +We can convert our set of sampled graphs to a histogram +(empirical posterior over all the DAGs) thus +<pre> +all_dags = mk_all_dags(N); +mcmc_post = mcmc_sample_to_hist(sampled_graphs, all_dags); +</pre> +To see how well this performs, let us compute the exact posterior exhaustively. +<p> +<pre> +score = score_dags(data, ns, all_dags); +post = normalise(exp(score)); % assuming uniform structural prior +</pre> +We plot the results below. +(The data set was 100 samples drawn from a random 4 node bnet; see the +file BNT/examples/static/mcmc1.) +<pre> +subplot(2,1,1) +bar(post) +subplot(2,1,2) +bar(mcmc_post) +</pre> +<img src="Figures/mcmc_post.jpg" width="800" height="500"> +<p> +We can also plot the acceptance ratio versus number of MCMC steps, +as a crude convergence diagnostic. +<pre> +clf +plot(accept_ratio) +</pre> +<img src="Figures/mcmc_accept.jpg" width="800" height="300"> +<p> +Even though the number of samples needed by MCMC is theoretically +polynomial (not exponential) in the dimensionality of the search space, in practice it has been +found that MCMC does not converge in reasonable time for graphs with +more than about 10 nodes. + + + + +<h2><a name="active">Active structure learning</h2> + +As was mentioned <a href="#markov_equiv">above</a>, +one can only learn a DAG up to Markov equivalence, even given infinite data. +If one is interested in learning the structure of a causal network, +one needs interventional data. +(By "intervention" we mean forcing a node to take on a specific value, +thereby effectively severing its incoming arcs.) +<p> +Most of the scoring functions accept an optional argument +that specifies whether a node was observed to have a certain value, or +was forced to have that value: we set clamped(i,m)=1 if node i was +forced in training case m. e.g., see the file +BNT/examples/static/cooper_yoo. +<p> +An interesting question is to decide which interventions to perform +(c.f., design of experiments). For details, see the following tech +report +<ul> +<li> <a href = "../../Papers/alearn.ps.gz"> +Active learning of causal Bayes net structure</a>, Kevin Murphy, March +2001. +</ul> + + +<h2><a name="struct_em">Structural EM</h2> + +Computing the Bayesian score when there is partial observability is +computationally challenging, because the parameter posterior becomes +multimodal (the hidden nodes induce a mixture distribution). +One therefore needs to use approximations such as BIC. +Unfortunately, search algorithms are still expensive, because we need +to run EM at each step to compute the MLE, which is needed to compute +the score of each model. An alternative approach is +to do the local search steps inside of the M step of EM, which is more +efficient since the data has been "filled in" - this is +called the structural EM algorithm (Friedman 1997), and provably +converges to a local maximum of the BIC score. +<p> +Wei Hu has implemented SEM for discrete nodes. +You can download his package from +<a href="../SEM.zip">here</a>. +Please address all questions about this code to +wei.hu@intel.com. +See also <a href="#phl">Phl's implementation of SEM</a>. + +<!-- +<h2><a name="reveal">REVEAL algorithm</h2> + +A simple way to learn the structure of a fully observed, discrete, +factored DBN from a time series is described <a +href="usage_dbn.html#struct_learn">here</a>. +--> + + +<h2><a name="graphdraw">Visualizing the graph</h2> + +You can visualize an arbitrary graph (such as one learned using the +structure learning routines) with Matlab code contributed by +<a href="http://www.mbfys.kun.nl/~cemgil/matlab/layout.html">Ali +Taylan Cemgil</a> +from the University of Nijmegen. +For static BNs, call it as follows: +<pre> +draw_graph(bnet.dag); +</pre> +For example, this is the output produced on a +<a href="#qmr">random QMR-like model</a>: +<p> +<img src="Figures/qmr.rnd.jpg"> +<p> +If you install the excellent <a +href="http://www.research.att.com/sw/tools/graphviz">graphhviz</a>, an +open-source graph visualization package from AT&T, +you can create a much better visualization as follows +<pre> +graph_to_dot(bnet.dag) +</pre> +This works by converting the adjacency matrix to a file suitable +for input to graphviz (using the dot format), +then converting the output of graphviz to postscript, and displaying the results using +ghostview. +You can do each of these steps separately for more control, as shown +below. +<pre> +graph_to_dot(bnet.dag, 'filename', 'foo.dot'); +dot -Tps foo.dot -o foo.ps +ghostview foo.ps & +</pre> + +<h2><a name = "constraint">Constraint-based methods</h2> + +The IC algorithm (Pearl and Verma, 1991), +and the faster, but otherwise equivalent, PC algorithm (Spirtes, Glymour, and Scheines 1993), +computes many conditional independence tests, +and combines these constraints into a +PDAG to represent the whole +<a href="#markov_equiv">Markov equivalence class</a>. +<p> +IC*/FCI extend IC/PC to handle latent variables: see <a href="#ic_star">below</a>. +(IC stands for inductive causation; PC stands for Peter and Clark, +the first names of Spirtes and Glymour; FCI stands for fast causal +inference. +What we, following Pearl (2000), call IC* was called +IC in the original Pearl and Verma paper.) +For details, see +<ul> +<li> +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Causation, +Prediction, and Search</a>, Spirtes, Glymour and +Scheines (SGS), 2001 (2nd edition), MIT Press. +<li> +<a href="http://bayes.cs.ucla.edu/BOOK-2K/index.html">Causality: Models, Reasoning and Inference</a>, J. Pearl, +2000, Cambridge University Press. +</ul> + +<p> + +The PC algorithm takes as arguments a function f, the number of nodes N, +the maximum fan in K, and additional arguments A which are passed to f. +The function f(X,Y,S,A) returns 1 if X is conditionally independent of Y given S, and 0 +otherwise. +For example, suppose we cheat by +passing in a CI "oracle" which has access to the true DAG; the oracle +tests for d-separation in this DAG, i.e., +f(X,Y,S) calls dsep(X,Y,S,dag). We can to this as follows. +<pre> +pdag = learn_struct_pdag_pc('dsep', N, max_fan_in, dag); +</pre> +pdag(i,j) = -1 if there is definitely an i->j arc, +and pdag(i,j) = 1 if there is either an i->j or and i<-j arc. +<p> +Applied to the sprinkler network, this returns +<pre> +pdag = + 0 1 1 0 + 1 0 0 -1 + 1 0 0 -1 + 0 0 0 0 +</pre> +So as expected, we see that the V-structure at the W node is uniquely identified, +but the other arcs have ambiguous orientation. +<p> +We now give an example from p141 (1st edn) / p103 (2nd end) of the SGS +book. +This example concerns the female orgasm. +We are given a correlation matrix C between 7 measured factors (such +as subjective experiences of coital and masturbatory experiences), +derived from 281 samples, and want to learn a causal model of the +data. We will not discuss the merits of this type of work here, but +merely show how to reproduce the results in the SGS book. +Their program, +<a href="http://hss.cmu.edu/html/departments/philosophy/TETRAD/tetrad.html">Tetrad</a>, +makes use of the Fisher Z-test for conditional +independence, so we do the same: +<pre> +max_fan_in = 4; +nsamples = 281; +alpha = 0.05; +pdag = learn_struct_pdag_pc('cond_indep_fisher_z', n, max_fan_in, C, nsamples, alpha); +</pre> +In this case, the CI test is +<pre> +f(X,Y,S) = cond_indep_fisher_z(X,Y,S, C,nsamples,alpha) +</pre> +The results match those of Fig 12a of SGS apart from two edge +differences; presumably this is due to rounding error (although it +could be a bug, either in BNT or in Tetrad). +This example can be found in the file BNT/examples/static/pc2.m. + +<p> + +The IC* algorithm (Pearl and Verma, 1991), +and the faster FCI algorithm (Spirtes, Glymour, and Scheines 1993), +are like the IC/PC algorithm, except that they can detect the presence +of latent variables. +See the file <tt>learn_struct_pdag_ic_star</tt> written by Tamar +Kushnir. The output is a matrix P, defined as follows +(see Pearl (2000), p52 for details): +<pre> +% P(i,j) = -1 if there is either a latent variable L such that i <-L->j OR there is a directed edge from i->j. +% P(i,j) = -2 if there is a marked directed i-*>j edge. +% P(i,j) = P(j,i) = 1 if there is and undirected edge i--j +% P(i,j) = P(j,i) = 2 if there is a latent variable L such that i<-L->j. +</pre> + + +<h2><a name="phl">Philippe Leray's structure learning package</h2> + +Philippe Leray has written a +<a href="http://bnt.insa-rouen.fr/ajouts.html"> +structure learning package</a> that uses BNT. + +It currently (Juen 2003) has the following features: +<ul> +<li>PC with Chi2 statistical test +<li> MWST : Maximum weighted Spanning Tree +<li> Hill Climbing +<li> Greedy Search +<li> Structural EM +<li> hist_ic : optimal Histogram based on IC information criterion +<li> cpdag_to_dag +<li> dag_to_cpdag +<li> ... +</ul> + + +</a> + + +<!-- +<h2><a name="read_learning">Further reading on learning</h2> + +I recommend the following tutorials for more details on learning. +<ul> +<li> <a +href="http://www.cs.berkeley.edu/~murphyk/Papers/intel.ps.gz">My short +tutorial</a> on graphical models, which contains an overview of learning. + +<li> +<A HREF="ftp://ftp.research.microsoft.com/pub/tr/TR-95-06.PS"> +A tutorial on learning with Bayesian networks</a>, D. Heckerman, +Microsoft Research Tech Report, 1995. + +<li> <A HREF="http://www-cad.eecs.berkeley.edu/~wray/Mirror/lwgmja"> +Operations for Learning with Graphical Models</a>, +W. L. Buntine, JAIR'94, 159--225. +</ul> +<p> +--> + + + + + +<h1><a name="engines">Inference engines</h1> + +Up until now, we have used the junction tree algorithm for inference. +However, sometimes this is too slow, or not even applicable. +In general, there are many inference algorithms each of which make +different tradeoffs between speed, accuracy, complexity and +generality. Furthermore, there might be many implementations of the +same algorithm; for instance, a general purpose, readable version, +and a highly-optimized, specialized one. +To cope with this variety, we treat each inference algorithm as an +object, which we call an inference engine. + +<p> +An inference engine is an object that contains a bnet and supports the +'enter_evidence' and 'marginal_nodes' methods. The engine constructor +takes the bnet as argument and may do some model-specific processing. +When 'enter_evidence' is called, the engine may do some +evidence-specific processing. Finally, when 'marginal_nodes' is +called, the engine may do some query-specific processing. + +<p> +The amount of work done when each stage is specified -- structure, +parameters, evidence, and query -- depends on the engine. The cost of +work done early in this sequence can be amortized. On the other hand, +one can make better optimizations if one waits until later in the +sequence. +For example, the parameters might imply +conditional indpendencies that are not evident in the graph structure, +but can nevertheless be exploited; the evidence indicates which nodes +are observed and hence can effectively be disconnected from the +graph; and the query might indicate that large parts of the network +are d-separated from the query nodes. (Since it is not the actual +<em>values</em> of the evidence that matters, just which nodes are observed, +many engines allow you to specify which nodes will be observed when they are constructed, +i.e., before calling 'enter_evidence'. Some engines can still cope if +the actual pattern of evidence is different, e.g., if there is missing +data.) +<p> + +Although being maximally lazy (i.e., only doing work when a query is +issued) may seem desirable, +this is not always the most efficient. +For example, +when learning using EM, we need to call marginal_nodes N times, where N is the +number of nodes. <a href="varelim">Variable elimination</a> would end +up repeating a lot of work +each time marginal_nodes is called, making it inefficient for +learning. The junction tree algorithm, by contrast, uses dynamic +programming to avoid this redundant computation --- it calculates all +marginals in two passes during 'enter_evidence', so calling +'marginal_nodes' takes constant time. +<p> +We will discuss some of the inference algorithms implemented in BNT +below, and finish with a <a href="#engine_summary">summary</a> of all +of them. + + + + + + + +<h2><a name="varelim">Variable elimination</h2> + +The variable elimination algorithm, also known as bucket elimination +or peeling, is one of the simplest inference algorithms. +The basic idea is to "push sums inside of products"; this is explained +in more detail +<a +href="http://HTTP.CS.Berkeley.EDU/~murphyk/Bayes/bayes.html#infer">here</a>. +<p> +The principle of distributing sums over products can be generalized +greatly to apply to any commutative semiring. +This forms the basis of many common algorithms, such as Viterbi +decoding and the Fast Fourier Transform. For details, see + +<ul> +<li> R. McEliece and S. M. Aji, 2000. +<!--<a href="http://www.systems.caltech.edu/EE/Faculty/rjm/papers/GDL.ps">--> +<a href="GDL.pdf"> +The Generalized Distributive Law</a>, +IEEE Trans. Inform. Theory, vol. 46, no. 2 (March 2000), +pp. 325--343. + + +<li> +F. R. Kschischang, B. J. Frey and H.-A. Loeliger, 2001. +<a href="http://www.cs.toronto.edu/~frey/papers/fgspa.abs.html"> +Factor graphs and the sum-product algorithm</a> +IEEE Transactions on Information Theory, February, 2001. + +</ul> + +<p> +Choosing an order in which to sum out the variables so as to minimize +computational cost is known to be NP-hard. +The implementation of this algorithm in +<tt>var_elim_inf_engine</tt> makes no attempt to optimize this +ordering (in contrast, say, to <tt>jtree_inf_engine</tt>, which uses a +greedy search procedure to find a good ordering). +<p> +Note: unlike most algorithms, var_elim does all its computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + + + +<h2><a name="global">Global inference methods</h2> + +The simplest inference algorithm of all is to explicitely construct +the joint distribution over all the nodes, and then to marginalize it. +This is implemented in <tt>global_joint_inf_engine</tt>. +Since the size of the joint is exponential in the +number of discrete (hidden) nodes, this is not a very practical algorithm. +It is included merely for pedagogical and debugging purposes. +<p> +Three specialized versions of this algorithm have also been implemented, +corresponding to the cases where all the nodes are discrete (D), all +are Gaussian (G), and some are discrete and some Gaussian (CG). +They are called <tt>enumerative_inf_engine</tt>, +<tt>gaussian_inf_engine</tt>, +and <tt>cond_gauss_inf_engine</tt> respectively. +<p> +Note: unlike most algorithms, these global inference algorithms do all their computational work +inside of <tt>marginal_nodes</tt>, not inside of +<tt>enter_evidence</tt>. + + +<h2><a name="quickscore">Quickscore</h2> + +The junction tree algorithm is quite slow on the <a href="#qmr">QMR</a> network, +since the cliques are so big. +One simple trick we can use is to notice that hidden leaves do not +affect the posteriors on the roots, and hence do not need to be +included in the network. +A second trick is to notice that the negative findings can be +"absorbed" into the prior: +see the file +BNT/examples/static/mk_minimal_qmr_bnet for details. +<p> + +A much more significant speedup is obtained by exploiting special +properties of the noisy-or node, as done by the quickscore +algorithm. For details, see +<ul> +<li> Heckerman, "A tractable inference algorithm for diagnosing multiple diseases", UAI 89. +<li> Rish and Dechter, "On the impact of causal independence", UCI +tech report, 1998. +</ul> + +This has been implemented in BNT as a special-purpose inference +engine, which can be created and used as follows: +<pre> +engine = quickscore_inf_engine(inhibit, leak, prior); +engine = enter_evidence(engine, pos, neg); +m = marginal_nodes(engine, i); +</pre> + + +<h2><a name="belprop">Belief propagation</h2> + +Even using quickscore, exact inference takes time that is exponential +in the number of positive findings. +Hence for large networks we need to resort to approximate inference techniques. +See for example +<ul> +<li> T. Jaakkola and M. Jordan, "Variational probabilistic inference and the +QMR-DT network", JAIR 10, 1999. + +<li> K. Murphy, Y. Weiss and M. Jordan, "Loopy belief propagation for approximate inference: an empirical study", + UAI 99. +</ul> +The latter approximation +entails applying Pearl's belief propagation algorithm to a model even +if it has loops (hence the name loopy belief propagation). +Pearl's algorithm, implemented as <tt>pearl_inf_engine</tt>, gives +exact results when applied to singly-connected graphs +(a.k.a. polytrees, since +the underlying undirected topology is a tree, but a node may have +multiple parents). +To apply this algorithm to a graph with loops, +use <tt>pearl_inf_engine</tt>. +This can use a centralized or distributed message passing protocol. +You can use it as in the following example. +<pre> +engine = pearl_inf_engine(bnet, 'max_iter', 30); +engine = enter_evidence(engine, evidence); +m = marginal_nodes(engine, i); +</pre> +We found that this algorithm often converges, and when it does, often +is very accurate, but it depends on the precise setting of the +parameter values of the network. +(See the file BNT/examples/static/qmr1 to repeat the experiment for yourself.) +Understanding when and why belief propagation converges/ works +is a topic of ongoing research. +<p> +<tt>pearl_inf_engine</tt> can exploit special structure in noisy-or +and gmux nodes to compute messages efficiently. +<p> +<tt>belprop_inf_engine</tt> is like pearl, but uses potentials to +represent messages. Hence this is slower. +<p> +<tt>belprop_fg_inf_engine</tt> is like belprop, +but is designed for factor graphs. + + + +<h2><a name="sampling">Sampling</h2> + +BNT now (Mar '02) has two sampling (Monte Carlo) inference algorithms: +<ul> +<li> <tt>likelihood_weighting_inf_engine</tt> which does importance +sampling and can handle any node type. +<li> <tt>gibbs_sampling_inf_engine</tt>, written by Bhaskara Marthi. +Currently this can only handle tabular CPDs. +For a much faster and more powerful Gibbs sampling program, see +<a href="http://www.mrc-bsu.cam.ac.uk/bugs">BUGS</a>. +</ul> +Note: To generate samples from a network (which is not the same as inference!), +use <tt>sample_bnet</tt>. + + + +<h2><a name="engine_summary">Summary of inference engines</h2> + + +The inference engines differ in many ways. Here are +some of the major "axes": +<ul> +<li> Works for all topologies or makes restrictions? +<li> Works for all node types or makes restrictions? +<li> Exact or approximate inference? +</ul> + +<p> +In terms of topology, most engines handle any kind of DAG. +<tt>belprop_fg</tt> does approximate inference on factor graphs (FG), which +can be used to represent directed, undirected, and mixed (chain) +graphs. +(In the future, we plan to support exact inference on chain graphs.) +<tt>quickscore</tt> only works on QMR-like models. +<p> +In terms of node types: algorithms that use potentials can handle +discrete (D), Gaussian (G) or conditional Gaussian (CG) models. +Sampling algorithms can essentially handle any kind of node (distribution). +Other algorithms make more restrictive assumptions in exchange for +speed. +<p> +Finally, most algorithms are designed to give the exact answer. +The belief propagation algorithms are exact if applied to trees, and +in some other cases. +Sampling is considered approximate, even though, in the limit of an +infinite number of samples, it gives the exact answer. + +<p> + +Here is a summary of the properties +of all the engines in BNT which work on static networks. +<p> +<table> +<table border units = pixels><tr> +<td align=left width=0>Name +<td align=left width=0>Exact? +<td align=left width=0>Node type? +<td align=left width=0>topology +<tr> +<tr> +<td align=left> belprop +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> belprop_fg +<td align=left> approx +<td align=left> D +<td align=left> factor graph +<tr> +<td align=left> cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> enumerative +<td align=left> exact +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> gaussian +<td align=left> exact +<td align=left> G +<td align=left> DAG +<tr> +<td align=left> gibbs +<td align=left> approx +<td align=left> D +<td align=left> DAG +<tr> +<td align=left> global_joint +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +<tr> +<td align=left> jtree +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +b<tr> +<td align=left> likelihood_weighting +<td align=left> approx +<td align=left> any +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> approx +<td align=left> D,G +<td align=left> DAG +<tr> +<td align=left> pearl +<td align=left> exact +<td align=left> D,G +<td align=left> polytree +<tr> +<td align=left> quickscore +<td align=left> exact +<td align=left> noisy-or +<td align=left> QMR +<tr> +<td align=left> stab_cond_gauss +<td align=left> exact +<td align=left> CG +<td align=left> DAG +<tr> +<td align=left> var_elim +<td align=left> exact +<td align=left> D,G,CG +<td align=left> DAG +</table> + + + +<h1><a name="influence">Influence diagrams/ decision making</h1> + +BNT implements an exact algorithm for solving LIMIDs (limited memory +influence diagrams), described in +<ul> +<li> S. L. Lauritzen and D. Nilsson. +<a href="http://www.math.auc.dk/~steffen/papers/limids.pdf"> +Representing and solving decision problems with limited +information</a> +Management Science, 47, 1238 - 1251. September 2001. +</ul> +LIMIDs explicitely show all information arcs, rather than implicitely +assuming no forgetting. This allows them to model forgetful +controllers. +<p> +See the examples in <tt>BNT/examples/limids</tt> for details. + + + + +<h1>DBNs, HMMs, Kalman filters and all that</h1> + +Click <a href="usage_dbn.html">here</a> for documentation about how to +use BNT for dynamical systems and sequence data. + + +</BODY> diff --git a/sourcecodes/bnt-master/docs/whyNotSourceforge.html b/sourcecodes/bnt-master/docs/whyNotSourceforge.html new file mode 100644 index 00000000..7c17e425 --- /dev/null +++ b/sourcecodes/bnt-master/docs/whyNotSourceforge.html @@ -0,0 +1,17 @@ +On 4 October 2007, I decided to move BNT back from +<a href="http://bnt.sourceforge.net/">sourceforge</a> to my +own website, where I could modify it more easily. I have no plans to +add new functionality, but I at least want it to ensure it runs +correctly on new versions of matlab. Despite new users complaining on +the Yahoo group that BNT would +not work on the latest version of Matlab, nobody in the sourceforce +community did anything about it. As an opensource project, it was basically +a failure (although I would like +to thank Nicholas Saunier and Tom Murray (yozhik) for +making code updates, +as well as Hiroaki Ogawa and mayliszt, +and Bob Welch for answering many questions on the newsgroup.) +For historical purposes, +v1.0.3 (as of 4 Oct 07) has been cached +<a href="http://www.cs.ubc.ca/~murphyk/Software/BNT/FullBNT-1.0.3.zip">here</a>. + |
