diff options
| author | ziejd2 | 2017-09-28 15:04:40 -0500 |
|---|---|---|
| committer | ziejd2 | 2017-09-28 15:04:40 -0500 |
| commit | 8070dc963753142bb86c4ed698d91fd623ed28e7 (patch) | |
| tree | d0f6dd8fc46a49b819aa55c1a90faa14d8448883 /sourcecodes/bnt-master/nethelp3.3 | |
| parent | 7cc31810d53176e805532b2789955f4eedbce6bb (diff) | |
| download | BNW-8070dc963753142bb86c4ed698d91fd623ed28e7.tar.gz | |
BNW using Octave instead of Matlab.
This version of BNW should perform the same as the original version. The only difference is that it uses Octave instead of Matlab when running BayesNet Toolbox during parameter learning. I am calling this BNW_1.02. It can be accessed at: compbio.uthsc.edu/BNW_1.02
Diffstat (limited to 'sourcecodes/bnt-master/nethelp3.3')
176 files changed, 10080 insertions, 0 deletions
diff --git a/sourcecodes/bnt-master/nethelp3.3/CVS/Entries b/sourcecodes/bnt-master/nethelp3.3/CVS/Entries new file mode 100644 index 00000000..f617382b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/CVS/Entries @@ -0,0 +1,174 @@ +/conffig.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/confmat.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/conjgrad.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/consist.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/convertoldnet.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/datread.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/datwrite.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/dem2ddat.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/demard.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/demev1.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/demev2.htm/1.1.1.1/Wed Apr 27 17:58:54 2005// +/demev3.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgauss.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demglm1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demglm2.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgmm1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgmm2.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgmm3.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgmm4.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgmm5.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgp.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgpard.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgpot.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgtm1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demgtm2.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demhint.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demhmc1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demhmc2.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demhmc3.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demkmn1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demknn1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demmdn1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demmet1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demmlp1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demmlp2.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demnlab.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demns1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demolgd1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demopt1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/dempot.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demprgp.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demprior.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demrbf1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demsom1.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/demtrain.htm/1.1.1.1/Wed Apr 27 17:58:56 2005// +/dist2.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/eigdec.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/errbayes.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/evidence.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/fevbayes.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gauss.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gbayes.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glm.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmderiv.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmerr.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmevfwd.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmfwd.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmgrad.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmhess.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glminit.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmpak.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmtrain.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/glmunpak.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmm.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmactiv.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmem.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmminit.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmpak.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmpost.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmprob.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmsamp.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gmmunpak.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gp.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gpcovar.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gpcovarf.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gpcovarp.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gperr.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gpfwd.htm/1.1.1.1/Wed Apr 27 17:58:58 2005// +/gpgrad.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gpinit.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gppak.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gpunpak.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gradchek.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/graddesc.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gsamp.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtm.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmem.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmfwd.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtminit.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmlmean.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmlmode.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmmag.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmpost.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/gtmprob.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/hbayes.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/hesschek.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/hintmat.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/hinton.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/histp.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/hmc.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/index.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/kmeans.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/knn.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/knnfwd.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/linef.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/linemin.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/maxitmess.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/mdn.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/mdn2gmm.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/mdndist2.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/mdnerr.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/mdnfwd.htm/1.1.1.1/Wed Apr 27 17:59:00 2005// +/mdngrad.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mdninit.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mdnpak.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mdnpost.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mdnprob.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mdnunpak.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/metrop.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/minbrack.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlp.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpbkp.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpderiv.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlperr.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpevfwd.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpfwd.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpgrad.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlphdotv.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlphess.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlphint.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpinit.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlppak.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpprior.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlptrain.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/mlpunpak.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netderiv.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/neterr.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netevfwd.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netgrad.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/nethelp3.3.zip/1.1.1.1/Wed Apr 27 17:59:02 2005// +/nethess.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netinit.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netopt.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netpak.htm/1.1.1.1/Wed Apr 27 17:59:02 2005// +/netunpak.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/olgd.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/pca.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/plotmat.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/ppca.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/quasinew.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbf.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfbkp.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfderiv.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbferr.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfevfwd.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbffwd.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfgrad.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfhess.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfjacob.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfpak.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfprior.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfsetbf.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfsetfw.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbftrain.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rbfunpak.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rosegrad.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/rosen.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/scg.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/som.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/somfwd.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/sompak.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/somtrain.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +/somunpak.htm/1.1.1.1/Wed Apr 27 17:59:04 2005// +D diff --git a/sourcecodes/bnt-master/nethelp3.3/CVS/Repository b/sourcecodes/bnt-master/nethelp3.3/CVS/Repository new file mode 100644 index 00000000..d6658c37 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/CVS/Repository @@ -0,0 +1 @@ +FullBNT/nethelp3.3 diff --git a/sourcecodes/bnt-master/nethelp3.3/CVS/Root b/sourcecodes/bnt-master/nethelp3.3/CVS/Root new file mode 100644 index 00000000..f3bd14a6 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/CVS/Root @@ -0,0 +1 @@ +:ext:nsaunier@bnt.cvs.sourceforge.net:/cvsroot/bnt diff --git a/sourcecodes/bnt-master/nethelp3.3/conffig.htm b/sourcecodes/bnt-master/nethelp3.3/conffig.htm new file mode 100644 index 00000000..52aca7b9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/conffig.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual conffig +</title> +</head> +<body> +<H1> conffig +</H1> +<h2> +Purpose +</h2> +Display a confusion matrix. + +<p><h2> +Synopsis +</h2> +<PRE> +conffig(y, t) +fh = conffig(y, t)</PRE> + + +<p><h2> +Description +</h2> +<CODE>conffig(y, t)</CODE> displays the confusion matrix +and classification performance for the predictions mat{y} +compared with the targets <CODE>t</CODE>. The data is assumed to be in a +1-of-N encoding, unless there is just one column, when it is assumed to +be a 2 class problem with a 0-1 encoding. Each row of <CODE>y</CODE> and <CODE>t</CODE> +corresponds to a single example. + +<p>In the confusion matrix, the rows represent the true classes and the +columns the predicted classes. + +<p><CODE>fh = conffig(y, t)</CODE> also returns the figure handle <CODE>fh</CODE> which +can be used, for instance, to delete the figure when it is no longer needed. + +<p><h2> +See Also +</h2> +<CODE><a href="confmat.htm">confmat</a></CODE>, <CODE><a href="demtrain.htm">demtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/confmat.htm b/sourcecodes/bnt-master/nethelp3.3/confmat.htm new file mode 100644 index 00000000..2162293d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/confmat.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual confmat +</title> +</head> +<body> +<H1> confmat +</H1> +<h2> +Purpose +</h2> +Compute a confusion matrix. + +<p><h2> +Synopsis +</h2> +<PRE> +[C, rate] = confmat(y, t)</PRE> + + +<p><h2> +Description +</h2> +<CODE>[C, rate] = confmat(y, t)</CODE> computes the confusion matrix <CODE>C</CODE> +and classification performance <CODE>rate</CODE> for the predictions mat{y} +compared with the targets <CODE>t</CODE>. The data is assumed to be in a +1-of-N encoding, unless there is just one column, when it is assumed to +be a 2 class problem with a 0-1 encoding. Each row of <CODE>y</CODE> and <CODE>t</CODE> +corresponds to a single example. + +<p>In the confusion matrix, the rows represent the true classes and the +columns the predicted classes. The vector <CODE>rate</CODE> has two entries: +the percentage of correct classifications and the total number of +correct classifications. + +<p><h2> +See Also +</h2> +<CODE><a href="conffig.htm">conffig</a></CODE>, <CODE><a href="demtrain.htm">demtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/conjgrad.htm b/sourcecodes/bnt-master/nethelp3.3/conjgrad.htm new file mode 100644 index 00000000..4a70c330 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/conjgrad.htm @@ -0,0 +1,101 @@ +<html> +<head> +<title> +Netlab Reference Manual conjgrad +</title> +</head> +<body> +<H1> conjgrad +</H1> +<h2> +Purpose +</h2> +Conjugate gradients optimization. + +<p><h2> +Description +</h2> +<CODE>[x, options, flog, pointlog] = conjgrad(f, x, options, gradf)</CODE> uses a +conjugate gradients +algorithm to find the minimum of the function <CODE>f(x)</CODE> whose +gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector +and <CODE>f</CODE> returns a scalar value. +The point at which <CODE>f</CODE> has a local minimum +is returned as <CODE>x</CODE>. The function value at that point is returned +in <CODE>options(8)</CODE>. A log of the function values +after each cycle is (optionally) returned in <CODE>flog</CODE>, and a log +of the points visited is (optionally) returned in <CODE>pointlog</CODE>. + +<p><CODE>conjgrad(f, x, options, gradf, p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>, and the points visited +in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value +of <CODE>x</CODE> at the solution. If the absolute difference between +the values of <CODE>x</CODE> between two successive steps is less than +<CODE>options(2)</CODE>, then this condition is satisfied. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. If the absolute difference between the +objective function values between two successive steps is less than +<CODE>options(3)</CODE>, then this condition is satisfied. +Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(9)</CODE> is set to 1 to check the user defined gradient function. + +<p><CODE>options(10)</CODE> returns the total number of function evaluations (including +those in any line searches). + +<p><CODE>options(11)</CODE> returns the total number of gradient evaluations. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p><CODE>options(15)</CODE> is the precision in parameter space of the line search; +default <CODE>1e-4</CODE>. + +<p><h2> +Examples +</h2> +An example of +the use of the additional arguments is the minimization of an error +function for a neural network: +<PRE> + +w = quasinew('neterr', w, options, 'netgrad', net, x, t); +</PRE> + + +<p><h2> +Algorithm +</h2> + +The conjugate gradients algorithm constructs search +directions <CODE>di</CODE> that are conjugate: i.e. <CODE>di*H*d(i-1) = 0</CODE>, +where <CODE>H</CODE> is the Hessian matrix. This means that minimising along +<CODE>di</CODE> does not undo the effect of minimising along the previous +direction. The Polak-Ribiere formula is used to calculate new search +directions. The Hessian is not calculated, so there is only an +<CODE>O(W)</CODE> storage requirement (where <CODE>W</CODE> is the number of +parameters). However, relatively accurate line searches must be used +(default is <CODE>1e-04</CODE>). + +<p><h2> +See Also +</h2> +<CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/consist.htm b/sourcecodes/bnt-master/nethelp3.3/consist.htm new file mode 100644 index 00000000..40bc374c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/consist.htm @@ -0,0 +1,78 @@ +<html> +<head> +<title> +Netlab Reference Manual consist +</title> +</head> +<body> +<H1> consist +</H1> +<h2> +Purpose +</h2> +Check that arguments are consistent. + +<p><h2> +Synopsis +</h2> +<PRE> +errstring = consist(net, type, inputs, outputs) +errstring = consist(net, type, inputs) +errstring = consist(net, type) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>errstring = consist(net, type, inputs)</CODE> takes a network +data structure <CODE>net</CODE> together with a string <CODE>type</CODE> containing +the correct network type, a matrix <CODE>inputs</CODE> of input vectors and checks +that the data structure is consistent with the other arguments. An empty +string is returned if there is no error, otherwise the string contains the +relevant error message. If the <CODE>type</CODE> string is empty, then any +type of network is allowed. + +<p><CODE>errstring = consist(net, type)</CODE> takes a network data structure +<CODE>net</CODE> together with a string <CODE>type</CODE> containing the correct +network type, and checks that the two types match. + +<p><CODE>errstring = consist(net, type, inputs, outputs)</CODE> also checks that the +network has the correct number of outputs, and that the number of patterns +in the <CODE>inputs</CODE> and <CODE>outputs</CODE> is the same. The fields in <CODE>net</CODE> +that are used are +<PRE> + type + nin + nout +</PRE> + + +<p><h2> +Example +</h2> + +<p><CODE>mlpfwd</CODE>, the function that propagates values forward through an MLP +network, has the following check at the head of the file: +<PRE> + +errstring = consist(net, 'mlp', x, t); +if ~isempty(errstring) + error(errstring) +end +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="mlpfwd.htm">mlpfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/convertoldnet.htm b/sourcecodes/bnt-master/nethelp3.3/convertoldnet.htm new file mode 100644 index 00000000..c3266367 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/convertoldnet.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual convertoldnet +</title> +</head> +<body> +<H1> convertoldnet +</H1> +<h2> +Purpose +</h2> +Convert pre-2.3 release MLP and MDN nets to new format + +<p><h2> +Synopsis +</h2> +<PRE> +net = convertoldnet(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = convertoldnet(net)</CODE> takes a network <CODE>net</CODE> and, if appropriate, +converts it from pre-2.3 to the current format. The difference is simply +that in MLPs and the MLP sub-net of MDNs the field <CODE>actfn</CODE> has been +renamed <CODE>outfn</CODE> to make it consistent with GLM and RBF networks. +If the network is not old-format or an MLP or MDN it is left unchanged. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mdn.htm">mdn</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/datread.htm b/sourcecodes/bnt-master/nethelp3.3/datread.htm new file mode 100644 index 00000000..78f087eb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/datread.htm @@ -0,0 +1,57 @@ +<html> +<head> +<title> +Netlab Reference Manual datread +</title> +</head> +<body> +<H1> datread +</H1> +<h2> +Purpose +</h2> +Read data from an ascii file. + +<p><h2> +Synopsis +</h2> +<PRE> +[x, t, nin, nout, ndata] = datread(filename) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>[x, t, nin, nout, ndata] = datread(filename)</CODE> reads from +the file <CODE>filename</CODE> and returns a matrix <CODE>x</CODE> of input vectors, +a matrix <CODE>t</CODE> of target vectors, and integers <CODE>nin</CODE>, <CODE>nout</CODE> +and <CODE>ndata</CODE> specifying the number of inputs, the number of outputs +and the number of data points respectively. + +<p>The format of the data file is as follows: the first row contains the +string <CODE>nin</CODE> followed by the number of inputs, the second row +contains the string <CODE>nout</CODE> followed by the number of outputs, and +the third row contains the string <CODE>ndata</CODE> followed by the number +of data vectors. Subsequent lines each contain one input vector +followed by one output vector, with individual values separated by +spaces. + +<p><h2> +Example +</h2> +For the XOR data set we have + +<p><h2> +See Also +</h2> +<CODE><a href="nin2nout1ndata40.000000e+000.000000e+001.000000e+000.000000e+001.000000e+000.000000e+001.000000e+000.000000e+000.000000e+001.000000e+001.000000e+001.000000e+00SeeAlsodatwrite.htm">nin2nout1ndata40.000000e+000.000000e+001.000000e+000.000000e+001.000000e+000.000000e+001.000000e+000.000000e+000.000000e+001.000000e+001.000000e+001.000000e+00SeeAlsodatwrite</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/datwrite.htm b/sourcecodes/bnt-master/nethelp3.3/datwrite.htm new file mode 100644 index 00000000..1006b911 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/datwrite.htm @@ -0,0 +1,65 @@ +<html> +<head> +<title> +Netlab Reference Manual datwrite +</title> +</head> +<body> +<H1> datwrite +</H1> +<h2> +Purpose +</h2> +Write data to ascii file. + +<p><h2> +Synopsis +</h2> +<PRE> +datwrite(filename, x, t) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>datwrite(filename, x, t)</CODE> takes a matrix <CODE>x</CODE> of input vectors +and a matrix <CODE>t</CODE> of target vectors and writes them to an ascii +file named <CODE>filename</CODE>. The file format is as follows: the first +row contains the string <CODE>nin</CODE> followed by the number of inputs, +the second row contains the string <CODE>nout</CODE> followed by the number +of outputs, and the third row contains the string <CODE>ndata</CODE> followed +by the number of data vectors. Subsequent lines each contain one input +vector followed by one output vector, with individual values separated +by spaces. + +<p><h2> +Example +</h2> +For the XOR data set we have + +<p><PRE> + + nin 2 + nout 1 + ndata 4 + 0.000000e+00 0.000000e+00 1.000000e+00 + 0.000000e+00 1.000000e+00 0.000000e+00 + 1.000000e+00 0.000000e+00 0.000000e+00 + 1.000000e+00 1.000000e+00 1.000000e+00 +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="datread.htm">datread</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/dem2ddat.htm b/sourcecodes/bnt-master/nethelp3.3/dem2ddat.htm new file mode 100644 index 00000000..5dd49ba5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/dem2ddat.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual dem2ddat +</title> +</head> +<body> +<H1> dem2ddat +</H1> +<h2> +Purpose +</h2> +Generates two dimensional data for demos. + +<p><h2> +Synopsis +</h2> +<PRE> +data = dem2ddat(ndata)</PRE> + +<PRE> +[data, c] = dem2ddat(ndata)</PRE> + + +<p><h2> +Description +</h2> +The data is +drawn from three spherical Gaussian distributions with priors 0.3, +0.5 and 0.2; centres (2, 3.5), (0, 0) and (0,2); and standard deviations +0.2, 0.5 and 1.0. <CODE>data = dem2ddat(ndata)</CODE> generates <CODE>ndata</CODE> +points. + +<p><CODE>[data, c] = dem2ddat(ndata)</CODE> also returns a matrix containing the +centres of the Gaussian distributions. + +<p><h2> +See Also +</h2> +<CODE><a href="demgmm1.htm">demgmm1</a></CODE>, <CODE><a href="demkmean.htm">demkmean</a></CODE>, <CODE><a href="demknn1.htm">demknn1</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demard.htm b/sourcecodes/bnt-master/nethelp3.3/demard.htm new file mode 100644 index 00000000..43923377 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demard.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual demard +</title> +</head> +<body> +<H1> demard +</H1> +<h2> +Purpose +</h2> +Automatic relevance determination using the MLP. + +<p><h2> +Synopsis +</h2> +<PRE> +demmlp1</PRE> + + +<p><h2> +Description +</h2> +This script demonstrates the technique of automatic relevance +determination (ARD) using a synthetic problem having three input +variables: <CODE>x1</CODE> is sampled uniformly from the range (0,1) and has +a low level of added Gaussian noise, <CODE>x2</CODE> is a copy of <CODE>x1</CODE> +with a higher level of added noise, and <CODE>x3</CODE> is sampled randomly +from a Gaussian distribution. The single target variable is determined +by <CODE>sin(2*pi*x1)</CODE> with additive Gaussian noise. Thus <CODE>x1</CODE> is +very relevant for determining the target value, <CODE>x2</CODE> is of some +relevance, while <CODE>x3</CODE> is irrelevant. The prior over weights is +given by the ARD Gaussian prior with a separate hyper-parameter for +the group of weights associated with each input. A multi-layer +perceptron is trained on this data, with re-estimation of the +hyper-parameters using <CODE>evidence</CODE>. The final values for the +hyper-parameters reflect the relative importance of the three inputs. + +<p><h2> +See Also +</h2> +<CODE><a href="demmlp1.htm">demmlp1</a></CODE>, <CODE><a href="demev1.htm">demev1</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demev1.htm b/sourcecodes/bnt-master/nethelp3.3/demev1.htm new file mode 100644 index 00000000..f7b05fbb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demev1.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual demev1 +</title> +</head> +<body> +<H1> demev1 +</H1> +<h2> +Purpose +</h2> +Demonstrate Bayesian regression for the MLP. + +<p><h2> +Synopsis +</h2> +<PRE> +demev1</PRE> + + +<p><h2> +Description +</h2> +The problem consists an input variable <CODE>x</CODE> which sampled from a +Gaussian distribution, and a target variable <CODE>t</CODE> generated by +computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian noise. A 2-layer +network with linear outputs is trained by minimizing a sum-of-squares +error function with isotropic Gaussian regularizer, using the scaled +conjugate gradient optimizer. The hyperparameters <CODE>alpha</CODE> and +<CODE>beta</CODE> are re-estimated using the function <CODE>evidence</CODE>. A graph +is plotted of the original function, the training data, the trained +network function, and the error bars. + +<p><h2> +See Also +</h2> +<CODE><a href="evidence.htm">evidence</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE>, <CODE><a href="demard.htm">demard</a></CODE>, <CODE><a href="demmlp1.htm">demmlp1</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demev2.htm b/sourcecodes/bnt-master/nethelp3.3/demev2.htm new file mode 100644 index 00000000..c1db9e65 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demev2.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual demev2 +</title> +</head> +<body> +<H1> demev2 +</H1> +<h2> +Purpose +</h2> +Demonstrate Bayesian classification for the MLP. + +<p><h2> +Synopsis +</h2> +<PRE> +demev2</PRE> + + +<p><h2> +Description +</h2> +A synthetic two class two-dimensional dataset <CODE>x</CODE> is sampled +from a mixture of four Gaussians. Each class is +associated with two of the Gaussians so that the optimal decision +boundary is non-linear. +A 2-layer +network with logistic outputs is trained by minimizing the cross-entropy +error function with isotroipc Gaussian regularizer (one hyperparameter for +each of the four standard weight groups), using the scaled +conjugate gradient optimizer. The hyperparameter vectors <CODE>alpha</CODE> and +<CODE>beta</CODE> are re-estimated using the function <CODE>evidence</CODE>. A graph +is plotted of the optimal, regularised, and unregularised decision +boundaries. A further plot of the moderated versus unmoderated contours +is generated. + +<p><h2> +See Also +</h2> +<CODE><a href="evidence.htm">evidence</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE>, <CODE><a href="demard.htm">demard</a></CODE>, <CODE><a href="demmlp2.htm">demmlp2</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demev3.htm b/sourcecodes/bnt-master/nethelp3.3/demev3.htm new file mode 100644 index 00000000..55af4d64 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demev3.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual demev3 +</title> +</head> +<body> +<H1> demev3 +</H1> +<h2> +Purpose +</h2> +Demonstrate Bayesian regression for the RBF. + +<p><h2> +Synopsis +</h2> +<PRE> +demev3</PRE> + + +<p><h2> +Description +</h2> +The problem consists an input variable <CODE>x</CODE> which sampled from a +Gaussian distribution, and a target variable <CODE>t</CODE> generated by +computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian noise. An RBF +network with linear outputs is trained by minimizing a sum-of-squares +error function with isotropic Gaussian regularizer, using the scaled +conjugate gradient optimizer. The hyperparameters <CODE>alpha</CODE> and +<CODE>beta</CODE> are re-estimated using the function <CODE>evidence</CODE>. A graph +is plotted of the original function, the training data, the trained +network function, and the error bars. + +<p><h2> +See Also +</h2> +<CODE><a href="demev1.htm">demev1</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE>, <CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE>, <CODE><a href="netevfwd.htm">netevfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgauss.htm b/sourcecodes/bnt-master/nethelp3.3/demgauss.htm new file mode 100644 index 00000000..4ddc8059 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgauss.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual demgauss +</title> +</head> +<body> +<H1> demgauss +</H1> +<h2> +Purpose +</h2> +Demonstrate sampling from Gaussian distributions. + +<p><h2> +Synopsis +</h2> +<PRE> +demgauss +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>demgauss</CODE> provides a simple illustration of the generation of +data from Gaussian distributions. It first samples from a +one-dimensional distribution using <CODE>randn</CODE>, and then plots a +normalized histogram estimate of the distribution using <CODE>histp</CODE> +together with the true density calculated using <CODE>gauss</CODE>. + +<p><CODE>demgauss</CODE> then demonstrates sampling from a Gaussian distribution +in two dimensions. It creates a mean vector and a covariance matrix, +and then plots contours of constant density using the function +<CODE>gauss</CODE>. A sample of points drawn from this distribution, obtained +using the function <CODE>gsamp</CODE>, is then superimposed on the contours. + +<p><h2> +See Also +</h2> +<CODE><a href="gauss.htm">gauss</a></CODE>, <CODE><a href="gsamp.htm">gsamp</a></CODE>, <CODE><a href="histp.htm">histp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demglm1.htm b/sourcecodes/bnt-master/nethelp3.3/demglm1.htm new file mode 100644 index 00000000..7740b361 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demglm1.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demglm1 +</title> +</head> +<body> +<H1> demglm1 +</H1> +<h2> +Purpose +</h2> +Demonstrate simple classification using a generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +demglm1</PRE> + + +<p><h2> +Description +</h2> + +The problem consists of a two dimensional input +matrix <CODE>data</CODE> and a vector of classifications <CODE>t</CODE>. The data is +generated from two Gaussian clusters, and a generalized linear model +with logistic output is trained using iterative reweighted least squares. +A plot of the data together with the 0.1, 0.5 and 0.9 contour lines +of the conditional probability is generated. + +<p><h2> +See Also +</h2> +<CODE><a href="demglm2.htm">demglm2</a></CODE>, <CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demglm2.htm b/sourcecodes/bnt-master/nethelp3.3/demglm2.htm new file mode 100644 index 00000000..274d5b61 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demglm2.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demglm2 +</title> +</head> +<body> +<H1> demglm2 +</H1> +<h2> +Purpose +</h2> +Demonstrate simple classification using a generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +demglm1</PRE> + + +<p><h2> +Description +</h2> + +The problem consists of a two dimensional input +matrix <CODE>data</CODE> and a vector of classifications <CODE>t</CODE>. The data is +generated from three Gaussian clusters, and a generalized linear model +with softmax output is trained using iterative reweighted least squares. +A plot of the data together with regions shaded by the classification +given by the network is generated. + +<p><h2> +See Also +</h2> +<CODE><a href="demglm1.htm">demglm1</a></CODE>, <CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm1.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm1.htm new file mode 100644 index 00000000..7476becb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm1.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual demgmm1 +</title> +</head> +<body> +<H1> demgmm1 +</H1> +<h2> +Purpose +</h2> +Demonstrate EM for Gaussian mixtures. + +<p><h2> +Synopsis +</h2> +<PRE> +demgmm1</PRE> + + +<p><h2> +Description +</h2> +This script demonstrates the use of the EM algorithm to fit a mixture +of Gaussians to a set of data using maximum likelihood. A colour +coding scheme is used to illustrate the evaluation of the posterior +probabilities in the E-step of the EM algorithm. + +<p><h2> +See Also +</h2> +<CODE><a href="demgmm2.htm">demgmm2</a></CODE>, <CODE><a href="demgmm3.htm">demgmm3</a></CODE>, <CODE><a href="demgmm4.htm">demgmm4</a></CODE>, <CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="gmmpost.htm">gmmpost</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm2.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm2.htm new file mode 100644 index 00000000..f8a3f61d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm2.htm @@ -0,0 +1,52 @@ +<html> +<head> +<title> +Netlab Reference Manual demgmm1 +</title> +</head> +<body> +<H1> demgmm1 +</H1> +<h2> +Purpose +</h2> +Demonstrate density modelling with a Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> +demgmm1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of modelling data generated by a mixture of three +Gaussians in 2 dimensions. The priors are 0.3, +0.5 and 0.2; the centres are (2, 3.5), (0, 0) and (0,2); the variances +are 0.2, 0.5 and 1.0. The first figure contains a + scatter plot of the data. + +<p>A Gaussian mixture model with three components is trained using EM. The +parameter vector is printed before training and after training. The user +should press any key to continue at these points. The parameter vector +consists of priors (the column), centres (given as (x, y) pairs as +the next two columns), and variances (the last column). + +<p>The second figure is a 3 dimensional view of the density function, while +the third shows the 1-standard deviation circles for the three components of +the mixture model. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE>, <CODE><a href="gmmunpak.htm">gmmunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm3.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm3.htm new file mode 100644 index 00000000..bbd7465c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm3.htm @@ -0,0 +1,55 @@ +<html> +<head> +<title> +Netlab Reference Manual demgmm3 +</title> +</head> +<body> +<H1> demgmm3 +</H1> +<h2> +Purpose +</h2> +Demonstrate density modelling with a Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> +demgmm3</PRE> + + +<p><h2> +Description +</h2> + +The problem consists of modelling data generated +by a mixture of three Gaussians in 2 dimensions with a mixture model +using diagonal covariance matrices. The priors are 0.3, 0.5 and 0.2; the +centres are (2, 3.5), (0, 0) and (0,2); the covariances are all axis aligned +(0.16, 0.64), (0.25, 1) and the identity +matrix. The first figure contains a scatter plot of the data. + +<p>A Gaussian mixture model with three components is trained using EM. The +parameter vector is printed before training and after training. The user +should press any key to continue at these points. The parameter vector +consists of priors (the column), and centres (given as (x, y) pairs as +the next two columns). The diagonal entries of the +covariance matrices are printed separately. + +<p>The second figure is a 3 dimensional view of the density function, +while the third shows the axes of the 1-standard deviation circles +for the three components of the mixture model. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE>, <CODE><a href="gmmunpak.htm">gmmunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm4.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm4.htm new file mode 100644 index 00000000..95f4bc87 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm4.htm @@ -0,0 +1,54 @@ +<html> +<head> +<title> +Netlab Reference Manual demgmm4 +</title> +</head> +<body> +<H1> demgmm4 +</H1> +<h2> +Purpose +</h2> +Demonstrate density modelling with a Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> +demgmm4</PRE> + + +<p><h2> +Description +</h2> + +The problem consists of modelling data generated +by a mixture of three Gaussians in 2 dimensions with a mixture model +using full covariance matrices. The priors are 0.3, 0.5 and 0.2; the +centres are (2, 3.5), (0, 0) and (0,2); the variances are (0.16, 0.64) +axis aligned, (0.25, 1) rotated by 30 degrees and the identity +matrix. The first figure contains a scatter plot of the data. + +<p>A Gaussian mixture model with three components is trained using EM. The +parameter vector is printed before training and after training. The user +should press any key to continue at these points. The parameter vector +consists of priors (the column), and centres (given as (x, y) pairs as +the next two columns). The covariance matrices are printed separately. + +<p>The second figure is a 3 dimensional view of the density function, +while the third shows the axes of the 1-standard deviation ellipses +for the three components of the mixture model. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE>, <CODE><a href="gmmunpak.htm">gmmunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm5.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm5.htm new file mode 100644 index 00000000..ed3caf1e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm5.htm @@ -0,0 +1,55 @@ +<html> +<head> +<title> +Netlab Reference Manual demgmm5 +</title> +</head> +<body> +<H1> demgmm5 +</H1> +<h2> +Purpose +</h2> +Demonstrate density modelling with a PPCA mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> +demgmm5</PRE> + + +<p><h2> +Description +</h2> + +The problem consists of modelling data generated +by a mixture of three Gaussians in 2 dimensions with a mixture model +using full covariance matrices. The priors are 0.3, 0.5 and 0.2; the +centres are (2, 3.5), (0, 0) and (0,2); the variances are (0.16, 0.64) +axis aligned, (0.25, 1) rotated by 30 degrees and the identity +matrix. The first figure contains a scatter plot of the data. + +<p>A mixture model with three one-dimensional PPCA components is trained +using EM. The parameter vector is printed before training and after +training. The parameter vector consists of priors (the column), and +centres (given as (x, y) pairs as the next two columns). + +<p>The second figure is a 3 dimensional view of the density function, +while the third shows the axes of the 1-standard deviation ellipses +for the three components of the mixture model together with the one +standard deviation along the principal component of each mixture +model component. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE>, <CODE><a href="ppca.htm">ppca</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgp.htm b/sourcecodes/bnt-master/nethelp3.3/demgp.htm new file mode 100644 index 00000000..9103bfb8 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgp.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demgp +</title> +</head> +<body> +<H1> demgp +</H1> +<h2> +Purpose +</h2> +Demonstrate simple regression using a Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +demgp</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable <CODE>x</CODE> and one target variable +<CODE>t</CODE>. The values in <CODE>x</CODE> are chosen in two separated clusters and the +target data is generated by computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian +noise. Two Gaussian Processes, each with different covariance functions +are trained by optimising the hyperparameters +using the scaled conjugate gradient algorithm. The final predictions are +plotted together with 2 standard deviation error bars. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE>, <CODE><a href="gpinit.htm">gpinit</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgpard.htm b/sourcecodes/bnt-master/nethelp3.3/demgpard.htm new file mode 100644 index 00000000..eb188c19 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgpard.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual demgpard +</title> +</head> +<body> +<H1> demgpard +</H1> +<h2> +Purpose +</h2> +Demonstrate ARD using a Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +demgpare</PRE> + + +<p><h2> +Description +</h2> +The data consists of three input variables <CODE>x1</CODE>, <CODE>x2</CODE> and +<CODE>x3</CODE>, and one target variable +<CODE>t</CODE>. The +target data is generated by computing <CODE>sin(2*pi*x1)</CODE> and adding Gaussian +noise, x2 is a copy of x1 with a higher level of added +noise, and x3 is sampled randomly from a Gaussian distribution. +A Gaussian Process, is +trained by optimising the hyperparameters +using the scaled conjugate gradient algorithm. The final values of the +hyperparameters show that the model successfully identifies the importance +of each input. + +<p><h2> +See Also +</h2> +<CODE><a href="demgp.htm">demgp</a></CODE>, <CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE>, <CODE><a href="gpinit.htm">gpinit</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgpot.htm b/sourcecodes/bnt-master/nethelp3.3/demgpot.htm new file mode 100644 index 00000000..368f222e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgpot.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demgpot +</title> +</head> +<body> +<H1> demgpot +</H1> +<h2> +Purpose +</h2> +Computes the gradient of the negative log likelihood for a mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> +g = demgpot(x, mix)</PRE> + + +<p><h2> +Description +</h2> +This function computes the gradient of the negative log of the unconditional data +density <CODE>p(x)</CODE> with respect to the coefficients of the +data vector <CODE>x</CODE> for a Gaussian mixture model. The data structure +<CODE>mix</CODE> defines the mixture model, while the matrix <CODE>x</CODE> contains +the data vector as a row vector. Note the unusual order of the arguments: +this is so that the function can be used in <CODE>demhmc1</CODE> directly for +sampling from the distribution <CODE>p(x)</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="demhmc1.htm">demhmc1</a></CODE>, <CODE><a href="demmet1.htm">demmet1</a></CODE>, <CODE><a href="dempot.htm">dempot</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgtm1.htm b/sourcecodes/bnt-master/nethelp3.3/demgtm1.htm new file mode 100644 index 00000000..1422c764 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgtm1.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual demgtm1 +</title> +</head> +<body> +<H1> demgtm1 +</H1> +<h2> +Purpose +</h2> +Demonstrate EM for GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +demgtm1</PRE> + + +<p><h2> +Description +</h2> + +This script demonstrates the use of the EM +algorithm to fit a one-dimensional GTM to a two-dimensional set of data +using maximum likelihood. The location and spread of the Gaussian kernels +in the data space is shown during training. + +<p><h2> +See Also +</h2> +<CODE><a href="demgtm2.htm">demgtm2</a></CODE>, <CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmem.htm">gtmem</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgtm2.htm b/sourcecodes/bnt-master/nethelp3.3/demgtm2.htm new file mode 100644 index 00000000..4696b152 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgtm2.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual demgtm2 +</title> +</head> +<body> +<H1> demgtm2 +</H1> +<h2> +Purpose +</h2> +Demonstrate GTM for visualisation. + +<p><h2> +Synopsis +</h2> +<PRE> +demgtm2</PRE> + + +<p><h2> +Description +</h2> + +This script demonstrates the use of a +GTM with a two-dimensional latent space to visualise data in a higher +dimensional space. +This is done through the use of the mean responsibility and magnification +factors. + +<p><h2> +See Also +</h2> +<CODE><a href="demgtm1.htm">demgtm1</a></CODE>, <CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmem.htm">gtmem</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhint.htm b/sourcecodes/bnt-master/nethelp3.3/demhint.htm new file mode 100644 index 00000000..ac54f77a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhint.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual demhint +</title> +</head> +<body> +<H1> demhint +</H1> +<h2> +Purpose +</h2> +Demonstration of Hinton diagram for 2-layer feed-forward network. + +<p><h2> +Synopsis +</h2> +<PRE> +demhint +demhint(nin, nhidden, nout)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>demhint</CODE> plots a Hinton diagram for a 2-layer feedforward network +with 5 inputs, 4 hidden units and 3 outputs. The weight vector is +chosen from a Gaussian distribution as described under <CODE>mlp</CODE>. + +<p><CODE>demhint(nin, nhidden, nout)</CODE> allows the user to specify the +number of inputs, hidden units and outputs. + +<p><h2> +See Also +</h2> +<CODE><a href="hinton.htm">hinton</a></CODE>, <CODE><a href="hintmat.htm">hintmat</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhmc1.htm b/sourcecodes/bnt-master/nethelp3.3/demhmc1.htm new file mode 100644 index 00000000..4ad8b73b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhmc1.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual demhmc1 +</title> +</head> +<body> +<H1> demhmc1 +</H1> +<h2> +Purpose +</h2> +Demonstrate Hybrid Monte Carlo sampling on mixture of two Gaussians. + +<p><h2> +Synopsis +</h2> +<PRE> +demhmc1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of generating data from a mixture of two Gaussians +in two dimensions using a hybrid Monte Carlo algorithm with persistence. +A mixture model is then fitted to the sample to compare it with the +true underlying generator. + +<p><h2> +See Also +</h2> +<CODE><a href="demhmc3.htm">demhmc3</a></CODE>, <CODE><a href="hmc.htm">hmc</a></CODE>, <CODE><a href="dempot.htm">dempot</a></CODE>, <CODE><a href="demgpot.htm">demgpot</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhmc2.htm b/sourcecodes/bnt-master/nethelp3.3/demhmc2.htm new file mode 100644 index 00000000..b5657cbe --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhmc2.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual demhmc2 +</title> +</head> +<body> +<H1> demhmc2 +</H1> +<h2> +Purpose +</h2> +Demonstrate Bayesian regression with Hybrid Monte Carlo sampling. + +<p><h2> +Synopsis +</h2> +<PRE> +demhmc2</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable <CODE>x</CODE> and one target variable +<CODE>t</CODE> with data generated by sampling <CODE>x</CODE> at equal intervals and then +generating target data by computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian +noise. The model is a 2-layer network with linear outputs, and the hybrid Monte +Carlo algorithm (without persistence) is used to sample from the posterior +distribution of the weights. The graph shows the underlying function, +100 samples from the function given by the posterior distribution of the +weights, and the average prediction (weighted by the posterior probabilities). + +<p><h2> +See Also +</h2> +<CODE><a href="demhmc3.htm">demhmc3</a></CODE>, <CODE><a href="hmc.htm">hmc</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhmc3.htm b/sourcecodes/bnt-master/nethelp3.3/demhmc3.htm new file mode 100644 index 00000000..9ccc16c9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhmc3.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual demhmc3 +</title> +</head> +<body> +<H1> demhmc3 +</H1> +<h2> +Purpose +</h2> +Demonstrate Bayesian regression with Hybrid Monte Carlo sampling. + +<p><h2> +Synopsis +</h2> +<PRE> +demhmc3</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable <CODE>x</CODE> and one target variable +<CODE>t</CODE> with data generated by sampling <CODE>x</CODE> at equal intervals and then +generating target data by computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian +noise. The model is a 2-layer network with linear outputs, and the hybrid Monte +Carlo algorithm (with persistence) is used to sample from the posterior +distribution of the weights. The graph shows the underlying function, +300 samples from the function given by the posterior distribution of the +weights, and the average prediction (weighted by the posterior probabilities). + +<p><h2> +See Also +</h2> +<CODE><a href="demhmc2.htm">demhmc2</a></CODE>, <CODE><a href="hmc.htm">hmc</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demkmn1.htm b/sourcecodes/bnt-master/nethelp3.3/demkmn1.htm new file mode 100644 index 00000000..f22787df --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demkmn1.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual demkmean +</title> +</head> +<body> +<H1> demkmean +</H1> +<h2> +Purpose +</h2> +Demonstrate simple clustering model trained with K-means. + +<p><h2> +Synopsis +</h2> +<PRE> +demkmean</PRE> + + +<p><h2> +Description +</h2> +The problem consists of data in a two-dimensional space. +The data is +drawn from three spherical Gaussian distributions with priors 0.3, +0.5 and 0.2; centres (2, 3.5), (0, 0) and (0,2); and standard deviations +0.2, 0.5 and 1.0. The first figure contains a +scatter plot of the data. The data is the same as in <CODE>demgmm1</CODE>. + +<p>A cluster model with three components is trained using the batch +K-means algorithm. The matrix of centres is printed after training. +The second +figure shows the data labelled with a colour derived from the corresponding +cluster + +<p><h2> +See Also +</h2> +<CODE><a href="dem2ddat.htm">dem2ddat</a></CODE>, <CODE><a href="demgmm1.htm">demgmm1</a></CODE>, <CODE><a href="knn1.htm">knn1</a></CODE>, <CODE><a href="kmeans.htm">kmeans</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demknn1.htm b/sourcecodes/bnt-master/nethelp3.3/demknn1.htm new file mode 100644 index 00000000..6c0d4031 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demknn1.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual demknn1 +</title> +</head> +<body> +<H1> demknn1 +</H1> +<h2> +Purpose +</h2> +Demonstrate nearest neighbour classifier. + +<p><h2> +Synopsis +</h2> +<PRE> +demknn1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of data in a two-dimensional space. +The data is +drawn from three spherical Gaussian distributions with priors 0.3, +0.5 and 0.2; centres (2, 3.5), (0, 0) and (0,2); and standard deviations +0.2, 0.5 and 1.0. The first figure contains a +scatter plot of the data. The data is the same as in <CODE>demgmm1</CODE>. + +<p>The second +figure shows the data labelled with the corresponding class given +by the classifier. + +<p><h2> +See Also +</h2> +<CODE><a href="dem2ddat.htm">dem2ddat</a></CODE>, <CODE><a href="demgmm1.htm">demgmm1</a></CODE>, <CODE><a href="knn.htm">knn</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmdn1.htm b/sourcecodes/bnt-master/nethelp3.3/demmdn1.htm new file mode 100644 index 00000000..7401d220 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmdn1.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual demmdn1 +</title> +</head> +<body> +<H1> demmdn1 +</H1> +<h2> +Purpose +</h2> +Demonstrate fitting a multi-valued function using a Mixture Density Network. + +<p><h2> +Synopsis +</h2> +<PRE> +demmdn1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable +<CODE>x</CODE> and one target variable <CODE>t</CODE> with data generated by +sampling <CODE>t</CODE> at equal intervals and then generating target data by +computing <CODE>t + 0.3*sin(2*pi*t)</CODE> and adding Gaussian noise. A +Mixture Density Network with 3 centres in the mixture model is trained +by minimizing a negative log likelihood error function using the scaled +conjugate gradient optimizer. + +<p>The conditional means, mixing coefficients and variances are plotted +as a function of <CODE>x</CODE>, and a contour plot of the full conditional +density is also generated. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmet1.htm b/sourcecodes/bnt-master/nethelp3.3/demmet1.htm new file mode 100644 index 00000000..ff5423df --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmet1.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demmet1 +</title> +</head> +<body> +<H1> demmet1 +</H1> +<h2> +Purpose +</h2> +Demonstrate Markov Chain Monte Carlo sampling on a Gaussian. + +<p><h2> +Synopsis +</h2> +<PRE> +demmet1 +demmet1(plotwait)</PRE> + + +<p><h2> +Description +</h2> +The problem consists of generating data from a Gaussian +in two dimensions using a Markov Chain Monte Carlo algorithm. The points are +plotted one after another to show the path taken by the chain. + +<p><CODE>demmet1(plotwait)</CODE> allows the user to set the time (in a whole number +of seconds) between the plotting of points. This is passed to <CODE>pause</CODE> + +<p><h2> +See Also +</h2> +<CODE><a href="demhmc1.htm">demhmc1</a></CODE>, <CODE><a href="metrop.htm">metrop</a></CODE>, <CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="dempot.htm">dempot</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmlp1.htm b/sourcecodes/bnt-master/nethelp3.3/demmlp1.htm new file mode 100644 index 00000000..3d9c2079 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmlp1.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual demmlp1 +</title> +</head> +<body> +<H1> demmlp1 +</H1> +<h2> +Purpose +</h2> +Demonstrate simple regression using a multi-layer perceptron + +<p><h2> +Synopsis +</h2> +<PRE> +demmlp1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable <CODE>x</CODE> and one target variable +<CODE>t</CODE> with data generated by sampling <CODE>x</CODE> at equal intervals and then +generating target data by computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian +noise. A 2-layer network with linear outputs is trained by minimizing a +sum-of-squares error function using the scaled conjugate gradient optimizer. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmlp2.htm b/sourcecodes/bnt-master/nethelp3.3/demmlp2.htm new file mode 100644 index 00000000..ecb5e002 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmlp2.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual demmlp2 +</title> +</head> +<body> +<H1> demmlp2 +</H1> +<h2> +Purpose +</h2> +Demonstrate simple classification using a multi-layer perceptron + +<p><h2> +Synopsis +</h2> +<PRE> +demmlp2</PRE> + + +<p><h2> +Description +</h2> +The problem consists of input data in two dimensions drawn from a mixture +of three Gaussians: two of which are assigned to a single class. An MLP +with logistic outputs trained with a quasi-Newton optimisation algorithm is +compared with the optimal Bayesian decision rule. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="neterr.htm">neterr</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demnlab.htm b/sourcecodes/bnt-master/nethelp3.3/demnlab.htm new file mode 100644 index 00000000..1d46b12c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demnlab.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual demnlab +</title> +</head> +<body> +<H1> demnlab +</H1> +<h2> +Purpose +</h2> +A front-end Graphical User Interface to the demos + +<p><h2> +Synopsis +</h2> +<PRE> +demnlab</PRE> + + +<p><h2> +Description +</h2> +This function will start a user interface allowing the user to select +different demonstration functions to view. The demos are divided into 4 +groups, with the demo being executed by selecting the desired option +from a pop-up menu. + +<p><h2> +See Also +</h2> +<CODE><a href=".htm"></a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demns1.htm b/sourcecodes/bnt-master/nethelp3.3/demns1.htm new file mode 100644 index 00000000..9a6bcae7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demns1.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual demns1 +</title> +</head> +<body> +<H1> demns1 +</H1> +<h2> +Purpose +</h2> +Demonstrate Neuroscale for visualisation. + +<p><h2> +Synopsis +</h2> +<PRE> +demns1</PRE> + + +<p><h2> +Description +</h2> +This script demonstrates the use of the Neuroscale algorithm for +topographic projection and visualisation. A data sample is generated +from a mixture of two Gaussians in 4d space, and an RBF is trained +with the stress error function to project the data into 2d. The training +data and a test sample are both plotted in this projection. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbftrain.htm">rbftrain</a></CODE>, <CODE><a href="rbfprior.htm">rbfprior</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demolgd1.htm b/sourcecodes/bnt-master/nethelp3.3/demolgd1.htm new file mode 100644 index 00000000..76c55303 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demolgd1.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual demolgd1 +</title> +</head> +<body> +<H1> demolgd1 +</H1> +<h2> +Purpose +</h2> +Demonstrate simple MLP optimisation with on-line gradient descent + +<p><h2> +Synopsis +</h2> +<PRE> +demolgd1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable <CODE>x</CODE> and one target variable +<CODE>t</CODE> with data generated by sampling <CODE>x</CODE> at equal intervals and then +generating target data by computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian +noise. A 2-layer network with linear outputs is trained by minimizing a +sum-of-squares error function using on-line gradient descent. + +<p><h2> +See Also +</h2> +<CODE><a href="demmlp1.htm">demmlp1</a></CODE>, <CODE><a href="olgd.htm">olgd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demopt1.htm b/sourcecodes/bnt-master/nethelp3.3/demopt1.htm new file mode 100644 index 00000000..b2511711 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demopt1.htm @@ -0,0 +1,52 @@ +<html> +<head> +<title> +Netlab Reference Manual demopt1 +</title> +</head> +<body> +<H1> demopt1 +</H1> +<h2> +Purpose +</h2> +Demonstrate different optimisers on Rosenbrock's function. + +<p><h2> +Synopsis +</h2> +<PRE> +demopt1 +demopt1(xinit) +</PRE> + + +<p><h2> +Description +</h2> +The four general optimisers (quasi-Newton, conjugate gradients, +scaled conjugate gradients, and gradient descent) are applied to +the minimisation of Rosenbrock's well known `banana' function. +Each optimiser is run for at most 100 cycles, and a stopping +criterion of 1.0e-4 is used for both position and function value. +At the end, the trajectory of each algorithm is shown on a contour +plot of the function. + +<p><CODE>demopt1(xinit)</CODE> allows the user to specify a row vector with +two columns as the starting point. The default is the point [-1 1]. +Note that the contour plot has an x range of [-1.5, 1.5] and a y +range of [-0.5, 2.1], so it is best to choose a starting point in the +same region. + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE>, <CODE><a href="rosen.htm">rosen</a></CODE>, <CODE><a href="rosegrad.htm">rosegrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/dempot.htm b/sourcecodes/bnt-master/nethelp3.3/dempot.htm new file mode 100644 index 00000000..10bc3907 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/dempot.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual dempot +</title> +</head> +<body> +<H1> dempot +</H1> +<h2> +Purpose +</h2> +Computes the negative log likelihood for a mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> +e = dempot(x, mix)</PRE> + + +<p><h2> +Description +</h2> +This function computes the negative log of the unconditional data +density <CODE>p(x)</CODE> for a Gaussian mixture model. The data structure +<CODE>mix</CODE> defines the mixture model, while the matrix <CODE>x</CODE> contains +the data vectors. + +<p><h2> +See Also +</h2> +<CODE><a href="demgpot.htm">demgpot</a></CODE>, <CODE><a href="demhmc1.htm">demhmc1</a></CODE>, <CODE><a href="demmet1.htm">demmet1</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demprgp.htm b/sourcecodes/bnt-master/nethelp3.3/demprgp.htm new file mode 100644 index 00000000..1d7abeca --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demprgp.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demprgp +</title> +</head> +<body> +<H1> demprgp +</H1> +<h2> +Purpose +</h2> +Demonstrate sampling from a Gaussian Process prior. + +<p><h2> +Synopsis +</h2> +<PRE> +demprgp</PRE> + + +<p><h2> +Description +</h2> +This function plots the functions represented by a Gaussian Process +model. The hyperparameter values can be adjusted +on a linear scale using the sliders (though the exponential +of the parameters is used in the covariance function), or +by typing values into the text boxes and pressing the return key. +Both types of covariance function are supported. An extra function +specific parameter is needed for the rational quadratic function. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demprior.htm b/sourcecodes/bnt-master/nethelp3.3/demprior.htm new file mode 100644 index 00000000..02f5a000 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demprior.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual demprior +</title> +</head> +<body> +<H1> demprior +</H1> +<h2> +Purpose +</h2> +Demonstrate sampling from a multi-parameter Gaussian prior. + +<p><h2> +Synopsis +</h2> +<PRE> +demprior</PRE> + + +<p><h2> +Description +</h2> +This function plots the functions represented by a multi-layer perceptron +network when the weights are set to values drawn from a Gaussian prior +distribution. The parameters <CODE>aw1</CODE>, <CODE>ab1</CODE> <CODE>aw2</CODE> and <CODE>ab2</CODE> +control the inverse variances of the first-layer weights, the hidden unit +biases, the second-layer weights and the output unit biases respectively. +Their values can be adjusted on a logarithmic scale using the sliders, or +by typing values into the text boxes and pressing the return key. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demrbf1.htm b/sourcecodes/bnt-master/nethelp3.3/demrbf1.htm new file mode 100644 index 00000000..6c36ed49 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demrbf1.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual demrbf1 +</title> +</head> +<body> +<H1> demrbf1 +</H1> +<h2> +Purpose +</h2> +Demonstrate simple regression using a radial basis function network. + +<p><h2> +Synopsis +</h2> +<PRE> +demrbf1</PRE> + + +<p><h2> +Description +</h2> +The problem consists of one input variable <CODE>x</CODE> and one target variable +<CODE>t</CODE> with data generated by sampling <CODE>x</CODE> at equal intervals and then +generating target data by computing <CODE>sin(2*pi*x)</CODE> and adding Gaussian +noise. This data is the same as that used in demmlp1. + +<p>Three different RBF networks (with different activation functions) +are trained in two stages. First, a Gaussian mixture model is trained using +the EM algorithm, and the centres of this model are used to set the centres +of the RBF. Second, the output weights (and biases) are determined using the +pseudo-inverse of the design matrix. + +<p><h2> +See Also +</h2> +<CODE><a href="demmlp1.htm">demmlp1</a></CODE>, <CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbffwd.htm">rbffwd</a></CODE>, <CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demsom1.htm b/sourcecodes/bnt-master/nethelp3.3/demsom1.htm new file mode 100644 index 00000000..4cc1bbf0 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demsom1.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual demsom1 +</title> +</head> +<body> +<H1> demsom1 +</H1> +<h2> +Purpose +</h2> +Demonstrate SOM for visualisation. + +<p><h2> +Synopsis +</h2> +<PRE> +demsom1</PRE> + + +<p><h2> +Description +</h2> + +This script demonstrates the use of a +SOM with a two-dimensional grid to map onto data in +two-dimensional space. Both on-line and batch training algorithms +are shown. + +<p><h2> +See Also +</h2> +<CODE><a href="som.htm">som</a></CODE>, <CODE><a href="sompak.htm">sompak</a></CODE>, <CODE><a href="somtrain.htm">somtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demtrain.htm b/sourcecodes/bnt-master/nethelp3.3/demtrain.htm new file mode 100644 index 00000000..f3c8d8d5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demtrain.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual demtrain +</title> +</head> +<body> +<H1> demtrain +</H1> +<h2> +Purpose +</h2> +Demonstrate training of MLP network. + +<p><h2> +Synopsis +</h2> +<PRE> +demtrain</PRE> + + +<p><h2> +Description +</h2> +<CODE>demtrain</CODE> brings up a simple GUI to show the training of +an MLP network on classification and regression problems. The user +should load in a dataset (which should be in Netlab format: see +<CODE>datread</CODE>), select the output activation function, the + number of cycles and hidden units and then +train the network. The scaled conjugate gradient algorithm is used. +A graph shows the evolution of the error: the value is shown +<CODE>max(ceil(iterations / 50), 5)</CODE> cycles. + +<p>Once the network is trained, it is saved to the file <CODE>mlptrain.net</CODE>. +The results can then be viewed as a confusion matrix (for classification +problems) or a plot of output versus target (for regression problems). + +<p><h2> +See Also +</h2> +<CODE><a href="confmat.htm">confmat</a></CODE>, <CODE><a href="datread.htm">datread</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="netopt.htm">netopt</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/dist2.htm b/sourcecodes/bnt-master/nethelp3.3/dist2.htm new file mode 100644 index 00000000..80dd82b9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/dist2.htm @@ -0,0 +1,57 @@ +<html> +<head> +<title> +Netlab Reference Manual dist2 +</title> +</head> +<body> +<H1> dist2 +</H1> +<h2> +Purpose +</h2> +Calculates squared distance between two sets of points. + +<p><h2> +Synopsis +</h2> +<PRE> +d = dist2(x, c) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>d = dist2(x, c)</CODE> takes two matrices of vectors and calculates the +squared Euclidean distance between them. Both matrices must be of the +same column dimension. If <CODE>x</CODE> has <CODE>m</CODE> rows and <CODE>n</CODE> columns, and +<CODE>c</CODE> has <CODE>l</CODE> rows and <CODE>n</CODE> columns, then the result has +<CODE>m</CODE> rows and <CODE>l</CODE> columns. The <CODE>i, j</CODE>th entry is the +squared distance from the <CODE>i</CODE>th row of <CODE>x</CODE> to the <CODE>j</CODE>th +row of <CODE>c</CODE>. + +<p><h2> +Example +</h2> +The following code is used in <CODE>rbffwd</CODE> to calculate the activation of +a thin plate spline function. +<PRE> + +n2 = dist2(x, c); +z = log(n2.^(n2.^2)); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gmmactiv.htm">gmmactiv</a></CODE>, <CODE><a href="kmeans.htm">kmeans</a></CODE>, <CODE><a href="rbffwd.htm">rbffwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/eigdec.htm b/sourcecodes/bnt-master/nethelp3.3/eigdec.htm new file mode 100644 index 00000000..b981a96c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/eigdec.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual eigdec +</title> +</head> +<body> +<H1> eigdec +</H1> +<h2> +Purpose +</h2> +Sorted eigendecomposition + +<p><h2> +Synopsis +</h2> +<PRE> +evals = eigdec(x, N) +[evals, evec] = eigdec(x, N) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>evals = eigdec(x, N</CODE> computes the largest <CODE>N</CODE> eigenvalues of the +matrix <CODE>x</CODE> in descending order. <CODE>[evals, evec] = eigdec(x, N)</CODE> +also computes the corresponding eigenvectors. + +<p><h2> +See Also +</h2> +<CODE><a href="pca.htm">pca</a></CODE>, <CODE><a href="ppca.htm">ppca</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/errbayes.htm b/sourcecodes/bnt-master/nethelp3.3/errbayes.htm new file mode 100644 index 00000000..d3aaa1da --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/errbayes.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual errbayes +</title> +</head> +<body> +<H1> errbayes +</H1> +<h2> +Purpose +</h2> +Evaluate Bayesian error function for network. + +<p><h2> +Synopsis +</h2> +<PRE> +e = errbayes(net, edata) +[e, edata, eprior] = errbayes(net, edata) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>e = errbayes(net, edata)</CODE> takes a network data structure +<CODE>net</CODE> together +the data contribution to the error for a set of inputs and targets. +It returns the regularised error using any zero mean Gaussian priors +on the weights defined in +<CODE>net</CODE>. + +<p><CODE>[e, edata, eprior] = errbayes(net, x, t)</CODE> additionally returns the +data and prior components of the error. + +<p><h2> +See Also +</h2> +<CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="rbferr.htm">rbferr</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/evidence.htm b/sourcecodes/bnt-master/nethelp3.3/evidence.htm new file mode 100644 index 00000000..7e319484 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/evidence.htm @@ -0,0 +1,57 @@ +<html> +<head> +<title> +Netlab Reference Manual evidence +</title> +</head> +<body> +<H1> evidence +</H1> +<h2> +Purpose +</h2> +Re-estimate hyperparameters using evidence approximation. + +<p><h2> +Synopsis +</h2> +<PRE> +[net] = evidence(net, x, t) +[net, gamma, logev] = evidence(net, x, t, num) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>[net] = evidence(net, x, t)</CODE> re-estimates the +hyperparameters <CODE>alpha</CODE> and <CODE>beta</CODE> by applying Bayesian +re-estimation formulae for <CODE>num</CODE> iterations. The hyperparameter +<CODE>alpha</CODE> can be a simple scalar associated with an isotropic prior +on the weights, or can be a vector in which each component is +associated with a group of weights as defined by the <CODE>index</CODE> +matrix in the <CODE>net</CODE> data structure. These more complex priors can +be set up for an MLP using <CODE>mlpprior</CODE>. Initial values for the iterative +re-estimation are taken from the network data structure <CODE>net</CODE> +passed as an input argument, while the return argument <CODE>net</CODE> +contains the re-estimated values. + +<p><CODE>[net, gamma, logev] = evidence(net, x, t, num)</CODE> allows the re-estimation +formula to be applied for <CODE>num</CODE> cycles in which the re-estimated +values for the hyperparameters from each cycle are used to re-evaluate +the Hessian matrix for the next cycle. The return value <CODE>gamma</CODE> is +the number of well-determined parameters and <CODE>logev</CODE> is the log +of the evidence. + +<p><h2> +See Also +</h2> +<CODE><a href="mlpprior.htm">mlpprior</a></CODE>, <CODE><a href="netgrad.htm">netgrad</a></CODE>, <CODE><a href="nethess.htm">nethess</a></CODE>, <CODE><a href="demev1.htm">demev1</a></CODE>, <CODE><a href="demard.htm">demard</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/fevbayes.htm b/sourcecodes/bnt-master/nethelp3.3/fevbayes.htm new file mode 100644 index 00000000..42289df9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/fevbayes.htm @@ -0,0 +1,53 @@ +<html> +<head> +<title> +Netlab Reference Manual fevbayes +</title> +</head> +<body> +<H1> fevbayes +</H1> +<h2> +Purpose +</h2> +Evaluate Bayesian regularisation for network forward propagation. + +<p><h2> +Synopsis +</h2> +<PRE> +extra = fevbayes(net, y, a, x, t, x_test) +[extra, invhess] = fevbayes(net, y, a, x, t, x_test, invhess) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>extra = fevbayes(net, y, a, x, t, x_test)</CODE> takes a network data structure +<CODE>net</CODE> together with a set of hidden unit activations <CODE>a</CODE> from +test inputs <CODE>x_test</CODE>, training data inputs <CODE>x</CODE> and <CODE>t</CODE> and +outputs a matrix of extra information <CODE>extra</CODE> that consists of +error bars (variance) +for a regression problem or moderated outputs for a classification problem. +The optional argument (and return value) +<CODE>invhess</CODE> is the inverse of the network Hessian +computed on the training data inputs and targets. Passing it in avoids +recomputing it, which can be a significant saving for large training sets. + +<p>This is called by network-specific functions such as <CODE>mlpevfwd</CODE> which +are needed since the return values (predictions and hidden unit activations) +for different network types are in different orders (for good reasons). + +<p><h2> +See Also +</h2> +<CODE><a href="mlpevfwd.htm">mlpevfwd</a></CODE>, <CODE><a href="rbfevfwd.htm">rbfevfwd</a></CODE>, <CODE><a href="glmevfwd.htm">glmevfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gauss.htm b/sourcecodes/bnt-master/nethelp3.3/gauss.htm new file mode 100644 index 00000000..4f0f0d88 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gauss.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual gauss +</title> +</head> +<body> +<H1> gauss +</H1> +<h2> +Purpose +</h2> +Evaluate a Gaussian distribution. + +<p><h2> +Synopsis +</h2> +<PRE> +y = gauss(mu, covar, x) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>y = gauss(mu, covar, x)</CODE> evaluates a multi-variate Gaussian +density in <CODE>d</CODE>-dimensions at a set of points given by the rows +of the matrix <CODE>x</CODE>. The Gaussian density has mean vector <CODE>mu</CODE> +and covariance matrix <CODE>covar</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gsamp.htm">gsamp</a></CODE>, <CODE><a href="demgauss.htm">demgauss</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gbayes.htm b/sourcecodes/bnt-master/nethelp3.3/gbayes.htm new file mode 100644 index 00000000..3e123fdf --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gbayes.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual gbayes +</title> +</head> +<body> +<H1> gbayes +</H1> +<h2> +Purpose +</h2> +Evaluate gradient of Bayesian error function for network. + +<p><h2> +Synopsis +</h2> +<PRE> +g = gbayes(net, gdata) +[g, gdata, gprior] = gbayes(net, gdata) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = gbayes(net, gdata)</CODE> takes a network data structure <CODE>net</CODE> together +the data contribution to the error gradient +for a set of inputs and targets. +It returns the regularised error gradient using any zero mean Gaussian priors +on the weights defined in +<CODE>net</CODE>. In addition, if a <CODE>mask</CODE> is defined in <CODE>net</CODE>, then +the entries in <CODE>g</CODE> that correspond to weights with a 0 in the +mask are removed. + +<p><CODE>[g, gdata, gprior] = gbayes(net, gdata)</CODE> additionally returns the +data and prior components of the error. + +<p><h2> +See Also +</h2> +<CODE><a href="errbayes.htm">errbayes</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glm.htm b/sourcecodes/bnt-master/nethelp3.3/glm.htm new file mode 100644 index 00000000..a860372f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glm.htm @@ -0,0 +1,85 @@ +<html> +<head> +<title> +Netlab Reference Manual glm +</title> +</head> +<body> +<H1> glm +</H1> +<h2> +Purpose +</h2> +Create a generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +net = glm(nin, nout, func) +net = glm(nin, nout, func, prior) +net = glm(nin, nout, func, prior, beta) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = glm(nin, nout, func)</CODE> takes the number of inputs +and outputs for a generalized linear model, together +with a string <CODE>func</CODE> which specifies the output unit activation function, +and returns a data structure <CODE>net</CODE>. The weights are drawn from a zero mean, +isotropic Gaussian, with variance scaled by the fan-in of the +output units. This makes use of the Matlab function +<CODE>randn</CODE> and so the seed for the random weight initialization can be +set using <CODE>randn('state', s)</CODE> where <CODE>s</CODE> is the seed value. The optional +argument <CODE>alpha</CODE> sets the inverse variance for the weight +initialization. + +<p>The fields in <CODE>net</CODE> are +<PRE> + type = 'glm' + nin = number of inputs + nout = number of outputs + nwts = total number of weights and biases + actfn = string describing the output unit activation function: + 'linear' + 'logistic' + 'softmax' + w1 = first-layer weight matrix + b1 = first-layer bias vector +</PRE> + + +<p><CODE>net = glm(nin, nout, func, prior)</CODE>, in which <CODE>prior</CODE> is +a scalar, allows the field +<CODE>net.alpha</CODE> in the data structure <CODE>net</CODE> to be set, corresponding +to a zero-mean isotropic Gaussian prior with inverse variance with +value <CODE>prior</CODE>. Alternatively, <CODE>prior</CODE> can consist of a data +structure with fields <CODE>alpha</CODE> and <CODE>index</CODE>, allowing individual +Gaussian priors to be set over groups of weights in the network. Here +<CODE>alpha</CODE> is a column vector in which each element corresponds to a +separate group of weights, which need not be mutually exclusive. The +membership of the groups is defined by the matrix <CODE>index</CODE> in which +the columns correspond to the elements of <CODE>alpha</CODE>. Each column has +one element for each weight in the matrix, in the order defined by the +function <CODE>glmpak</CODE>, and each element is 1 or 0 according to whether +the weight is a member of the corresponding group or not. + +<p><CODE>net = glm(nin, nout, func, prior, beta)</CODE> also sets the +additional field <CODE>net.beta</CODE> in the data structure <CODE>net</CODE>, where +beta corresponds to the inverse noise variance. + +<p><h2> +See Also +</h2> +<CODE><a href="glmpak.htm">glmpak</a></CODE>, <CODE><a href="glmunpak.htm">glmunpak</a></CODE>, <CODE><a href="glmfwd.htm">glmfwd</a></CODE>, <CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmderiv.htm b/sourcecodes/bnt-master/nethelp3.3/glmderiv.htm new file mode 100644 index 00000000..56887889 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmderiv.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual glmderiv +</title> +</head> +<body> +<H1> glmderiv +</H1> +<h2> +Purpose +</h2> +Evaluate derivatives of GLM outputs with respect to weights. + +<p><h2> +Synopsis +</h2> +<PRE> + +g = glmderiv(net, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = glmderiv(net, x)</CODE> takes a network data structure <CODE>net</CODE> and a matrix +of input vectors <CODE>x</CODE> and returns a three-index matrix mat{g} whose +<CODE>i</CODE>, <CODE>j</CODE>, <CODE>k</CODE> +element contains the derivative of network output <CODE>k</CODE> with respect to +weight or bias parameter <CODE>j</CODE> for input pattern <CODE>i</CODE>. The ordering of the +weight and bias parameters is defined by <CODE>glmunpak</CODE>. + +<p><h2> +See also +</h2> +<PRE> +glm, glmunpak, glmgrad</PRE> + + +<p><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmerr.htm b/sourcecodes/bnt-master/nethelp3.3/glmerr.htm new file mode 100644 index 00000000..fde8a08b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmerr.htm @@ -0,0 +1,56 @@ +<html> +<head> +<title> +Netlab Reference Manual glmerr +</title> +</head> +<body> +<H1> glmerr +</H1> +<h2> +Purpose +</h2> +Evaluate error function for generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +e = glmerr(net, x, t) +[e, edata, eprior] = glmerr(net, x, t) +[e, edata, eprior, y, a] = glmerr(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>e = glmerr(net, x, t)</CODE> takes a generalized +linear model data structure <CODE>net</CODE> together with a matrix <CODE>x</CODE> +of input vectors and a matrix <CODE>t</CODE> of target vectors, and evaluates +the error function <CODE>e</CODE>. The choice of error function corresponds +to the output unit activation function. Each row of <CODE>x</CODE> +corresponds to one input vector and each row of <CODE>t</CODE> corresponds to +one target vector. + +<p><CODE>[e, edata, eprior, y, a] = glmerr(net, x, t)</CODE> also returns +the data and prior components of the total error. + +<p><CODE>[e, edata, eprior, y, a] = glmerr(net, x)</CODE> also returns a matrix <CODE>y</CODE> +giving the outputs of the models and a matrix <CODE>a</CODE> +giving the summed inputs to each output unit, where each row +corresponds to one pattern. + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmpak.htm">glmpak</a></CODE>, <CODE><a href="glmunpak.htm">glmunpak</a></CODE>, <CODE><a href="glmfwd.htm">glmfwd</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/glmevfwd.htm new file mode 100644 index 00000000..fc2f4749 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmevfwd.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual glmevfwd +</title> +</head> +<body> +<H1> glmevfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation with evidence for GLM + +<p><h2> +Synopsis +</h2> +<PRE> + +[y, extra] = glmevfwd(net, x, t, x_test) +[y, extra, invhess] = glmevfwd(net, x, t, x_test, invhess) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = glmevfwd(net, x, t, x_test)</CODE> takes a network data structure +<CODE>net</CODE> together with the input <CODE>x</CODE> and target <CODE>t</CODE> training data +and input test data <CODE>x_test</CODE>. +It returns the normal forward propagation through the network <CODE>y</CODE> +together with a matrix <CODE>extra</CODE> which consists of error bars (variance) +for a regression problem or moderated outputs for a classification problem. + +<p>The optional argument (and return value) +<CODE>invhess</CODE> is the inverse of the network Hessian +computed on the training data inputs and targets. Passing it in avoids +recomputing it, which can be a significant saving for large training sets. + +<p><h2> +See Also +</h2> +<CODE><a href="fevbayes.htm">fevbayes</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmfwd.htm b/sourcecodes/bnt-master/nethelp3.3/glmfwd.htm new file mode 100644 index 00000000..9e8f1061 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmfwd.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual glmfwd +</title> +</head> +<body> +<H1> glmfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +y = glmfwd(net, x) +[y, a] = glmfwd(net, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = glmfwd(net, x)</CODE> takes a generalized linear model +data structure <CODE>net</CODE> together with +a matrix <CODE>x</CODE> of input vectors, and forward propagates the inputs +through the network to generate a matrix <CODE>y</CODE> of output +vectors. Each row of <CODE>x</CODE> corresponds to one input vector and each +row of <CODE>y</CODE> corresponds to one output vector. + +<p><CODE>[y, a] = glmfwd(net, x)</CODE> also returns a matrix <CODE>a</CODE> +giving the summed inputs to each output unit, where each row +corresponds to one pattern. + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmpak.htm">glmpak</a></CODE>, <CODE><a href="glmunpak.htm">glmunpak</a></CODE>, <CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmgrad.htm b/sourcecodes/bnt-master/nethelp3.3/glmgrad.htm new file mode 100644 index 00000000..024a5d19 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmgrad.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual glmgrad +</title> +</head> +<body> +<H1> glmgrad +</H1> +<h2> +Purpose +</h2> +Evaluate gradient of error function for generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> + +g = glmgrad(net, x, t) +[g, gdata, gprior] = glmgrad(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = glmgrad(net, x, t)</CODE> takes a generalized linear model +data structure <CODE>net</CODE> +together with a matrix <CODE>x</CODE> of input vectors and a matrix <CODE>t</CODE> +of target vectors, and evaluates the gradient <CODE>g</CODE> of the error +function with respect to the network weights. The error function +corresponds to the choice of output unit activation function. Each row +of <CODE>x</CODE> corresponds to one input vector and each row of <CODE>t</CODE> +corresponds to one target vector. + +<p><CODE>[g, gdata, gprior] = glmgrad(net, x, t)</CODE> also returns separately +the data and prior contributions to the gradient. + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmpak.htm">glmpak</a></CODE>, <CODE><a href="glmunpak.htm">glmunpak</a></CODE>, <CODE><a href="glmfwd.htm">glmfwd</a></CODE>, <CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmhess.htm b/sourcecodes/bnt-master/nethelp3.3/glmhess.htm new file mode 100644 index 00000000..481e3220 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmhess.htm @@ -0,0 +1,74 @@ +<html> +<head> +<title> +Netlab Reference Manual glmhess +</title> +</head> +<body> +<H1> glmhess +</H1> +<h2> +Purpose +</h2> +Evaluate the Hessian matrix for a generalised linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +h = glmhess(net, x, t) +[h, hdata] = glmhess(net, x, t) +h = glmhess(net, x, t, hdata) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>h = glmhess(net, x, t)</CODE> takes a GLM network data structure <CODE>net</CODE>, +a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of target +values and returns the full Hessian matrix <CODE>h</CODE> corresponding to +the second derivatives of the negative log posterior distribution, +evaluated for the current weight and bias values as defined by +<CODE>net</CODE>. Note that the target data is not required in the calculation, +but is included to make the interface uniform with <CODE>nethess</CODE>. For +linear and logistic outputs, the computation is very simple and is +done (in effect) in one line in <CODE>glmtrain</CODE>. + +<p><CODE>[h, hdata] = glmhess(net, x, t)</CODE> returns both the Hessian matrix +<CODE>h</CODE> and the contribution <CODE>hdata</CODE> arising from the data dependent +term in the Hessian. + +<p><CODE>h = glmhess(net, x, t, hdata)</CODE> takes a network data structure +<CODE>net</CODE>, a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of +target values, together with the contribution <CODE>hdata</CODE> arising from +the data dependent term in the Hessian, and returns the full Hessian +matrix <CODE>h</CODE> corresponding to the second derivatives of the negative +log posterior distribution. This version saves computation time if +<CODE>hdata</CODE> has already been evaluated for the current weight and bias +values. + +<p><h2> +Example +</h2> +The Hessian matrix is used by <CODE>glmtrain</CODE> to take a Newton step for +softmax outputs. +<PRE> + +Hessian = glmhess(net, x, t); +deltaw = -gradient*pinv(Hessian); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE>, <CODE><a href="nethess.htm">nethess</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glminit.htm b/sourcecodes/bnt-master/nethelp3.3/glminit.htm new file mode 100644 index 00000000..e19369d3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glminit.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual glminit +</title> +</head> +<body> +<H1> glminit +</H1> +<h2> +Purpose +</h2> +Initialise the weights in a generalized linear model. + +<p><h2> +Synopsis +</h2> +<PRE> +net = glminit(net, prior) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = glminit(net, prior)</CODE> takes a generalized linear model +<CODE>net</CODE> and sets the weights and biases by sampling from a Gaussian +distribution. If <CODE>prior</CODE> is a scalar, then all of the parameters +(weights and biases) are sampled from a single isotropic Gaussian with +inverse variance equal to <CODE>prior</CODE>. If <CODE>prior</CODE> is a data +structure similar to that in <CODE>mlpprior</CODE> but for a single layer of +weights, then the parameters +are sampled from multiple Gaussians according to their groupings +(defined by the <CODE>index</CODE> field) with corresponding variances +(defined by the <CODE>alpha</CODE> field). + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmpak.htm">glmpak</a></CODE>, <CODE><a href="glmunpak.htm">glmunpak</a></CODE>, <CODE><a href="mlpinit.htm">mlpinit</a></CODE>, <CODE><a href="mlpprior.htm">mlpprior</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmpak.htm b/sourcecodes/bnt-master/nethelp3.3/glmpak.htm new file mode 100644 index 00000000..8e1af1ad --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmpak.htm @@ -0,0 +1,40 @@ +<html> +<head> +<title> +Netlab Reference Manual glmpak +</title> +</head> +<body> +<H1> glmpak +</H1> +<h2> +Purpose +</h2> +Combines weights and biases into one weights vector. + +<p><h2> +Synopsis +</h2> +<PRE> +w = glmpak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>w = glmpak(net)</CODE> takes a network data structure <CODE>net</CODE> and +combines them into a single row vector <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmunpak.htm">glmunpak</a></CODE>, <CODE><a href="glmfwd.htm">glmfwd</a></CODE>, <CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmtrain.htm b/sourcecodes/bnt-master/nethelp3.3/glmtrain.htm new file mode 100644 index 00000000..a52ada6d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmtrain.htm @@ -0,0 +1,71 @@ +<html> +<head> +<title> +Netlab Reference Manual glmtrain +</title> +</head> +<body> +<H1> glmtrain +</H1> +<h2> +Purpose +</h2> +Specialised training of generalized linear model + +<p><h2> +Description +</h2> +<CODE>net = glmtrain(net, options, x, t)</CODE> uses +the iterative reweighted least squares (IRLS) +algorithm to set the weights in the generalized linear model structure +<CODE>net</CODE>. This is a more efficient alternative to using <CODE>glmerr</CODE> +and <CODE>glmgrad</CODE> and a non-linear optimisation routine through +<CODE>netopt</CODE>. +Note that for linear outputs, a single pass through the +algorithm is all that is required, since the error function is quadratic in +the weights. The algorithm also handles scalar <CODE>alpha</CODE> and <CODE>beta</CODE> +terms. If you want to use more complicated priors, you should use +general-purpose non-linear optimisation algorithms. + +<p>For logistic and softmax outputs, general priors can be handled, although +this requires the pseudo-inverse of the Hessian, giving up the better +conditioning and some of the speed advantage of the normal form equations. + +<p>The error function value at the final set of weights is returned +in <CODE>options(8)</CODE>. +Each row of <CODE>x</CODE> corresponds to one +input vector and each row of <CODE>t</CODE> corresponds to one target vector. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values during training. +If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is a measure of the precision required for the value +of the weights <CODE>w</CODE> at the solution. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(5)</CODE> is set to 1 if an approximation to the Hessian (which assumes +that all outputs are independent) is used for softmax outputs. With the default +value of 0 the exact Hessian (which is more expensive to compute) is used. + +<p><CODE>options(14)</CODE> is the maximum number of iterations for the IRLS algorithm; +default 100. + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmunpak.htm b/sourcecodes/bnt-master/nethelp3.3/glmunpak.htm new file mode 100644 index 00000000..73fd5054 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmunpak.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual glmunpak +</title> +</head> +<body> +<H1> glmunpak +</H1> +<h2> +Purpose +</h2> +Separates weights vector into weight and bias matrices. + +<p><h2> +Synopsis +</h2> +<PRE> +net = glmunpak(net, w) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = glmunpak(net, w)</CODE> takes a glm network data structure <CODE>net</CODE> and +a weight vector <CODE>w</CODE>, and returns a network data structure identical to +the input network, except that the first-layer weight matrix +<CODE>w1</CODE> and the first-layer bias vector <CODE>b1</CODE> have +been set to the corresponding elements of <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmpak.htm">glmpak</a></CODE>, <CODE><a href="glmfwd.htm">glmfwd</a></CODE>, <CODE><a href="glmerr.htm">glmerr</a></CODE>, <CODE><a href="glmgrad.htm">glmgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmm.htm b/sourcecodes/bnt-master/nethelp3.3/gmm.htm new file mode 100644 index 00000000..c815b3cb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmm.htm @@ -0,0 +1,96 @@ +<html> +<head> +<title> +Netlab Reference Manual gmm +</title> +</head> +<body> +<H1> gmm +</H1> +<h2> +Purpose +</h2> +Creates a Gaussian mixture model with specified architecture. + +<p><h2> +Synopsis +</h2> +<PRE> +mix = gmm(dim, ncentres, covartype) +mix = gmm(dim, ncentres, covartype, ppca_dim) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>mix = gmm(dim, ncentres, covartype)</CODE> takes +the dimension of the space <CODE>dim</CODE>, the number of centres in the +mixture model and the type of the mixture model, and returns a data +structure <CODE>mix</CODE>. +The mixture model type defines the covariance structure of each component +Gaussian: +<PRE> + + 'spherical' = single variance parameter for each component: stored as a vector + 'diag' = diagonal matrix for each component: stored as rows of a matrix + 'full' = full matrix for each component: stored as 3d array + 'ppca' = probabilistic PCA: stored as principal components (in a 3d array + and associated variances and off-subspace noise +</PRE> + +<CODE>mix = gmm(dim, ncentres, covartype, ppca_dim)</CODE> also sets the dimension of +the PPCA sub-spaces: the default value is one. + +<p>The priors are initialised to equal values summing to one, and the covariances +are all the identity matrix (or equivalent). The centres are +initialised randomly from a zero mean unit variance Gaussian. This makes use +of the MATLAB function <CODE>randn</CODE> and so the seed for the random weight +initialisation can be set using <CODE>randn('state', s)</CODE> where <CODE>s</CODE> is the +state value. + +<p>The fields in <CODE>mix</CODE> are +<PRE> + + type = 'gmm' + nin = the dimension of the space + ncentres = number of mixture components + covartype = string for type of variance model + priors = mixing coefficients + centres = means of Gaussians: stored as rows of a matrix + covars = covariances of Gaussians +</PRE> + +The additional fields for mixtures of PPCA are +<PRE> + + U = principal component subspaces + lambda = in-space covariances: stored as rows of a matrix +</PRE> + +The off-subspace noise is stored in <CODE>covars</CODE>. + +<p><h2> +Example +</h2> +<PRE> + +mix = gmm(2, 4, 'spherical'); +</PRE> + +This creates a Gaussian mixture model with 4 components in 2 dimensions. +The covariance structure is a spherical model. + +<p><h2> +See Also +</h2> +<CODE><a href="gmmpak.htm">gmmpak</a></CODE>, <CODE><a href="gmmunpak.htm">gmmunpak</a></CODE>, <CODE><a href="gmmsamp.htm">gmmsamp</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="gmmactiv.htm">gmmactiv</a></CODE>, <CODE><a href="gmmpost.htm">gmmpost</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmactiv.htm b/sourcecodes/bnt-master/nethelp3.3/gmmactiv.htm new file mode 100644 index 00000000..0c3c5c8f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmactiv.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmactiv +</title> +</head> +<body> +<H1> gmmactiv +</H1> +<h2> +Purpose +</h2> +Computes the activations of a Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> + +a = gmmactiv(mix, x) +</PRE> + + +<p><h2> +Description +</h2> +This function computes the activations <CODE>a</CODE> (i.e. the +probability <CODE>p(x|j)</CODE> of the data conditioned on each component density) +for a Gaussian mixture model. For the PPCA model, each activation +is the conditional probability of <CODE>x</CODE> given that it is generated +by the component subspace. +The data structure <CODE>mix</CODE> defines the mixture model, while the matrix +<CODE>x</CODE> contains the data vectors. Each row of <CODE>x</CODE> represents a single +vector. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmpost.htm">gmmpost</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmem.htm b/sourcecodes/bnt-master/nethelp3.3/gmmem.htm new file mode 100644 index 00000000..6c0f46dd --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmem.htm @@ -0,0 +1,87 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmem +</title> +</head> +<body> +<H1> gmmem +</H1> +<h2> +Purpose +</h2> +EM algorithm for Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> + +[mix, options, errlog] = gmmem(mix, x, options) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>[mix, options, errlog] = gmmem(mix, x, options)</CODE> uses the Expectation +Maximization algorithm of Dempster et al. to estimate the parameters of +a Gaussian mixture model defined by a data structure <CODE>mix</CODE>. +The matrix <CODE>x</CODE> represents the data whose expectation +is maximized, with each row corresponding to a vector. + +The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>. +If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(3)</CODE> is a measure of the absolute precision required of the error +function at the solution. If the change in log likelihood between two steps of +the EM algorithm is less than this value, then the function terminates. + +<p><CODE>options(5)</CODE> is set to 1 if a covariance matrix is reset to its +original value when any of its singular values are too small (less +than MIN_COVAR which has the value eps). +With the default value of 0 no action is taken. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p>The optional return value <CODE>options</CODE> contains the final error value +(i.e. data log likelihood) in +<CODE>options(8)</CODE>. + +<p><h2> +Examples +</h2> +The following code fragment sets up a Gaussian mixture model, initialises +the parameters from the data, sets the options and trains the model. +<PRE> + +mix = gmm(inputdim, ncentres, 'full'); + +<p>options = foptions; +options(14) = 5; +mix = gmminit(mix, data, options); + +<p>options(1) = 1; % Prints out error values. +options(14) = 30; % Max. number of iterations. + +<p>mix = gmmem(mix, data, options); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmminit.htm b/sourcecodes/bnt-master/nethelp3.3/gmminit.htm new file mode 100644 index 00000000..3108da74 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmminit.htm @@ -0,0 +1,65 @@ +<html> +<head> +<title> +Netlab Reference Manual gmminit +</title> +</head> +<body> +<H1> gmminit +</H1> +<h2> +Purpose +</h2> +Initialises Gaussian mixture model from data + +<p><h2> +Synopsis +</h2> +<PRE> + +mix = gmminit(mix, x, options) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>mix = gmminit(mix, x, options)</CODE> uses a dataset <CODE>x</CODE> +to initialise the parameters of a Gaussian mixture +model defined by the data structure <CODE>mix</CODE>. The k-means algorithm +is used to determine the centres. The priors are computed from the +proportion of examples belonging to each cluster. +The covariance matrices are calculated as the sample covariance of the +points associated with (i.e. closest to) the corresponding centres. +For a mixture of PPCA model, the PPCA decomposition is calculated +for the points closest to a given centre. +This initialisation can be used as the starting point for training the +model using the EM algorithm. + +<p><h2> +Example +</h2> +<PRE> + +mix = gmm(3, 2); +options = foptions; +options(14) = 5; +mix = gmminit(mix, data, options); +</PRE> + +This code sets up a Gaussian mixture model with 3 centres in 2 dimensions, and +then initialises the parameters from the data set <CODE>data</CODE> with 5 iterations +of the k means algorithm. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmpak.htm b/sourcecodes/bnt-master/nethelp3.3/gmmpak.htm new file mode 100644 index 00000000..0ef370a9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmpak.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmpak +</title> +</head> +<body> +<H1> gmmpak +</H1> +<h2> +Purpose +</h2> +Combines all the parameters in a Gaussian mixture model into one vector. + +<p><h2> +Synopsis +</h2> +<PRE> +p = gmmpak(mix) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>p = gmmpak(net)</CODE> takes a mixture data structure <CODE>mix</CODE> +and combines the component parameter matrices into a single row +vector <CODE>p</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmunpak.htm">gmmunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmpost.htm b/sourcecodes/bnt-master/nethelp3.3/gmmpost.htm new file mode 100644 index 00000000..d323115f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmpost.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmpost +</title> +</head> +<body> +<H1> gmmpost +</H1> +<h2> +Purpose +</h2> +Computes the class posterior probabilities of a Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> + +function post = gmmpost(mix, x) +</PRE> + + +<p><h2> +Description +</h2> +This function computes the posteriors <CODE>post</CODE> (i.e. the probability of each +component conditioned on the data <CODE>p(j|x)</CODE>) for a Gaussian mixture model. +The data structure <CODE>mix</CODE> defines the mixture model, while the matrix +<CODE>x</CODE> contains the data vectors. Each row of <CODE>x</CODE> represents a single +vector. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmactiv.htm">gmmactiv</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmprob.htm b/sourcecodes/bnt-master/nethelp3.3/gmmprob.htm new file mode 100644 index 00000000..acc3fb65 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmprob.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmprob +</title> +</head> +<body> +<H1> gmmprob +</H1> +<h2> +Purpose +</h2> +Computes the data probability for a Gaussian mixture model. + +<p><h2> +Synopsis +</h2> +<PRE> + +function prob = gmmprob(mix, x) +</PRE> + + +<p><h2> +Description +</h2> + +This function computes the unconditional data +density <CODE>p(x)</CODE> for a Gaussian mixture model. The data structure +<CODE>mix</CODE> defines the mixture model, while the matrix <CODE>x</CODE> contains +the data vectors. Each row of <CODE>x</CODE> represents a single vector. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmpost.htm">gmmpost</a></CODE>, <CODE><a href="gmmactiv.htm">gmmactiv</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmsamp.htm b/sourcecodes/bnt-master/nethelp3.3/gmmsamp.htm new file mode 100644 index 00000000..30f6821c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmsamp.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmsamp +</title> +</head> +<body> +<H1> gmmsamp +</H1> +<h2> +Purpose +</h2> +Sample from a Gaussian mixture distribution. + +<p><h2> +Synopsis +</h2> +<PRE> +data = gmmsamp(mix, n) +[data, label] = gmmsamp(mix, n) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>data = gsamp(mix, n)</CODE> generates a sample of size <CODE>n</CODE> from a +Gaussian mixture distribution defined by the <CODE>mix</CODE> data +structure. The matrix <CODE>x</CODE> has <CODE>n</CODE> +rows in which each row represents a <CODE>mix.nin</CODE>-dimensional sample vector. + +<p><CODE>[data, label] = gmmsamp(mix, n)</CODE> also returns a column vector of +classes (as an index 1..N) <CODE>label</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gsamp.htm">gsamp</a></CODE>, <CODE><a href="gmm.htm">gmm</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmunpak.htm b/sourcecodes/bnt-master/nethelp3.3/gmmunpak.htm new file mode 100644 index 00000000..2287fb3b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmunpak.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual gmmunpak +</title> +</head> +<body> +<H1> gmmunpak +</H1> +<h2> +Purpose +</h2> +Separates a vector of Gaussian mixture model parameters into its components. + +<p><h2> +Synopsis +</h2> +<PRE> +mix = gmmunpak(mix, p) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>mix = gmmunpak(mix, p)</CODE> +takes a GMM data structure <CODE>mix</CODE> and +a single row vector of parameters <CODE>p</CODE> and returns a mixture data structure +identical to the input <CODE>mix</CODE>, except that the mixing coefficients +<CODE>priors</CODE>, centres <CODE>centres</CODE> and covariances <CODE>covars</CODE> +(and, for PPCA, the lambdas and U (PCA sub-spaces)) are all set +to the corresponding elements of <CODE>p</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="gmmpak.htm">gmmpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gp.htm b/sourcecodes/bnt-master/nethelp3.3/gp.htm new file mode 100644 index 00000000..77d6ab1a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gp.htm @@ -0,0 +1,80 @@ +<html> +<head> +<title> +Netlab Reference Manual gp +</title> +</head> +<body> +<H1> gp +</H1> +<h2> +Purpose +</h2> +Create a Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +net = gp(nin, covarfn) +net = gp(nin, covarfn, prior) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = gp(nin, covarfn)</CODE> takes the number of inputs <CODE>nin</CODE> +for a Gaussian Process model with a single output, together +with a string <CODE>covarfn</CODE> which specifies the type of the covariance function, +and returns a data structure <CODE>net</CODE>. The parameters are set to zero. + +<p>The fields in <CODE>net</CODE> are +<PRE> + type = 'gp' + nin = number of inputs + nout = number of outputs: always 1 + nwts = total number of weights and covariance function parameters + bias = logarithm of constant offset in covariance function + noise = logarithm of output noise variance + inweights = logarithm of inverse length scale for each input + covarfn = string describing the covariance function: + 'sqexp' + 'ratquad' + fpar = covariance function specific parameters (1 for squared exponential, + 2 for rational quadratic) + trin = training input data (initially empty) + trtargets = training target data (initially empty) +</PRE> + + +<p><CODE>net = gp(nin, covarfn, prior)</CODE> sets a Gaussian prior on the +parameters of the model. <CODE>prior</CODE> must contain the fields +<CODE>pr_mean</CODE> and <CODE>pr_variance</CODE>. If <CODE>pr_mean</CODE> is a scalar, +then the Gaussian is assumed to be isotropic and the additional fields +<CODE>net.pr_mean</CODE> and <CODE>pr_variance</CODE> are set. Otherwise, +the Gaussian prior has a mean +defined by a column vector of parameters <CODE>prior.pr_mean</CODE> and +covariance defined by a column vector of parameters <CODE>prior.pr_variance</CODE>. +Each element of <CODE>prmean</CODE> corresponds to a separate group of parameters, which +need not be mutually exclusive. The membership of the groups is defined +by the matrix <CODE>prior.index</CODE> in which the columns correspond to the elements of +<CODE>prmean</CODE>. Each column has one element for each weight in the matrix, +in the order defined by the function <CODE>gppak</CODE>, and each element +is 1 or 0 according to whether the parameter is a member of the +corresponding group or not. The additional field <CODE>net.index</CODE> is set +in this case. + +<p><h2> +See Also +</h2> +<CODE><a href="gppak.htm">gppak</a></CODE>, <CODE><a href="gpunpak.htm">gpunpak</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpcovar.htm">gpcovar</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpcovar.htm b/sourcecodes/bnt-master/nethelp3.3/gpcovar.htm new file mode 100644 index 00000000..f626f8fd --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpcovar.htm @@ -0,0 +1,64 @@ +<html> +<head> +<title> +Netlab Reference Manual gpcovar +</title> +</head> +<body> +<H1> gpcovar +</H1> +<h2> +Purpose +</h2> +Calculate the covariance for a Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +cov = gpcovar(net, x) +[cov, covf] = gpcovar(net, x) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>cov = gpcovar(net, x)</CODE> takes +a Gaussian Process data structure <CODE>net</CODE> together with +a matrix <CODE>x</CODE> of input vectors, and computes the covariance +matrix <CODE>cov</CODE>. The inverse of this matrix is used when calculating +the mean and variance of the predictions made by <CODE>net</CODE>. + +<p><CODE>[cov, covf] = gpcovar(net, x)</CODE> also generates the covariance +matrix due to the covariance function specified by <CODE>net.covarfn</CODE> +as calculated by <CODE>gpcovarf</CODE>. + +<p><h2> +Example +</h2> +In the following example, the inverse covariance matrix is calculated +for a set of training inputs <CODE>x</CODE> and is then +passed to <CODE>gpfwd</CODE> so that predictions (with mean <CODE>ytest</CODE> and +variance <CODE>sigsq</CODE>) can be made for the test inputs +<CODE>xtest</CODE>. +<PRE> + +cninv = inv(gpcovar(net, x)); +[ytest, sigsq] = gpfwd(net, xtest, cninv); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gppak.htm">gppak</a></CODE>, <CODE><a href="gpunpak.htm">gpunpak</a></CODE>, <CODE><a href="gpcovarp.htm">gpcovarp</a></CODE>, <CODE><a href="gpcovarf.htm">gpcovarf</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpcovarf.htm b/sourcecodes/bnt-master/nethelp3.3/gpcovarf.htm new file mode 100644 index 00000000..5bce6b98 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpcovarf.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual gpcovarf +</title> +</head> +<body> +<H1> gpcovarf +</H1> +<h2> +Purpose +</h2> +Calculate the covariance function for a Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +covf = gpcovarf(net, x1, x2) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>covf = gpcovarf(net, x1, x2)</CODE> takes +a Gaussian Process data structure <CODE>net</CODE> together with +two matrices <CODE>x1</CODE> and <CODE>x2</CODE> of input vectors, +and computes the matrix of the covariance function values +<CODE>covf</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gpcovar.htm">gpcovar</a></CODE>, <CODE><a href="gpcovarp.htm">gpcovarp</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpcovarp.htm b/sourcecodes/bnt-master/nethelp3.3/gpcovarp.htm new file mode 100644 index 00000000..616664ce --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpcovarp.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual gpcovarp +</title> +</head> +<body> +<H1> gpcovarp +</H1> +<h2> +Purpose +</h2> +Calculate the prior covariance for a Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +covp = gpcovarp(net, x1, x2) +[covp, covf] = gpcovarp(net, x1, x2) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>covp = gpcovarp(net, x1, x2)</CODE> takes +a Gaussian Process data structure <CODE>net</CODE> together with +two matrices <CODE>x1</CODE> and <CODE>x2</CODE> of input vectors, +and computes the matrix of the prior covariance. This is +the function component of the covariance plus the exponential of the bias +term. + +<p><CODE>[covp, covf] = gpcovarp(net, x1, x2)</CODE> also returns the function +component of the covariance. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gpcovar.htm">gpcovar</a></CODE>, <CODE><a href="gpcovarf.htm">gpcovarf</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gperr.htm b/sourcecodes/bnt-master/nethelp3.3/gperr.htm new file mode 100644 index 00000000..de8d9831 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gperr.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual gperr +</title> +</head> +<body> +<H1> gperr +</H1> +<h2> +Purpose +</h2> +Evaluate error function for Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +edata = gperr(net, x, t) +[e, edata, eprior] = gperr(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>e = gperr(net, x, t)</CODE> takes a Gaussian Process data structure <CODE>net</CODE> together +with a matrix <CODE>x</CODE> of input vectors and a matrix <CODE>t</CODE> of target +vectors, and evaluates the error function <CODE>e</CODE>. Each row +of <CODE>x</CODE> corresponds to one input vector and each row of <CODE>t</CODE> +corresponds to one target vector. + +<p><CODE>[e, edata, eprior] = gperr(net, x, t)</CODE> additionally returns the +data and hyperprior components of the error, assuming a Gaussian +prior on the weights with mean and variance parameters <CODE>prmean</CODE> and +<CODE>prvariance</CODE> taken from the network data structure <CODE>net</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gpcovar.htm">gpcovar</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpfwd.htm b/sourcecodes/bnt-master/nethelp3.3/gpfwd.htm new file mode 100644 index 00000000..51fc239c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpfwd.htm @@ -0,0 +1,73 @@ +<html> +<head> +<title> +Netlab Reference Manual gpfwd +</title> +</head> +<body> +<H1> gpfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +y = gpfwd(net, x) +[y, sigsq] = gpfwd(net, x) +[y, sigsq] = gpfwd(net, x, cninv) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = gpfwd(net, x)</CODE> takes a Gaussian Process data structure <CODE>net</CODE> +together +with a matrix <CODE>x</CODE> of input vectors, and forward propagates the inputs +through the model to generate a matrix <CODE>y</CODE> of output +vectors. Each row of <CODE>x</CODE> corresponds to one input vector and each +row of <CODE>y</CODE> corresponds to one output vector. This assumes that the +training data (both inputs and targets) has been stored in <CODE>net</CODE> by +a call to <CODE>gpinit</CODE>; these are needed to compute the training +data covariance matrix. + +<p><CODE>[y, sigsq] = gpfwd(net, x)</CODE> also generates a column vector <CODE>sigsq</CODE> of +conditional variances (or squared error bars) where each value corresponds to a pattern. + +<p><CODE>[y, sigsq] = gpfwd(net, x, cninv)</CODE> uses the pre-computed inverse covariance +matrix <CODE>cninv</CODE> in the forward propagation. This increases efficiency if +several calls to <CODE>gpfwd</CODE> are made. + +<p><h2> +Example +</h2> +The following code creates a Gaussian Process, trains it, and then plots the +predictions on a test set with one standard deviation error bars: +<PRE> + +net = gp(1, 'sqexp'); +net = gpinit(net, x, t); +net = netopt(net, options, x, t, 'scg'); +[pred, sigsq] = gpfwd(net, xtest); +plot(xtest, pred, '-k'); +hold on +plot(xtest, pred+sqrt(sigsq), '-b', xtest, pred-sqrt(sigsq), '-b'); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="demgp.htm">demgp</a></CODE>, <CODE><a href="gpinit.htm">gpinit</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpgrad.htm b/sourcecodes/bnt-master/nethelp3.3/gpgrad.htm new file mode 100644 index 00000000..887cdf1b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpgrad.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual gpgrad +</title> +</head> +<body> +<H1> gpgrad +</H1> +<h2> +Purpose +</h2> +Evaluate error gradient for Gaussian Process. + +<p><h2> +Synopsis +</h2> +<PRE> +g = gpgrad(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = gpgrad(net, x, t)</CODE> takes a Gaussian Process data structure <CODE>net</CODE> together +with a matrix <CODE>x</CODE> of input vectors and a matrix <CODE>t</CODE> of target +vectors, and evaluates the error gradient <CODE>g</CODE>. Each row +of <CODE>x</CODE> corresponds to one input vector and each row of <CODE>t</CODE> +corresponds to one target vector. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gpcovar.htm">gpcovar</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpinit.htm b/sourcecodes/bnt-master/nethelp3.3/gpinit.htm new file mode 100644 index 00000000..1874dde7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpinit.htm @@ -0,0 +1,76 @@ +<html> +<head> +<title> +Netlab Reference Manual gpinit +</title> +</head> +<body> +<H1> gpinit +</H1> +<h2> +Purpose +</h2> +Initialise Gaussian Process model. + +<p><h2> +Synopsis +</h2> +<PRE> +net = gpinit(net, trin, trtargets, prior) +net = gpinit(net, trin, trtargets, prior) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = gpinit(net, trin, trtargets)</CODE> takes a Gaussian Process data structure <CODE>net</CODE> +together +with a matrix <CODE>trin</CODE> of training input vectors and a matrix <CODE>trtargets</CODE> of +training target +vectors, and stores them in <CODE>net</CODE>. These datasets are required if +the corresponding inverse covariance matrix is not supplied to <CODE>gpfwd</CODE>. +This is important if the data structure is saved and then reloaded before +calling <CODE>gpfwd</CODE>. +Each row +of <CODE>trin</CODE> corresponds to one input vector and each row of <CODE>trtargets</CODE> +corresponds to one target vector. + +<p><CODE>net = gpinit(net, trin, trtargets, prior)</CODE> additionally initialises the +parameters in <CODE>net</CODE> from the <CODE>prior</CODE> data structure which contains the +mean and variance of the Gaussian distribution which is sampled from. + +<p><h2> +Example +</h2> +Suppose that a Gaussian Process model is created and trained with input data <CODE>x</CODE> +and targets <CODE>t</CODE>: +<PRE> + +net = gp(2, 'sqexp'); +net = gpinit(net, x, t); +% Train the network +save 'gp.net' net; +</PRE> + +Another Matlab program can now read in the network and make predictions on a data set +<CODE>testin</CODE>: +<PRE> + +load 'gp.net'; +pred = gpfwd(net, testin); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gppak.htm b/sourcecodes/bnt-master/nethelp3.3/gppak.htm new file mode 100644 index 00000000..e913dd2c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gppak.htm @@ -0,0 +1,40 @@ +<html> +<head> +<title> +Netlab Reference Manual gppak +</title> +</head> +<body> +<H1> gppak +</H1> +<h2> +Purpose +</h2> +Combines GP hyperparameters into one vector. + +<p><h2> +Synopsis +</h2> +<PRE> +hp = gppak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>hp = gppak(net)</CODE> takes a Gaussian Process data structure <CODE>net</CODE> and +combines the hyperparameters into a single row vector <CODE>hp</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gpunpak.htm">gpunpak</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpunpak.htm b/sourcecodes/bnt-master/nethelp3.3/gpunpak.htm new file mode 100644 index 00000000..593dbb04 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpunpak.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual gpunpak +</title> +</head> +<body> +<H1> gpunpak +</H1> +<h2> +Purpose +</h2> +Separates hyperparameter vector into components. + +<p><h2> +Synopsis +</h2> +<PRE> +net = gpunpak(net, hp) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = gpunpak(net, hp)</CODE> takes an Gaussian Process data structure <CODE>net</CODE> and +a hyperparameter vector <CODE>hp</CODE>, and returns a Gaussian Process data structure +identical to +the input model, except that the covariance bias +<CODE>bias</CODE>, output noise <CODE>noise</CODE>, the input weight vector +<CODE>inweights</CODE> and the vector of covariance function specific parameters + <CODE>fpar</CODE> have all +been set to the corresponding elements of <CODE>hp</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gp.htm">gp</a></CODE>, <CODE><a href="gppak.htm">gppak</a></CODE>, <CODE><a href="gpfwd.htm">gpfwd</a></CODE>, <CODE><a href="gperr.htm">gperr</a></CODE>, <CODE><a href="gpgrad.htm">gpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gradchek.htm b/sourcecodes/bnt-master/nethelp3.3/gradchek.htm new file mode 100644 index 00000000..0a7df6ea --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gradchek.htm @@ -0,0 +1,58 @@ +<html> +<head> +<title> +Netlab Reference Manual gradchek +</title> +</head> +<body> +<H1> gradchek +</H1> +<h2> +Purpose +</h2> +Checks a user-defined gradient function using finite differences. + +<p><h2> +Synopsis +</h2> +<PRE> + +gradchek(w, func, grad) +[gradient, delta] = gradchek(w, func, grad) +gradchek(w, func, grad, p1, p2, ...) +</PRE> + + +<p><h2> +Description +</h2> +This function is intended as a utility for other netlab functions +(particularly optimisation functions) to use. It enables the user +to check whether a gradient calculation has been correctly implmented +for a given function. +<CODE>gradchek(w, func, grad)</CODE> checks how accurate the gradient +<CODE>grad</CODE> of a function <CODE>func</CODE> is at a parameter vector <CODE>x</CODE>. +A central +difference formula with step size 1.0e-6 is used, and the results +for both gradient function and finite difference approximation are +printed. +The optional return value <CODE>gradient</CODE> is the gradient calculated +using the function <CODE>grad</CODE> and the return value <CODE>delta</CODE> is the +difference between the functional and finite difference methods of +calculating the graident. + +<p><CODE>gradchek(x, func, grad, p1, p2, ...)</CODE> allows additional arguments +to be passed to <CODE>func</CODE> and <CODE>grad</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="hmc.htm">hmc</a></CODE>, <CODE><a href="olgd.htm">olgd</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/graddesc.htm b/sourcecodes/bnt-master/nethelp3.3/graddesc.htm new file mode 100644 index 00000000..fe8a3df1 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/graddesc.htm @@ -0,0 +1,104 @@ +<html> +<head> +<title> +Netlab Reference Manual graddesc +</title> +</head> +<body> +<H1> graddesc +</H1> +<h2> +Purpose +</h2> +Gradient descent optimization. + +<p><h2> +Description +</h2> +<CODE>[x, options, flog, pointlog] = graddesc(f, x, options, gradf)</CODE> uses +batch gradient descent to find a local minimum of the function +<CODE>f(x)</CODE> whose gradient is given by <CODE>gradf(x)</CODE>. A log of the function values +after each cycle is (optionally) returned in <CODE>errlog</CODE>, and a log +of the points visited is (optionally) returned in <CODE>pointlog</CODE>. + +<p>Note that <CODE>x</CODE> is a row vector +and <CODE>f</CODE> returns a scalar value. +The point at which <CODE>f</CODE> has a local minimum +is returned as <CODE>x</CODE>. The function value at that point is returned +in <CODE>options(8)</CODE>. + +<p><CODE>graddesc(f, x, options, gradf, p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>, and the points visited +in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is the absolute precision required for the value +of <CODE>x</CODE> at the solution. If the absolute difference between +the values of <CODE>x</CODE> between two successive steps is less than +<CODE>options(2)</CODE>, then this condition is satisfied. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. If the absolute difference between the +objective function values between two successive steps is less than +<CODE>options(3)</CODE>, then this condition is satisfied. +Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(7)</CODE> determines the line minimisation method used. If it +is set to 1 then a line minimiser is used (in the direction of the negative +gradient). If it is 0 (the default), then each parameter update +is a fixed multiple (the learning rate) +of the negative gradient added to a fixed multiple (the momentum) of +the previous parameter update. + +<p><CODE>options(9)</CODE> should be set to 1 to check the user defined gradient +function <CODE>gradf</CODE> with <CODE>gradchek</CODE>. This is carried out at +the initial parameter vector <CODE>x</CODE>. + +<p><CODE>options(10)</CODE> returns the total number of function evaluations (including +those in any line searches). + +<p><CODE>options(11)</CODE> returns the total number of gradient evaluations. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p><CODE>options(15)</CODE> is the precision in parameter space of the line search; +default <CODE>foptions(2)</CODE>. + +<p><CODE>options(17)</CODE> is the momentum; default 0.5. It should be scaled by the +inverse of the number of data points. + +<p><CODE>options(18)</CODE> is the learning rate; default 0.01. It should be +scaled by the inverse of the number of data points. + +<p><h2> +Examples +</h2> +An example of how this function can be used to train a neural network is: +<PRE> + +options = zeros(1, 18); +options(17) = 0.1/size(x, 1); +net = netopt(net, options, x, t, 'graddesc'); +</PRE> + +Note how the learning rate is scaled by the number of data points. + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="olgd.htm">olgd</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gsamp.htm b/sourcecodes/bnt-master/nethelp3.3/gsamp.htm new file mode 100644 index 00000000..0f5e5ead --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gsamp.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual gsamp +</title> +</head> +<body> +<H1> gsamp +</H1> +<h2> +Purpose +</h2> +Sample from a Gaussian distribution. + +<p><h2> +Synopsis +</h2> +<PRE> +x = gsamp(mu, covar, nsamp) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>x = gsamp(mu, covar, nsamp)</CODE> generates a sample of size <CODE>nsamp</CODE> +from a <CODE>d</CODE>-dimensional Gaussian distribution. The Gaussian density +has mean vector <CODE>mu</CODE> and covariance matrix <CODE>covar</CODE>, and the +matrix <CODE>x</CODE> has <CODE>nsamp</CODE> rows in which each row represents a +<CODE>d</CODE>-dimensional sample vector. + +<p><h2> +See Also +</h2> +<CODE><a href="gauss.htm">gauss</a></CODE>, <CODE><a href="demgauss.htm">demgauss</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtm.htm b/sourcecodes/bnt-master/nethelp3.3/gtm.htm new file mode 100644 index 00000000..e02ef536 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtm.htm @@ -0,0 +1,66 @@ +<html> +<head> +<title> +Netlab Reference Manual gtm +</title> +</head> +<body> +<H1> gtm +</H1> +<h2> +Purpose +</h2> +Create a Generative Topographic Map. + +<p><h2> +Synopsis +</h2> +<PRE> +net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc) +net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc, prior) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc)</CODE>, +takes the dimension of the latent space <CODE>dimlatent</CODE>, the +number of data points sampled in the latent space <CODE>nlatent</CODE>, the +dimension of the data space <CODE>dimdata</CODE>, the number of centres in the +RBF model <CODE>ncentres</CODE>, the activation function for the RBF +<CODE>rbfunc</CODE> +and returns a data structure <CODE>net</CODE>. The parameters in the +RBF and GMM sub-models are set by calls to the corresponding creation routines +<CODE>rbf</CODE> and <CODE>gmm</CODE>. + +<p>The fields in <CODE>net</CODE> are +<PRE> + type = 'gtm' + nin = dimension of data space + dimlatent = dimension of latent space + rbfnet = RBF network data structure + gmmnet = GMM data structure + X = sample of latent points +</PRE> + + +<p><CODE>net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc, prior)</CODE>, + sets a Gaussian zero mean prior on the +parameters of the RBF model. <CODE>prior</CODE> must be a scalar and represents +the inverse variance of the prior distribution. This gives rise to +a weight decay term in the error function. + +<p><h2> +See Also +</h2> +<CODE><a href="gtmfwd.htm">gtmfwd</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE>, <CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="gmm.htm">gmm</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmem.htm b/sourcecodes/bnt-master/nethelp3.3/gtmem.htm new file mode 100644 index 00000000..95c385b1 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmem.htm @@ -0,0 +1,88 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmem +</title> +</head> +<body> +<H1> gtmem +</H1> +<h2> +Purpose +</h2> +EM algorithm for Generative Topographic Mapping. + +<p><h2> +Synopsis +</h2> +<PRE> + +[net, options, errlog] = gtmem(net, t, options) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>[net, options, errlog] = gtmem(net, t, options)</CODE> uses the Expectation +Maximization algorithm to estimate the parameters of +a GTM defined by a data structure <CODE>net</CODE>. +The matrix <CODE>t</CODE> represents the data whose expectation +is maximized, with each row corresponding to a vector. It is assumed +that the latent data <CODE>net.X</CODE> has been set following a call to +<CODE>gtminit</CODE>, for example. + +The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>. +If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(3)</CODE> is a measure of the absolute precision required of the error +function at the solution. If the change in log likelihood between two steps of +the EM algorithm is less than this value, then the function terminates. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p>The optional return value <CODE>options</CODE> contains the final error value +(i.e. data log likelihood) in +<CODE>options(8)</CODE>. + +<p><h2> +Examples +</h2> +The following code fragment sets up a GTM, initialises +the latent data sample and RBF +parameters from the data, sets the options and trains the model. +<PRE> + +% Create and initialise GTM model +net = gtm(latentdim, nlatent, datadim, numrbfcentres, ... + 'gaussian', 0.1); + +<p>options = foptions; +options(1) = -1; +options(7) = 1; % Set width factor of RBF +net = gtminit(net, options, data, 'regular', latentshape, [4 4]); + +<p>options = foptions; +options(14) = 30; +options(1) = 1; +[net, options] = gtmem(net, data, options); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtminit.htm">gtminit</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmfwd.htm b/sourcecodes/bnt-master/nethelp3.3/gtmfwd.htm new file mode 100644 index 00000000..2ba02a35 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmfwd.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmfwd +</title> +</head> +<body> +<H1> gtmfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +mix = gtmfwd(net) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>mix = gtmfwd(net)</CODE> takes a GTM +structure <CODE>net</CODE>, and forward +propagates the latent data sample <CODE>net.X</CODE> through the GTM to generate +the structure +<CODE>mix</CODE> which represents the Gaussian mixture model in data space. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtminit.htm b/sourcecodes/bnt-master/nethelp3.3/gtminit.htm new file mode 100644 index 00000000..8ac58331 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtminit.htm @@ -0,0 +1,66 @@ +<html> +<head> +<title> +Netlab Reference Manual gtminit +</title> +</head> +<body> +<H1> gtminit +</H1> +<h2> +Purpose +</h2> +Initialise the weights and latent sample in a GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +net = gtminit(net, options, data, samptype) +net = gtminit(net, options, data, samptype, lsampsize, rbfsampsize) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = gtminit(net, options, data, samptype)</CODE> takes a GTM <CODE>net</CODE> +and generates a sample of latent data points and sets the centres (and +widths if appropriate) of +<CODE>net.rbfnet</CODE>. + +<p>If the <CODE>samptype</CODE> is <CODE>'regular'</CODE>, then regular grids of latent +data points and RBF centres are created. The dimension of the latent data +space must be +1 or 2. For one-dimensional latent space, the <CODE>lsampsize</CODE> parameter +gives the number of latent points and the <CODE>rbfsampsize</CODE> parameter +gives the number of RBF centres. For a two-dimensional latent space, +these parameters must be vectors of length 2 with the number of points +in each of the x and y directions to create a rectangular grid. The +widths of the RBF basis functions are set by a call to <CODE>rbfsetfw</CODE> +passing <CODE>options(7)</CODE> as the scaling parameter. + +<p>If the <CODE>samptype</CODE> is <CODE>'uniform'</CODE> or <CODE>'gaussian'</CODE> then the +latent data is found by sampling from a uniform or +Gaussian distribution correspondingly. The RBF basis function parameters +are set +by a call to <CODE>rbfsetbf</CODE> with the <CODE>data</CODE> parameter +as dataset and the <CODE>options</CODE> vector. + +<p>Finally, the output layer weights of the RBF are initialised by +mapping the mean of the latent variable to the mean of the target variable, +and the L-dimensional latent variale variance to the variance of the +targets along the first L principal components. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmem.htm">gtmem</a></CODE>, <CODE><a href="pca.htm">pca</a></CODE>, <CODE><a href="rbfsetbf.htm">rbfsetbf</a></CODE>, <CODE><a href="rbfsetfw.htm">rbfsetfw</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmlmean.htm b/sourcecodes/bnt-master/nethelp3.3/gtmlmean.htm new file mode 100644 index 00000000..1736c0ed --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmlmean.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmlmean +</title> +</head> +<body> +<H1> gtmlmean +</H1> +<h2> +Purpose +</h2> +Mean responsibility for data in a GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +means = gtmlmean(net, data) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>means = gtmlmean(net, data)</CODE> takes a GTM +structure <CODE>net</CODE>, and computes the means of the responsibility +distributions for each data point in <CODE>data</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE>, <CODE><a href="gtmlmode.htm">gtmlmode</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmlmode.htm b/sourcecodes/bnt-master/nethelp3.3/gtmlmode.htm new file mode 100644 index 00000000..ab37c6ff --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmlmode.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmlmode +</title> +</head> +<body> +<H1> gtmlmode +</H1> +<h2> +Purpose +</h2> +Mode responsibility for data in a GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +modes = gtmlmode(net, data) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>modes = gtmlmode(net, data)</CODE> takes a GTM +structure <CODE>net</CODE>, and computes the modes of the responsibility +distributions for each data point in <CODE>data</CODE>. These will always lie +at one of the latent space sample points <CODE>net.X</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE>, <CODE><a href="gtmlmean.htm">gtmlmean</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmmag.htm b/sourcecodes/bnt-master/nethelp3.3/gtmmag.htm new file mode 100644 index 00000000..f8590da7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmmag.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmmag +</title> +</head> +<body> +<H1> gtmmag +</H1> +<h2> +Purpose +</h2> +Magnification factors for a GTM + +<p><h2> +Synopsis +</h2> +<PRE> +mags = gtmmag(net, latentdata) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>mags = gtmmag(net, latentdata)</CODE> takes a GTM +structure <CODE>net</CODE>, and computes the magnification factors +for each point the latent space contained in <CODE>latentdata</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE>, <CODE><a href="gtmlmean.htm">gtmlmean</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmpost.htm b/sourcecodes/bnt-master/nethelp3.3/gtmpost.htm new file mode 100644 index 00000000..a64ce88a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmpost.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmpost +</title> +</head> +<body> +<H1> gtmpost +</H1> +<h2> +Purpose +</h2> +Latent space responsibility for data in a GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +post = gtmpost(net, data) +[post, a] = gtmpost(net, data) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>post = gtmpost(net, data)</CODE> takes a GTM +structure <CODE>net</CODE>, and computes the responsibility at each latent space +sample point <CODE>net.X</CODE> +for each data point in <CODE>data</CODE>. + +<p><CODE>[post, a] = gtmpost(net, data)</CODE> also returns the activations +<CODE>a</CODE> of the GMM <CODE>net.gmmnet</CODE> as computed by <CODE>gmmpost</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmem.htm">gtmem</a></CODE>, <CODE><a href="gtmlmean.htm">gtmlmean</a></CODE>, <CODE><a href="gmlmode.htm">gmlmode</a></CODE>, <CODE><a href="gmmprob.htm">gmmprob</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmprob.htm b/sourcecodes/bnt-master/nethelp3.3/gtmprob.htm new file mode 100644 index 00000000..d3a44e30 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmprob.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual gtmprob +</title> +</head> +<body> +<H1> gtmprob +</H1> +<h2> +Purpose +</h2> +Probability for data under a GTM. + +<p><h2> +Synopsis +</h2> +<PRE> +prob = gtmprob(net, data) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>prob = gtmprob(net, data)</CODE> takes a GTM +structure <CODE>net</CODE>, and computes the probability of each point in the +dataset <CODE>data</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gtm.htm">gtm</a></CODE>, <CODE><a href="gtmem.htm">gtmem</a></CODE>, <CODE><a href="gtmlmean.htm">gtmlmean</a></CODE>, <CODE><a href="gtmlmode.htm">gtmlmode</a></CODE>, <CODE><a href="gtmpost.htm">gtmpost</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hbayes.htm b/sourcecodes/bnt-master/nethelp3.3/hbayes.htm new file mode 100644 index 00000000..5fa0783d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hbayes.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual hbayes +</title> +</head> +<body> +<H1> hbayes +</H1> +<h2> +Purpose +</h2> +Evaluate Hessian of Bayesian error function for network. + +<p><h2> +Synopsis +</h2> +<PRE> +h = hbayes(net, hdata) +[h, hdata] = hbayes(net, hdata) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>h = hbayes(net, hdata)</CODE> takes a network data structure <CODE>net</CODE> together +the data contribution to the Hessian +for a set of inputs and targets. +It returns the regularised Hessian using any zero mean Gaussian priors +on the weights defined in +<CODE>net</CODE>. In addition, if a <CODE>mask</CODE> is defined in <CODE>net</CODE>, then +the entries in <CODE>h</CODE> that correspond to weights with a 0 in the +mask are removed. + +<p><CODE>[h, hdata] = hbayes(net, hdata)</CODE> additionally returns the +data component of the Hessian. + +<p><h2> +See Also +</h2> +<CODE><a href="gbayes.htm">gbayes</a></CODE>, <CODE><a href="glmhess.htm">glmhess</a></CODE>, <CODE><a href="mlphess.htm">mlphess</a></CODE>, <CODE><a href="rbfhess.htm">rbfhess</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hesschek.htm b/sourcecodes/bnt-master/nethelp3.3/hesschek.htm new file mode 100644 index 00000000..34a4f591 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hesschek.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual hesschek +</title> +</head> +<body> +<H1> hesschek +</H1> +<h2> +Purpose +</h2> +Use central differences to confirm correct evaluation of Hessian matrix. + +<p><h2> +Synopsis +</h2> +<PRE> +hesschek(net, x, t) +h = hesschek(net, x, t)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>hesschek(net, x, t)</CODE> takes a network data structure <CODE>net</CODE>, together +with input and target data matrices <CODE>x</CODE> and <CODE>t</CODE>, and compares +the evaluation of the Hessian matrix using the function <CODE>nethess</CODE> +and using central differences with the function <CODE>neterr</CODE>. + +<p>The optional return value <CODE>h</CODE> is the Hessian computed using +<CODE>nethess</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="nethess.htm">nethess</a></CODE>, <CODE><a href="neterr.htm">neterr</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hintmat.htm b/sourcecodes/bnt-master/nethelp3.3/hintmat.htm new file mode 100644 index 00000000..5363385d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hintmat.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual hintmat +</title> +</head> +<body> +<H1> hintmat +</H1> +<h2> +Purpose +</h2> +Evaluates the coordinates of the patches for a Hinton diagram. + +<p><h2> +Synopsis +</h2> +<PRE> +[xvals, yvals, color] = hintmat(w)</PRE> + + +<p><h2> +Description +</h2> +<PRE> +[xvals, yvals, color] = hintmat(w)</PRE> + +takes a matrix <CODE>w</CODE> and +returns coordinates <CODE>xvals, yvals</CODE> for the patches comrising the +Hinton diagram, together with a vector <CODE>color</CODE> labelling the color +(black or white) of the corresponding elements according to their +sign. + +<p><h2> +See Also +</h2> +<CODE><a href="hinton.htm">hinton</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hinton.htm b/sourcecodes/bnt-master/nethelp3.3/hinton.htm new file mode 100644 index 00000000..570f3fb0 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hinton.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual hinton +</title> +</head> +<body> +<H1> hinton +</H1> +<h2> +Purpose +</h2> +Plot Hinton diagram for a weight matrix. + +<p><h2> +Synopsis +</h2> +<PRE> +hinton(w) +h = hinton(w)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>hinton(w)</CODE> takes a matrix <CODE>w</CODE> +and plots the Hinton diagram. + +<p><CODE>h = hinton(net)</CODE> also returns the figure handle <CODE>h</CODE> +which can be used, for instance, to delete the +figure when it is no longer needed. + +<p>To print the figure correctly in black and white, you should call +<CODE>set(h, 'InvertHardCopy', 'off')</CODE> before printing. + +<p><h2> +See Also +</h2> +<CODE><a href="demhint.htm">demhint</a></CODE>, <CODE><a href="hintmat.htm">hintmat</a></CODE>, <CODE><a href="mlphint.htm">mlphint</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/histp.htm b/sourcecodes/bnt-master/nethelp3.3/histp.htm new file mode 100644 index 00000000..fd1fa111 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/histp.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual histp +</title> +</head> +<body> +<H1> histp +</H1> +<h2> +Purpose +</h2> +Histogram estimate of 1-dimensional probability distribution. + +<p><h2> +Synopsis +</h2> +<PRE> +h = histp(x, xmin, xmax, nbins) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>histp(x, xmin, xmax, nbins)</CODE> takes a column vector <CODE>x</CODE> +of data values and generates a normalized histogram plot of the +distribution. The histogram has <CODE>nbins</CODE> bins lying in the +range <CODE>xmin</CODE> to <CODE>xmax</CODE>. + +<p><CODE>h = histp(...)</CODE> returns a vector of patch handles. + +<p><h2> +See Also +</h2> +<CODE><a href="demgauss.htm">demgauss</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hmc.htm b/sourcecodes/bnt-master/nethelp3.3/hmc.htm new file mode 100644 index 00000000..53afeb31 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hmc.htm @@ -0,0 +1,129 @@ +<html> +<head> +<title> +Netlab Reference Manual hmc +</title> +</head> +<body> +<H1> hmc +</H1> +<h2> +Purpose +</h2> +Hybrid Monte Carlo sampling. + +<p><h2> +Synopsis +</h2> +<PRE> + +samples = hmc(f, x, options, gradf) +samples = hmc(f, x, options, gradf, P1, P2, ...) +[samples, energies, diagn] = hmc(f, x, options, gradf) +s = hmc('state') +hmc('state', s) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>samples = hmc(f, x, options, gradf)</CODE> uses a +hybrid Monte Carlo algorithm to sample from the distribution <CODE>p ~ exp(-f)</CODE>, +where <CODE>f</CODE> is the first argument to <CODE>hmc</CODE>. +The Markov chain starts at the point <CODE>x</CODE>, and the function <CODE>gradf</CODE> +is the gradient of the `energy' function <CODE>f</CODE>. + +<p><CODE>hmc(f, x, options, gradf, p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>. + +<p><CODE>[samples, energies, diagn] = hmc(f, x, options, gradf)</CODE> also returns +a log of the energy values (i.e. negative log probabilities) for the +samples in <CODE>energies</CODE> and <CODE>diagn</CODE>, a structure containing +diagnostic information (position, momentum and +acceptance threshold) for each step of the chain in <CODE>diagn.pos</CODE>, +<CODE>diagn.mom</CODE> and +<CODE>diagn.acc</CODE> respectively. All candidate states (including rejected ones) +are stored in <CODE>diagn.pos</CODE>. + +<p><CODE>[samples, energies, diagn] = hmc(f, x, options, gradf)</CODE> also returns the +<CODE>energies</CODE> (i.e. negative log probabilities) corresponding to the samples. +The <CODE>diagn</CODE> structure contains three fields: + +<p><CODE>pos</CODE> the position vectors of the dynamic process. + +<p><CODE>mom</CODE> the momentum vectors of the dynamic process. + +<p><CODE>acc</CODE> the acceptance thresholds. + +<p><CODE>s = hmc('state')</CODE> returns a state structure that contains the state of the +two random number generators <CODE>rand</CODE> and <CODE>randn</CODE> and the momentum of +the dynamic process. These are contained in fields +<CODE>randstate</CODE>, <CODE>randnstate</CODE> +and <CODE>mom</CODE> respectively. The momentum state is +only used for a persistent momentum update. + +<p><CODE>hmc('state', s)</CODE> resets the state to <CODE>s</CODE>. If <CODE>s</CODE> is an integer, +then it is passed to <CODE>rand</CODE> and <CODE>randn</CODE> and the momentum variable +is randomised. If <CODE>s</CODE> is a structure returned by <CODE>hmc('state')</CODE> then +it resets the generator to exactly the same state. + +<p>The optional parameters in the <CODE>options</CODE> vector have the following +interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display the energy values and rejection +threshold at each step of the Markov chain. If the value is 2, then the +position vectors at each step are also displayed. + +<p><CODE>options(5)</CODE> is set to 1 if momentum persistence is used; default 0, for +complete replacement of momentum variables. + +<p><CODE>options(7)</CODE> defines the trajectory length (i.e. the number of leap-frog +steps at each iteration). Minimum value 1. + +<p><CODE>options(9)</CODE> is set to 1 to check the user defined gradient function. + +<p><CODE>options(14)</CODE> is the number of samples retained from the Markov chain; +default 100. + +<p><CODE>options(15)</CODE> is the number of samples omitted from the start of the +chain; default 0. + +<p><CODE>options(17)</CODE> defines the momentum used when a persistent update of +(leap-frog) momentum is used. This is bounded to the interval [0, 1). + +<p><CODE>options(18)</CODE> is the step size used in leap-frogs; default 1/trajectory +length. + +<p><h2> +Examples +</h2> +The following code fragment samples from the posterior distribution of +weights for a neural network. +<PRE> + +w = mlppak(net); +[samples, energies] = hmc('neterr', w, options, 'netgrad', net, x, t); +</PRE> + + +<p><h2> +Algorithm +</h2> + +The algroithm follows the procedure outlined in Radford Neal's technical +report CRG-TR-93-1 from the University of Toronto. The stochastic update of +momenta samples from a zero mean unit covariance gaussian. + +<p><h2> +See Also +</h2> +<CODE><a href="metrop.htm">metrop</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/index.htm b/sourcecodes/bnt-master/nethelp3.3/index.htm new file mode 100644 index 00000000..816e24eb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/index.htm @@ -0,0 +1,537 @@ +<html> +<head> +<title> +NETLAB Reference Documentation +</title> +</head> +<body> +<H1> NETLAB Online Reference Documentation </H1> +Welcome to the NETLAB online reference documentation. +The NETLAB simulation software is designed to provide all the tools necessary +for principled and theoretically well founded application development. The +NETLAB library is based on the approach and techniques described in <I>Neural +Networks for Pattern Recognition </I>(Bishop, 1995). The library includes software +implementations of a wide range of data analysis techniques, many of which are +not widely available, and are rarely, if ever, included in standard neural +network simulation packages. +<p>The online reference documentation provides direct hypertext links to specific Netlab function descriptions. +<p>If you have any comments or problems to report, please contact Ian Nabney (<a href="mailto:i.t.nabney@aston.ac.uk"><tt>i.t.nabney@aston.ac.uk</tt></a>) or Christopher Bishop (<a href="mailto:c.m.bishop@aston.ac.uk"><tt>c.m.bishop@aston.ac.uk</tt></a>).<H1> Index +</H1> +An alphabetic list of functions in Netlab.<p> +<DL> +<DT> +<CODE><a href="conffig.htm">conffig</a></CODE><DD> + Display a confusion matrix. +<DT> +<CODE><a href="confmat.htm">confmat</a></CODE><DD> + Compute a confusion matrix. +<DT> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE><DD> + Conjugate gradients optimization. +<DT> +<CODE><a href="consist.htm">consist</a></CODE><DD> + Check that arguments are consistent. +<DT> +<CODE><a href="convertoldnet.htm">convertoldnet</a></CODE><DD> + Convert pre-2.3 release MLP and MDN nets to new format +<DT> +<CODE><a href="datread.htm">datread</a></CODE><DD> + Read data from an ascii file. +<DT> +<CODE><a href="datwrite.htm">datwrite</a></CODE><DD> + Write data to ascii file. +<DT> +<CODE><a href="dem2ddat.htm">dem2ddat</a></CODE><DD> + Generates two dimensional data for demos. +<DT> +<CODE><a href="demard.htm">demard</a></CODE><DD> + Automatic relevance determination using the MLP. +<DT> +<CODE><a href="demev1.htm">demev1</a></CODE><DD> + Demonstrate Bayesian regression for the MLP. +<DT> +<CODE><a href="demev2.htm">demev2</a></CODE><DD> + Demonstrate Bayesian classification for the MLP. +<DT> +<CODE><a href="demev3.htm">demev3</a></CODE><DD> + Demonstrate Bayesian regression for the RBF. +<DT> +<CODE><a href="demgauss.htm">demgauss</a></CODE><DD> + Demonstrate sampling from Gaussian distributions. +<DT> +<CODE><a href="demglm1.htm">demglm1</a></CODE><DD> + Demonstrate simple classification using a generalized linear model. +<DT> +<CODE><a href="demglm2.htm">demglm2</a></CODE><DD> + Demonstrate simple classification using a generalized linear model. +<DT> +<CODE><a href="demgmm1.htm">demgmm1</a></CODE><DD> + Demonstrate density modelling with a Gaussian mixture model. +<DT> +<CODE><a href="demgmm3.htm">demgmm3</a></CODE><DD> + Demonstrate density modelling with a Gaussian mixture model. +<DT> +<CODE><a href="demgmm4.htm">demgmm4</a></CODE><DD> + Demonstrate density modelling with a Gaussian mixture model. +<DT> +<CODE><a href="demgmm5.htm">demgmm5</a></CODE><DD> + Demonstrate density modelling with a PPCA mixture model. +<DT> +<CODE><a href="demgp.htm">demgp</a></CODE><DD> + Demonstrate simple regression using a Gaussian Process. +<DT> +<CODE><a href="demgpard.htm">demgpard</a></CODE><DD> + Demonstrate ARD using a Gaussian Process. +<DT> +<CODE><a href="demgpot.htm">demgpot</a></CODE><DD> + Computes the gradient of the negative log likelihood for a mixture model. +<DT> +<CODE><a href="demgtm1.htm">demgtm1</a></CODE><DD> + Demonstrate EM for GTM. +<DT> +<CODE><a href="demgtm2.htm">demgtm2</a></CODE><DD> + Demonstrate GTM for visualisation. +<DT> +<CODE><a href="demhint.htm">demhint</a></CODE><DD> + Demonstration of Hinton diagram for 2-layer feed-forward network. +<DT> +<CODE><a href="demhmc1.htm">demhmc1</a></CODE><DD> + Demonstrate Hybrid Monte Carlo sampling on mixture of two Gaussians. +<DT> +<CODE><a href="demhmc2.htm">demhmc2</a></CODE><DD> + Demonstrate Bayesian regression with Hybrid Monte Carlo sampling. +<DT> +<CODE><a href="demhmc3.htm">demhmc3</a></CODE><DD> + Demonstrate Bayesian regression with Hybrid Monte Carlo sampling. +<DT> +<CODE><a href="demkmean.htm">demkmean</a></CODE><DD> + Demonstrate simple clustering model trained with K-means. +<DT> +<CODE><a href="demknn1.htm">demknn1</a></CODE><DD> + Demonstrate nearest neighbour classifier. +<DT> +<CODE><a href="demmdn1.htm">demmdn1</a></CODE><DD> + Demonstrate fitting a multi-valued function using a Mixture Density Network. +<DT> +<CODE><a href="demmet1.htm">demmet1</a></CODE><DD> + Demonstrate Markov Chain Monte Carlo sampling on a Gaussian. +<DT> +<CODE><a href="demmlp1.htm">demmlp1</a></CODE><DD> + Demonstrate simple regression using a multi-layer perceptron +<DT> +<CODE><a href="demmlp2.htm">demmlp2</a></CODE><DD> + Demonstrate simple classification using a multi-layer perceptron +<DT> +<CODE><a href="demnlab.htm">demnlab</a></CODE><DD> + A front-end Graphical User Interface to the demos +<DT> +<CODE><a href="demns1.htm">demns1</a></CODE><DD> + Demonstrate Neuroscale for visualisation. +<DT> +<CODE><a href="demolgd1.htm">demolgd1</a></CODE><DD> + Demonstrate simple MLP optimisation with on-line gradient descent +<DT> +<CODE><a href="demopt1.htm">demopt1</a></CODE><DD> + Demonstrate different optimisers on Rosenbrock's function. +<DT> +<CODE><a href="dempot.htm">dempot</a></CODE><DD> + Computes the negative log likelihood for a mixture model. +<DT> +<CODE><a href="demprgp.htm">demprgp</a></CODE><DD> + Demonstrate sampling from a Gaussian Process prior. +<DT> +<CODE><a href="demprior.htm">demprior</a></CODE><DD> + Demonstrate sampling from a multi-parameter Gaussian prior. +<DT> +<CODE><a href="demrbf1.htm">demrbf1</a></CODE><DD> + Demonstrate simple regression using a radial basis function network. +<DT> +<CODE><a href="demsom1.htm">demsom1</a></CODE><DD> + Demonstrate SOM for visualisation. +<DT> +<CODE><a href="demtrain.htm">demtrain</a></CODE><DD> + Demonstrate training of MLP network. +<DT> +<CODE><a href="dist2.htm">dist2</a></CODE><DD> + Calculates squared distance between two sets of points. +<DT> +<CODE><a href="eigdec.htm">eigdec</a></CODE><DD> + Sorted eigendecomposition +<DT> +<CODE><a href="errbayes.htm">errbayes</a></CODE><DD> + Evaluate Bayesian error function for network. +<DT> +<CODE><a href="evidence.htm">evidence</a></CODE><DD> + Re-estimate hyperparameters using evidence approximation. +<DT> +<CODE><a href="fevbayes.htm">fevbayes</a></CODE><DD> + Evaluate Bayesian regularisation for network forward propagation. +<DT> +<CODE><a href="gauss.htm">gauss</a></CODE><DD> + Evaluate a Gaussian distribution. +<DT> +<CODE><a href="gbayes.htm">gbayes</a></CODE><DD> + Evaluate gradient of Bayesian error function for network. +<DT> +<CODE><a href="glm.htm">glm</a></CODE><DD> + Create a generalized linear model. +<DT> +<CODE><a href="glmderiv.htm">glmderiv</a></CODE><DD> + Evaluate derivatives of GLM outputs with respect to weights. +<DT> +<CODE><a href="glmerr.htm">glmerr</a></CODE><DD> + Evaluate error function for generalized linear model. +<DT> +<CODE><a href="glmevfwd.htm">glmevfwd</a></CODE><DD> + Forward propagation with evidence for GLM +<DT> +<CODE><a href="glmfwd.htm">glmfwd</a></CODE><DD> + Forward propagation through generalized linear model. +<DT> +<CODE><a href="glmgrad.htm">glmgrad</a></CODE><DD> + Evaluate gradient of error function for generalized linear model. +<DT> +<CODE><a href="glmhess.htm">glmhess</a></CODE><DD> + Evaluate the Hessian matrix for a generalised linear model. +<DT> +<CODE><a href="glminit.htm">glminit</a></CODE><DD> + Initialise the weights in a generalized linear model. +<DT> +<CODE><a href="glmpak.htm">glmpak</a></CODE><DD> + Combines weights and biases into one weights vector. +<DT> +<CODE><a href="glmtrain.htm">glmtrain</a></CODE><DD> + Specialised training of generalized linear model +<DT> +<CODE><a href="glmunpak.htm">glmunpak</a></CODE><DD> + Separates weights vector into weight and bias matrices. +<DT> +<CODE><a href="gmm.htm">gmm</a></CODE><DD> + Creates a Gaussian mixture model with specified architecture. +<DT> +<CODE><a href="gmmactiv.htm">gmmactiv</a></CODE><DD> + Computes the activations of a Gaussian mixture model. +<DT> +<CODE><a href="gmmem.htm">gmmem</a></CODE><DD> + EM algorithm for Gaussian mixture model. +<DT> +<CODE><a href="gmminit.htm">gmminit</a></CODE><DD> + Initialises Gaussian mixture model from data +<DT> +<CODE><a href="gmmpak.htm">gmmpak</a></CODE><DD> + Combines all the parameters in a Gaussian mixture model into one vector. +<DT> +<CODE><a href="gmmpost.htm">gmmpost</a></CODE><DD> + Computes the class posterior probabilities of a Gaussian mixture model. +<DT> +<CODE><a href="gmmprob.htm">gmmprob</a></CODE><DD> + Computes the data probability for a Gaussian mixture model. +<DT> +<CODE><a href="gmmsamp.htm">gmmsamp</a></CODE><DD> + Sample from a Gaussian mixture distribution. +<DT> +<CODE><a href="gmmunpak.htm">gmmunpak</a></CODE><DD> + Separates a vector of Gaussian mixture model parameters into its components. +<DT> +<CODE><a href="gp.htm">gp</a></CODE><DD> + Create a Gaussian Process. +<DT> +<CODE><a href="gpcovar.htm">gpcovar</a></CODE><DD> + Calculate the covariance for a Gaussian Process. +<DT> +<CODE><a href="gpcovarf.htm">gpcovarf</a></CODE><DD> + Calculate the covariance function for a Gaussian Process. +<DT> +<CODE><a href="gpcovarp.htm">gpcovarp</a></CODE><DD> + Calculate the prior covariance for a Gaussian Process. +<DT> +<CODE><a href="gperr.htm">gperr</a></CODE><DD> + Evaluate error function for Gaussian Process. +<DT> +<CODE><a href="gpfwd.htm">gpfwd</a></CODE><DD> + Forward propagation through Gaussian Process. +<DT> +<CODE><a href="gpgrad.htm">gpgrad</a></CODE><DD> + Evaluate error gradient for Gaussian Process. +<DT> +<CODE><a href="gpinit.htm">gpinit</a></CODE><DD> + Initialise Gaussian Process model. +<DT> +<CODE><a href="gppak.htm">gppak</a></CODE><DD> + Combines GP hyperparameters into one vector. +<DT> +<CODE><a href="gpunpak.htm">gpunpak</a></CODE><DD> + Separates hyperparameter vector into components. +<DT> +<CODE><a href="gradchek.htm">gradchek</a></CODE><DD> + Checks a user-defined gradient function using finite differences. +<DT> +<CODE><a href="graddesc.htm">graddesc</a></CODE><DD> + Gradient descent optimization. +<DT> +<CODE><a href="gsamp.htm">gsamp</a></CODE><DD> + Sample from a Gaussian distribution. +<DT> +<CODE><a href="gtm.htm">gtm</a></CODE><DD> + Create a Generative Topographic Map. +<DT> +<CODE><a href="gtmem.htm">gtmem</a></CODE><DD> + EM algorithm for Generative Topographic Mapping. +<DT> +<CODE><a href="gtmfwd.htm">gtmfwd</a></CODE><DD> + Forward propagation through GTM. +<DT> +<CODE><a href="gtminit.htm">gtminit</a></CODE><DD> + Initialise the weights and latent sample in a GTM. +<DT> +<CODE><a href="gtmlmean.htm">gtmlmean</a></CODE><DD> + Mean responsibility for data in a GTM. +<DT> +<CODE><a href="gtmlmode.htm">gtmlmode</a></CODE><DD> + Mode responsibility for data in a GTM. +<DT> +<CODE><a href="gtmmag.htm">gtmmag</a></CODE><DD> + Magnification factors for a GTM +<DT> +<CODE><a href="gtmpost.htm">gtmpost</a></CODE><DD> + Latent space responsibility for data in a GTM. +<DT> +<CODE><a href="gtmprob.htm">gtmprob</a></CODE><DD> + Probability for data under a GTM. +<DT> +<CODE><a href="hbayes.htm">hbayes</a></CODE><DD> + Evaluate Hessian of Bayesian error function for network. +<DT> +<CODE><a href="hesschek.htm">hesschek</a></CODE><DD> + Use central differences to confirm correct evaluation of Hessian matrix. +<DT> +<CODE><a href="hintmat.htm">hintmat</a></CODE><DD> + Evaluates the coordinates of the patches for a Hinton diagram. +<DT> +<CODE><a href="hinton.htm">hinton</a></CODE><DD> + Plot Hinton diagram for a weight matrix. +<DT> +<CODE><a href="histp.htm">histp</a></CODE><DD> + Histogram estimate of 1-dimensional probability distribution. +<DT> +<CODE><a href="hmc.htm">hmc</a></CODE><DD> + Hybrid Monte Carlo sampling. +<DT> +<CODE><a href="kmeans.htm">kmeans</a></CODE><DD> + Trains a k means cluster model. +<DT> +<CODE><a href="knn.htm">knn</a></CODE><DD> + Creates a K-nearest-neighbour classifier. +<DT> +<CODE><a href="knnfwd.htm">knnfwd</a></CODE><DD> + Forward propagation through a K-nearest-neighbour classifier. +<DT> +<CODE><a href="linef.htm">linef</a></CODE><DD> + Calculate function value along a line. +<DT> +<CODE><a href="linemin.htm">linemin</a></CODE><DD> + One dimensional minimization. +<DT> +<CODE><a href="maxitmess.htm">maxitmess</a></CODE><DD> + Create a standard error message when training reaches max. iterations. +<DT> +<CODE><a href="mdn.htm">mdn</a></CODE><DD> + Creates a Mixture Density Network with specified architecture. +<DT> +<CODE><a href="mdn2gmm.htm">mdn2gmm</a></CODE><DD> + Converts an MDN mixture data structure to array of GMMs. +<DT> +<CODE><a href="mdndist2.htm">mdndist2</a></CODE><DD> + Calculates squared distance between centres of Gaussian kernels and data +<DT> +<CODE><a href="mdnerr.htm">mdnerr</a></CODE><DD> + Evaluate error function for Mixture Density Network. +<DT> +<CODE><a href="mdnfwd.htm">mdnfwd</a></CODE><DD> + Forward propagation through Mixture Density Network. +<DT> +<CODE><a href="mdngrad.htm">mdngrad</a></CODE><DD> + Evaluate gradient of error function for Mixture Density Network. +<DT> +<CODE><a href="mdninit.htm">mdninit</a></CODE><DD> + Initialise the weights in a Mixture Density Network. +<DT> +<CODE><a href="mdnpak.htm">mdnpak</a></CODE><DD> + Combines weights and biases into one weights vector. +<DT> +<CODE><a href="mdnpost.htm">mdnpost</a></CODE><DD> + Computes the posterior probability for each MDN mixture component. +<DT> +<CODE><a href="mdnprob.htm">mdnprob</a></CODE><DD> + Computes the data probability likelihood for an MDN mixture structure. +<DT> +<CODE><a href="mdnunpak.htm">mdnunpak</a></CODE><DD> + Separates weights vector into weight and bias matrices. +<DT> +<CODE><a href="metrop.htm">metrop</a></CODE><DD> + Markov Chain Monte Carlo sampling with Metropolis algorithm. +<DT> +<CODE><a href="minbrack.htm">minbrack</a></CODE><DD> + Bracket a minimum of a function of one variable. +<DT> +<CODE><a href="mlp.htm">mlp</a></CODE><DD> + Create a 2-layer feedforward network. +<DT> +<CODE><a href="mlpbkp.htm">mlpbkp</a></CODE><DD> + Backpropagate gradient of error function for 2-layer network. +<DT> +<CODE><a href="mlpderiv.htm">mlpderiv</a></CODE><DD> + Evaluate derivatives of network outputs with respect to weights. +<DT> +<CODE><a href="mlperr.htm">mlperr</a></CODE><DD> + Evaluate error function for 2-layer network. +<DT> +<CODE><a href="mlpevfwd.htm">mlpevfwd</a></CODE><DD> + Forward propagation with evidence for MLP +<DT> +<CODE><a href="mlpfwd.htm">mlpfwd</a></CODE><DD> + Forward propagation through 2-layer network. +<DT> +<CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><DD> + Evaluate gradient of error function for 2-layer network. +<DT> +<CODE><a href="mlphdotv.htm">mlphdotv</a></CODE><DD> + Evaluate the product of the data Hessian with a vector. +<DT> +<CODE><a href="mlphess.htm">mlphess</a></CODE><DD> + Evaluate the Hessian matrix for a multi-layer perceptron network. +<DT> +<CODE><a href="mlphint.htm">mlphint</a></CODE><DD> + Plot Hinton diagram for 2-layer feed-forward network. +<DT> +<CODE><a href="mlpinit.htm">mlpinit</a></CODE><DD> + Initialise the weights in a 2-layer feedforward network. +<DT> +<CODE><a href="mlppak.htm">mlppak</a></CODE><DD> + Combines weights and biases into one weights vector. +<DT> +<CODE><a href="mlpprior.htm">mlpprior</a></CODE><DD> + Create Gaussian prior for mlp. +<DT> +<CODE><a href="mlptrain.htm">mlptrain</a></CODE><DD> + Utility to train an MLP network for demtrain +<DT> +<CODE><a href="mlpunpak.htm">mlpunpak</a></CODE><DD> + Separates weights vector into weight and bias matrices. +<DT> +<CODE><a href="netderiv.htm">netderiv</a></CODE><DD> + Evaluate derivatives of network outputs by weights generically. +<DT> +<CODE><a href="neterr.htm">neterr</a></CODE><DD> + Evaluate network error function for generic optimizers +<DT> +<CODE><a href="netevfwd.htm">netevfwd</a></CODE><DD> + Generic forward propagation with evidence for network +<DT> +<CODE><a href="netgrad.htm">netgrad</a></CODE><DD> + Evaluate network error gradient for generic optimizers +<DT> +<CODE><a href="nethess.htm">nethess</a></CODE><DD> + Evaluate network Hessian +<DT> +<CODE><a href="netinit.htm">netinit</a></CODE><DD> + Initialise the weights in a network. +<DT> +<CODE><a href="netopt.htm">netopt</a></CODE><DD> + Optimize the weights in a network model. +<DT> +<CODE><a href="netpak.htm">netpak</a></CODE><DD> + Combines weights and biases into one weights vector. +<DT> +<CODE><a href="netunpak.htm">netunpak</a></CODE><DD> + Separates weights vector into weight and bias matrices. +<DT> +<CODE><a href="olgd.htm">olgd</a></CODE><DD> + On-line gradient descent optimization. +<DT> +<CODE><a href="pca.htm">pca</a></CODE><DD> + Principal Components Analysis +<DT> +<CODE><a href="plotmat.htm">plotmat</a></CODE><DD> + Display a matrix. +<DT> +<CODE><a href="ppca.htm">ppca</a></CODE><DD> + Probabilistic Principal Components Analysis +<DT> +<CODE><a href="quasinew.htm">quasinew</a></CODE><DD> + Quasi-Newton optimization. +<DT> +<CODE><a href="rbf.htm">rbf</a></CODE><DD> + Creates an RBF network with specified architecture +<DT> +<CODE><a href="rbfbkp.htm">rbfbkp</a></CODE><DD> + Backpropagate gradient of error function for RBF network. +<DT> +<CODE><a href="rbfderiv.htm">rbfderiv</a></CODE><DD> + Evaluate derivatives of RBF network outputs with respect to weights. +<DT> +<CODE><a href="rbferr.htm">rbferr</a></CODE><DD> + Evaluate error function for RBF network. +<DT> +<CODE><a href="rbfevfwd.htm">rbfevfwd</a></CODE><DD> + Forward propagation with evidence for RBF +<DT> +<CODE><a href="rbffwd.htm">rbffwd</a></CODE><DD> + Forward propagation through RBF network with linear outputs. +<DT> +<CODE><a href="rbfgrad.htm">rbfgrad</a></CODE><DD> + Evaluate gradient of error function for RBF network. +<DT> +<CODE><a href="rbfhess.htm">rbfhess</a></CODE><DD> + Evaluate the Hessian matrix for RBF network. +<DT> +<CODE><a href="rbfjacob.htm">rbfjacob</a></CODE><DD> + Evaluate derivatives of RBF network outputs with respect to inputs. +<DT> +<CODE><a href="rbfpak.htm">rbfpak</a></CODE><DD> + Combines all the parameters in an RBF network into one weights vector. +<DT> +<CODE><a href="rbfprior.htm">rbfprior</a></CODE><DD> + Create Gaussian prior and output layer mask for RBF. +<DT> +<CODE><a href="rbfsetbf.htm">rbfsetbf</a></CODE><DD> + Set basis functions of RBF from data. +<DT> +<CODE><a href="rbfsetfw.htm">rbfsetfw</a></CODE><DD> + Set basis function widths of RBF. +<DT> +<CODE><a href="rbftrain.htm">rbftrain</a></CODE><DD> + Two stage training of RBF network. +<DT> +<CODE><a href="rbfunpak.htm">rbfunpak</a></CODE><DD> + Separates a vector of RBF weights into its components. +<DT> +<CODE><a href="rosegrad.htm">rosegrad</a></CODE><DD> + Calculate gradient of Rosenbrock's function. +<DT> +<CODE><a href="rosen.htm">rosen</a></CODE><DD> + Calculate Rosenbrock's function. +<DT> +<CODE><a href="scg.htm">scg</a></CODE><DD> + Scaled conjugate gradient optimization. +<DT> +<CODE><a href="som.htm">som</a></CODE><DD> + Creates a Self-Organising Map. +<DT> +<CODE><a href="somfwd.htm">somfwd</a></CODE><DD> + Forward propagation through a Self-Organising Map. +<DT> +<CODE><a href="sompak.htm">sompak</a></CODE><DD> + Combines node weights into one weights matrix. +<DT> +<CODE><a href="somtrain.htm">somtrain</a></CODE><DD> + Kohonen training algorithm for SOM. +<DT> +<CODE><a href="somunpak.htm">somunpak</a></CODE><DD> + Replaces node weights in SOM. +</DL> + +<hr> +<p>Copyright (c) Christopher M Bishop, Ian T Nabney (1996, 1997) +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/kmeans.htm b/sourcecodes/bnt-master/nethelp3.3/kmeans.htm new file mode 100644 index 00000000..092e51d3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/kmeans.htm @@ -0,0 +1,89 @@ +<html> +<head> +<title> +Netlab Reference Manual kmeans +</title> +</head> +<body> +<H1> kmeans +</H1> +<h2> +Purpose +</h2> +Trains a k means cluster model. + +<p><h2> +Synopsis +</h2> +<PRE> +centres = kmeans(centres, data, options) +[centres, options] = kmeans(centres, data, options) +[centres, options, post, errlog] = kmeans(centres, data, options) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>centres = kmeans(centres, data, options)</CODE> +uses the batch K-means algorithm to set the centres of a cluster model. +The matrix <CODE>data</CODE> represents the data +which is being clustered, with each row corresponding to a vector. +The sum of squares error function is used. The point at which +a local minimum is achieved is returned as <CODE>centres</CODE>. The +error value at that point is returned in <CODE>options(8)</CODE>. + +<p><CODE>[centres, options, post, errlog] = kmeans(centres, data, options)</CODE> +also returns the cluster number (in a one-of-N encoding) for each data +point in <CODE>post</CODE> and a log of the error values after each cycle in +<CODE>errlog</CODE>. + +The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>. +If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value +of <CODE>centres</CODE> at the solution. If the absolute difference between +the values of <CODE>centres</CODE> between two successive steps is less than +<CODE>options(2)</CODE>, then this condition is satisfied. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the error +function at the solution. If the absolute difference between the +error functions between two successive steps is less than +<CODE>options(3)</CODE>, then this condition is satisfied. +Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p><h2> +Example +</h2> +<CODE>kmeans</CODE> can be used to initialise the centres of a Gaussian +mixture model that is then trained with the EM algorithm. +<PRE> + +[priors, centres, var] = gmmunpak(p, md); +centres = kmeans(centres, data, options); +p = gmmpak(priors, centres, var); +p = gmmem(p, md, data, options); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="gmminit.htm">gmminit</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/knn.htm b/sourcecodes/bnt-master/nethelp3.3/knn.htm new file mode 100644 index 00000000..31f032bc --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/knn.htm @@ -0,0 +1,54 @@ +<html> +<head> +<title> +Netlab Reference Manual knn +</title> +</head> +<body> +<H1> knn +</H1> +<h2> +Purpose +</h2> +Creates a K-nearest-neighbour classifier. + +<p><h2> +Synopsis +</h2> +<PRE> + +net = knn(nin, nout, k, tr_in, tr_targets) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = knn(nin, nout, k, tr_in, tr_targets)</CODE> creates a KNN model <CODE>net</CODE> +with input dimension <CODE>nin</CODE>, output dimension <CODE>nout</CODE> and <CODE>k</CODE> +neighbours. The training data is also stored in the data structure and the +targets are assumed to be using a 1-of-N coding. + +<p>The fields in <CODE>net</CODE> are +<PRE> + + type = 'knn' + nin = number of inputs + nout = number of outputs + tr_in = training input data + tr_targets = training target data +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="kmeans.htm">kmeans</a></CODE>, <CODE><a href="knnfwd.htm">knnfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/knnfwd.htm b/sourcecodes/bnt-master/nethelp3.3/knnfwd.htm new file mode 100644 index 00000000..6670920f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/knnfwd.htm @@ -0,0 +1,66 @@ +<html> +<head> +<title> +Netlab Reference Manual knnfwd +</title> +</head> +<body> +<H1> knnfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through a K-nearest-neighbour classifier. + +<p><h2> +Synopsis +</h2> +<PRE> + +[y, l] = knnfwd(net, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>[y, l] = knnfwd(net, x)</CODE> takes a matrix <CODE>x</CODE> +of input vectors (one vector per row) + and uses the <CODE>k</CODE>-nearest-neighbour rule on the training data contained +in <CODE>net</CODE> to +produce +a matrix <CODE>y</CODE> of outputs and a matrix <CODE>l</CODE> of classification +labels. +The nearest neighbours are determined using Euclidean distance. +The <CODE>ij</CODE>th entry of <CODE>y</CODE> counts the number of occurrences that +an example from class <CODE>j</CODE> is among the <CODE>k</CODE> closest training +examples to example <CODE>i</CODE> from <CODE>x</CODE>. +The matrix <CODE>l</CODE> contains the predicted class labels +as an index 1..N, not as 1-of-N coding. + +<p><h2> +Example +</h2> +<PRE> + +net = knn(size(xtrain, 2), size(t_train, 2), 3, xtrain, t_train); +y = knnfwd(net, xtest); +conffig(y, t_test); +</PRE> + +Creates a 3 nearest neighbour model <CODE>net</CODE> and then applies it to +the data <CODE>xtest</CODE>. The results are plotted as a confusion matrix with +<CODE>conffig</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="kmeans.htm">kmeans</a></CODE>, <CODE><a href="knn.htm">knn</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/linef.htm b/sourcecodes/bnt-master/nethelp3.3/linef.htm new file mode 100644 index 00000000..c6eb9188 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/linef.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual linef +</title> +</head> +<body> +<H1> linef +</H1> +<h2> +Purpose +</h2> +Calculate function value along a line. + +<p><h2> +Description +</h2> +<CODE>linef(lambda, fn, x, d)</CODE> calculates the value of the function +<CODE>fn</CODE> at the point <CODE>x+lambda*d</CODE>. Here <CODE>x</CODE> is a row vector +and <CODE>lambda</CODE> is a scalar. + +<p><CODE>linef(lambda, fn, x, d, p1, p2, ...)</CODE> allows additional +arguments to be passed to <CODE>fn()</CODE>. +This function is used for convenience in some of the optimisation routines. + +<p><h2> +Examples +</h2> +An example of +the use of this function can be found in the function <CODE>linemin</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gradchek.htm">gradchek</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/linemin.htm b/sourcecodes/bnt-master/nethelp3.3/linemin.htm new file mode 100644 index 00000000..e4d7a8b9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/linemin.htm @@ -0,0 +1,74 @@ +<html> +<head> +<title> +Netlab Reference Manual linemin +</title> +</head> +<body> +<H1> linemin +</H1> +<h2> +Purpose +</h2> +One dimensional minimization. + +<p><h2> +Description +</h2> +<CODE>[x, options] = linemin(f, pt, dir, fpt, options)</CODE> uses Brent's +algorithm to find the minimum of the function <CODE>f(x)</CODE> along the +line <CODE>dir</CODE> through the point <CODE>pt</CODE>. The function value at the +starting point is <CODE>fpt</CODE>. The point at which <CODE>f</CODE> has a local minimum +is returned as <CODE>x</CODE>. The function value at that point is returned +in <CODE>options(8)</CODE>. + +<p><CODE>linemin(f, pt, dir, fpt, options, p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE>. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values. + +<p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value +of <CODE>x</CODE> at the solution. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p><h2> +Examples +</h2> +An example of the use of this function to find the minimum of a function +<CODE>f</CODE> in the direction <CODE>sd</CODE> can be found in <CODE>conjgrad</CODE> +<PRE> + +x = linemin(f, xold, sd, fold, lineoptions); +</PRE> + + +<p><h2> +Algorithm +</h2> + +Brent's algorithm uses a mixture of quadratic interpolation and golden +section search to find the minimum of a function of a single variable once +it has been bracketed (which is done with <CODE>minbrack</CODE>). This is adapted +to minimize a function along a line. +This implementation +is based on that in Numerical Recipes. + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/maxitmess.htm b/sourcecodes/bnt-master/nethelp3.3/maxitmess.htm new file mode 100644 index 00000000..a42b1f8b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/maxitmess.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual maxitmess +</title> +</head> +<body> +<H1> maxitmess +</H1> +<h2> +Purpose +</h2> +Create a standard error message when training reaches max. iterations. + +<p><h2> +Synopsis +</h2> +<PRE> +s = maxitmess +</PRE> + + +<p><h2> +Description +</h2> +<CODE>s = maxitmess</CODE> returns a standard string that it used by training +algorithms when the maximum number of iterations (as specified in +<CODE>options(14)</CODE> is reached. + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="gtmem.htm">gtmem</a></CODE>, <CODE><a href="kmeans.htm">kmeans</a></CODE>, <CODE><a href="olgd.htm">olgd</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdn.htm b/sourcecodes/bnt-master/nethelp3.3/mdn.htm new file mode 100644 index 00000000..5e5b83a8 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdn.htm @@ -0,0 +1,84 @@ +<html> +<head> +<title> +Netlab Reference Manual mdn +</title> +</head> +<body> +<H1> mdn +</H1> +<h2> +Purpose +</h2> +Creates a Mixture Density Network with specified architecture. + +<p><h2> +Synopsis +</h2> +<PRE> +net = mdn(nin, nhidden, ncentres, dimtarget) +net = mdn(nin, nhidden, ncentres, dimtarget, mixtype, ... + prior, beta) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = mdn(nin, nhidden, ncentres, dimtarget)</CODE> takes the number of +inputs, +hidden units for a 2-layer feed-forward +network and the number of centres and target dimension for the +mixture model whose parameters are set from the outputs of the neural network. +The fifth argument <CODE>mixtype</CODE> is used to define the type of mixture +model. (Currently there is only one type supported: a mixture of Gaussians with +a single covariance parameter for each component.) For this model, +the mixture coefficients are computed from a group of softmax outputs, +the centres are equal to a group of linear outputs, and the variances are +obtained by applying the exponential function to a third group of outputs. + +<p>The network is initialised by a call to <CODE>mlp</CODE>, and the arguments +<CODE>prior</CODE>, and <CODE>beta</CODE> have the same role as for that function. +Weight initialisation uses the Matlab function <CODE>randn</CODE> + and so the seed for the random weight initialization can be +set using <CODE>randn('state', s)</CODE> where <CODE>s</CODE> is the seed value. +A specialised data structure (rather than <CODE>gmm</CODE>) +is used for the mixture model outputs to improve +the efficiency of error and gradient calculations in network training. +The fields are described in <CODE>mdnfwd</CODE> where they are set up. + +<p>The fields in <CODE>net</CODE> are +<PRE> + + type = 'mdn' + nin = number of input variables + nout = dimension of target space (not number of network outputs) + nwts = total number of weights and biases + mdnmixes = data structure for mixture model output + mlp = data structure for MLP network +</PRE> + + +<p><h2> +Example +</h2> +<PRE> + +net = mdn(2, 4, 3, 1, 'spherical'); +</PRE> + +This creates a Mixture Density Network with 2 inputs and 4 hidden units. +The mixture model has 3 components and the target space has dimension 1. + +<p><h2> +See Also +</h2> +<CODE><a href="mdnfwd.htm">mdnfwd</a></CODE>, <CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdn2gmm.htm">mdn2gmm</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE>, <CODE><a href="mdnpak.htm">mdnpak</a></CODE>, <CODE><a href="mdnunpak.htm">mdnunpak</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdn2gmm.htm b/sourcecodes/bnt-master/nethelp3.3/mdn2gmm.htm new file mode 100644 index 00000000..51369a50 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdn2gmm.htm @@ -0,0 +1,58 @@ +<html> +<head> +<title> +Netlab Reference Manual mdn2gmm +</title> +</head> +<body> +<H1> mdn2gmm +</H1> +<h2> +Purpose +</h2> +Converts an MDN mixture data structure to array of GMMs. + +<p><h2> +Synopsis +</h2> +<PRE> +gmmmixes = mdn2gmm(mdnmixes) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>gmmmixes = mdn2gmm(mdnmixes)</CODE> takes an MDN mixture data structure +<CODE>mdnmixes</CODE> +containing three matrices (for priors, centres and variances) where each +row represents the corresponding parameter values for a different mixture model +and creates an array of GMMs. These can then be used with the standard +Netlab Gaussian mixture model functions. + +<p><h2> +Example +</h2> +<PRE> + +mdnmixes = mdnfwd(net, x); +mixes = mdn2gmm(mdnmixes); +p = gmmprob(mixes(1), y); +</PRE> + +This creates an array GMM mixture models (one for each data point in +<CODE>x</CODE>). The vector <CODE>p</CODE> is then filled with the conditional +probabilities of the values <CODE>y</CODE> given <CODE>x(1,:)</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="gmm.htm">gmm</a></CODE>, <CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdnfwd.htm">mdnfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdndist2.htm b/sourcecodes/bnt-master/nethelp3.3/mdndist2.htm new file mode 100644 index 00000000..76713ae4 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdndist2.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual mdndist2 +</title> +</head> +<body> +<H1> mdndist2 +</H1> +<h2> +Purpose +</h2> +Calculates squared distance between centres of Gaussian kernels and data + +<p><h2> +Synopsis +</h2> +<PRE> +n2 = mdndist2(mixparams, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>n2 = mdndist2(mixparams, t)</CODE> takes takes the centres of the Gaussian +contained in + <CODE>mixparams</CODE> and the target data matrix, <CODE>t</CODE>, and computes the squared +Euclidean distance between them. If <CODE>t</CODE> has <CODE>m</CODE> rows and <CODE>n</CODE> +columns, then the <CODE>centres</CODE> field in +the <CODE>mixparams</CODE> structure should have <CODE>m</CODE> rows and +<CODE>n*mixparams.ncentres</CODE> columns: the centres in each row relate to +the corresponding row in <CODE>t</CODE>. +The result has <CODE>m</CODE> rows and <CODE>mixparams.ncentres</CODE> columns. +The <CODE>i, j</CODE>th entry is the +squared distance from the <CODE>i</CODE>th row of <CODE>x</CODE> to the <CODE>j</CODE>th +centre in the <CODE>i</CODE>th row of <CODE>mixparams.centres</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mdnfwd.htm">mdnfwd</a></CODE>, <CODE><a href="mdnprob.htm">mdnprob</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnerr.htm b/sourcecodes/bnt-master/nethelp3.3/mdnerr.htm new file mode 100644 index 00000000..41d2f260 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnerr.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual mdnerr +</title> +</head> +<body> +<H1> mdnerr +</H1> +<h2> +Purpose +</h2> +Evaluate error function for Mixture Density Network. + +<p><h2> +Synopsis +</h2> +<PRE> +e = mdnerr(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>e = mdnerr(net, x, t)</CODE> takes a mixture density network data +structure <CODE>net</CODE>, a matrix <CODE>x</CODE> of input vectors and a matrix +<CODE>t</CODE> of target vectors, and evaluates the error function +<CODE>e</CODE>. The error function is the negative log likelihood of the +target data under the conditional density given by the mixture model +parameterised by the MLP. Each row of <CODE>x</CODE> corresponds to one +input vector and each row of <CODE>t</CODE> corresponds to one target vector. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdnfwd.htm">mdnfwd</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnfwd.htm b/sourcecodes/bnt-master/nethelp3.3/mdnfwd.htm new file mode 100644 index 00000000..67c5c5d3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnfwd.htm @@ -0,0 +1,71 @@ +<html> +<head> +<title> +Netlab Reference Manual mdnfwd +</title> +</head> +<body> +<H1> mdnfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through Mixture Density Network. + +<p><h2> +Synopsis +</h2> +<PRE> +mixparams = mdnfwd(net, x) +[mixparams, y, z] = mdnfwd(net, x) +[mixparams, y, z, a] = mdnfwd(net, x) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>mixparams = mdnfwd(net, x)</CODE> takes a mixture density network data +structure <CODE>net</CODE> and a matrix <CODE>x</CODE> of input vectors, and forward +propagates the inputs through the network to generate a structure +<CODE>mixparams</CODE> which contains the parameters of several mixture models. +Each row of <CODE>x</CODE> represents +one input vector and the corresponding row of the matrices in <CODE>mixparams</CODE> +represents the parameters of a mixture model for the conditional probability +of target vectors given the input vector. This is not represented as an array +of <CODE>gmm</CODE> structures to improve the efficiency of MDN training. + +<p>The fields in <CODE>mixparams</CODE> are +<PRE> + + type = 'mdnmixes' + ncentres = number of mixture components + dimtarget = dimension of target space + mixcoeffs = mixing coefficients + centres = means of Gaussians: stored as one row per pattern + covars = covariances of Gaussians + nparams = number of parameters +</PRE> + + +<p><CODE>[mixparams, y, z] = mdnfwd(net, x)</CODE> also generates a matrix <CODE>y</CODE> of +the outputs of the MLP and a matrix <CODE>z</CODE> of the hidden +unit activations where each row corresponds to one pattern. + +<p><CODE>[mixparams, y, z, a] = mlpfwd(net, x)</CODE> also returns a matrix <CODE>a</CODE> +giving the summed inputs to each output unit, where each row +corresponds to one pattern. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdn2gmm.htm">mdn2gmm</a></CODE>, <CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdngrad.htm b/sourcecodes/bnt-master/nethelp3.3/mdngrad.htm new file mode 100644 index 00000000..b3842991 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdngrad.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual mdngrad +</title> +</head> +<body> +<H1> mdngrad +</H1> +<h2> +Purpose +</h2> +Evaluate gradient of error function for Mixture Density Network. + +<p><h2> +Synopsis +</h2> +<PRE> + +g = mdngrad(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>g = mdngrad(net, x, t)</CODE> takes a mixture density network data +structure <CODE>net</CODE>, a matrix <CODE>x</CODE> of input vectors and a matrix +<CODE>t</CODE> of target vectors, and evaluates the gradient <CODE>g</CODE> of the +error function with respect to the network weights. The error function +is negative log likelihood of the target data. Each row of <CODE>x</CODE> +corresponds to one input vector and each row of <CODE>t</CODE> corresponds to +one target vector. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdnfwd.htm">mdnfwd</a></CODE>, <CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdnprob.htm">mdnprob</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdninit.htm b/sourcecodes/bnt-master/nethelp3.3/mdninit.htm new file mode 100644 index 00000000..9e205c51 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdninit.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual mdninit +</title> +</head> +<body> +<H1> mdninit +</H1> +<h2> +Purpose +</h2> +Initialise the weights in a Mixture Density Network. + +<p><h2> +Synopsis +</h2> +<PRE> +net = mdninit(net, prior) +net = mdninit(net, prior, t, options) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = mdninit(net, prior)</CODE> takes a Mixture Density Network +<CODE>net</CODE> and sets the weights and biases by sampling from a Gaussian +distribution. It calls <CODE>mlpinit</CODE> for the MLP component of <CODE>net</CODE>. + +<p><CODE>net = mdninit(net, prior, t, options)</CODE> uses the target data <CODE>t</CODE> to +initialise the biases for the output units after initialising the +other weights as above. It calls <CODE>gmminit</CODE>, with <CODE>t</CODE> and <CODE>options</CODE> +as arguments, to obtain a model of the unconditional density of <CODE>t</CODE>. The +biases are then set so that <CODE>net</CODE> will output the values in the Gaussian +mixture model. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlpinit.htm">mlpinit</a></CODE>, <CODE><a href="gmminit.htm">gmminit</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnpak.htm b/sourcecodes/bnt-master/nethelp3.3/mdnpak.htm new file mode 100644 index 00000000..d5e39033 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnpak.htm @@ -0,0 +1,41 @@ +<html> +<head> +<title> +Netlab Reference Manual mdnpak +</title> +</head> +<body> +<H1> mdnpak +</H1> +<h2> +Purpose +</h2> +Combines weights and biases into one weights vector. + +<p><h2> +Synopsis +</h2> +<PRE> +w = mdnpak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>w = mdnpak(net)</CODE> takes a mixture density +network data structure <CODE>net</CODE> and +combines the network weights into a single row vector <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdnunpak.htm">mdnunpak</a></CODE>, <CODE><a href="mdnfwd.htm">mdnfwd</a></CODE>, <CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnpost.htm b/sourcecodes/bnt-master/nethelp3.3/mdnpost.htm new file mode 100644 index 00000000..3ae071a0 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnpost.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual mdnpost +</title> +</head> +<body> +<H1> mdnpost +</H1> +<h2> +Purpose +</h2> +Computes the posterior probability for each MDN mixture component. + +<p><h2> +Synopsis +</h2> +<PRE> +post = mdnpost(mixparams, t) +[post, a] = mdnpost(mixparams, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>post = mdnpost(mixparams, t)</CODE> computes the posterior +probability <CODE>p(j|t)</CODE> of each +data vector in <CODE>t</CODE> under the Gaussian mixture model represented by the +corresponding entries in <CODE>mixparams</CODE>. Each row of <CODE>t</CODE> represents a +single vector. + +<p><CODE>[post, a] = mdnpost(mixparams, t)</CODE> also computes the activations +<CODE>a</CODE> (i.e. the probability <CODE>p(t|j)</CODE> of the data conditioned on +each component density) for a Gaussian mixture model. + +<p><h2> +See Also +</h2> +<CODE><a href="mdngrad.htm">mdngrad</a></CODE>, <CODE><a href="mdnprob.htm">mdnprob</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnprob.htm b/sourcecodes/bnt-master/nethelp3.3/mdnprob.htm new file mode 100644 index 00000000..cffa2517 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnprob.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual mdnprob +</title> +</head> +<body> +<H1> mdnprob +</H1> +<h2> +Purpose +</h2> +Computes the data probability likelihood for an MDN mixture structure. + +<p><h2> +Synopsis +</h2> +<PRE> +prob = mdnprob(mixparams, t) +[prob, a] = mdnprob(mixparams, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>prob = mdnprob(mixparams, t)</CODE> computes the probability <CODE>p(t)</CODE> of each +data vector in <CODE>t</CODE> under the Gaussian mixture model represented by the +corresponding entries in <CODE>mixparams</CODE>. Each row of <CODE>t</CODE> represents a +single vector. + +<p><CODE>[prob, a] = mdnprob(mixparams, t)</CODE> also computes the activations +<CODE>a</CODE> (i.e. the probability <CODE>p(t|j)</CODE> of the data conditioned on +each component density) for a Gaussian mixture model. + +<p><h2> +See Also +</h2> +<CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdnpost.htm">mdnpost</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnunpak.htm b/sourcecodes/bnt-master/nethelp3.3/mdnunpak.htm new file mode 100644 index 00000000..de6d2610 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnunpak.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual mdnunpak +</title> +</head> +<body> +<H1> mdnunpak +</H1> +<h2> +Purpose +</h2> +Separates weights vector into weight and bias matrices. + +<p><h2> +Synopsis +</h2> +<PRE> +net = mdnunpak(net, w) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = mdnunpak(net, w)</CODE> takes an mdn network data structure <CODE>net</CODE> and +a weight vector <CODE>w</CODE>, and returns a network data structure identical to +the input network, except that the weights in the MLP sub-structure are +set to the corresponding elements of <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mdn.htm">mdn</a></CODE>, <CODE><a href="mdnpak.htm">mdnpak</a></CODE>, <CODE><a href="mdnfwd.htm">mdnfwd</a></CODE>, <CODE><a href="mdnerr.htm">mdnerr</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) +<p>David J Evans (1998) + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/metrop.htm b/sourcecodes/bnt-master/nethelp3.3/metrop.htm new file mode 100644 index 00000000..93c3ed49 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/metrop.htm @@ -0,0 +1,108 @@ +<html> +<head> +<title> +Netlab Reference Manual metrop +</title> +</head> +<body> +<H1> metrop +</H1> +<h2> +Purpose +</h2> +Markov Chain Monte Carlo sampling with Metropolis algorithm. + +<p><h2> +Synopsis +</h2> +<PRE> + +samples = metrop(f, x, options) +samples = metrop(f, x, options, [], P1, P2, ...) +[samples, energies, diagn] = metrop(f, x, options) +s = metrop('state') +metrop('state', s) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>samples = metrop(f, x, options)</CODE> uses +the Metropolis algorithm to sample from the distribution +<CODE>p ~ exp(-f)</CODE>, where <CODE>f</CODE> is the first argument to <CODE>metrop</CODE>. +The Markov chain starts at the point <CODE>x</CODE> and each +candidate state is picked from a Gaussian proposal distribution and +accepted or rejected according to the Metropolis criterion. + +<p><CODE>samples = metrop(f, x, options, [], p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE>. The fourth argument is +ignored, but is included for compatibility with <CODE>hmc</CODE> and the +optimisers. + +<p><CODE>[samples, energies, diagn] = metrop(f, x, options)</CODE> also returns +a log of the energy values (i.e. negative log probabilities) for the +samples in <CODE>energies</CODE> and <CODE>diagn</CODE>, a structure containing +diagnostic information (position and +acceptance threshold) for each step of the chain in <CODE>diagn.pos</CODE> and +<CODE>diagn.acc</CODE> respectively. All candidate states (including rejected +ones) are stored in <CODE>diagn.pos</CODE>. + +<p><CODE>s = metrop('state')</CODE> returns a state structure that contains the +state of the two random number generators <CODE>rand</CODE> and <CODE>randn</CODE>. +These are contained in fields +<CODE>randstate</CODE>, +<CODE>randnstate</CODE>. + +<p><CODE>metrop('state', s)</CODE> resets the state to <CODE>s</CODE>. If <CODE>s</CODE> is an integer, +then it is passed to <CODE>rand</CODE> and <CODE>randn</CODE>. +If <CODE>s</CODE> is a structure returned by <CODE>metrop('state')</CODE> then +it resets the generator to exactly the same state. + +<p>The optional parameters in the <CODE>options</CODE> vector have the following +interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display the energy values and rejection +threshold at each step of the Markov chain. If the value is 2, then the +position vectors at each step are also displayed. + +<p><CODE>options(14)</CODE> is the number of samples retained from the Markov chain; +default 100. + +<p><CODE>options(15)</CODE> is the number of samples omitted from the start of the +chain; default 0. + +<p><CODE>options(18)</CODE> is the variance of the proposal distribution; default 1. + +<p><h2> +Examples +</h2> +The following code fragment samples from the posterior distribution of +weights for a neural network. +<PRE> + +w = mlppak(net); +[samples, energies] = metrop('neterr', w, options, 'netgrad', net, x, t); +</PRE> + + +<p><h2> +Algorithm +</h2> + +The algorithm follows the procedure outlined in Radford Neal's technical +report CRG-TR-93-1 from the University of Toronto. + +<p><h2> +See Also +</h2> +<CODE><a href="hmc.htm">hmc</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/minbrack.htm b/sourcecodes/bnt-master/nethelp3.3/minbrack.htm new file mode 100644 index 00000000..f49d18f5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/minbrack.htm @@ -0,0 +1,65 @@ +<html> +<head> +<title> +Netlab Reference Manual minbrack +</title> +</head> +<body> +<H1> minbrack +</H1> +<h2> +Purpose +</h2> +Bracket a minimum of a function of one variable. + +<p><h2> +Description +</h2> +<CODE>brmin, brmid, brmax, numevals] = minbrack(f, a, b, fa)</CODE> +finds a bracket of three points around a local minimum of +<CODE>f</CODE>. The function <CODE>f</CODE> must have a one dimensional domain. +<CODE>a < b</CODE> is an initial guess at the minimum and maximum points +of a bracket, but <CODE>minbrack</CODE> will search outside this interval if +necessary. The bracket consists of three points (in increasing order) +such that <CODE>f(brmid) < f(brmin)</CODE> and <CODE>f(brmid) < f(brmax)</CODE>. +<CODE>fa</CODE> is the value of the function at <CODE>a</CODE>: it is included to +avoid unnecessary function evaluations in the optimization routines. +The return value <CODE>numevals</CODE> is the number of function evaluations +in <CODE>minbrack</CODE>. + +<p><CODE>minbrack(f, a, b, fa, p1, p2, ...)</CODE> allows additional +arguments to be passed to <CODE>f</CODE> + +<p><h2> +Examples +</h2> +An example of the use of this function to bracket the minimum of a function +<CODE>f</CODE> in the direction <CODE>sd</CODE> can be found in <CODE>linemin</CODE> +<PRE> + +[min, mid, max, nevals]] = minbrack('linef', 0.0, 1.0, fa, f, pt, dir); +</PRE> + +where the function <CODE>linef</CODE> is used to turn a general function <CODE>f</CODE> +into a one dimensional one. + +<p><h2> +Algorithm +</h2> + +Quadratic extrapolation with a limit to the maximum step size is +used to find the outside points of the bracket. This implementation +is based on that in Numerical Recipes. + +<p><h2> +See Also +</h2> +<CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="linef.htm">linef</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlp.htm b/sourcecodes/bnt-master/nethelp3.3/mlp.htm new file mode 100644 index 00000000..b8237991 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlp.htm @@ -0,0 +1,94 @@ +<html> +<head> +<title> +Netlab Reference Manual mlp +</title> +</head> +<body> +<H1> mlp +</H1> +<h2> +Purpose +</h2> +Create a 2-layer feedforward network. + +<p><h2> +Synopsis +</h2> +<PRE> +net = mlp(nin, nhidden, nout, func) +net = mlp(nin, nhidden, nout, func, prior) +net = mlp(nin, nhidden, nout, func, prior, beta) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = mlp(nin, nhidden, nout, func)</CODE> takes the number of inputs, +hidden units and output units for a 2-layer feed-forward network, +together with a string <CODE>func</CODE> which specifies the output unit +activation function, and returns a data structure <CODE>net</CODE>. The +weights are drawn from a zero mean, unit variance isotropic Gaussian, +with varianced scaled by the fan-in of the hidden or output units as +appropriate. This makes use of the Matlab function +<CODE>randn</CODE> and so the seed for the random weight initialization can be +set using <CODE>randn('state', s)</CODE> where <CODE>s</CODE> is the seed value. +The hidden units use the <CODE>tanh</CODE> activation function. + +<p>The fields in <CODE>net</CODE> are +<PRE> + + type = 'mlp' + nin = number of inputs + nhidden = number of hidden units + nout = number of outputs + nwts = total number of weights and biases + actfn = string describing the output unit activation function: + 'linear' + 'logistic + 'softmax' + w1 = first-layer weight matrix + b1 = first-layer bias vector + w2 = second-layer weight matrix + b2 = second-layer bias vector +</PRE> + +Here <CODE>w1</CODE> has dimensions <CODE>nin</CODE> times <CODE>nhidden</CODE>, <CODE>b1</CODE> has +dimensions <CODE>1</CODE> times <CODE>nhidden</CODE>, <CODE>w2</CODE> has +dimensions <CODE>nhidden</CODE> times <CODE>nout</CODE>, and <CODE>b2</CODE> has +dimensions <CODE>1</CODE> times <CODE>nout</CODE>. + +<p><CODE>net = mlp(nin, nhidden, nout, func, prior)</CODE>, in which <CODE>prior</CODE> is +a scalar, allows the field <CODE>net.alpha</CODE> in the data structure +<CODE>net</CODE> to be set, corresponding to a zero-mean isotropic Gaussian +prior with inverse variance with value <CODE>prior</CODE>. Alternatively, +<CODE>prior</CODE> can consist of a data structure with fields <CODE>alpha</CODE> +and <CODE>index</CODE>, allowing individual Gaussian priors to be set over +groups of weights in the network. Here <CODE>alpha</CODE> is a column vector +in which each element corresponds to a separate group of weights, +which need not be mutually exclusive. The membership of the groups is +defined by the matrix <CODE>indx</CODE> in which the columns correspond to +the elements of <CODE>alpha</CODE>. Each column has one element for each +weight in the matrix, in the order defined by the function +<CODE>mlppak</CODE>, and each element is 1 or 0 according to whether the +weight is a member of the corresponding group or not. A utility +function <CODE>mlpprior</CODE> is provided to help in setting up the +<CODE>prior</CODE> data structure. + +<p><CODE>net = mlp(nin, nhidden, nout, func, prior, beta)</CODE> also sets the +additional field <CODE>net.beta</CODE> in the data structure <CODE>net</CODE>, where +beta corresponds to the inverse noise variance. + +<p><h2> +See Also +</h2> +<CODE><a href="mlpprior.htm">mlpprior</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpbkp.htm b/sourcecodes/bnt-master/nethelp3.3/mlpbkp.htm new file mode 100644 index 00000000..00fd6846 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpbkp.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpbkp +</title> +</head> +<body> +<H1> mlpbkp +</H1> +<h2> +Purpose +</h2> +Backpropagate gradient of error function for 2-layer network. + +<p><h2> +Synopsis +</h2> +<PRE> +g = mlpbkp(net, x, z, deltas)</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = mlpbkp(net, x, z, deltas)</CODE> takes a network data structure +<CODE>net</CODE> together with a matrix <CODE>x</CODE> of input vectors, a matrix +<CODE>z</CODE> of hidden unit activations, and a matrix <CODE>deltas</CODE> of the +gradient of the error function with respect to the values of the +output units (i.e. the summed inputs to the output units, before the +activation function is applied). The return value is the gradient +<CODE>g</CODE> of the error function with respect to the network +weights. Each row of <CODE>x</CODE> corresponds to one input vector. + +<p>This function is provided so that the common backpropagation algorithm +can be used by multi-layer perceptron network models to compute +gradients for mixture density networks as well as standard error +functions. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE>, <CODE><a href="mlpderiv.htm">mlpderiv</a></CODE>, <CODE><a href="mdngrad.htm">mdngrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpderiv.htm b/sourcecodes/bnt-master/nethelp3.3/mlpderiv.htm new file mode 100644 index 00000000..514dafc6 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpderiv.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpderiv +</title> +</head> +<body> +<H1> mlpderiv +</H1> +<h2> +Purpose +</h2> +Evaluate derivatives of network outputs with respect to weights. + +<p><h2> +Synopsis +</h2> +<PRE> +g = mlpderiv(net, x)</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = mlpderiv(net, x)</CODE> takes a network data structure <CODE>net</CODE> +and a matrix of input vectors <CODE>x</CODE> and returns a three-index matrix +<CODE>g</CODE> whose <CODE>i</CODE>, <CODE>j</CODE>, <CODE>k</CODE> element contains the +derivative of network output <CODE>k</CODE> with respect to weight or bias +parameter <CODE>j</CODE> for input pattern <CODE>i</CODE>. The ordering of the +weight and bias parameters is defined by <CODE>mlpunpak</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlperr.htm b/sourcecodes/bnt-master/nethelp3.3/mlperr.htm new file mode 100644 index 00000000..33d748da --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlperr.htm @@ -0,0 +1,49 @@ +<html> +<head> +<title> +Netlab Reference Manual mlperr +</title> +</head> +<body> +<H1> mlperr +</H1> +<h2> +Purpose +</h2> +Evaluate error function for 2-layer network. + +<p><h2> +Synopsis +</h2> +<PRE> +e = mlperr(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>e = mlperr(net, x, t)</CODE> takes a network data structure <CODE>net</CODE> together +with a matrix <CODE>x</CODE> of input vectors and a matrix <CODE>t</CODE> of target +vectors, and evaluates the error function <CODE>e</CODE>. The choice of error +function corresponds to the output unit activation function. Each row +of <CODE>x</CODE> corresponds to one input vector and each row of <CODE>t</CODE> +corresponds to one target vector. + +<p><CODE>[e, edata, eprior] = mlperr(net, x, t)</CODE> additionally returns the +data and prior components of the error, assuming a zero mean Gaussian +prior on the weights with inverse variance parameters <CODE>alpha</CODE> and +<CODE>beta</CODE> taken from the network data structure <CODE>net</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/mlpevfwd.htm new file mode 100644 index 00000000..1cf75307 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpevfwd.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpevfwd +</title> +</head> +<body> +<H1> mlpevfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation with evidence for MLP + +<p><h2> +Synopsis +</h2> +<PRE> + +[y, extra] = mlpevfwd(net, x, t, x_test) +[y, extra, invhess] = mlpevfwd(net, x, t, x_test, invhess) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = mlpevfwd(net, x, t, x_test)</CODE> takes a network data structure +<CODE>net</CODE> together with the input <CODE>x</CODE> and target <CODE>t</CODE> training data +and input test data <CODE>x_test</CODE>. +It returns the normal forward propagation through the network <CODE>y</CODE> +together with a matrix <CODE>extra</CODE> which consists of error bars (variance) +for a regression problem or moderated outputs for a classification problem. +The optional argument (and return value) +<CODE>invhess</CODE> is the inverse of the network Hessian +computed on the training data inputs and targets. Passing it in avoids +recomputing it, which can be a significant saving for large training sets. + +<p><h2> +See Also +</h2> +<CODE><a href="fevbayes.htm">fevbayes</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpfwd.htm b/sourcecodes/bnt-master/nethelp3.3/mlpfwd.htm new file mode 100644 index 00000000..2f2be47a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpfwd.htm @@ -0,0 +1,52 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpfwd +</title> +</head> +<body> +<H1> mlpfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through 2-layer network. + +<p><h2> +Synopsis +</h2> +<PRE> +y = mlpfwd(net, x) +[y, z] = mlpfwd(net, x) +[y, z, a] = mlpfwd(net, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = mlpfwd(net, x)</CODE> takes a network data structure <CODE>net</CODE> together with +a matrix <CODE>x</CODE> of input vectors, and forward propagates the inputs +through the network to generate a matrix <CODE>y</CODE> of output +vectors. Each row of <CODE>x</CODE> corresponds to one input vector and each +row of <CODE>y</CODE> corresponds to one output vector. + +<p><CODE>[y, z] = mlpfwd(net, x)</CODE> also generates a matrix <CODE>z</CODE> of the hidden +unit activations where each row corresponds to one pattern. + +<p><CODE>[y, z, a] = mlpfwd(net, x)</CODE> also returns a matrix <CODE>a</CODE> +giving the summed inputs to each output unit, where each row +corresponds to one pattern. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpgrad.htm b/sourcecodes/bnt-master/nethelp3.3/mlpgrad.htm new file mode 100644 index 00000000..08943cc9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpgrad.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpgrad +</title> +</head> +<body> +<H1> mlpgrad +</H1> +<h2> +Purpose +</h2> +Evaluate gradient of error function for 2-layer network. + +<p><h2> +Synopsis +</h2> +<PRE> + +g = mlpgrad(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = mlpgrad(net, x, t)</CODE> takes a network data structure <CODE>net</CODE> +together with a matrix <CODE>x</CODE> of input vectors and a matrix <CODE>t</CODE> +of target vectors, and evaluates the gradient <CODE>g</CODE> of the error +function with respect to the network weights. The error funcion +corresponds to the choice of output unit activation function. Each row +of <CODE>x</CODE> corresponds to one input vector and each row of <CODE>t</CODE> +corresponds to one target vector. + +<p><CODE>[g, gdata, gprior] = mlpgrad(net, x, t)</CODE> also returns separately +the data and prior contributions to the gradient. In the case of +multiple groups in the prior, <CODE>gprior</CODE> is a matrix with a row +for each group and a column for each weight parameter. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlphdotv.htm b/sourcecodes/bnt-master/nethelp3.3/mlphdotv.htm new file mode 100644 index 00000000..7ecf2765 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlphdotv.htm @@ -0,0 +1,45 @@ +<html> +<head> +<title> +Netlab Reference Manual mlphdotv +</title> +</head> +<body> +<H1> mlphdotv +</H1> +<h2> +Purpose +</h2> +Evaluate the product of the data Hessian with a vector. + +<p><h2> +Synopsis +</h2> +<PRE> +hdv = mlphdotv(net, x, t, v)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>hdv = mlphdotv(net, x, t, v)</CODE> takes an MLP network data structure +<CODE>net</CODE>, together with the matrix <CODE>x</CODE> of input vectors, the +matrix <CODE>t</CODE> of target vectors and an arbitrary row vector <CODE>v</CODE> +whose length equals the number of parameters in the network, and +returns the product of the data-dependent contribution to the Hessian +matrix with <CODE>v</CODE>. The implementation is based on the R-propagation +algorithm of Pearlmutter. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlphess.htm">mlphess</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlphess.htm b/sourcecodes/bnt-master/nethelp3.3/mlphess.htm new file mode 100644 index 00000000..59a6ec91 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlphess.htm @@ -0,0 +1,73 @@ +<html> +<head> +<title> +Netlab Reference Manual mlphess +</title> +</head> +<body> +<H1> mlphess +</H1> +<h2> +Purpose +</h2> +Evaluate the Hessian matrix for a multi-layer perceptron network. + +<p><h2> +Synopsis +</h2> +<PRE> +h = mlphess(net, x, t) +[h, hdata] = mlphess(net, x, t) +h = mlphess(net, x, t, hdata) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>h = mlphess(net, x, t)</CODE> takes an MLP network data structure <CODE>net</CODE>, +a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of target +values and returns the full Hessian matrix <CODE>h</CODE> corresponding to +the second derivatives of the negative log posterior distribution, +evaluated for the current weight and bias values as defined by +<CODE>net</CODE>. + +<p><CODE>[h, hdata] = mlphess(net, x, t)</CODE> returns both the Hessian matrix +<CODE>h</CODE> and the contribution <CODE>hdata</CODE> arising from the data dependent +term in the Hessian. + +<p><CODE>h = mlphess(net, x, t, hdata)</CODE> takes a network data structure +<CODE>net</CODE>, a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of +target values, together with the contribution <CODE>hdata</CODE> arising from +the data dependent term in the Hessian, and returns the full Hessian +matrix <CODE>h</CODE> corresponding to the second derivatives of the negative +log posterior distribution. This version saves computation time if +<CODE>hdata</CODE> has already been evaluated for the current weight and bias +values. + +<p><h2> +Example +</h2> +For the standard regression framework with a Gaussian conditional +distribution of target values given input values, and a simple +Gaussian prior over weights, the Hessian takes the form +<PRE> + + h = beta*hd + alpha*I +</PRE> + +where the contribution <CODE>hd</CODE> is evaluated by calls to <CODE>mlphdotv</CODE> and +<CODE>h</CODE> is the full Hessian. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE>, <CODE><a href="mlphdotv.htm">mlphdotv</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlphint.htm b/sourcecodes/bnt-master/nethelp3.3/mlphint.htm new file mode 100644 index 00000000..82411403 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlphint.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual mlphint +</title> +</head> +<body> +<H1> mlphint +</H1> +<h2> +Purpose +</h2> +Plot Hinton diagram for 2-layer feed-forward network. + +<p><h2> +Synopsis +</h2> +<PRE> +mlphint(net) +[h1, h2] = mlphint(net)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>mlphint(net)</CODE> takes a network structure <CODE>net</CODE> +and plots the Hinton diagram comprised of two +figure windows, one displaying the first-layer weights and biases, and +one displaying the second-layer weights and biases. + +<p><CODE>[h1, h2] = mlphint(net)</CODE> also returns handles <CODE>h1</CODE> and +<CODE>h2</CODE> to the figures which can be used, for instance, to delete the +figures when they are no longer needed. + +<p>To print the figure correctly, you should call +<CODE>set(h, 'InvertHardCopy', 'on')</CODE> before printing. + +<p><h2> +See Also +</h2> +<CODE><a href="demhint.htm">demhint</a></CODE>, <CODE><a href="hintmat.htm">hintmat</a></CODE>, <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpinit.htm b/sourcecodes/bnt-master/nethelp3.3/mlpinit.htm new file mode 100644 index 00000000..2f7dbd4e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpinit.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpinit +</title> +</head> +<body> +<H1> mlpinit +</H1> +<h2> +Purpose +</h2> +Initialise the weights in a 2-layer feedforward network. + +<p><h2> +Synopsis +</h2> +<PRE> +net = mlpinit(net, prior) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = mlpinit(net, prior)</CODE> takes a 2-layer feedforward network +<CODE>net</CODE> and sets the weights and biases by sampling from a Gaussian +distribution. If <CODE>prior</CODE> is a scalar, then all of the parameters +(weights and biases) are sampled from a single isotropic Gaussian with +inverse variance equal to <CODE>prior</CODE>. If <CODE>prior</CODE> is a data +structure of the kind generated by <CODE>mlpprior</CODE>, then the parameters +are sampled from multiple Gaussians according to their groupings +(defined by the <CODE>index</CODE> field) with corresponding variances +(defined by the <CODE>alpha</CODE> field). + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlpprior.htm">mlpprior</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlppak.htm b/sourcecodes/bnt-master/nethelp3.3/mlppak.htm new file mode 100644 index 00000000..aa038f46 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlppak.htm @@ -0,0 +1,55 @@ +<html> +<head> +<title> +Netlab Reference Manual mlppak +</title> +</head> +<body> +<H1> mlppak +</H1> +<h2> +Purpose +</h2> +Combines weights and biases into one weights vector. + +<p><h2> +Synopsis +</h2> +<PRE> +w = mlppak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>w = mlppak(net)</CODE> takes a network data structure <CODE>net</CODE> and +combines the component weight matrices bias vectors into a single row +vector <CODE>w</CODE>. The facility to switch between these two +representations for the network parameters is useful, for example, in +training a network by error function minimization, since a single +vector of parameters can be handled by general-purpose optimization +routines. + +<p>The ordering of the paramters in <CODE>w</CODE> is defined by +<PRE> + + w = [net.w1(:)', net.b1, net.w2(:)', net.b2]; +</PRE> + +where <CODE>w1</CODE> is the first-layer weight matrix, <CODE>b1</CODE> is the +first-layer bias vector, <CODE>w2</CODE> is the second-layer weight matrix, +and <CODE>b2</CODE> is the second-layer bias vector. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpprior.htm b/sourcecodes/bnt-master/nethelp3.3/mlpprior.htm new file mode 100644 index 00000000..1821e919 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpprior.htm @@ -0,0 +1,58 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpprior +</title> +</head> +<body> +<H1> mlpprior +</H1> +<h2> +Purpose +</h2> +Create Gaussian prior for mlp. + +<p><h2> +Synopsis +</h2> +<PRE> +prior = mlpprior(nin, nhidden, nout, aw1, ab1, aw2, ab2)</PRE> + + +<p><h2> +Description +</h2> +<CODE>prior = mlpprior(nin, nhidden, nout, aw1, ab1, aw2, ab2)</CODE> +generates a data structure +<CODE>prior</CODE>, with fields <CODE>prior.alpha</CODE> and <CODE>prior.index</CODE>, which +specifies a Gaussian prior distribution for the network weights in a +two-layer feedforward network. Two different cases are possible. In +the first case, <CODE>aw1</CODE>, <CODE>ab1</CODE>, <CODE>aw2</CODE> and <CODE>ab2</CODE> are all +scalars and represent the regularization coefficients for four groups +of parameters in the network corresponding to first-layer weights, +first-layer biases, second-layer weights, and second-layer biases +respectively. Then <CODE>prior.alpha</CODE> represents a column vector of +length 4 containing the parameters, and <CODE>prior.index</CODE> is a matrix +specifying which weights belong in each group. Each column has one +element for each weight in the matrix, using the standard ordering as +defined in <CODE>mlppak</CODE>, and each element is 1 or 0 according to +whether the weight is a member of the corresponding group or not. In +the second case the parameter <CODE>aw1</CODE> is a vector of length equal to +the number of inputs in the network, and the corresponding matrix +<CODE>prior.index</CODE> now partitions the first-layer weights into groups +corresponding to the weights fanning out of each input unit. This +prior is appropriate for the technique of automatic relevance +determination. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlptrain.htm b/sourcecodes/bnt-master/nethelp3.3/mlptrain.htm new file mode 100644 index 00000000..ad89786b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlptrain.htm @@ -0,0 +1,35 @@ +<html> +<head> +<title> +Netlab Reference Manual mlptrain +</title> +</head> +<body> +<H1> mlptrain +</H1> +<h2> +Purpose +</h2> +Utility to train an MLP network for demtrain + +<p><h2> +Description +</h2> + +<p><CODE>[net, error] = mlptrain(net, x, t, its)</CODE> trains a network data +structure <CODE>net</CODE> using the scaled conjugate gradient algorithm +for <CODE>its</CODE> cycles with +input data <CODE>x</CODE>, target data <CODE>t</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="demtrain.htm">demtrain</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE>, <CODE><a href="netopt.htm">netopt</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpunpak.htm b/sourcecodes/bnt-master/nethelp3.3/mlpunpak.htm new file mode 100644 index 00000000..3c6bba21 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpunpak.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual mlpunpak +</title> +</head> +<body> +<H1> mlpunpak +</H1> +<h2> +Purpose +</h2> +Separates weights vector into weight and bias matrices. + +<p><h2> +Synopsis +</h2> +<PRE> +net = mlpunpak(net, w) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = mlpunpak(net, w)</CODE> takes an mlp network data structure <CODE>net</CODE> and +a weight vector <CODE>w</CODE>, and returns a network data structure identical to +the input network, except that the first-layer weight matrix +<CODE>w1</CODE>, the first-layer bias vector <CODE>b1</CODE>, the second-layer +weight matrix <CODE>w2</CODE> and the second-layer bias vector <CODE>b2</CODE> have all +been set to the corresponding elements of <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netderiv.htm b/sourcecodes/bnt-master/nethelp3.3/netderiv.htm new file mode 100644 index 00000000..3275eced --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netderiv.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual netderiv +</title> +</head> +<body> +<H1> netderiv +</H1> +<h2> +Purpose +</h2> +Evaluate derivatives of network outputs by weights generically. + +<p><h2> +Synopsis +</h2> +<PRE> +g = netderiv(w, net, x)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>g = netderiv(w, net, x)</CODE> takes a weight vector <CODE>w</CODE> and a network +data structure <CODE>net</CODE>, together with the matrix <CODE>x</CODE> of input +vectors, and returns the +gradient of the outputs with respect to the weights evaluated at <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="netevfwd.htm">netevfwd</a></CODE>, <CODE><a href="netopt.htm">netopt</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/neterr.htm b/sourcecodes/bnt-master/nethelp3.3/neterr.htm new file mode 100644 index 00000000..2ec53e61 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/neterr.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual neterr +</title> +</head> +<body> +<H1> neterr +</H1> +<h2> +Purpose +</h2> +Evaluate network error function for generic optimizers + +<p><h2> +Synopsis +</h2> +<PRE> +e = neterr(w, net, x, t) +[e, varargout] = neterr(w, net, x, t) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>e = neterr(w, net, x, t)</CODE> takes a weight vector <CODE>w</CODE> and a network +data structure <CODE>net</CODE>, together with the matrix <CODE>x</CODE> of input +vectors and the matrix <CODE>t</CODE> of target vectors, and returns the +value of the error function evaluated at <CODE>w</CODE>. + +<p><CODE>[e, varargout] = neterr(w, net, x, t)</CODE> also returns any additional +return values from the error function. + +<p><h2> +See Also +</h2> +<CODE><a href="netgrad.htm">netgrad</a></CODE>, <CODE><a href="nethess.htm">nethess</a></CODE>, <CODE><a href="netopt.htm">netopt</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/netevfwd.htm new file mode 100644 index 00000000..ea876bb2 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netevfwd.htm @@ -0,0 +1,52 @@ +<html> +<head> +<title> +Netlab Reference Manual netevfwd +</title> +</head> +<body> +<H1> netevfwd +</H1> +<h2> +Purpose +</h2> +Generic forward propagation with evidence for network + +<p><h2> +Synopsis +</h2> +<PRE> + +[y, extra] = netevfwd(w, net, x, t, x_test) +[y, extra, invhess] = netevfwd(w, net, x, t, x_test, invhess) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>[y, extra] = netevfwd(w, net, x, t, x_test)</CODE> takes a network data +structure +<CODE>net</CODE> together with the input <CODE>x</CODE> and target <CODE>t</CODE> training data +and input test data <CODE>x_test</CODE>. +It returns the normal forward propagation through the network <CODE>y</CODE> +together with a matrix <CODE>extra</CODE> which consists of error bars (variance) +for a regression problem or moderated outputs for a classification problem. + +<p>The optional argument (and return value) +<CODE>invhess</CODE> is the inverse of the network Hessian +computed on the training data inputs and targets. Passing it in avoids +recomputing it, which can be a significant saving for large training sets. + +<p><h2> +See Also +</h2> +<CODE><a href="mlpevfwd.htm">mlpevfwd</a></CODE>, <CODE><a href="rbfevfwd.htm">rbfevfwd</a></CODE>, <CODE><a href="glmevfwd.htm">glmevfwd</a></CODE>, <CODE><a href="fevbayes.htm">fevbayes</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netgrad.htm b/sourcecodes/bnt-master/nethelp3.3/netgrad.htm new file mode 100644 index 00000000..a9ce1571 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netgrad.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual netgrad +</title> +</head> +<body> +<H1> netgrad +</H1> +<h2> +Purpose +</h2> +Evaluate network error gradient for generic optimizers + +<p><h2> +Synopsis +</h2> +<PRE> +g = netgrad(w, net, x, t)</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>g = netgrad(w, net, x, t)</CODE> takes a weight vector <CODE>w</CODE> and a network +data structure <CODE>net</CODE>, together with the matrix <CODE>x</CODE> of input +vectors and the matrix <CODE>t</CODE> of target vectors, and returns the +gradient of the error function evaluated at <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="neterr.htm">neterr</a></CODE>, <CODE><a href="netopt.htm">netopt</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/nethelp3.3.zip b/sourcecodes/bnt-master/nethelp3.3/nethelp3.3.zip new file mode 100644 index 00000000..5c316a1d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/nethelp3.3.zip Binary files differdiff --git a/sourcecodes/bnt-master/nethelp3.3/nethess.htm b/sourcecodes/bnt-master/nethelp3.3/nethess.htm new file mode 100644 index 00000000..b918fe0a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/nethess.htm @@ -0,0 +1,66 @@ +<html> +<head> +<title> +Netlab Reference Manual nethess +</title> +</head> +<body> +<H1> nethess +</H1> +<h2> +Purpose +</h2> +Evaluate network Hessian + +<p><h2> +Synopsis +</h2> +<PRE> +h = neterr(w, net, x, t) +[h, varargout] = neterr(w, net, x, t, varargin) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>h = nethess(w, net, x, t)</CODE> takes a weight vector <CODE>w</CODE> and a network +data structure <CODE>net</CODE>, together with the matrix <CODE>x</CODE> of input +vectors and the matrix <CODE>t</CODE> of target vectors, and returns the +value of the Hessian evaluated at <CODE>w</CODE>. + +<p><CODE>[e, varargout] = nethess(w, net, x, t, varargin)</CODE> also returns any additional +return values from the network Hessian function, and passes additional arguments +to that function. + +<p><h2> +Example +</h2> + +<p>In <CODE>evidence</CODE>, this function is called once to compute the +data contribution to the Hessian +<PRE> + +[h, dh] = nethess(w, net, x, t, dh); +</PRE> + +and again to update the Hessian for new values of the hyper-parameters +<PRE> + +h = nethess(w, net, x, t, dh); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="neterr.htm">neterr</a></CODE>, <CODE><a href="netgrad.htm">netgrad</a></CODE>, <CODE><a href="netopt.htm">netopt</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netinit.htm b/sourcecodes/bnt-master/nethelp3.3/netinit.htm new file mode 100644 index 00000000..3612b574 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netinit.htm @@ -0,0 +1,48 @@ +<html> +<head> +<title> +Netlab Reference Manual netinit +</title> +</head> +<body> +<H1> netinit +</H1> +<h2> +Purpose +</h2> +Initialise the weights in a network. + +<p><h2> +Synopsis +</h2> +<PRE> +net = netinit(net, prior) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>net = netinit(net, prior)</CODE> takes a network data structure +<CODE>net</CODE> and sets the weights and biases by sampling from a Gaussian +distribution. If <CODE>prior</CODE> is a scalar, then all of the parameters +(weights and biases) are sampled from a single isotropic Gaussian with +inverse variance equal to <CODE>prior</CODE>. If <CODE>prior</CODE> is a data +structure of the kind generated by <CODE>mlpprior</CODE>, then the parameters +are sampled from multiple Gaussians according to their groupings +(defined by the <CODE>index</CODE> field) with corresponding variances +(defined by the <CODE>alpha</CODE> field). + +<p><h2> +See Also +</h2> +<CODE><a href="mlpprior.htm">mlpprior</a></CODE>, <CODE><a href="netunpak.htm">netunpak</a></CODE>, <CODE><a href="rbfprior.htm">rbfprior</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netopt.htm b/sourcecodes/bnt-master/nethelp3.3/netopt.htm new file mode 100644 index 00000000..369c959c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netopt.htm @@ -0,0 +1,75 @@ +<html> +<head> +<title> +Netlab Reference Manual netopt +</title> +</head> +<body> +<H1> netopt +</H1> +<h2> +Purpose +</h2> +Optimize the weights in a network model. + +<p><h2> +Synopsis +</h2> +<PRE> +[net, options] = netopt(net, options, x, t, alg) +[net, options, varargout] = netopt(net, options, x, t, alg) +</PRE> + + +<p><h2> +Description +</h2> + +<p><CODE>netopt</CODE> is a helper function which facilitates the training of +networks using the general purpose optimizers as well as sampling from the +posterior distribution of parameters using general purpose Markov chain +Monte Carlo sampling algorithms. It can be used with any function that +searches in parameter space using error and gradient functions. + +<p><CODE>[net, options] = netopt(net, options, x, t, alg)</CODE> takes a network +data structure <CODE>net</CODE>, together with a vector <CODE>options</CODE> of +parameters governing the behaviour of the optimization algorithm, a +matrix <CODE>x</CODE> of input vectors and a matrix <CODE>t</CODE> of target +vectors, and returns the trained network as well as an updated +<CODE>options</CODE> vector. The string <CODE>alg</CODE> determines which optimization +algorithm (<CODE>conjgrad</CODE>, <CODE>quasinew</CODE>, <CODE>scg</CODE>, etc.) or Monte +Carlo algorithm (such as <CODE>hmc</CODE>) will be used. + +<p><CODE>[net, options, varargout] = netopt(net, options, x, t, alg)</CODE> +also returns any additional return values from the optimisation algorithm. + +<p><h2> +Examples +</h2> +Suppose we create a 4-input, 3 hidden unit, 2-output feed-forward +network using <CODE>net = mlp(4, 3, 2, 'linear')</CODE>. We can then train +the network with the scaled conjugate gradient algorithm by using +<CODE>net = netopt(net, options, x, t, 'scg')</CODE> where <CODE>x</CODE> and +<CODE>t</CODE> are the input and target data matrices respectively, and the +options vector is set appropriately for <CODE>scg</CODE>. + +<p>If we also wish to plot the learning curve, we can use the additional +return value <CODE>errlog</CODE> given by <CODE>scg</CODE>: +<PRE> + +[net, options, errlog] = netopt(net, options, x, t, 'scg'); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="netgrad.htm">netgrad</a></CODE>, <CODE><a href="bfgs.htm">bfgs</a></CODE>, <CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="hmc.htm">hmc</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netpak.htm b/sourcecodes/bnt-master/nethelp3.3/netpak.htm new file mode 100644 index 00000000..29286b7d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netpak.htm @@ -0,0 +1,47 @@ +<html> +<head> +<title> +Netlab Reference Manual netpak +</title> +</head> +<body> +<H1> netpak +</H1> +<h2> +Purpose +</h2> +Combines weights and biases into one weights vector. + +<p><h2> +Synopsis +</h2> +<PRE> +w = netpak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>w = netpak(net)</CODE> takes a network data structure <CODE>net</CODE> and +combines the component weight matrices into a single row +vector <CODE>w</CODE>. The facility to switch between these two +representations for the network parameters is useful, for example, in +training a network by error function minimization, since a single +vector of parameters can be handled by general-purpose optimization +routines. This function also takes into account a <CODE>mask</CODE> defined +as a field in <CODE>net</CODE> by removing any weights that correspond to +entries of 0 in the mask. + +<p><h2> +See Also +</h2> +<CODE><a href="net.htm">net</a></CODE>, <CODE><a href="netunpak.htm">netunpak</a></CODE>, <CODE><a href="netfwd.htm">netfwd</a></CODE>, <CODE><a href="neterr.htm">neterr</a></CODE>, <CODE><a href="netgrad.htm">netgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netunpak.htm b/sourcecodes/bnt-master/nethelp3.3/netunpak.htm new file mode 100644 index 00000000..fef34ed7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netunpak.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual netunpak +</title> +</head> +<body> +<H1> netunpak +</H1> +<h2> +Purpose +</h2> +Separates weights vector into weight and bias matrices. + +<p><h2> +Synopsis +</h2> +<PRE> +net = netunpak(net, w) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = netunpak(net, w)</CODE> takes an net network data structure <CODE>net</CODE> and +a weight vector <CODE>w</CODE>, and returns a network data structure identical to +the input network, except that the componenet weight matrices have all +been set to the corresponding elements of <CODE>w</CODE>. If there is +a <CODE>mask</CODE> field in the <CODE>net</CODE> data structure, then the weights in +<CODE>w</CODE> are placed in locations corresponding to non-zero entries in the +mask (so <CODE>w</CODE> should have the same length as the number of non-zero +entries in the <CODE>mask</CODE>). + +<p><h2> +See Also +</h2> +<CODE><a href="netpak.htm">netpak</a></CODE>, <CODE><a href="netfwd.htm">netfwd</a></CODE>, <CODE><a href="neterr.htm">neterr</a></CODE>, <CODE><a href="netgrad.htm">netgrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/olgd.htm b/sourcecodes/bnt-master/nethelp3.3/olgd.htm new file mode 100644 index 00000000..15417b70 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/olgd.htm @@ -0,0 +1,101 @@ +<html> +<head> +<title> +Netlab Reference Manual olgd +</title> +</head> +<body> +<H1> olgd +</H1> +<h2> +Purpose +</h2> +On-line gradient descent optimization. + +<p><h2> +Description +</h2> +<CODE>[net, options, errlog, pointlog] = olgd(net, options, x, t)</CODE> uses +on-line gradient descent to find a local minimum of the error function for the +network +<CODE>net</CODE> computed on the input data <CODE>x</CODE> and target values +<CODE>t</CODE>. A log of the error values +after each cycle is (optionally) returned in <CODE>errlog</CODE>, and a log +of the points visited is (optionally) returned in <CODE>pointlog</CODE>. +Because the gradient is computed on-line (i.e. after each pattern) +this can be quite inefficient in Matlab. + +<p>The error function value at final weight vector is returned +in <CODE>options(8)</CODE>. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>, and the points visited +in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is the precision required for the value +of <CODE>x</CODE> at the solution. If the absolute difference between +the values of <CODE>x</CODE> between two successive steps is less than +<CODE>options(2)</CODE>, then this condition is satisfied. + +<p><CODE>options(3)</CODE> is the precision required of the objective +function at the solution. If the absolute difference between the +error functions between two successive steps is less than +<CODE>options(3)</CODE>, then this condition is satisfied. +Both this and the previous condition must be +satisfied for termination. Note that testing the function value at each +iteration roughly halves the speed of the algorithm. + +<p><CODE>options(5)</CODE> determines whether the patterns are sampled randomly +with replacement. If it is 0 (the default), then patterns are sampled +in order. + +<p><CODE>options(6)</CODE> determines if the learning rate decays. If it is 1 +then the learning rate decays at a rate of <CODE>1/t</CODE>. If it is 0 +(the default) then the learning rate is constant. + +<p><CODE>options(9)</CODE> should be set to 1 to check the user defined gradient +function. + +<p><CODE>options(10)</CODE> returns the total number of function evaluations (including +those in any line searches). + +<p><CODE>options(11)</CODE> returns the total number of gradient evaluations. + +<p><CODE>options(14)</CODE> is the maximum number of iterations (passes through +the complete pattern set); default 100. + +<p><CODE>options(17)</CODE> is the momentum; default 0.5. + +<p><CODE>options(18)</CODE> is the learning rate; default 0.01. + +<p><h2> +Examples +</h2> +The following example performs on-line gradient descent on an MLP with +random sampling from the pattern set. +<PRE> + +net = mlp(5, 3, 1, 'linear'); +options = foptions; +options(18) = 0.01; +options(5) = 1; +net = olgd(net, options, x, t); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="graddesc.htm">graddesc</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/pca.htm b/sourcecodes/bnt-master/nethelp3.3/pca.htm new file mode 100644 index 00000000..84fe3e38 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/pca.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual pca +</title> +</head> +<body> +<H1> pca +</H1> +<h2> +Purpose +</h2> +Principal Components Analysis + +<p><h2> +Synopsis +</h2> +<PRE> +PCcoeff = pca(data) +PCcoeff = pca(data, N) +[PCcoeff, PCvec] = pca(data) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>PCcoeff = pca(data)</CODE> computes the eigenvalues of the covariance +matrix of the dataset <CODE>data</CODE> and returns them as <CODE>PCcoeff</CODE>. These +coefficients give the variance of <CODE>data</CODE> along the corresponding +principal components. + +<p><CODE>PCcoeff = pca(data, N)</CODE> returns the largest <CODE>N</CODE> eigenvalues. + +<p><CODE>[PCcoeff, PCvec] = pca(data)</CODE> returns the principal components as +well as the coefficients. This is considerably more computationally +demanding than just computing the eigenvalues. + +<p><h2> +See Also +</h2> +<CODE><a href="eigdec.htm">eigdec</a></CODE>, <CODE><a href="gtminit.htm">gtminit</a></CODE>, <CODE><a href="ppca.htm">ppca</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/plotmat.htm b/sourcecodes/bnt-master/nethelp3.3/plotmat.htm new file mode 100644 index 00000000..a9a38cc9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/plotmat.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual plotmat +</title> +</head> +<body> +<H1> plotmat +</H1> +<h2> +Purpose +</h2> +Display a matrix. + +<p><h2> +Synopsis +</h2> +<PRE> +plotmat(matrix, textcolour, gridcolour, fontsize)</PRE> + + +<p><h2> +Description +</h2> +<CODE>plotmat(matrix, textcolour, gridcolour, fontsize)</CODE> displays the matrix +<CODE>matrix</CODE> on the current figure. The <CODE>textcolour</CODE> and <CODE>gridcolour</CODE> +arguments control the colours of the numbers and grid labels respectively and +should follow the usual Matlab specification. +The parameter <CODE>fontsize</CODE> should be an integer. + +<p><h2> +See Also +</h2> +<CODE><a href="conffig.htm">conffig</a></CODE>, <CODE><a href="demmlp2.htm">demmlp2</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/ppca.htm b/sourcecodes/bnt-master/nethelp3.3/ppca.htm new file mode 100644 index 00000000..b55a61c4 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/ppca.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual ppca +</title> +</head> +<body> +<H1> ppca +</H1> +<h2> +Purpose +</h2> +Probabilistic Principal Components Analysis + +<p><h2> +Synopsis +</h2> +<PRE> +[var, U, lambda] = pca(x, ppca_dim) +</PRE> + + +<p><h2> +Description +</h2> + +<CODE>[var, U, lambda] = ppca(x, ppca_dim)</CODE> computes the principal component +subspace <CODE>U</CODE> of dimension <CODE>ppca_dim</CODE> using a centred +covariance matrix <CODE>x</CODE>. The variable <CODE>var</CODE> contains +the off-subspace variance (which is assumed to be spherical), while the +vector <CODE>lambda</CODE> contains the variances of each of the principal +components. This is computed using the eigenvalue and eigenvector +decomposition of <CODE>x</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="eigdec.htm">eigdec</a></CODE>, <CODE><a href="pca.htm">pca</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/quasinew.htm b/sourcecodes/bnt-master/nethelp3.3/quasinew.htm new file mode 100644 index 00000000..528ecb1d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/quasinew.htm @@ -0,0 +1,99 @@ +<html> +<head> +<title> +Netlab Reference Manual quasinew +</title> +</head> +<body> +<H1> quasinew +</H1> +<h2> +Purpose +</h2> +Quasi-Newton optimization. + +<p><h2> +Description +</h2> +<CODE>[x, options, flog, pointlog] = quasinew(f, x, options, gradf)</CODE> +uses a quasi-Newton +algorithm to find a local minimum of the function <CODE>f(x)</CODE> whose +gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector +and <CODE>f</CODE> returns a scalar value. +The point at which <CODE>f</CODE> has a local minimum +is returned as <CODE>x</CODE>. The function value at that point is returned +in <CODE>options(8)</CODE>. A log of the function values +after each cycle is (optionally) returned in <CODE>flog</CODE>, and a log +of the points visited is (optionally) returned in <CODE>pointlog</CODE>. + +<p><CODE>quasinew(f, x, options, gradf, p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>, and the points visited +in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value +of <CODE>x</CODE> at the solution. If the absolute difference between +the values of <CODE>x</CODE> between two successive steps is less than +<CODE>options(2)</CODE>, then this condition is satisfied. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. If the absolute difference between the +objective function values between two successive steps is less than +<CODE>options(3)</CODE>, then this condition is satisfied. +Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(9)</CODE> should be set to 1 to check the user defined gradient +function. + +<p><CODE>options(10)</CODE> returns the total number of function evaluations (including +those in any line searches). + +<p><CODE>options(11)</CODE> returns the total number of gradient evaluations. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p><CODE>options(15)</CODE> is the precision in parameter space of the line search; +default <CODE>1e-2</CODE>. + +<p><h2> +Examples +</h2> +An example of +the use of the additional arguments is the minimization of an error +function for a neural network: +<PRE> + +w = quasinew('neterr', w, options, 'netgrad', net, x, t); +</PRE> + + +<p><h2> +Algorithm +</h2> + +The quasi-Newton algorithm builds up an +approximation to the inverse Hessian over a number of steps. The +method requires order W squared storage, where W is the number of function +parameters. The Broyden-Fletcher-Goldfarb-Shanno formula for the +inverse Hessian updates is used. The line searches are carried out to +a relatively low precision (1.0e-2). + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbf.htm b/sourcecodes/bnt-master/nethelp3.3/rbf.htm new file mode 100644 index 00000000..97a7f925 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbf.htm @@ -0,0 +1,114 @@ +<html> +<head> +<title> +Netlab Reference Manual rbf +</title> +</head> +<body> +<H1> rbf +</H1> +<h2> +Purpose +</h2> +Creates an RBF network with specified architecture + +<p><h2> +Synopsis +</h2> +<PRE> + +net = rbf(nin, nhidden, nout, rbfunc) +net = rbf(nin, nhidden, nout, rbfunc, outfunc) +net = rbf(nin, nhidden, nout, rbfunc, outfunc, prior, beta) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = rbf(nin, nhidden, nout, rbfunc)</CODE> constructs and initialises +a radial basis function network returning a data structure <CODE>net</CODE>. +The weights are all initialised with a zero mean, unit variance normal +distribution, with the exception of the variances, which are set to one. +This makes use of the Matlab function +<CODE>randn</CODE> and so the seed for the random weight initialization can be +set using <CODE>randn('state', s)</CODE> where <CODE>s</CODE> is the seed value. The +activation functions are defined in terms of the distance between +the data point and the corresponding centre. Note that the functions are +computed to a convenient constant multiple: for example, the Gaussian +is not normalised. (Normalisation is not needed as the function outputs +are linearly combined in the next layer.) + +<p>The fields in <CODE>net</CODE> are +<PRE> + + type = 'rbf' + nin = number of inputs + nhidden = number of hidden units + nout = number of outputs + nwts = total number of weights and biases + actfn = string defining hidden unit activation function: + 'gaussian' for a radially symmetric Gaussian function. + 'tps' for r^2 log r, the thin plate spline function. + 'r4logr' for r^4 log r. + outfn = string defining output error function: + 'linear' for linear outputs (default) and SoS error. + 'neuroscale' for Sammon stress measure. + c = centres + wi = squared widths (null for rlogr and tps) + w2 = second layer weight matrix + b2 = second layer bias vector +</PRE> + + +<p><CODE>net = rbf(nin, nhidden, nout, rbfund, outfunc)</CODE> allows the user to +specify the type of error function to be used. The field <CODE>outfn</CODE> +is set to the value of this string. Linear outputs (for regression problems) +and Neuroscale outputs (for topographic mappings) are supported. + +<p><CODE>net = rbf(nin, nhidden, nout, rbfunc, outfunc, prior, beta)</CODE>, +in which <CODE>prior</CODE> is +a scalar, allows the field <CODE>net.alpha</CODE> in the data structure +<CODE>net</CODE> to be set, corresponding to a zero-mean isotropic Gaussian +prior with inverse variance with value <CODE>prior</CODE>. Alternatively, +<CODE>prior</CODE> can consist of a data structure with fields <CODE>alpha</CODE> +and <CODE>index</CODE>, allowing individual Gaussian priors to be set over +groups of weights in the network. Here <CODE>alpha</CODE> is a column vector +in which each element corresponds to a separate group of weights, +which need not be mutually exclusive. The membership of the groups is +defined by the matrix <CODE>indx</CODE> in which the columns correspond to +the elements of <CODE>alpha</CODE>. Each column has one element for each +weight in the matrix, in the order defined by the function +<CODE>rbfpak</CODE>, and each element is 1 or 0 according to whether the +weight is a member of the corresponding group or not. A utility +function <CODE>rbfprior</CODE> is provided to help in setting up the +<CODE>prior</CODE> data structure. + +<p><CODE>net = rbf(nin, nhidden, nout, func, prior, beta)</CODE> also sets the +additional field <CODE>net.beta</CODE> in the data structure <CODE>net</CODE>, where +beta corresponds to the inverse noise variance. + +<p><h2> +Example +</h2> +The following code constructs an RBF network with 1 input and output node +and 5 hidden nodes and then propagates some data <CODE>x</CODE> through it. +<PRE> + +net = rbf(1, 5, 1, 'tps'); +[y, act] = rbffwd(net, x); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="rbferr.htm">rbferr</a></CODE>, <CODE><a href="rbffwd.htm">rbffwd</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbftrain.htm">rbftrain</a></CODE>, <CODE><a href="rbfunpak.htm">rbfunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfbkp.htm b/sourcecodes/bnt-master/nethelp3.3/rbfbkp.htm new file mode 100644 index 00000000..cf3a9d52 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfbkp.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfbkp +</title> +</head> +<body> +<H1> rbfbkp +</H1> +<h2> +Purpose +</h2> +Backpropagate gradient of error function for RBF network. + +<p><h2> +Synopsis +</h2> +<PRE> +g = rbfbkp(net, x, z, n2, deltas)</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = rbfbkp(net, x, z, n2, deltas)</CODE> takes a network data structure +<CODE>net</CODE> together with a matrix <CODE>x</CODE> of input vectors, a matrix +<CODE>z</CODE> of hidden unit activations, a matrix <CODE>n2</CODE> of the squared +distances between centres and inputs, and a matrix <CODE>deltas</CODE> of the +gradient of the error function with respect to the values of the +output units (i.e. the summed inputs to the output units, before the +activation function is applied). The return value is the gradient +<CODE>g</CODE> of the error function with respect to the network +weights. Each row of <CODE>x</CODE> corresponds to one input vector. + +<p>This function is provided so that the common backpropagation algorithm +can be used by RBF network models to compute +gradients for the output values (in <CODE>rbfderiv</CODE>) as well as standard error +functions. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfderiv.htm">rbfderiv</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfderiv.htm b/sourcecodes/bnt-master/nethelp3.3/rbfderiv.htm new file mode 100644 index 00000000..f9c0438a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfderiv.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfderiv +</title> +</head> +<body> +<H1> rbfderiv +</H1> +<h2> +Purpose +</h2> +Evaluate derivatives of RBF network outputs with respect to weights. + +<p><h2> +Synopsis +</h2> +<PRE> +g = rbfderiv(net, x)</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = rbfderiv(net, x)</CODE> takes a network data structure <CODE>net</CODE> +and a matrix of input vectors <CODE>x</CODE> and returns a three-index matrix +<CODE>g</CODE> whose <CODE>i</CODE>, <CODE>j</CODE>, <CODE>k</CODE> element contains the +derivative of network output <CODE>k</CODE> with respect to weight or bias +parameter <CODE>j</CODE> for input pattern <CODE>i</CODE>. The ordering of the +weight and bias parameters is defined by <CODE>rbfunpak</CODE>. This +function also takes into account any mask in the network data structure. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfbkp.htm">rbfbkp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbferr.htm b/sourcecodes/bnt-master/nethelp3.3/rbferr.htm new file mode 100644 index 00000000..5a97de46 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbferr.htm @@ -0,0 +1,50 @@ +<html> +<head> +<title> +Netlab Reference Manual rbferr +</title> +</head> +<body> +<H1> rbferr +</H1> +<h2> +Purpose +</h2> +Evaluate error function for RBF network. + +<p><h2> +Synopsis +</h2> +<PRE> +e = rbferr(net, x, t) +[e, edata, eprior] = rbferr(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>e = rbferr(net, x, t)</CODE> takes a network data structure <CODE>net</CODE> together +with a matrix <CODE>x</CODE> of input +vectors and a matrix <CODE>t</CODE> of target vectors, and evaluates the +appropriate error function <CODE>e</CODE> depending on <CODE>net.outfn</CODE>. +Each row of <CODE>x</CODE> corresponds to one +input vector and each row of <CODE>t</CODE> contains the corresponding target vector. + +<p><CODE>[e, edata, eprior] = rbferr(net, x, t)</CODE> additionally returns the +data and prior components of the error, assuming a zero mean Gaussian +prior on the weights with inverse variance parameters <CODE>alpha</CODE> and +<CODE>beta</CODE> taken from the network data structure <CODE>net</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbffwd.htm">rbffwd</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbftrain.htm">rbftrain</a></CODE>, <CODE><a href="rbfunpak.htm">rbfunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/rbfevfwd.htm new file mode 100644 index 00000000..21513c59 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfevfwd.htm @@ -0,0 +1,51 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfevfwd +</title> +</head> +<body> +<H1> rbfevfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation with evidence for RBF + +<p><h2> +Synopsis +</h2> +<PRE> + +[y, extra] = rbfevfwd(net, x, t, x_test) +[y, extra, invhess] = rbfevfwd(net, x, t, x_test, invhess) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = rbfevfwd(net, x, t, x_test)</CODE> takes a network data structure +<CODE>net</CODE> together with the input <CODE>x</CODE> and target <CODE>t</CODE> training data +and input test data <CODE>x_test</CODE>. +It returns the normal forward propagation through the network <CODE>y</CODE> +together with a matrix <CODE>extra</CODE> which consists of error bars (variance) +for a regression problem or moderated outputs for a classification problem. + +<p>The optional argument (and return value) +<CODE>invhess</CODE> is the inverse of the network Hessian +computed on the training data inputs and targets. Passing it in avoids +recomputing it, which can be a significant saving for large training sets. + +<p><h2> +See Also +</h2> +<CODE><a href="fevbayes.htm">fevbayes</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbffwd.htm b/sourcecodes/bnt-master/nethelp3.3/rbffwd.htm new file mode 100644 index 00000000..02e56931 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbffwd.htm @@ -0,0 +1,67 @@ +<html> +<head> +<title> +Netlab Reference Manual rbffwd +</title> +</head> +<body> +<H1> rbffwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through RBF network with linear outputs. + +<p><h2> +Synopsis +</h2> +<PRE> +a = rbffwd(net, x) +function [a, z, n2] = rbffwd(net, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>a = rbffwd(net, x)</CODE> takes a network data structure +<CODE>net</CODE> and a matrix <CODE>x</CODE> of input +vectors and forward propagates the inputs through the network to generate +a matrix <CODE>a</CODE> of output vectors. Each row of <CODE>x</CODE> corresponds to one +input vector and each row of <CODE>a</CODE> contains the corresponding output vector. +The activation function that is used is determined by <CODE>net.actfn</CODE>. + +<p><CODE>[a, z, n2] = rbffwd(net, x)</CODE> also generates a matrix <CODE>z</CODE> of +the hidden unit activations where each row corresponds to one pattern. +These hidden unit activations represent the <CODE>design matrix</CODE> for +the RBF. The matrix <CODE>n2</CODE> is the squared distances between each +basis function centre and each pattern in which each row corresponds +to a data point. + +<p><h2> +Examples +</h2> +<PRE> + +[a, z] = rbffwd(net, x); + +<p>temp = pinv([z ones(size(x, 1), 1)]) * t; +net.w2 = temp(1: nd(2), :); +net.b2 = temp(size(x, nd(2)) + 1, :); +</PRE> + +Here <CODE>x</CODE> is the input data, <CODE>t</CODE> are the target values, and we use the +pseudo-inverse to find the output weights and biases. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbferr.htm">rbferr</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbftrain.htm">rbftrain</a></CODE>, <CODE><a href="rbfunpak.htm">rbfunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfgrad.htm b/sourcecodes/bnt-master/nethelp3.3/rbfgrad.htm new file mode 100644 index 00000000..30759dfe --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfgrad.htm @@ -0,0 +1,55 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfgrad +</title> +</head> +<body> +<H1> rbfgrad +</H1> +<h2> +Purpose +</h2> +Evaluate gradient of error function for RBF network. + +<p><h2> +Synopsis +</h2> +<PRE> + +g = rbfgrad(net, x, t) +[g, gdata, gprior] = rbfgrad(net, x, t) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = rbfgrad(net, x, t)</CODE> takes a network data structure <CODE>net</CODE> +together with a matrix <CODE>x</CODE> of input +vectors and a matrix <CODE>t</CODE> of target vectors, and evaluates the gradient +<CODE>g</CODE> of the error function with respect to the network weights (i.e. +including the hidden unit parameters). The error +function is sum of squares. +Each row of <CODE>x</CODE> corresponds to one +input vector and each row of <CODE>t</CODE> contains the corresponding target vector. +If the output function is <CODE>'neuroscale'</CODE> then the gradient is only +computed for the output layer weights and biases. + +<p><CODE>[g, gdata, gprior] = rbfgrad(net, x, t)</CODE> also returns separately +the data and prior contributions to the gradient. In the case of +multiple groups in the prior, <CODE>gprior</CODE> is a matrix with a row +for each group and a column for each weight parameter. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbffwd.htm">rbffwd</a></CODE>, <CODE><a href="rbferr.htm">rbferr</a></CODE>, <CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbfunpak.htm">rbfunpak</a></CODE>, <CODE><a href="rbfbkp.htm">rbfbkp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfhess.htm b/sourcecodes/bnt-master/nethelp3.3/rbfhess.htm new file mode 100644 index 00000000..a9c6be79 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfhess.htm @@ -0,0 +1,72 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfhess +</title> +</head> +<body> +<H1> rbfhess +</H1> +<h2> +Purpose +</h2> +Evaluate the Hessian matrix for RBF network. + +<p><h2> +Synopsis +</h2> +<PRE> +h = rbfhess(net, x, t) +[h, hdata] = rbfhess(net, x, t) +h = rbfhess(net, x, t, hdata) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>h = rbfhess(net, x, t)</CODE> takes an RBF network data structure <CODE>net</CODE>, +a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of target +values and returns the full Hessian matrix <CODE>h</CODE> corresponding to +the second derivatives of the negative log posterior distribution, +evaluated for the current weight and bias values as defined by +<CODE>net</CODE>. Currently, the implementation only computes the +Hessian for the output layer weights. + +<p><CODE>[h, hdata] = rbfhess(net, x, t)</CODE> returns both the Hessian matrix +<CODE>h</CODE> and the contribution <CODE>hdata</CODE> arising from the data dependent +term in the Hessian. + +<p><CODE>h = rbfhess(net, x, t, hdata)</CODE> takes a network data structure +<CODE>net</CODE>, a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of +target values, together with the contribution <CODE>hdata</CODE> arising from +the data dependent term in the Hessian, and returns the full Hessian +matrix <CODE>h</CODE> corresponding to the second derivatives of the negative +log posterior distribution. This version saves computation time if +<CODE>hdata</CODE> has already been evaluated for the current weight and bias +values. + +<p><h2> +Example +</h2> +For the standard regression framework with a Gaussian conditional +distribution of target values given input values, and a simple +Gaussian prior over weights, the Hessian takes the form +<PRE> + + h = beta*hdata + alpha*I +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="mlphess.htm">mlphess</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfjacob.htm b/sourcecodes/bnt-master/nethelp3.3/rbfjacob.htm new file mode 100644 index 00000000..853e59a3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfjacob.htm @@ -0,0 +1,42 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfjacob +</title> +</head> +<body> +<H1> rbfjacob +</H1> +<h2> +Purpose +</h2> +Evaluate derivatives of RBF network outputs with respect to inputs. + +<p><h2> +Synopsis +</h2> +<PRE> +g = rbfjacob(net, x)</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = rbfjacob(net, x)</CODE> takes a network data structure <CODE>net</CODE> +and a matrix of input vectors <CODE>x</CODE> and returns a three-index matrix +<CODE>g</CODE> whose <CODE>i</CODE>, <CODE>j</CODE>, <CODE>k</CODE> element contains the +derivative of network output <CODE>k</CODE> with respect to input +parameter <CODE>j</CODE> for input pattern <CODE>i</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfbkp.htm">rbfbkp</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfpak.htm b/sourcecodes/bnt-master/nethelp3.3/rbfpak.htm new file mode 100644 index 00000000..08769b16 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfpak.htm @@ -0,0 +1,40 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfpak +</title> +</head> +<body> +<H1> rbfpak +</H1> +<h2> +Purpose +</h2> +Combines all the parameters in an RBF network into one weights vector. + +<p><h2> +Synopsis +</h2> +<PRE> +w = rbfpak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>w = rbfpak(net)</CODE> takes a network data structure <CODE>net</CODE> and combines +the component parameter matrices into a single row vector <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="rbfunpak.htm">rbfunpak</a></CODE>, <CODE><a href="rbf.htm">rbf</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfprior.htm b/sourcecodes/bnt-master/nethelp3.3/rbfprior.htm new file mode 100644 index 00000000..202bdde7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfprior.htm @@ -0,0 +1,59 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfprior +</title> +</head> +<body> +<H1> rbfprior +</H1> +<h2> +Purpose +</h2> +Create Gaussian prior and output layer mask for RBF. + +<p><h2> +Synopsis +</h2> +<PRE> +[mask, prior] = rbfprior(rbfunc, nin, nhidden, nout, aw2, ab2)</PRE> + + +<p><h2> +Description +</h2> +<CODE>[mask, prior] = rbfprior(rbfunc, nin, nhidden, nout, aw2, ab2)</CODE> +generates a vector +<CODE>mask</CODE> that selects only the output +layer weights. This is because most uses of RBF networks in a Bayesian +context have fixed basis functions with the output layer as the only +adjustable parameters. In particular, the Neuroscale output error function +is designed to work only with this mask. + +<p>The return value +<CODE>prior</CODE> is a data structure, +with fields <CODE>prior.alpha</CODE> and <CODE>prior.index</CODE>, which +specifies a Gaussian prior distribution for the network weights in an +RBF network. The parameters <CODE>aw2</CODE> and <CODE>ab2</CODE> are all +scalars and represent the regularization coefficients for two groups +of parameters in the network corresponding to + second-layer weights, and second-layer biases +respectively. Then <CODE>prior.alpha</CODE> represents a column vector of +length 2 containing the parameters, and <CODE>prior.index</CODE> is a matrix +specifying which weights belong in each group. Each column has one +element for each weight in the matrix, using the standard ordering as +defined in <CODE>rbfpak</CODE>, and each element is 1 or 0 according to +whether the weight is a member of the corresponding group or not. + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbferr.htm">rbferr</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfsetbf.htm b/sourcecodes/bnt-master/nethelp3.3/rbfsetbf.htm new file mode 100644 index 00000000..c4fbf443 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfsetbf.htm @@ -0,0 +1,43 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfsetbf +</title> +</head> +<body> +<H1> rbfsetbf +</H1> +<h2> +Purpose +</h2> +Set basis functions of RBF from data. + +<p><h2> +Synopsis +</h2> +<PRE> +net = rbfsetbf(net, options, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = rbfsetbf(net, options, x)</CODE> sets the basis functions of the +RBF network <CODE>net</CODE> so that they model the unconditional density of the +dataset <CODE>x</CODE>. This is done by training a GMM with spherical covariances +using <CODE>gmmem</CODE>. The <CODE>options</CODE> vector is passed to <CODE>gmmem</CODE>. +The widths of the functions are set by a call to <CODE>rbfsetfw</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="rbftrain.htm">rbftrain</a></CODE>, <CODE><a href="rbfsetfw.htm">rbfsetfw</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfsetfw.htm b/sourcecodes/bnt-master/nethelp3.3/rbfsetfw.htm new file mode 100644 index 00000000..ae3fab56 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfsetfw.htm @@ -0,0 +1,46 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfsetfw +</title> +</head> +<body> +<H1> rbfsetfw +</H1> +<h2> +Purpose +</h2> +Set basis function widths of RBF. + +<p><h2> +Synopsis +</h2> +<PRE> +net = rbfsetfw(net, scale) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = rbfsetfw(net, scale)</CODE> sets the widths of +the basis functions of the +RBF network <CODE>net</CODE>. +If Gaussian basis functions are used, then the variances are set to +the largest squared distance between centres if <CODE>scale</CODE> is non-positive +and <CODE>scale</CODE> times the mean distance of each centre to its nearest +neighbour if <CODE>scale</CODE> is positive. Non-Gaussian basis functions do +not have a width. + +<p><h2> +See Also +</h2> +<CODE><a href="rbftrain.htm">rbftrain</a></CODE>, <CODE><a href="rbfsetbf.htm">rbfsetbf</a></CODE>, <CODE><a href="gmmem.htm">gmmem</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbftrain.htm b/sourcecodes/bnt-master/nethelp3.3/rbftrain.htm new file mode 100644 index 00000000..04c2bef5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbftrain.htm @@ -0,0 +1,90 @@ +<html> +<head> +<title> +Netlab Reference Manual rbftrain +</title> +</head> +<body> +<H1> rbftrain +</H1> +<h2> +Purpose +</h2> +Two stage training of RBF network. + +<p><h2> +Description +</h2> +<CODE>net = rbftrain(net, options, x, t)</CODE> uses a +two stage training +algorithm to set the weights in the RBF model structure <CODE>net</CODE>. +Each row of <CODE>x</CODE> corresponds to one +input vector and each row of <CODE>t</CODE> contains the corresponding target vector. +The centres are determined by fitting a Gaussian mixture model +with circular covariances using the EM algorithm through a call to +<CODE>rbfsetbf</CODE>. (The mixture model is +initialised using a small number of iterations of the K-means algorithm.) +If the activation functions are Gaussians, then the basis function widths +are then set to the maximum inter-centre squared distance. + +<p>For linear outputs, +the hidden to output +weights that give rise to the least squares solution +can then be determined using the pseudo-inverse. For neuroscale outputs, +the hidden to output weights are determined using the iterative shadow +targets algorithm. + Although this two stage +procedure may not give solutions with as low an error as using general +purpose non-linear optimisers, it is much faster. + +<p>The options vector may have two rows: if this is the case, then the second row +is passed to <CODE>rbfsetbf</CODE>, which allows the user to specify a different +number iterations for RBF and GMM training. +The optional parameters to <CODE>rbftrain</CODE> have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values during EM training. + +<p><CODE>options(2)</CODE> is a measure of the precision required for the value +of the weights <CODE>w</CODE> at the solution. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(5)</CODE> is set to 1 if the basis functions parameters should remain +unchanged; default 0. + +<p><CODE>options(6)</CODE> is set to 1 if the output layer weights should be should +set using PCA. This is only relevant for Neuroscale outputs; default 0. + +<p><CODE>options(14)</CODE> is the maximum number of iterations for the shadow +targets algorithm; +default 100. + +<p><h2> +Example +</h2> +The following example creates an RBF network and then trains it: +<PRE> + +net = rbf(1, 4, 1, 'gaussian'); +options(1, :) = foptions; +options(2, :) = foptions; +options(2, 14) = 10; % 10 iterations of EM +options(2, 5) = 1; % Check for covariance collapse in EM +net = rbftrain(net, options, x, t); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="rbf.htm">rbf</a></CODE>, <CODE><a href="rbferr.htm">rbferr</a></CODE>, <CODE><a href="rbffwd.htm">rbffwd</a></CODE>, <CODE><a href="rbfgrad.htm">rbfgrad</a></CODE>, <CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbfunpak.htm">rbfunpak</a></CODE>, <CODE><a href="rbfsetbf.htm">rbfsetbf</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfunpak.htm b/sourcecodes/bnt-master/nethelp3.3/rbfunpak.htm new file mode 100644 index 00000000..20464916 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfunpak.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual rbfunpak +</title> +</head> +<body> +<H1> rbfunpak +</H1> +<h2> +Purpose +</h2> +Separates a vector of RBF weights into its components. + +<p><h2> +Synopsis +</h2> +<PRE> +net = rbfunpak(net, w) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = rbfunpak(net, w)</CODE> takes an RBF network data structure <CODE>net</CODE> and +a weight vector <CODE>w</CODE>, and returns a network data structure identical to +the input network, except that the centres +<CODE>c</CODE>, the widths <CODE>wi</CODE>, the second-layer +weight matrix <CODE>w2</CODE> and the second-layer bias vector <CODE>b2</CODE> have all +been set to the corresponding elements of <CODE>w</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="rbfpak.htm">rbfpak</a></CODE>, <CODE><a href="rbf.htm">rbf</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rosegrad.htm b/sourcecodes/bnt-master/nethelp3.3/rosegrad.htm new file mode 100644 index 00000000..fc4fa9d6 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rosegrad.htm @@ -0,0 +1,40 @@ +<html> +<head> +<title> +Netlab Reference Manual rosegrad +</title> +</head> +<body> +<H1> rosegrad +</H1> +<h2> +Purpose +</h2> +Calculate gradient of Rosenbrock's function. + +<p><h2> +Synopsis +</h2> +<PRE> +g = rosegrad(x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>g = rosegrad(x)</CODE> computes the gradient of Rosenbrock's function +at each row of <CODE>x</CODE>, which should have two columns. + +<p><h2> +See Also +</h2> +<CODE><a href="demopt1.htm">demopt1</a></CODE>, <CODE><a href="rosen.htm">rosen</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rosen.htm b/sourcecodes/bnt-master/nethelp3.3/rosen.htm new file mode 100644 index 00000000..2b8f37aa --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rosen.htm @@ -0,0 +1,40 @@ +<html> +<head> +<title> +Netlab Reference Manual rosen +</title> +</head> +<body> +<H1> rosen +</H1> +<h2> +Purpose +</h2> +Calculate Rosenbrock's function. + +<p><h2> +Synopsis +</h2> +<PRE> +y = rosen(x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>y = rosen(x)</CODE> computes the value of Rosenbrock's function +at each row of <CODE>x</CODE>, which should have two columns. + +<p><h2> +See Also +</h2> +<CODE><a href="demopt1.htm">demopt1</a></CODE>, <CODE><a href="rosegrad.htm">rosegrad</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/scg.htm b/sourcecodes/bnt-master/nethelp3.3/scg.htm new file mode 100644 index 00000000..e2e512e2 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/scg.htm @@ -0,0 +1,97 @@ +<html> +<head> +<title> +Netlab Reference Manual scg +</title> +</head> +<body> +<H1> scg +</H1> +<h2> +Purpose +</h2> +Scaled conjugate gradient optimization. + +<p><h2> +Description +</h2> +<CODE>[x, options] = scg(f, x, options, gradf)</CODE> uses a scaled conjugate +gradients +algorithm to find a local minimum of the function <CODE>f(x)</CODE> whose +gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector +and <CODE>f</CODE> returns a scalar value. +The point at which <CODE>f</CODE> has a local minimum +is returned as <CODE>x</CODE>. The function value at that point is returned +in <CODE>options(8)</CODE>. + +<p><CODE>[x, options, flog, pointlog, scalelog] = scg(f, x, options, gradf)</CODE> +also returns (optionally) a log of the function values +after each cycle in <CODE>flog</CODE>, a log +of the points visited in <CODE>pointlog</CODE>, and a log of the scale values +in the algorithm in <CODE>scalelog</CODE>. + +<p><CODE>scg(f, x, options, gradf, p1, p2, ...)</CODE> allows +additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>. + +The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error +values in the return argument <CODE>errlog</CODE>, and the points visited +in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0, +then only warning messages are displayed. If <CODE>options(1)</CODE> is -1, +then nothing is displayed. + +<p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value +of <CODE>x</CODE> at the solution. If the absolute difference between +the values of <CODE>x</CODE> between two successive steps is less than +<CODE>options(2)</CODE>, then this condition is satisfied. + +<p><CODE>options(3)</CODE> is a measure of the precision required of the objective +function at the solution. If the absolute difference between the +objective function values between two successive steps is less than +<CODE>options(3)</CODE>, then this condition is satisfied. +Both this and the previous condition must be +satisfied for termination. + +<p><CODE>options(9)</CODE> is set to 1 to check the user defined gradient function. + +<p><CODE>options(10)</CODE> returns the total number of function evaluations (including +those in any line searches). + +<p><CODE>options(11)</CODE> returns the total number of gradient evaluations. + +<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100. + +<p><h2> +Examples +</h2> +An example of +the use of the additional arguments is the minimization of an error +function for a neural network: +<PRE> + +w = scg('neterr', w, options, 'netgrad', net, x, t); +</PRE> + + +<p><h2> +Algorithm +</h2> +The search direction is re-started after every <CODE>nparams</CODE> +successful weight updates where <CODE>nparams</CODE> is the total number of +parameters in <CODE>x</CODE>. The algorithm is based on that given by Williams +(1991), with a simplified procedure for updating <CODE>lambda</CODE> when +<CODE>rho < 0.25</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/som.htm b/sourcecodes/bnt-master/nethelp3.3/som.htm new file mode 100644 index 00000000..637e752a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/som.htm @@ -0,0 +1,58 @@ +<html> +<head> +<title> +Netlab Reference Manual som +</title> +</head> +<body> +<H1> som +</H1> +<h2> +Purpose +</h2> +Creates a Self-Organising Map. + +<p><h2> +Synopsis +</h2> +<PRE> + +net = som(nin, map_size) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = som(nin, map_size)</CODE> creates a SOM <CODE>net</CODE> +with input dimension (i.e. data dimension) <CODE>nin</CODE> and map dimensions +<CODE>map_size</CODE>. Only two-dimensional maps are currently implemented. + +<p>The fields in <CODE>net</CODE> are +<PRE> + + type = 'som' + nin = number of inputs + map_dim = dimension of map (constrained to be 2) + map_size = grid size: number of nodes in each dimension + num_nodes = number of nodes: the product of values in map_size + map = map_dim+1 dimensional array containing nodes + inode_dist = map of inter-node distances using Manhatten metric +</PRE> + + +<p>The map contains the node vectors arranged column-wise in the first +dimension of the array. + +<p><h2> +See Also +</h2> +<CODE><a href="kmeans.htm">kmeans</a></CODE>, <CODE><a href="somfwd.htm">somfwd</a></CODE>, <CODE><a href="somtrain.htm">somtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/somfwd.htm b/sourcecodes/bnt-master/nethelp3.3/somfwd.htm new file mode 100644 index 00000000..82102b4c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/somfwd.htm @@ -0,0 +1,59 @@ +<html> +<head> +<title> +Netlab Reference Manual somfwd +</title> +</head> +<body> +<H1> somfwd +</H1> +<h2> +Purpose +</h2> +Forward propagation through a Self-Organising Map. + +<p><h2> +Synopsis +</h2> +<PRE> + +d2 = somfwd(net, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>d2 = somfwd(net, x)</CODE> propagates the data matrix <CODE>x</CODE> through + a SOM <CODE>net</CODE>, returning the squared distance matrix <CODE>d2</CODE> with +dimension <CODE>nin</CODE> by <CODE>num_nodes</CODE>. The $i$th row represents the +squared Euclidean distance to each of the nodes of the SOM. + +<p><CODE>[d2, win_nodes] = somfwd(net, x)</CODE> also returns the indices of the +winning nodes for each pattern. + +<p><h2> +Example +</h2> + +<p>The following code fragment creates a SOM with a $5times 5$ map for an +8-dimensional data space. It then applies the test data to the map. +<PRE> + +net = som(8, [5, 5]); +[d2, wn] = somfwd(net, test_data); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="som.htm">som</a></CODE>, <CODE><a href="somtrain.htm">somtrain</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/sompak.htm b/sourcecodes/bnt-master/nethelp3.3/sompak.htm new file mode 100644 index 00000000..902ef52e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/sompak.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual sompak +</title> +</head> +<body> +<H1> sompak +</H1> +<h2> +Purpose +</h2> +Combines node weights into one weights matrix. + +<p><h2> +Synopsis +</h2> +<PRE> +c = sompak(net) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>c = sompak(net)</CODE> takes a SOM data structure <CODE>net</CODE> and +combines the node weights into a matrix of centres +<CODE>c</CODE> where each row represents the node vector. + +<p>The ordering of the parameters in <CODE>w</CODE> is defined by the indexing of the +multi-dimensional array <CODE>net.map</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="som.htm">som</a></CODE>, <CODE><a href="somunpak.htm">somunpak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/somtrain.htm b/sourcecodes/bnt-master/nethelp3.3/somtrain.htm new file mode 100644 index 00000000..ecf4e2cb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/somtrain.htm @@ -0,0 +1,104 @@ +<html> +<head> +<title> +Netlab Reference Manual somtrain +</title> +</head> +<body> +<H1> somtrain +</H1> +<h2> +Purpose +</h2> +Kohonen training algorithm for SOM. + +<p><h2> +Synopsis +</h2> +<PRE> + +net = somtrain{net, options, x) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = somtrain{net, options, x)</CODE> uses Kohonen's algorithm to +train a SOM. Both on-line and batch algorithms are implemented. +The learning rate (for on-line) and neighbourhood size decay linearly. +There is no error function minimised during training (so there is +no termination criterion other than the number of epochs), but the +sum-of-squares is computed and returned in <CODE>options(8)</CODE>. + +<p>The optional parameters have the following interpretations. + +<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs learning +rate <CODE>alpha</CODE> and neighbourhood size <CODE>nsize</CODE>. +Otherwise nothing is displayed. + +<p><CODE>options(5)</CODE> determines whether the patterns are sampled randomly +with replacement. If it is 0 (the default), then patterns are sampled +in order. This is only relevant to the on-line algorithm. + +<p><CODE>options(6)</CODE> determines if the on-line or batch algorithm is +used. If it is 1 +then the batch algorithm is used. If it is 0 +(the default) then the on-line algorithm is used. + +<p><CODE>options(14)</CODE> is the maximum number of iterations (passes through +the complete pattern set); default 100. + +<p><CODE>options(15)</CODE> is the final neighbourhood size; default value is the +same as the initial neighbourhood size. + +<p><CODE>options(16)</CODE> is the final learning rate; default value is the same +as the initial learning rate. + +<p><CODE>options(17)</CODE> is the initial neighbourhood size; default 0.5*maximum +map size. + +<p><CODE>options(18)</CODE> is the initial learning rate; default 0.9. This parameter +must be positive. + +<p><h2> +Examples +</h2> +The following example performs on-line training on a SOM in two stages: +ordering and convergence. +<PRE> + +net = som(nin, [8, 7]); +options = foptions; + +<p>% Ordering phase +options(1) = 1; +options(14) = 50; +options(18) = 0.9; % Initial learning rate +options(16) = 0.05; % Final learning rate +options(17) = 8; % Initial neighbourhood size +options(15) = 1; % Final neighbourhood size +net2 = somtrain(net, options, x); + +<p>% Convergence phase +options(14) = 400; +options(18) = 0.05; +options(16) = 0.01; +options(17) = 0; +options(15) = 0; +net3 = somtrain(net2, options, x); +</PRE> + + +<p><h2> +See Also +</h2> +<CODE><a href="kmeans.htm">kmeans</a></CODE>, <CODE><a href="som.htm">som</a></CODE>, <CODE><a href="somfwd.htm">somfwd</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/somunpak.htm b/sourcecodes/bnt-master/nethelp3.3/somunpak.htm new file mode 100644 index 00000000..68032339 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/somunpak.htm @@ -0,0 +1,44 @@ +<html> +<head> +<title> +Netlab Reference Manual somunpak +</title> +</head> +<body> +<H1> somunpak +</H1> +<h2> +Purpose +</h2> +Replaces node weights in SOM. + +<p><h2> +Synopsis +</h2> +<PRE> +net = somunpak(net, w) +</PRE> + + +<p><h2> +Description +</h2> +<CODE>net = somunpak(net, w)</CODE> takes a SOM data structure <CODE>net</CODE> and +weight matrix <CODE>w</CODE> (each node represented by a row) and +puts the nodes back into the multi-dimensional array <CODE>net.map</CODE>. + +<p>The ordering of the parameters in <CODE>w</CODE> is defined by the indexing of the +multi-dimensional array <CODE>net.map</CODE>. + +<p><h2> +See Also +</h2> +<CODE><a href="som.htm">som</a></CODE>, <CODE><a href="sompak.htm">sompak</a></CODE><hr> +<b>Pages:</b> +<a href="index.htm">Index</a> +<hr> +<p>Copyright (c) Ian T Nabney (1996-9) + + +</body> +</html> \ No newline at end of file |