+conffig(y, t) +fh = conffig(y, t)+ + +
conffig(y, t) displays the confusion matrix
+and classification performance for the predictions mat{y}
+compared with the targets t. The data is assumed to be in a
+1-of-N encoding, unless there is just one column, when it is assumed to
+be a 2 class problem with a 0-1 encoding. Each row of y and t
+corresponds to a single example.
+
+In the confusion matrix, the rows represent the true classes and the +columns the predicted classes. + +
fh = conffig(y, t) also returns the figure handle fh which
+can be used, for instance, to delete the figure when it is no longer needed.
+
+
confmat, demtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/confmat.htm b/sourcecodes/bnt-master/nethelp3.3/confmat.htm new file mode 100644 index 00000000..2162293d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/confmat.htm @@ -0,0 +1,48 @@ + +
++[C, rate] = confmat(y, t)+ + +
[C, rate] = confmat(y, t) computes the confusion matrix C
+and classification performance rate for the predictions mat{y}
+compared with the targets t. The data is assumed to be in a
+1-of-N encoding, unless there is just one column, when it is assumed to
+be a 2 class problem with a 0-1 encoding. Each row of y and t
+corresponds to a single example.
+
+In the confusion matrix, the rows represent the true classes and the
+columns the predicted classes. The vector rate has two entries:
+the percentage of correct classifications and the total number of
+correct classifications.
+
+
conffig, demtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/conjgrad.htm b/sourcecodes/bnt-master/nethelp3.3/conjgrad.htm new file mode 100644 index 00000000..4a70c330 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/conjgrad.htm @@ -0,0 +1,101 @@ + +
+[x, options, flog, pointlog] = conjgrad(f, x, options, gradf) uses a
+conjugate gradients
+algorithm to find the minimum of the function f(x) whose
+gradient is given by gradf(x). Here x is a row vector
+and f returns a scalar value.
+The point at which f has a local minimum
+is returned as x. The function value at that point is returned
+in options(8). A log of the function values
+after each cycle is (optionally) returned in flog, and a log
+of the points visited is (optionally) returned in pointlog.
+
+conjgrad(f, x, options, gradf, p1, p2, ...) allows
+additional arguments to be passed to f() and gradf().
+
+
The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog, and the points visited
+in the return argument pointslog. If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is a measure of the absolute precision required for the value
+of x at the solution. If the absolute difference between
+the values of x between two successive steps is less than
+options(2), then this condition is satisfied.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. If the absolute difference between the
+objective function values between two successive steps is less than
+options(3), then this condition is satisfied.
+Both this and the previous condition must be
+satisfied for termination.
+
+
options(9) is set to 1 to check the user defined gradient function.
+
+
options(10) returns the total number of function evaluations (including
+those in any line searches).
+
+
options(11) returns the total number of gradient evaluations.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
options(15) is the precision in parameter space of the line search;
+default 1e-4.
+
+
+
+w = quasinew('neterr', w, options, 'netgrad', net, x, t);
+
+
+
+di that are conjugate: i.e. di*H*d(i-1) = 0,
+where H is the Hessian matrix. This means that minimising along
+di does not undo the effect of minimising along the previous
+direction. The Polak-Ribiere formula is used to calculate new search
+directions. The Hessian is not calculated, so there is only an
+O(W) storage requirement (where W is the number of
+parameters). However, relatively accurate line searches must be used
+(default is 1e-04).
+
+graddesc, linemin, minbrack, quasinew, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/consist.htm b/sourcecodes/bnt-master/nethelp3.3/consist.htm new file mode 100644 index 00000000..40bc374c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/consist.htm @@ -0,0 +1,78 @@ + +
++errstring = consist(net, type, inputs, outputs) +errstring = consist(net, type, inputs) +errstring = consist(net, type) ++ + +
errstring = consist(net, type, inputs) takes a network
+data structure net together with a string type containing
+the correct network type, a matrix inputs of input vectors and checks
+that the data structure is consistent with the other arguments. An empty
+string is returned if there is no error, otherwise the string contains the
+relevant error message. If the type string is empty, then any
+type of network is allowed.
+
+
errstring = consist(net, type) takes a network data structure
+net together with a string type containing the correct
+network type, and checks that the two types match.
+
+
errstring = consist(net, type, inputs, outputs) also checks that the
+network has the correct number of outputs, and that the number of patterns
+in the inputs and outputs is the same. The fields in net
+that are used are
+
+ type + nin + nout ++ + +
mlpfwd, the function that propagates values forward through an MLP
+network, has the following check at the head of the file:
+
+ +errstring = consist(net, 'mlp', x, t); +if ~isempty(errstring) + error(errstring) +end ++ + +
mlpfwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/convertoldnet.htm b/sourcecodes/bnt-master/nethelp3.3/convertoldnet.htm new file mode 100644 index 00000000..c3266367 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/convertoldnet.htm @@ -0,0 +1,43 @@ + +
++net = convertoldnet(net) ++ + +
net = convertoldnet(net) takes a network net and, if appropriate,
+converts it from pre-2.3 to the current format. The difference is simply
+that in MLPs and the MLP sub-net of MDNs the field actfn has been
+renamed outfn to make it consistent with GLM and RBF networks.
+If the network is not old-format or an MLP or MDN it is left unchanged.
+
+mlp, mdnCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/datread.htm b/sourcecodes/bnt-master/nethelp3.3/datread.htm new file mode 100644 index 00000000..78f087eb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/datread.htm @@ -0,0 +1,57 @@ + +
++[x, t, nin, nout, ndata] = datread(filename) ++ + +
[x, t, nin, nout, ndata] = datread(filename) reads from
+the file filename and returns a matrix x of input vectors,
+a matrix t of target vectors, and integers nin, nout
+and ndata specifying the number of inputs, the number of outputs
+and the number of data points respectively.
+
+
The format of the data file is as follows: the first row contains the
+string nin followed by the number of inputs, the second row
+contains the string nout followed by the number of outputs, and
+the third row contains the string ndata followed by the number
+of data vectors. Subsequent lines each contain one input vector
+followed by one output vector, with individual values separated by
+spaces.
+
+
nin2nout1ndata40.000000e+000.000000e+001.000000e+000.000000e+001.000000e+000.000000e+001.000000e+000.000000e+000.000000e+001.000000e+001.000000e+001.000000e+00SeeAlsodatwriteCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/datwrite.htm b/sourcecodes/bnt-master/nethelp3.3/datwrite.htm new file mode 100644 index 00000000..1006b911 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/datwrite.htm @@ -0,0 +1,65 @@ + +
++datwrite(filename, x, t) ++ + +
datwrite(filename, x, t) takes a matrix x of input vectors
+and a matrix t of target vectors and writes them to an ascii
+file named filename. The file format is as follows: the first
+row contains the string nin followed by the number of inputs,
+the second row contains the string nout followed by the number
+of outputs, and the third row contains the string ndata followed
+by the number of data vectors. Subsequent lines each contain one input
+vector followed by one output vector, with individual values separated
+by spaces.
+
+
+ + nin 2 + nout 1 + ndata 4 + 0.000000e+00 0.000000e+00 1.000000e+00 + 0.000000e+00 1.000000e+00 0.000000e+00 + 1.000000e+00 0.000000e+00 0.000000e+00 + 1.000000e+00 1.000000e+00 1.000000e+00 ++ + +
datreadCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/dem2ddat.htm b/sourcecodes/bnt-master/nethelp3.3/dem2ddat.htm new file mode 100644 index 00000000..5dd49ba5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/dem2ddat.htm @@ -0,0 +1,48 @@ + +
++data = dem2ddat(ndata)+ +
+[data, c] = dem2ddat(ndata)+ + +
data = dem2ddat(ndata) generates ndata
+points.
+
+[data, c] = dem2ddat(ndata) also returns a matrix containing the
+centres of the Gaussian distributions.
+
+
demgmm1, demkmean, demknn1Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demard.htm b/sourcecodes/bnt-master/nethelp3.3/demard.htm new file mode 100644 index 00000000..43923377 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demard.htm @@ -0,0 +1,51 @@ + +
++demmlp1+ + +
x1 is sampled uniformly from the range (0,1) and has
+a low level of added Gaussian noise, x2 is a copy of x1
+with a higher level of added noise, and x3 is sampled randomly
+from a Gaussian distribution. The single target variable is determined
+by sin(2*pi*x1) with additive Gaussian noise. Thus x1 is
+very relevant for determining the target value, x2 is of some
+relevance, while x3 is irrelevant. The prior over weights is
+given by the ARD Gaussian prior with a separate hyper-parameter for
+the group of weights associated with each input. A multi-layer
+perceptron is trained on this data, with re-estimation of the
+hyper-parameters using evidence. The final values for the
+hyper-parameters reflect the relative importance of the three inputs.
+
+demmlp1, demev1, mlp, evidenceCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demev1.htm b/sourcecodes/bnt-master/nethelp3.3/demev1.htm new file mode 100644 index 00000000..f7b05fbb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demev1.htm @@ -0,0 +1,46 @@ + +
++demev1+ + +
x which sampled from a
+Gaussian distribution, and a target variable t generated by
+computing sin(2*pi*x) and adding Gaussian noise. A 2-layer
+network with linear outputs is trained by minimizing a sum-of-squares
+error function with isotropic Gaussian regularizer, using the scaled
+conjugate gradient optimizer. The hyperparameters alpha and
+beta are re-estimated using the function evidence. A graph
+is plotted of the original function, the training data, the trained
+network function, and the error bars.
+
+evidence, mlp, scg, demard, demmlp1Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demev2.htm b/sourcecodes/bnt-master/nethelp3.3/demev2.htm new file mode 100644 index 00000000..c1db9e65 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demev2.htm @@ -0,0 +1,50 @@ + +
++demev2+ + +
x is sampled
+from a mixture of four Gaussians. Each class is
+associated with two of the Gaussians so that the optimal decision
+boundary is non-linear.
+A 2-layer
+network with logistic outputs is trained by minimizing the cross-entropy
+error function with isotroipc Gaussian regularizer (one hyperparameter for
+each of the four standard weight groups), using the scaled
+conjugate gradient optimizer. The hyperparameter vectors alpha and
+beta are re-estimated using the function evidence. A graph
+is plotted of the optimal, regularised, and unregularised decision
+boundaries. A further plot of the moderated versus unmoderated contours
+is generated.
+
+evidence, mlp, scg, demard, demmlp2Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demev3.htm b/sourcecodes/bnt-master/nethelp3.3/demev3.htm new file mode 100644 index 00000000..55af4d64 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demev3.htm @@ -0,0 +1,46 @@ + +
++demev3+ + +
x which sampled from a
+Gaussian distribution, and a target variable t generated by
+computing sin(2*pi*x) and adding Gaussian noise. An RBF
+network with linear outputs is trained by minimizing a sum-of-squares
+error function with isotropic Gaussian regularizer, using the scaled
+conjugate gradient optimizer. The hyperparameters alpha and
+beta are re-estimated using the function evidence. A graph
+is plotted of the original function, the training data, the trained
+network function, and the error bars.
+
+demev1, evidence, rbf, scg, netevfwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgauss.htm b/sourcecodes/bnt-master/nethelp3.3/demgauss.htm new file mode 100644 index 00000000..4ddc8059 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgauss.htm @@ -0,0 +1,50 @@ + +
++demgauss ++ + +
demgauss provides a simple illustration of the generation of
+data from Gaussian distributions. It first samples from a
+one-dimensional distribution using randn, and then plots a
+normalized histogram estimate of the distribution using histp
+together with the true density calculated using gauss.
+
+
demgauss then demonstrates sampling from a Gaussian distribution
+in two dimensions. It creates a mean vector and a covariance matrix,
+and then plots contours of constant density using the function
+gauss. A sample of points drawn from this distribution, obtained
+using the function gsamp, is then superimposed on the contours.
+
+
gauss, gsamp, histpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demglm1.htm b/sourcecodes/bnt-master/nethelp3.3/demglm1.htm new file mode 100644 index 00000000..7740b361 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demglm1.htm @@ -0,0 +1,44 @@ + +
++demglm1+ + +
data and a vector of classifications t. The data is
+generated from two Gaussian clusters, and a generalized linear model
+with logistic output is trained using iterative reweighted least squares.
+A plot of the data together with the 0.1, 0.5 and 0.9 contour lines
+of the conditional probability is generated.
+
+demglm2, glm, glmtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demglm2.htm b/sourcecodes/bnt-master/nethelp3.3/demglm2.htm new file mode 100644 index 00000000..274d5b61 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demglm2.htm @@ -0,0 +1,44 @@ + +
++demglm1+ + +
data and a vector of classifications t. The data is
+generated from three Gaussian clusters, and a generalized linear model
+with softmax output is trained using iterative reweighted least squares.
+A plot of the data together with regions shaded by the classification
+given by the network is generated.
+
+demglm1, glm, glmtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm1.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm1.htm new file mode 100644 index 00000000..7476becb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm1.htm @@ -0,0 +1,41 @@ + +
++demgmm1+ + +
demgmm2, demgmm3, demgmm4, gmm, gmmem, gmmpostCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm2.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm2.htm new file mode 100644 index 00000000..f8a3f61d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm2.htm @@ -0,0 +1,52 @@ + +
++demgmm1+ + +
A Gaussian mixture model with three components is trained using EM. The +parameter vector is printed before training and after training. The user +should press any key to continue at these points. The parameter vector +consists of priors (the column), centres (given as (x, y) pairs as +the next two columns), and variances (the last column). + +
The second figure is a 3 dimensional view of the density function, while +the third shows the 1-standard deviation circles for the three components of +the mixture model. + +
gmm, gmminit, gmmem, gmmprob, gmmunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm3.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm3.htm new file mode 100644 index 00000000..bbd7465c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm3.htm @@ -0,0 +1,55 @@ + +
++demgmm3+ + +
A Gaussian mixture model with three components is trained using EM. The +parameter vector is printed before training and after training. The user +should press any key to continue at these points. The parameter vector +consists of priors (the column), and centres (given as (x, y) pairs as +the next two columns). The diagonal entries of the +covariance matrices are printed separately. + +
The second figure is a 3 dimensional view of the density function, +while the third shows the axes of the 1-standard deviation circles +for the three components of the mixture model. + +
gmm, gmminit, gmmem, gmmprob, gmmunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm4.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm4.htm new file mode 100644 index 00000000..95f4bc87 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm4.htm @@ -0,0 +1,54 @@ + +
++demgmm4+ + +
A Gaussian mixture model with three components is trained using EM. The +parameter vector is printed before training and after training. The user +should press any key to continue at these points. The parameter vector +consists of priors (the column), and centres (given as (x, y) pairs as +the next two columns). The covariance matrices are printed separately. + +
The second figure is a 3 dimensional view of the density function, +while the third shows the axes of the 1-standard deviation ellipses +for the three components of the mixture model. + +
gmm, gmminit, gmmem, gmmprob, gmmunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgmm5.htm b/sourcecodes/bnt-master/nethelp3.3/demgmm5.htm new file mode 100644 index 00000000..ed3caf1e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgmm5.htm @@ -0,0 +1,55 @@ + +
++demgmm5+ + +
A mixture model with three one-dimensional PPCA components is trained +using EM. The parameter vector is printed before training and after +training. The parameter vector consists of priors (the column), and +centres (given as (x, y) pairs as the next two columns). + +
The second figure is a 3 dimensional view of the density function, +while the third shows the axes of the 1-standard deviation ellipses +for the three components of the mixture model together with the one +standard deviation along the principal component of each mixture +model component. + +
gmm, gmminit, gmmem, gmmprob, ppcaCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgp.htm b/sourcecodes/bnt-master/nethelp3.3/demgp.htm new file mode 100644 index 00000000..9103bfb8 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgp.htm @@ -0,0 +1,44 @@ + +
++demgp+ + +
x and one target variable
+t. The values in x are chosen in two separated clusters and the
+target data is generated by computing sin(2*pi*x) and adding Gaussian
+noise. Two Gaussian Processes, each with different covariance functions
+are trained by optimising the hyperparameters
+using the scaled conjugate gradient algorithm. The final predictions are
+plotted together with 2 standard deviation error bars.
+
+gp, gperr, gpfwd, gpgrad, gpinit, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgpard.htm b/sourcecodes/bnt-master/nethelp3.3/demgpard.htm new file mode 100644 index 00000000..eb188c19 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgpard.htm @@ -0,0 +1,48 @@ + +
++demgpare+ + +
x1, x2 and
+x3, and one target variable
+t. The
+target data is generated by computing sin(2*pi*x1) and adding Gaussian
+noise, x2 is a copy of x1 with a higher level of added
+noise, and x3 is sampled randomly from a Gaussian distribution.
+A Gaussian Process, is
+trained by optimising the hyperparameters
+using the scaled conjugate gradient algorithm. The final values of the
+hyperparameters show that the model successfully identifies the importance
+of each input.
+
+demgp, gp, gperr, gpfwd, gpgrad, gpinit, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgpot.htm b/sourcecodes/bnt-master/nethelp3.3/demgpot.htm new file mode 100644 index 00000000..368f222e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgpot.htm @@ -0,0 +1,44 @@ + +
++g = demgpot(x, mix)+ + +
p(x) with respect to the coefficients of the
+data vector x for a Gaussian mixture model. The data structure
+mix defines the mixture model, while the matrix x contains
+the data vector as a row vector. Note the unusual order of the arguments:
+this is so that the function can be used in demhmc1 directly for
+sampling from the distribution p(x).
+
+demhmc1, demmet1, dempotCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgtm1.htm b/sourcecodes/bnt-master/nethelp3.3/demgtm1.htm new file mode 100644 index 00000000..1422c764 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgtm1.htm @@ -0,0 +1,42 @@ + +
++demgtm1+ + +
demgtm2, gtm, gtmem, gtmpostCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demgtm2.htm b/sourcecodes/bnt-master/nethelp3.3/demgtm2.htm new file mode 100644 index 00000000..4696b152 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demgtm2.htm @@ -0,0 +1,43 @@ + +
++demgtm2+ + +
demgtm1, gtm, gtmem, gtmpostCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhint.htm b/sourcecodes/bnt-master/nethelp3.3/demhint.htm new file mode 100644 index 00000000..ac54f77a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhint.htm @@ -0,0 +1,45 @@ + +
++demhint +demhint(nin, nhidden, nout)+ + +
demhint plots a Hinton diagram for a 2-layer feedforward network
+with 5 inputs, 4 hidden units and 3 outputs. The weight vector is
+chosen from a Gaussian distribution as described under mlp.
+
+
demhint(nin, nhidden, nout) allows the user to specify the
+number of inputs, hidden units and outputs.
+
+
hinton, hintmat, mlp, mlppak, mlpunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhmc1.htm b/sourcecodes/bnt-master/nethelp3.3/demhmc1.htm new file mode 100644 index 00000000..4ad8b73b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhmc1.htm @@ -0,0 +1,41 @@ + +
++demhmc1+ + +
demhmc3, hmc, dempot, demgpotCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhmc2.htm b/sourcecodes/bnt-master/nethelp3.3/demhmc2.htm new file mode 100644 index 00000000..b5657cbe --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhmc2.htm @@ -0,0 +1,45 @@ + +
++demhmc2+ + +
x and one target variable
+t with data generated by sampling x at equal intervals and then
+generating target data by computing sin(2*pi*x) and adding Gaussian
+noise. The model is a 2-layer network with linear outputs, and the hybrid Monte
+Carlo algorithm (without persistence) is used to sample from the posterior
+distribution of the weights. The graph shows the underlying function,
+100 samples from the function given by the posterior distribution of the
+weights, and the average prediction (weighted by the posterior probabilities).
+
+demhmc3, hmc, mlp, mlperr, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demhmc3.htm b/sourcecodes/bnt-master/nethelp3.3/demhmc3.htm new file mode 100644 index 00000000..9ccc16c9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demhmc3.htm @@ -0,0 +1,45 @@ + +
++demhmc3+ + +
x and one target variable
+t with data generated by sampling x at equal intervals and then
+generating target data by computing sin(2*pi*x) and adding Gaussian
+noise. The model is a 2-layer network with linear outputs, and the hybrid Monte
+Carlo algorithm (with persistence) is used to sample from the posterior
+distribution of the weights. The graph shows the underlying function,
+300 samples from the function given by the posterior distribution of the
+weights, and the average prediction (weighted by the posterior probabilities).
+
+demhmc2, hmc, mlp, mlperr, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demkmn1.htm b/sourcecodes/bnt-master/nethelp3.3/demkmn1.htm new file mode 100644 index 00000000..f22787df --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demkmn1.htm @@ -0,0 +1,49 @@ + +
++demkmean+ + +
demgmm1.
+
+A cluster model with three components is trained using the batch +K-means algorithm. The matrix of centres is printed after training. +The second +figure shows the data labelled with a colour derived from the corresponding +cluster + +
dem2ddat, demgmm1, knn1, kmeansCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demknn1.htm b/sourcecodes/bnt-master/nethelp3.3/demknn1.htm new file mode 100644 index 00000000..6c0d4031 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demknn1.htm @@ -0,0 +1,47 @@ + +
++demknn1+ + +
demgmm1.
+
+The second +figure shows the data labelled with the corresponding class given +by the classifier. + +
dem2ddat, demgmm1, knnCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmdn1.htm b/sourcecodes/bnt-master/nethelp3.3/demmdn1.htm new file mode 100644 index 00000000..7401d220 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmdn1.htm @@ -0,0 +1,48 @@ + +
++demmdn1+ + +
x and one target variable t with data generated by
+sampling t at equal intervals and then generating target data by
+computing t + 0.3*sin(2*pi*t) and adding Gaussian noise. A
+Mixture Density Network with 3 centres in the mixture model is trained
+by minimizing a negative log likelihood error function using the scaled
+conjugate gradient optimizer.
+
+The conditional means, mixing coefficients and variances are plotted
+as a function of x, and a contour plot of the full conditional
+density is also generated.
+
+
mdn, mdnerr, mdngrad, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmet1.htm b/sourcecodes/bnt-master/nethelp3.3/demmet1.htm new file mode 100644 index 00000000..ff5423df --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmet1.htm @@ -0,0 +1,44 @@ + +
++demmet1 +demmet1(plotwait)+ + +
demmet1(plotwait) allows the user to set the time (in a whole number
+of seconds) between the plotting of points. This is passed to pause
+
+
demhmc1, metrop, gmm, dempotCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmlp1.htm b/sourcecodes/bnt-master/nethelp3.3/demmlp1.htm new file mode 100644 index 00000000..3d9c2079 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmlp1.htm @@ -0,0 +1,42 @@ + +
++demmlp1+ + +
x and one target variable
+t with data generated by sampling x at equal intervals and then
+generating target data by computing sin(2*pi*x) and adding Gaussian
+noise. A 2-layer network with linear outputs is trained by minimizing a
+sum-of-squares error function using the scaled conjugate gradient optimizer.
+
+mlp, mlperr, mlpgrad, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demmlp2.htm b/sourcecodes/bnt-master/nethelp3.3/demmlp2.htm new file mode 100644 index 00000000..ecb5e002 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demmlp2.htm @@ -0,0 +1,41 @@ + +
++demmlp2+ + +
mlp, mlpfwd, neterr, quasinewCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demnlab.htm b/sourcecodes/bnt-master/nethelp3.3/demnlab.htm new file mode 100644 index 00000000..1d46b12c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demnlab.htm @@ -0,0 +1,41 @@ + +
++demnlab+ + +
Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demns1.htm b/sourcecodes/bnt-master/nethelp3.3/demns1.htm new file mode 100644 index 00000000..9a6bcae7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demns1.htm @@ -0,0 +1,42 @@ + +
++demns1+ + +
rbf, rbftrain, rbfpriorCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demolgd1.htm b/sourcecodes/bnt-master/nethelp3.3/demolgd1.htm new file mode 100644 index 00000000..76c55303 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demolgd1.htm @@ -0,0 +1,42 @@ + +
++demolgd1+ + +
x and one target variable
+t with data generated by sampling x at equal intervals and then
+generating target data by computing sin(2*pi*x) and adding Gaussian
+noise. A 2-layer network with linear outputs is trained by minimizing a
+sum-of-squares error function using on-line gradient descent.
+
+demmlp1, olgdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demopt1.htm b/sourcecodes/bnt-master/nethelp3.3/demopt1.htm new file mode 100644 index 00000000..b2511711 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demopt1.htm @@ -0,0 +1,52 @@ + +
++demopt1 +demopt1(xinit) ++ + +
demopt1(xinit) allows the user to specify a row vector with
+two columns as the starting point. The default is the point [-1 1].
+Note that the contour plot has an x range of [-1.5, 1.5] and a y
+range of [-0.5, 2.1], so it is best to choose a starting point in the
+same region.
+
+
conjgrad, graddesc, quasinew, scg, rosen, rosegradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/dempot.htm b/sourcecodes/bnt-master/nethelp3.3/dempot.htm new file mode 100644 index 00000000..10bc3907 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/dempot.htm @@ -0,0 +1,41 @@ + +
++e = dempot(x, mix)+ + +
p(x) for a Gaussian mixture model. The data structure
+mix defines the mixture model, while the matrix x contains
+the data vectors.
+
+demgpot, demhmc1, demmet1Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demprgp.htm b/sourcecodes/bnt-master/nethelp3.3/demprgp.htm new file mode 100644 index 00000000..1d7abeca --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demprgp.htm @@ -0,0 +1,44 @@ + +
++demprgp+ + +
gpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demprior.htm b/sourcecodes/bnt-master/nethelp3.3/demprior.htm new file mode 100644 index 00000000..02f5a000 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demprior.htm @@ -0,0 +1,44 @@ + +
++demprior+ + +
aw1, ab1 aw2 and ab2
+control the inverse variances of the first-layer weights, the hidden unit
+biases, the second-layer weights and the output unit biases respectively.
+Their values can be adjusted on a logarithmic scale using the sliders, or
+by typing values into the text boxes and pressing the return key.
+
+mlpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demrbf1.htm b/sourcecodes/bnt-master/nethelp3.3/demrbf1.htm new file mode 100644 index 00000000..6c36ed49 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demrbf1.htm @@ -0,0 +1,47 @@ + +
++demrbf1+ + +
x and one target variable
+t with data generated by sampling x at equal intervals and then
+generating target data by computing sin(2*pi*x) and adding Gaussian
+noise. This data is the same as that used in demmlp1.
+
+Three different RBF networks (with different activation functions) +are trained in two stages. First, a Gaussian mixture model is trained using +the EM algorithm, and the centres of this model are used to set the centres +of the RBF. Second, the output weights (and biases) are determined using the +pseudo-inverse of the design matrix. + +
demmlp1, rbf, rbffwd, gmm, gmmemCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demsom1.htm b/sourcecodes/bnt-master/nethelp3.3/demsom1.htm new file mode 100644 index 00000000..4cc1bbf0 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demsom1.htm @@ -0,0 +1,42 @@ + +
++demsom1+ + +
som, sompak, somtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/demtrain.htm b/sourcecodes/bnt-master/nethelp3.3/demtrain.htm new file mode 100644 index 00000000..f3c8d8d5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/demtrain.htm @@ -0,0 +1,49 @@ + +
++demtrain+ + +
demtrain brings up a simple GUI to show the training of
+an MLP network on classification and regression problems. The user
+should load in a dataset (which should be in Netlab format: see
+datread), select the output activation function, the
+ number of cycles and hidden units and then
+train the network. The scaled conjugate gradient algorithm is used.
+A graph shows the evolution of the error: the value is shown
+max(ceil(iterations / 50), 5) cycles.
+
+Once the network is trained, it is saved to the file mlptrain.net.
+The results can then be viewed as a confusion matrix (for classification
+problems) or a plot of output versus target (for regression problems).
+
+
confmat, datread, mlp, netopt, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/dist2.htm b/sourcecodes/bnt-master/nethelp3.3/dist2.htm new file mode 100644 index 00000000..80dd82b9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/dist2.htm @@ -0,0 +1,57 @@ + +
++d = dist2(x, c) ++ + +
d = dist2(x, c) takes two matrices of vectors and calculates the
+squared Euclidean distance between them. Both matrices must be of the
+same column dimension. If x has m rows and n columns, and
+c has l rows and n columns, then the result has
+m rows and l columns. The i, jth entry is the
+squared distance from the ith row of x to the jth
+row of c.
+
+rbffwd to calculate the activation of
+a thin plate spline function.
++ +n2 = dist2(x, c); +z = log(n2.^(n2.^2)); ++ + +
gmmactiv, kmeans, rbffwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/eigdec.htm b/sourcecodes/bnt-master/nethelp3.3/eigdec.htm new file mode 100644 index 00000000..b981a96c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/eigdec.htm @@ -0,0 +1,43 @@ + +
++evals = eigdec(x, N) +[evals, evec] = eigdec(x, N) ++ + +
evals = eigdec(x, N computes the largest N eigenvalues of the
+matrix x in descending order. [evals, evec] = eigdec(x, N)
+also computes the corresponding eigenvectors.
+
+pca, ppcaCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/errbayes.htm b/sourcecodes/bnt-master/nethelp3.3/errbayes.htm new file mode 100644 index 00000000..d3aaa1da --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/errbayes.htm @@ -0,0 +1,48 @@ + +
++e = errbayes(net, edata) +[e, edata, eprior] = errbayes(net, edata) ++ + +
e = errbayes(net, edata) takes a network data structure
+net together
+the data contribution to the error for a set of inputs and targets.
+It returns the regularised error using any zero mean Gaussian priors
+on the weights defined in
+net.
+
+[e, edata, eprior] = errbayes(net, x, t) additionally returns the
+data and prior components of the error.
+
+
glmerr, mlperr, rbferrCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/evidence.htm b/sourcecodes/bnt-master/nethelp3.3/evidence.htm new file mode 100644 index 00000000..7e319484 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/evidence.htm @@ -0,0 +1,57 @@ + +
++[net] = evidence(net, x, t) +[net, gamma, logev] = evidence(net, x, t, num) ++ + +
[net] = evidence(net, x, t) re-estimates the
+hyperparameters alpha and beta by applying Bayesian
+re-estimation formulae for num iterations. The hyperparameter
+alpha can be a simple scalar associated with an isotropic prior
+on the weights, or can be a vector in which each component is
+associated with a group of weights as defined by the index
+matrix in the net data structure. These more complex priors can
+be set up for an MLP using mlpprior. Initial values for the iterative
+re-estimation are taken from the network data structure net
+passed as an input argument, while the return argument net
+contains the re-estimated values.
+
+[net, gamma, logev] = evidence(net, x, t, num) allows the re-estimation
+formula to be applied for num cycles in which the re-estimated
+values for the hyperparameters from each cycle are used to re-evaluate
+the Hessian matrix for the next cycle. The return value gamma is
+the number of well-determined parameters and logev is the log
+of the evidence.
+
+
mlpprior, netgrad, nethess, demev1, demardCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/fevbayes.htm b/sourcecodes/bnt-master/nethelp3.3/fevbayes.htm new file mode 100644 index 00000000..42289df9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/fevbayes.htm @@ -0,0 +1,53 @@ + +
++extra = fevbayes(net, y, a, x, t, x_test) +[extra, invhess] = fevbayes(net, y, a, x, t, x_test, invhess) ++ + +
extra = fevbayes(net, y, a, x, t, x_test) takes a network data structure
+net together with a set of hidden unit activations a from
+test inputs x_test, training data inputs x and t and
+outputs a matrix of extra information extra that consists of
+error bars (variance)
+for a regression problem or moderated outputs for a classification problem.
+The optional argument (and return value)
+invhess is the inverse of the network Hessian
+computed on the training data inputs and targets. Passing it in avoids
+recomputing it, which can be a significant saving for large training sets.
+
+This is called by network-specific functions such as mlpevfwd which
+are needed since the return values (predictions and hidden unit activations)
+for different network types are in different orders (for good reasons).
+
+
mlpevfwd, rbfevfwd, glmevfwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gauss.htm b/sourcecodes/bnt-master/nethelp3.3/gauss.htm new file mode 100644 index 00000000..4f0f0d88 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gauss.htm @@ -0,0 +1,43 @@ + +
++y = gauss(mu, covar, x) ++ + +
y = gauss(mu, covar, x) evaluates a multi-variate Gaussian
+density in d-dimensions at a set of points given by the rows
+of the matrix x. The Gaussian density has mean vector mu
+and covariance matrix covar.
+
+
gsamp, demgaussCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gbayes.htm b/sourcecodes/bnt-master/nethelp3.3/gbayes.htm new file mode 100644 index 00000000..3e123fdf --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gbayes.htm @@ -0,0 +1,50 @@ + +
++g = gbayes(net, gdata) +[g, gdata, gprior] = gbayes(net, gdata) ++ + +
g = gbayes(net, gdata) takes a network data structure net together
+the data contribution to the error gradient
+for a set of inputs and targets.
+It returns the regularised error gradient using any zero mean Gaussian priors
+on the weights defined in
+net. In addition, if a mask is defined in net, then
+the entries in g that correspond to weights with a 0 in the
+mask are removed.
+
+[g, gdata, gprior] = gbayes(net, gdata) additionally returns the
+data and prior components of the error.
+
+
errbayes, glmgrad, mlpgrad, rbfgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glm.htm b/sourcecodes/bnt-master/nethelp3.3/glm.htm new file mode 100644 index 00000000..a860372f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glm.htm @@ -0,0 +1,85 @@ + +
++net = glm(nin, nout, func) +net = glm(nin, nout, func, prior) +net = glm(nin, nout, func, prior, beta) ++ + +
net = glm(nin, nout, func) takes the number of inputs
+and outputs for a generalized linear model, together
+with a string func which specifies the output unit activation function,
+and returns a data structure net. The weights are drawn from a zero mean,
+isotropic Gaussian, with variance scaled by the fan-in of the
+output units. This makes use of the Matlab function
+randn and so the seed for the random weight initialization can be
+set using randn('state', s) where s is the seed value. The optional
+argument alpha sets the inverse variance for the weight
+initialization.
+
+
The fields in net are
+
+ type = 'glm' + nin = number of inputs + nout = number of outputs + nwts = total number of weights and biases + actfn = string describing the output unit activation function: + 'linear' + 'logistic' + 'softmax' + w1 = first-layer weight matrix + b1 = first-layer bias vector ++ + +
net = glm(nin, nout, func, prior), in which prior is
+a scalar, allows the field
+net.alpha in the data structure net to be set, corresponding
+to a zero-mean isotropic Gaussian prior with inverse variance with
+value prior. Alternatively, prior can consist of a data
+structure with fields alpha and index, allowing individual
+Gaussian priors to be set over groups of weights in the network. Here
+alpha is a column vector in which each element corresponds to a
+separate group of weights, which need not be mutually exclusive. The
+membership of the groups is defined by the matrix index in which
+the columns correspond to the elements of alpha. Each column has
+one element for each weight in the matrix, in the order defined by the
+function glmpak, and each element is 1 or 0 according to whether
+the weight is a member of the corresponding group or not.
+
+
net = glm(nin, nout, func, prior, beta) also sets the
+additional field net.beta in the data structure net, where
+beta corresponds to the inverse noise variance.
+
+
glmpak, glmunpak, glmfwd, glmerr, glmgrad, glmtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmderiv.htm b/sourcecodes/bnt-master/nethelp3.3/glmderiv.htm new file mode 100644 index 00000000..56887889 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmderiv.htm @@ -0,0 +1,49 @@ + +
++ +g = glmderiv(net, x) ++ + +
g = glmderiv(net, x) takes a network data structure net and a matrix
+of input vectors x and returns a three-index matrix mat{g} whose
+i, j, k
+element contains the derivative of network output k with respect to
+weight or bias parameter j for input pattern i. The ordering of the
+weight and bias parameters is defined by glmunpak.
+
++glm, glmunpak, glmgrad+ + +
Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmerr.htm b/sourcecodes/bnt-master/nethelp3.3/glmerr.htm new file mode 100644 index 00000000..fde8a08b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmerr.htm @@ -0,0 +1,56 @@ + +
++e = glmerr(net, x, t) +[e, edata, eprior] = glmerr(net, x, t) +[e, edata, eprior, y, a] = glmerr(net, x, t) ++ + +
e = glmerr(net, x, t) takes a generalized
+linear model data structure net together with a matrix x
+of input vectors and a matrix t of target vectors, and evaluates
+the error function e. The choice of error function corresponds
+to the output unit activation function. Each row of x
+corresponds to one input vector and each row of t corresponds to
+one target vector.
+
+[e, edata, eprior, y, a] = glmerr(net, x, t) also returns
+the data and prior components of the total error.
+
+
[e, edata, eprior, y, a] = glmerr(net, x) also returns a matrix y
+giving the outputs of the models and a matrix a
+giving the summed inputs to each output unit, where each row
+corresponds to one pattern.
+
+
glm, glmpak, glmunpak, glmfwd, glmgrad, glmtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/glmevfwd.htm new file mode 100644 index 00000000..fc2f4749 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmevfwd.htm @@ -0,0 +1,51 @@ + +
++ +[y, extra] = glmevfwd(net, x, t, x_test) +[y, extra, invhess] = glmevfwd(net, x, t, x_test, invhess) ++ + +
y = glmevfwd(net, x, t, x_test) takes a network data structure
+net together with the input x and target t training data
+and input test data x_test.
+It returns the normal forward propagation through the network y
+together with a matrix extra which consists of error bars (variance)
+for a regression problem or moderated outputs for a classification problem.
+
+The optional argument (and return value)
+invhess is the inverse of the network Hessian
+computed on the training data inputs and targets. Passing it in avoids
+recomputing it, which can be a significant saving for large training sets.
+
+
fevbayesCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmfwd.htm b/sourcecodes/bnt-master/nethelp3.3/glmfwd.htm new file mode 100644 index 00000000..9e8f1061 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmfwd.htm @@ -0,0 +1,49 @@ + +
++y = glmfwd(net, x) +[y, a] = glmfwd(net, x) ++ + +
y = glmfwd(net, x) takes a generalized linear model
+data structure net together with
+a matrix x of input vectors, and forward propagates the inputs
+through the network to generate a matrix y of output
+vectors. Each row of x corresponds to one input vector and each
+row of y corresponds to one output vector.
+
+[y, a] = glmfwd(net, x) also returns a matrix a
+giving the summed inputs to each output unit, where each row
+corresponds to one pattern.
+
+
glm, glmpak, glmunpak, glmerr, glmgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmgrad.htm b/sourcecodes/bnt-master/nethelp3.3/glmgrad.htm new file mode 100644 index 00000000..024a5d19 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmgrad.htm @@ -0,0 +1,51 @@ + +
++ +g = glmgrad(net, x, t) +[g, gdata, gprior] = glmgrad(net, x, t) ++ + +
g = glmgrad(net, x, t) takes a generalized linear model
+data structure net
+together with a matrix x of input vectors and a matrix t
+of target vectors, and evaluates the gradient g of the error
+function with respect to the network weights. The error function
+corresponds to the choice of output unit activation function. Each row
+of x corresponds to one input vector and each row of t
+corresponds to one target vector.
+
+[g, gdata, gprior] = glmgrad(net, x, t) also returns separately
+the data and prior contributions to the gradient.
+
+
glm, glmpak, glmunpak, glmfwd, glmerr, glmtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmhess.htm b/sourcecodes/bnt-master/nethelp3.3/glmhess.htm new file mode 100644 index 00000000..481e3220 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmhess.htm @@ -0,0 +1,74 @@ + +
++h = glmhess(net, x, t) +[h, hdata] = glmhess(net, x, t) +h = glmhess(net, x, t, hdata) ++ + +
h = glmhess(net, x, t) takes a GLM network data structure net,
+a matrix x of input values, and a matrix t of target
+values and returns the full Hessian matrix h corresponding to
+the second derivatives of the negative log posterior distribution,
+evaluated for the current weight and bias values as defined by
+net. Note that the target data is not required in the calculation,
+but is included to make the interface uniform with nethess. For
+linear and logistic outputs, the computation is very simple and is
+done (in effect) in one line in glmtrain.
+
+[h, hdata] = glmhess(net, x, t) returns both the Hessian matrix
+h and the contribution hdata arising from the data dependent
+term in the Hessian.
+
+
h = glmhess(net, x, t, hdata) takes a network data structure
+net, a matrix x of input values, and a matrix t of
+target values, together with the contribution hdata arising from
+the data dependent term in the Hessian, and returns the full Hessian
+matrix h corresponding to the second derivatives of the negative
+log posterior distribution. This version saves computation time if
+hdata has already been evaluated for the current weight and bias
+values.
+
+
glmtrain to take a Newton step for
+softmax outputs.
++ +Hessian = glmhess(net, x, t); +deltaw = -gradient*pinv(Hessian); ++ + +
glm, glmtrain, hesschek, nethessCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glminit.htm b/sourcecodes/bnt-master/nethelp3.3/glminit.htm new file mode 100644 index 00000000..e19369d3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glminit.htm @@ -0,0 +1,49 @@ + +
++net = glminit(net, prior) ++ + +
net = glminit(net, prior) takes a generalized linear model
+net and sets the weights and biases by sampling from a Gaussian
+distribution. If prior is a scalar, then all of the parameters
+(weights and biases) are sampled from a single isotropic Gaussian with
+inverse variance equal to prior. If prior is a data
+structure similar to that in mlpprior but for a single layer of
+weights, then the parameters
+are sampled from multiple Gaussians according to their groupings
+(defined by the index field) with corresponding variances
+(defined by the alpha field).
+
+
glm, glmpak, glmunpak, mlpinit, mlppriorCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmpak.htm b/sourcecodes/bnt-master/nethelp3.3/glmpak.htm new file mode 100644 index 00000000..8e1af1ad --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmpak.htm @@ -0,0 +1,40 @@ + +
++w = glmpak(net) ++ + +
w = glmpak(net) takes a network data structure net and
+combines them into a single row vector w.
+
+glm, glmunpak, glmfwd, glmerr, glmgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmtrain.htm b/sourcecodes/bnt-master/nethelp3.3/glmtrain.htm new file mode 100644 index 00000000..a52ada6d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmtrain.htm @@ -0,0 +1,71 @@ + +
+net = glmtrain(net, options, x, t) uses
+the iterative reweighted least squares (IRLS)
+algorithm to set the weights in the generalized linear model structure
+net. This is a more efficient alternative to using glmerr
+and glmgrad and a non-linear optimisation routine through
+netopt.
+Note that for linear outputs, a single pass through the
+algorithm is all that is required, since the error function is quadratic in
+the weights. The algorithm also handles scalar alpha and beta
+terms. If you want to use more complicated priors, you should use
+general-purpose non-linear optimisation algorithms.
+
+For logistic and softmax outputs, general priors can be handled, although +this requires the pseudo-inverse of the Hessian, giving up the better +conditioning and some of the speed advantage of the normal form equations. + +
The error function value at the final set of weights is returned
+in options(8).
+Each row of x corresponds to one
+input vector and each row of t corresponds to one target vector.
+
+
The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values during training.
+If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is a measure of the precision required for the value
+of the weights w at the solution.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. Both this and the previous condition must be
+satisfied for termination.
+
+
options(5) is set to 1 if an approximation to the Hessian (which assumes
+that all outputs are independent) is used for softmax outputs. With the default
+value of 0 the exact Hessian (which is more expensive to compute) is used.
+
+
options(14) is the maximum number of iterations for the IRLS algorithm;
+default 100.
+
+
glm, glmerr, glmgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/glmunpak.htm b/sourcecodes/bnt-master/nethelp3.3/glmunpak.htm new file mode 100644 index 00000000..73fd5054 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/glmunpak.htm @@ -0,0 +1,43 @@ + +
++net = glmunpak(net, w) ++ + +
net = glmunpak(net, w) takes a glm network data structure net and
+a weight vector w, and returns a network data structure identical to
+the input network, except that the first-layer weight matrix
+w1 and the first-layer bias vector b1 have
+been set to the corresponding elements of w.
+
+glm, glmpak, glmfwd, glmerr, glmgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmm.htm b/sourcecodes/bnt-master/nethelp3.3/gmm.htm new file mode 100644 index 00000000..c815b3cb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmm.htm @@ -0,0 +1,96 @@ + +
++mix = gmm(dim, ncentres, covartype) +mix = gmm(dim, ncentres, covartype, ppca_dim) ++ + +
mix = gmm(dim, ncentres, covartype) takes
+the dimension of the space dim, the number of centres in the
+mixture model and the type of the mixture model, and returns a data
+structure mix.
+The mixture model type defines the covariance structure of each component
+Gaussian:
++ + 'spherical' = single variance parameter for each component: stored as a vector + 'diag' = diagonal matrix for each component: stored as rows of a matrix + 'full' = full matrix for each component: stored as 3d array + 'ppca' = probabilistic PCA: stored as principal components (in a 3d array + and associated variances and off-subspace noise ++ +
mix = gmm(dim, ncentres, covartype, ppca_dim) also sets the dimension of
+the PPCA sub-spaces: the default value is one.
+
+The priors are initialised to equal values summing to one, and the covariances
+are all the identity matrix (or equivalent). The centres are
+initialised randomly from a zero mean unit variance Gaussian. This makes use
+of the MATLAB function randn and so the seed for the random weight
+initialisation can be set using randn('state', s) where s is the
+state value.
+
+
The fields in mix are
+
+ + type = 'gmm' + nin = the dimension of the space + ncentres = number of mixture components + covartype = string for type of variance model + priors = mixing coefficients + centres = means of Gaussians: stored as rows of a matrix + covars = covariances of Gaussians ++ +The additional fields for mixtures of PPCA are +
+ + U = principal component subspaces + lambda = in-space covariances: stored as rows of a matrix ++ +The off-subspace noise is stored in
covars.
+
++ +mix = gmm(2, 4, 'spherical'); ++ +This creates a Gaussian mixture model with 4 components in 2 dimensions. +The covariance structure is a spherical model. + +
gmmpak, gmmunpak, gmmsamp, gmminit, gmmem, gmmactiv, gmmpost, gmmprobCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmactiv.htm b/sourcecodes/bnt-master/nethelp3.3/gmmactiv.htm new file mode 100644 index 00000000..0c3c5c8f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmactiv.htm @@ -0,0 +1,47 @@ + +
++ +a = gmmactiv(mix, x) ++ + +
a (i.e. the
+probability p(x|j) of the data conditioned on each component density)
+for a Gaussian mixture model. For the PPCA model, each activation
+is the conditional probability of x given that it is generated
+by the component subspace.
+The data structure mix defines the mixture model, while the matrix
+x contains the data vectors. Each row of x represents a single
+vector.
+
+gmm, gmmpost, gmmprobCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmem.htm b/sourcecodes/bnt-master/nethelp3.3/gmmem.htm new file mode 100644 index 00000000..6c0f46dd --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmem.htm @@ -0,0 +1,87 @@ + +
++ +[mix, options, errlog] = gmmem(mix, x, options) ++ + +
[mix, options, errlog] = gmmem(mix, x, options) uses the Expectation
+Maximization algorithm of Dempster et al. to estimate the parameters of
+a Gaussian mixture model defined by a data structure mix.
+The matrix x represents the data whose expectation
+is maximized, with each row corresponding to a vector.
+
+The optional parameters have the following interpretations.
+
+options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog.
+If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(3) is a measure of the absolute precision required of the error
+function at the solution. If the change in log likelihood between two steps of
+the EM algorithm is less than this value, then the function terminates.
+
+
options(5) is set to 1 if a covariance matrix is reset to its
+original value when any of its singular values are too small (less
+than MIN_COVAR which has the value eps).
+With the default value of 0 no action is taken.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
The optional return value options contains the final error value
+(i.e. data log likelihood) in
+options(8).
+
+
+ +mix = gmm(inputdim, ncentres, 'full'); + ++ + +options = foptions; +options(14) = 5; +mix = gmminit(mix, data, options); + +
options(1) = 1; % Prints out error values. +options(14) = 30; % Max. number of iterations. + +
mix = gmmem(mix, data, options); +
gmm, gmminitCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmminit.htm b/sourcecodes/bnt-master/nethelp3.3/gmminit.htm new file mode 100644 index 00000000..3108da74 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmminit.htm @@ -0,0 +1,65 @@ + +
++ +mix = gmminit(mix, x, options) ++ + +
mix = gmminit(mix, x, options) uses a dataset x
+to initialise the parameters of a Gaussian mixture
+model defined by the data structure mix. The k-means algorithm
+is used to determine the centres. The priors are computed from the
+proportion of examples belonging to each cluster.
+The covariance matrices are calculated as the sample covariance of the
+points associated with (i.e. closest to) the corresponding centres.
+For a mixture of PPCA model, the PPCA decomposition is calculated
+for the points closest to a given centre.
+This initialisation can be used as the starting point for training the
+model using the EM algorithm.
+
++ +mix = gmm(3, 2); +options = foptions; +options(14) = 5; +mix = gmminit(mix, data, options); ++ +This code sets up a Gaussian mixture model with 3 centres in 2 dimensions, and +then initialises the parameters from the data set
data with 5 iterations
+of the k means algorithm.
+
+gmmCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmpak.htm b/sourcecodes/bnt-master/nethelp3.3/gmmpak.htm new file mode 100644 index 00000000..0ef370a9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmpak.htm @@ -0,0 +1,41 @@ + +
++p = gmmpak(mix) ++ + +
p = gmmpak(net) takes a mixture data structure mix
+and combines the component parameter matrices into a single row
+vector p.
+
+gmm, gmmunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmpost.htm b/sourcecodes/bnt-master/nethelp3.3/gmmpost.htm new file mode 100644 index 00000000..d323115f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmpost.htm @@ -0,0 +1,44 @@ + +
++ +function post = gmmpost(mix, x) ++ + +
post (i.e. the probability of each
+component conditioned on the data p(j|x)) for a Gaussian mixture model.
+The data structure mix defines the mixture model, while the matrix
+x contains the data vectors. Each row of x represents a single
+vector.
+
+gmm, gmmactiv, gmmprobCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmprob.htm b/sourcecodes/bnt-master/nethelp3.3/gmmprob.htm new file mode 100644 index 00000000..acc3fb65 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmprob.htm @@ -0,0 +1,44 @@ + +
++ +function prob = gmmprob(mix, x) ++ + +
p(x) for a Gaussian mixture model. The data structure
+mix defines the mixture model, while the matrix x contains
+the data vectors. Each row of x represents a single vector.
+
+gmm, gmmpost, gmmactivCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmsamp.htm b/sourcecodes/bnt-master/nethelp3.3/gmmsamp.htm new file mode 100644 index 00000000..30f6821c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmsamp.htm @@ -0,0 +1,47 @@ + +
++data = gmmsamp(mix, n) +[data, label] = gmmsamp(mix, n) ++ + +
data = gsamp(mix, n) generates a sample of size n from a
+Gaussian mixture distribution defined by the mix data
+structure. The matrix x has n
+rows in which each row represents a mix.nin-dimensional sample vector.
+
+
[data, label] = gmmsamp(mix, n) also returns a column vector of
+classes (as an index 1..N) label.
+
+
gsamp, gmmCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gmmunpak.htm b/sourcecodes/bnt-master/nethelp3.3/gmmunpak.htm new file mode 100644 index 00000000..2287fb3b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gmmunpak.htm @@ -0,0 +1,45 @@ + +
++mix = gmmunpak(mix, p) ++ + +
mix = gmmunpak(mix, p)
+takes a GMM data structure mix and
+a single row vector of parameters p and returns a mixture data structure
+identical to the input mix, except that the mixing coefficients
+priors, centres centres and covariances covars
+(and, for PPCA, the lambdas and U (PCA sub-spaces)) are all set
+to the corresponding elements of p.
+
+gmm, gmmpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gp.htm b/sourcecodes/bnt-master/nethelp3.3/gp.htm new file mode 100644 index 00000000..77d6ab1a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gp.htm @@ -0,0 +1,80 @@ + +
++net = gp(nin, covarfn) +net = gp(nin, covarfn, prior) ++ + +
net = gp(nin, covarfn) takes the number of inputs nin
+for a Gaussian Process model with a single output, together
+with a string covarfn which specifies the type of the covariance function,
+and returns a data structure net. The parameters are set to zero.
+
+
The fields in net are
+
+ type = 'gp' + nin = number of inputs + nout = number of outputs: always 1 + nwts = total number of weights and covariance function parameters + bias = logarithm of constant offset in covariance function + noise = logarithm of output noise variance + inweights = logarithm of inverse length scale for each input + covarfn = string describing the covariance function: + 'sqexp' + 'ratquad' + fpar = covariance function specific parameters (1 for squared exponential, + 2 for rational quadratic) + trin = training input data (initially empty) + trtargets = training target data (initially empty) ++ + +
net = gp(nin, covarfn, prior) sets a Gaussian prior on the
+parameters of the model. prior must contain the fields
+pr_mean and pr_variance. If pr_mean is a scalar,
+then the Gaussian is assumed to be isotropic and the additional fields
+net.pr_mean and pr_variance are set. Otherwise,
+the Gaussian prior has a mean
+defined by a column vector of parameters prior.pr_mean and
+covariance defined by a column vector of parameters prior.pr_variance.
+Each element of prmean corresponds to a separate group of parameters, which
+need not be mutually exclusive. The membership of the groups is defined
+by the matrix prior.index in which the columns correspond to the elements of
+prmean. Each column has one element for each weight in the matrix,
+in the order defined by the function gppak, and each element
+is 1 or 0 according to whether the parameter is a member of the
+corresponding group or not. The additional field net.index is set
+in this case.
+
+
gppak, gpunpak, gpfwd, gperr, gpcovar, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpcovar.htm b/sourcecodes/bnt-master/nethelp3.3/gpcovar.htm new file mode 100644 index 00000000..f626f8fd --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpcovar.htm @@ -0,0 +1,64 @@ + +
++cov = gpcovar(net, x) +[cov, covf] = gpcovar(net, x) ++ + +
cov = gpcovar(net, x) takes
+a Gaussian Process data structure net together with
+a matrix x of input vectors, and computes the covariance
+matrix cov. The inverse of this matrix is used when calculating
+the mean and variance of the predictions made by net.
+
+
[cov, covf] = gpcovar(net, x) also generates the covariance
+matrix due to the covariance function specified by net.covarfn
+as calculated by gpcovarf.
+
+
x and is then
+passed to gpfwd so that predictions (with mean ytest and
+variance sigsq) can be made for the test inputs
+xtest.
++ +cninv = inv(gpcovar(net, x)); +[ytest, sigsq] = gpfwd(net, xtest, cninv); ++ + +
gp, gppak, gpunpak, gpcovarp, gpcovarf, gpfwd, gperr, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpcovarf.htm b/sourcecodes/bnt-master/nethelp3.3/gpcovarf.htm new file mode 100644 index 00000000..5bce6b98 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpcovarf.htm @@ -0,0 +1,44 @@ + +
++covf = gpcovarf(net, x1, x2) ++ + +
covf = gpcovarf(net, x1, x2) takes
+a Gaussian Process data structure net together with
+two matrices x1 and x2 of input vectors,
+and computes the matrix of the covariance function values
+covf.
+
+
gp, gpcovar, gpcovarp, gperr, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpcovarp.htm b/sourcecodes/bnt-master/nethelp3.3/gpcovarp.htm new file mode 100644 index 00000000..616664ce --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpcovarp.htm @@ -0,0 +1,49 @@ + +
++covp = gpcovarp(net, x1, x2) +[covp, covf] = gpcovarp(net, x1, x2) ++ + +
covp = gpcovarp(net, x1, x2) takes
+a Gaussian Process data structure net together with
+two matrices x1 and x2 of input vectors,
+and computes the matrix of the prior covariance. This is
+the function component of the covariance plus the exponential of the bias
+term.
+
+
[covp, covf] = gpcovarp(net, x1, x2) also returns the function
+component of the covariance.
+
+
gp, gpcovar, gpcovarf, gperr, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gperr.htm b/sourcecodes/bnt-master/nethelp3.3/gperr.htm new file mode 100644 index 00000000..de8d9831 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gperr.htm @@ -0,0 +1,49 @@ + +
++edata = gperr(net, x, t) +[e, edata, eprior] = gperr(net, x, t) ++ + +
e = gperr(net, x, t) takes a Gaussian Process data structure net together
+with a matrix x of input vectors and a matrix t of target
+vectors, and evaluates the error function e. Each row
+of x corresponds to one input vector and each row of t
+corresponds to one target vector.
+
+[e, edata, eprior] = gperr(net, x, t) additionally returns the
+data and hyperprior components of the error, assuming a Gaussian
+prior on the weights with mean and variance parameters prmean and
+prvariance taken from the network data structure net.
+
+
gp, gpcovar, gpfwd, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpfwd.htm b/sourcecodes/bnt-master/nethelp3.3/gpfwd.htm new file mode 100644 index 00000000..51fc239c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpfwd.htm @@ -0,0 +1,73 @@ + +
++y = gpfwd(net, x) +[y, sigsq] = gpfwd(net, x) +[y, sigsq] = gpfwd(net, x, cninv) ++ + +
y = gpfwd(net, x) takes a Gaussian Process data structure net
+together
+with a matrix x of input vectors, and forward propagates the inputs
+through the model to generate a matrix y of output
+vectors. Each row of x corresponds to one input vector and each
+row of y corresponds to one output vector. This assumes that the
+training data (both inputs and targets) has been stored in net by
+a call to gpinit; these are needed to compute the training
+data covariance matrix.
+
+[y, sigsq] = gpfwd(net, x) also generates a column vector sigsq of
+conditional variances (or squared error bars) where each value corresponds to a pattern.
+
+
[y, sigsq] = gpfwd(net, x, cninv) uses the pre-computed inverse covariance
+matrix cninv in the forward propagation. This increases efficiency if
+several calls to gpfwd are made.
+
+
+ +net = gp(1, 'sqexp'); +net = gpinit(net, x, t); +net = netopt(net, options, x, t, 'scg'); +[pred, sigsq] = gpfwd(net, xtest); +plot(xtest, pred, '-k'); +hold on +plot(xtest, pred+sqrt(sigsq), '-b', xtest, pred-sqrt(sigsq), '-b'); ++ + +
gp, demgp, gpinitCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpgrad.htm b/sourcecodes/bnt-master/nethelp3.3/gpgrad.htm new file mode 100644 index 00000000..887cdf1b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpgrad.htm @@ -0,0 +1,43 @@ + +
++g = gpgrad(net, x, t) ++ + +
g = gpgrad(net, x, t) takes a Gaussian Process data structure net together
+with a matrix x of input vectors and a matrix t of target
+vectors, and evaluates the error gradient g. Each row
+of x corresponds to one input vector and each row of t
+corresponds to one target vector.
+
+gp, gpcovar, gpfwd, gperrCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpinit.htm b/sourcecodes/bnt-master/nethelp3.3/gpinit.htm new file mode 100644 index 00000000..1874dde7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpinit.htm @@ -0,0 +1,76 @@ + +
++net = gpinit(net, trin, trtargets, prior) +net = gpinit(net, trin, trtargets, prior) ++ + +
net = gpinit(net, trin, trtargets) takes a Gaussian Process data structure net
+together
+with a matrix trin of training input vectors and a matrix trtargets of
+training target
+vectors, and stores them in net. These datasets are required if
+the corresponding inverse covariance matrix is not supplied to gpfwd.
+This is important if the data structure is saved and then reloaded before
+calling gpfwd.
+Each row
+of trin corresponds to one input vector and each row of trtargets
+corresponds to one target vector.
+
+net = gpinit(net, trin, trtargets, prior) additionally initialises the
+parameters in net from the prior data structure which contains the
+mean and variance of the Gaussian distribution which is sampled from.
+
+
x
+and targets t:
++ +net = gp(2, 'sqexp'); +net = gpinit(net, x, t); +% Train the network +save 'gp.net' net; ++ +Another Matlab program can now read in the network and make predictions on a data set +
testin:
++ +load 'gp.net'; +pred = gpfwd(net, testin); ++ + +
gp, gpfwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gppak.htm b/sourcecodes/bnt-master/nethelp3.3/gppak.htm new file mode 100644 index 00000000..e913dd2c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gppak.htm @@ -0,0 +1,40 @@ + +
++hp = gppak(net) ++ + +
hp = gppak(net) takes a Gaussian Process data structure net and
+combines the hyperparameters into a single row vector hp.
+
+gp, gpunpak, gpfwd, gperr, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gpunpak.htm b/sourcecodes/bnt-master/nethelp3.3/gpunpak.htm new file mode 100644 index 00000000..593dbb04 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gpunpak.htm @@ -0,0 +1,46 @@ + +
++net = gpunpak(net, hp) ++ + +
net = gpunpak(net, hp) takes an Gaussian Process data structure net and
+a hyperparameter vector hp, and returns a Gaussian Process data structure
+identical to
+the input model, except that the covariance bias
+bias, output noise noise, the input weight vector
+inweights and the vector of covariance function specific parameters
+ fpar have all
+been set to the corresponding elements of hp.
+
+gp, gppak, gpfwd, gperr, gpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gradchek.htm b/sourcecodes/bnt-master/nethelp3.3/gradchek.htm new file mode 100644 index 00000000..0a7df6ea --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gradchek.htm @@ -0,0 +1,58 @@ + +
++ +gradchek(w, func, grad) +[gradient, delta] = gradchek(w, func, grad) +gradchek(w, func, grad, p1, p2, ...) ++ + +
gradchek(w, func, grad) checks how accurate the gradient
+grad of a function func is at a parameter vector x.
+A central
+difference formula with step size 1.0e-6 is used, and the results
+for both gradient function and finite difference approximation are
+printed.
+The optional return value gradient is the gradient calculated
+using the function grad and the return value delta is the
+difference between the functional and finite difference methods of
+calculating the graident.
+
+gradchek(x, func, grad, p1, p2, ...) allows additional arguments
+to be passed to func and grad.
+
+
conjgrad, graddesc, hmc, olgd, quasinew, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/graddesc.htm b/sourcecodes/bnt-master/nethelp3.3/graddesc.htm new file mode 100644 index 00000000..fe8a3df1 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/graddesc.htm @@ -0,0 +1,104 @@ + +
+[x, options, flog, pointlog] = graddesc(f, x, options, gradf) uses
+batch gradient descent to find a local minimum of the function
+f(x) whose gradient is given by gradf(x). A log of the function values
+after each cycle is (optionally) returned in errlog, and a log
+of the points visited is (optionally) returned in pointlog.
+
+Note that x is a row vector
+and f returns a scalar value.
+The point at which f has a local minimum
+is returned as x. The function value at that point is returned
+in options(8).
+
+
graddesc(f, x, options, gradf, p1, p2, ...) allows
+additional arguments to be passed to f() and gradf().
+
+
The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog, and the points visited
+in the return argument pointslog. If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is the absolute precision required for the value
+of x at the solution. If the absolute difference between
+the values of x between two successive steps is less than
+options(2), then this condition is satisfied.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. If the absolute difference between the
+objective function values between two successive steps is less than
+options(3), then this condition is satisfied.
+Both this and the previous condition must be
+satisfied for termination.
+
+
options(7) determines the line minimisation method used. If it
+is set to 1 then a line minimiser is used (in the direction of the negative
+gradient). If it is 0 (the default), then each parameter update
+is a fixed multiple (the learning rate)
+of the negative gradient added to a fixed multiple (the momentum) of
+the previous parameter update.
+
+
options(9) should be set to 1 to check the user defined gradient
+function gradf with gradchek. This is carried out at
+the initial parameter vector x.
+
+
options(10) returns the total number of function evaluations (including
+those in any line searches).
+
+
options(11) returns the total number of gradient evaluations.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
options(15) is the precision in parameter space of the line search;
+default foptions(2).
+
+
options(17) is the momentum; default 0.5. It should be scaled by the
+inverse of the number of data points.
+
+
options(18) is the learning rate; default 0.01. It should be
+scaled by the inverse of the number of data points.
+
+
+ +options = zeros(1, 18); +options(17) = 0.1/size(x, 1); +net = netopt(net, options, x, t, 'graddesc'); ++ +Note how the learning rate is scaled by the number of data points. + +
conjgrad, linemin, olgd, minbrack, quasinew, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gsamp.htm b/sourcecodes/bnt-master/nethelp3.3/gsamp.htm new file mode 100644 index 00000000..0f5e5ead --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gsamp.htm @@ -0,0 +1,44 @@ + +
++x = gsamp(mu, covar, nsamp) ++ + +
x = gsamp(mu, covar, nsamp) generates a sample of size nsamp
+from a d-dimensional Gaussian distribution. The Gaussian density
+has mean vector mu and covariance matrix covar, and the
+matrix x has nsamp rows in which each row represents a
+d-dimensional sample vector.
+
+
gauss, demgaussCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtm.htm b/sourcecodes/bnt-master/nethelp3.3/gtm.htm new file mode 100644 index 00000000..e02ef536 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtm.htm @@ -0,0 +1,66 @@ + +
++net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc) +net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc, prior) ++ + +
net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc),
+takes the dimension of the latent space dimlatent, the
+number of data points sampled in the latent space nlatent, the
+dimension of the data space dimdata, the number of centres in the
+RBF model ncentres, the activation function for the RBF
+rbfunc
+and returns a data structure net. The parameters in the
+RBF and GMM sub-models are set by calls to the corresponding creation routines
+rbf and gmm.
+
+
The fields in net are
+
+ type = 'gtm' + nin = dimension of data space + dimlatent = dimension of latent space + rbfnet = RBF network data structure + gmmnet = GMM data structure + X = sample of latent points ++ + +
net = gtm(dimlatent, nlatent, dimdata, ncentres, rbfunc, prior),
+ sets a Gaussian zero mean prior on the
+parameters of the RBF model. prior must be a scalar and represents
+the inverse variance of the prior distribution. This gives rise to
+a weight decay term in the error function.
+
+
gtmfwd, gtmpost, rbf, gmmCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmem.htm b/sourcecodes/bnt-master/nethelp3.3/gtmem.htm new file mode 100644 index 00000000..95c385b1 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmem.htm @@ -0,0 +1,88 @@ + +
++ +[net, options, errlog] = gtmem(net, t, options) ++ + +
[net, options, errlog] = gtmem(net, t, options) uses the Expectation
+Maximization algorithm to estimate the parameters of
+a GTM defined by a data structure net.
+The matrix t represents the data whose expectation
+is maximized, with each row corresponding to a vector. It is assumed
+that the latent data net.X has been set following a call to
+gtminit, for example.
+
+The optional parameters have the following interpretations.
+
+options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog.
+If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(3) is a measure of the absolute precision required of the error
+function at the solution. If the change in log likelihood between two steps of
+the EM algorithm is less than this value, then the function terminates.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
The optional return value options contains the final error value
+(i.e. data log likelihood) in
+options(8).
+
+
+ +% Create and initialise GTM model +net = gtm(latentdim, nlatent, datadim, numrbfcentres, ... + 'gaussian', 0.1); + ++ + +options = foptions; +options(1) = -1; +options(7) = 1; % Set width factor of RBF +net = gtminit(net, options, data, 'regular', latentshape, [4 4]); + +
options = foptions; +options(14) = 30; +options(1) = 1; +[net, options] = gtmem(net, data, options); +
gtm, gtminitCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmfwd.htm b/sourcecodes/bnt-master/nethelp3.3/gtmfwd.htm new file mode 100644 index 00000000..2ba02a35 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmfwd.htm @@ -0,0 +1,44 @@ + +
++mix = gtmfwd(net) ++ + +
mix = gtmfwd(net) takes a GTM
+structure net, and forward
+propagates the latent data sample net.X through the GTM to generate
+the structure
+mix which represents the Gaussian mixture model in data space.
+
+gtmCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtminit.htm b/sourcecodes/bnt-master/nethelp3.3/gtminit.htm new file mode 100644 index 00000000..8ac58331 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtminit.htm @@ -0,0 +1,66 @@ + +
++net = gtminit(net, options, data, samptype) +net = gtminit(net, options, data, samptype, lsampsize, rbfsampsize) ++ + +
net = gtminit(net, options, data, samptype) takes a GTM net
+and generates a sample of latent data points and sets the centres (and
+widths if appropriate) of
+net.rbfnet.
+
+If the samptype is 'regular', then regular grids of latent
+data points and RBF centres are created. The dimension of the latent data
+space must be
+1 or 2. For one-dimensional latent space, the lsampsize parameter
+gives the number of latent points and the rbfsampsize parameter
+gives the number of RBF centres. For a two-dimensional latent space,
+these parameters must be vectors of length 2 with the number of points
+in each of the x and y directions to create a rectangular grid. The
+widths of the RBF basis functions are set by a call to rbfsetfw
+passing options(7) as the scaling parameter.
+
+
If the samptype is 'uniform' or 'gaussian' then the
+latent data is found by sampling from a uniform or
+Gaussian distribution correspondingly. The RBF basis function parameters
+are set
+by a call to rbfsetbf with the data parameter
+as dataset and the options vector.
+
+
Finally, the output layer weights of the RBF are initialised by +mapping the mean of the latent variable to the mean of the target variable, +and the L-dimensional latent variale variance to the variance of the +targets along the first L principal components. + +
gtm, gtmem, pca, rbfsetbf, rbfsetfwCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmlmean.htm b/sourcecodes/bnt-master/nethelp3.3/gtmlmean.htm new file mode 100644 index 00000000..1736c0ed --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmlmean.htm @@ -0,0 +1,42 @@ + +
++means = gtmlmean(net, data) ++ + +
means = gtmlmean(net, data) takes a GTM
+structure net, and computes the means of the responsibility
+distributions for each data point in data.
+
+gtm, gtmpost, gtmlmodeCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmlmode.htm b/sourcecodes/bnt-master/nethelp3.3/gtmlmode.htm new file mode 100644 index 00000000..ab37c6ff --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmlmode.htm @@ -0,0 +1,43 @@ + +
++modes = gtmlmode(net, data) ++ + +
modes = gtmlmode(net, data) takes a GTM
+structure net, and computes the modes of the responsibility
+distributions for each data point in data. These will always lie
+at one of the latent space sample points net.X.
+
+gtm, gtmpost, gtmlmeanCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmmag.htm b/sourcecodes/bnt-master/nethelp3.3/gtmmag.htm new file mode 100644 index 00000000..f8590da7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmmag.htm @@ -0,0 +1,42 @@ + +
++mags = gtmmag(net, latentdata) ++ + +
mags = gtmmag(net, latentdata) takes a GTM
+structure net, and computes the magnification factors
+for each point the latent space contained in latentdata.
+
+gtm, gtmpost, gtmlmeanCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmpost.htm b/sourcecodes/bnt-master/nethelp3.3/gtmpost.htm new file mode 100644 index 00000000..a64ce88a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmpost.htm @@ -0,0 +1,47 @@ + +
++post = gtmpost(net, data) +[post, a] = gtmpost(net, data) ++ + +
post = gtmpost(net, data) takes a GTM
+structure net, and computes the responsibility at each latent space
+sample point net.X
+for each data point in data.
+
+[post, a] = gtmpost(net, data) also returns the activations
+a of the GMM net.gmmnet as computed by gmmpost.
+
+
gtm, gtmem, gtmlmean, gmlmode, gmmprobCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/gtmprob.htm b/sourcecodes/bnt-master/nethelp3.3/gtmprob.htm new file mode 100644 index 00000000..d3a44e30 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/gtmprob.htm @@ -0,0 +1,42 @@ + +
++prob = gtmprob(net, data) ++ + +
prob = gtmprob(net, data) takes a GTM
+structure net, and computes the probability of each point in the
+dataset data.
+
+gtm, gtmem, gtmlmean, gtmlmode, gtmpostCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hbayes.htm b/sourcecodes/bnt-master/nethelp3.3/hbayes.htm new file mode 100644 index 00000000..5fa0783d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hbayes.htm @@ -0,0 +1,50 @@ + +
++h = hbayes(net, hdata) +[h, hdata] = hbayes(net, hdata) ++ + +
h = hbayes(net, hdata) takes a network data structure net together
+the data contribution to the Hessian
+for a set of inputs and targets.
+It returns the regularised Hessian using any zero mean Gaussian priors
+on the weights defined in
+net. In addition, if a mask is defined in net, then
+the entries in h that correspond to weights with a 0 in the
+mask are removed.
+
+[h, hdata] = hbayes(net, hdata) additionally returns the
+data component of the Hessian.
+
+
gbayes, glmhess, mlphess, rbfhessCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hesschek.htm b/sourcecodes/bnt-master/nethelp3.3/hesschek.htm new file mode 100644 index 00000000..34a4f591 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hesschek.htm @@ -0,0 +1,46 @@ + +
++hesschek(net, x, t) +h = hesschek(net, x, t)+ + +
hesschek(net, x, t) takes a network data structure net, together
+with input and target data matrices x and t, and compares
+the evaluation of the Hessian matrix using the function nethess
+and using central differences with the function neterr.
+
+
The optional return value h is the Hessian computed using
+nethess.
+
+
nethess, neterrCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hintmat.htm b/sourcecodes/bnt-master/nethelp3.3/hintmat.htm new file mode 100644 index 00000000..5363385d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hintmat.htm @@ -0,0 +1,45 @@ + +
++[xvals, yvals, color] = hintmat(w)+ + +
+[xvals, yvals, color] = hintmat(w)+ +takes a matrix
w and
+returns coordinates xvals, yvals for the patches comrising the
+Hinton diagram, together with a vector color labelling the color
+(black or white) of the corresponding elements according to their
+sign.
+
+hintonCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hinton.htm b/sourcecodes/bnt-master/nethelp3.3/hinton.htm new file mode 100644 index 00000000..570f3fb0 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hinton.htm @@ -0,0 +1,48 @@ + +
++hinton(w) +h = hinton(w)+ + +
hinton(w) takes a matrix w
+and plots the Hinton diagram.
+
+
h = hinton(net) also returns the figure handle h
+which can be used, for instance, to delete the
+figure when it is no longer needed.
+
+
To print the figure correctly in black and white, you should call
+set(h, 'InvertHardCopy', 'off') before printing.
+
+
demhint, hintmat, mlphintCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/histp.htm b/sourcecodes/bnt-master/nethelp3.3/histp.htm new file mode 100644 index 00000000..fd1fa111 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/histp.htm @@ -0,0 +1,45 @@ + +
++h = histp(x, xmin, xmax, nbins) ++ + +
histp(x, xmin, xmax, nbins) takes a column vector x
+of data values and generates a normalized histogram plot of the
+distribution. The histogram has nbins bins lying in the
+range xmin to xmax.
+
+
h = histp(...) returns a vector of patch handles.
+
+
demgaussCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/hmc.htm b/sourcecodes/bnt-master/nethelp3.3/hmc.htm new file mode 100644 index 00000000..53afeb31 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/hmc.htm @@ -0,0 +1,129 @@ + +
+
+
+samples = hmc(f, x, options, gradf)
+samples = hmc(f, x, options, gradf, P1, P2, ...)
+[samples, energies, diagn] = hmc(f, x, options, gradf)
+s = hmc('state')
+hmc('state', s)
+
+
+
+samples = hmc(f, x, options, gradf) uses a
+hybrid Monte Carlo algorithm to sample from the distribution p ~ exp(-f),
+where f is the first argument to hmc.
+The Markov chain starts at the point x, and the function gradf
+is the gradient of the `energy' function f.
+
+hmc(f, x, options, gradf, p1, p2, ...) allows
+additional arguments to be passed to f() and gradf().
+
+
[samples, energies, diagn] = hmc(f, x, options, gradf) also returns
+a log of the energy values (i.e. negative log probabilities) for the
+samples in energies and diagn, a structure containing
+diagnostic information (position, momentum and
+acceptance threshold) for each step of the chain in diagn.pos,
+diagn.mom and
+diagn.acc respectively. All candidate states (including rejected ones)
+are stored in diagn.pos.
+
+
[samples, energies, diagn] = hmc(f, x, options, gradf) also returns the
+energies (i.e. negative log probabilities) corresponding to the samples.
+The diagn structure contains three fields:
+
+
pos the position vectors of the dynamic process.
+
+
mom the momentum vectors of the dynamic process.
+
+
acc the acceptance thresholds.
+
+
s = hmc('state') returns a state structure that contains the state of the
+two random number generators rand and randn and the momentum of
+the dynamic process. These are contained in fields
+randstate, randnstate
+and mom respectively. The momentum state is
+only used for a persistent momentum update.
+
+
hmc('state', s) resets the state to s. If s is an integer,
+then it is passed to rand and randn and the momentum variable
+is randomised. If s is a structure returned by hmc('state') then
+it resets the generator to exactly the same state.
+
+
The optional parameters in the options vector have the following
+interpretations.
+
+
options(1) is set to 1 to display the energy values and rejection
+threshold at each step of the Markov chain. If the value is 2, then the
+position vectors at each step are also displayed.
+
+
options(5) is set to 1 if momentum persistence is used; default 0, for
+complete replacement of momentum variables.
+
+
options(7) defines the trajectory length (i.e. the number of leap-frog
+steps at each iteration). Minimum value 1.
+
+
options(9) is set to 1 to check the user defined gradient function.
+
+
options(14) is the number of samples retained from the Markov chain;
+default 100.
+
+
options(15) is the number of samples omitted from the start of the
+chain; default 0.
+
+
options(17) defines the momentum used when a persistent update of
+(leap-frog) momentum is used. This is bounded to the interval [0, 1).
+
+
options(18) is the step size used in leap-frogs; default 1/trajectory
+length.
+
+
+
+w = mlppak(net);
+[samples, energies] = hmc('neterr', w, options, 'netgrad', net, x, t);
+
+
+
+metropCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/index.htm b/sourcecodes/bnt-master/nethelp3.3/index.htm new file mode 100644 index 00000000..816e24eb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/index.htm @@ -0,0 +1,537 @@ + +
+The online reference documentation provides direct hypertext links to specific Netlab function descriptions. +
If you have any comments or problems to report, please contact Ian Nabney (i.t.nabney@aston.ac.uk) or Christopher Bishop (c.m.bishop@aston.ac.uk).
+
conffigconfmatconjgradconsistconvertoldnetdatreaddatwritedem2ddatdemarddemev1demev2demev3demgaussdemglm1demglm2demgmm1demgmm3demgmm4demgmm5demgpdemgparddemgpotdemgtm1demgtm2demhintdemhmc1demhmc2demhmc3demkmeandemknn1demmdn1demmet1demmlp1demmlp2demnlabdemns1demolgd1demopt1dempotdemprgpdempriordemrbf1demsom1demtraindist2eigdecerrbayesevidencefevbayesgaussgbayesglmglmderivglmerrglmevfwdglmfwdglmgradglmhessglminitglmpakglmtrainglmunpakgmmgmmactivgmmemgmminitgmmpakgmmpostgmmprobgmmsampgmmunpakgpgpcovargpcovarfgpcovarpgperrgpfwdgpgradgpinitgppakgpunpakgradchekgraddescgsampgtmgtmemgtmfwdgtminitgtmlmeangtmlmodegtmmaggtmpostgtmprobhbayeshesschekhintmathintonhistphmckmeansknnknnfwdlineflineminmaxitmessmdnmdn2gmmmdndist2mdnerrmdnfwdmdngradmdninitmdnpakmdnpostmdnprobmdnunpakmetropminbrackmlpmlpbkpmlpderivmlperrmlpevfwdmlpfwdmlpgradmlphdotvmlphessmlphintmlpinitmlppakmlppriormlptrainmlpunpaknetderivneterrnetevfwdnetgradnethessnetinitnetoptnetpaknetunpakolgdpcaplotmatppcaquasinewrbfrbfbkprbfderivrbferrrbfevfwdrbffwdrbfgradrbfhessrbfjacobrbfpakrbfpriorrbfsetbfrbfsetfwrbftrainrbfunpakrosegradrosenscgsomsomfwdsompaksomtrainsomunpakCopyright (c) Christopher M Bishop, Ian T Nabney (1996, 1997) + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/kmeans.htm b/sourcecodes/bnt-master/nethelp3.3/kmeans.htm new file mode 100644 index 00000000..092e51d3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/kmeans.htm @@ -0,0 +1,89 @@ + +
++centres = kmeans(centres, data, options) +[centres, options] = kmeans(centres, data, options) +[centres, options, post, errlog] = kmeans(centres, data, options) ++ + +
centres = kmeans(centres, data, options)
+uses the batch K-means algorithm to set the centres of a cluster model.
+The matrix data represents the data
+which is being clustered, with each row corresponding to a vector.
+The sum of squares error function is used. The point at which
+a local minimum is achieved is returned as centres. The
+error value at that point is returned in options(8).
+
+[centres, options, post, errlog] = kmeans(centres, data, options)
+also returns the cluster number (in a one-of-N encoding) for each data
+point in post and a log of the error values after each cycle in
+errlog.
+
+The optional parameters have the following interpretations.
+
+
options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog.
+If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is a measure of the absolute precision required for the value
+of centres at the solution. If the absolute difference between
+the values of centres between two successive steps is less than
+options(2), then this condition is satisfied.
+
+
options(3) is a measure of the precision required of the error
+function at the solution. If the absolute difference between the
+error functions between two successive steps is less than
+options(3), then this condition is satisfied.
+Both this and the previous condition must be
+satisfied for termination.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
kmeans can be used to initialise the centres of a Gaussian
+mixture model that is then trained with the EM algorithm.
++ +[priors, centres, var] = gmmunpak(p, md); +centres = kmeans(centres, data, options); +p = gmmpak(priors, centres, var); +p = gmmem(p, md, data, options); ++ + +
gmminit, gmmemCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/knn.htm b/sourcecodes/bnt-master/nethelp3.3/knn.htm new file mode 100644 index 00000000..31f032bc --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/knn.htm @@ -0,0 +1,54 @@ + +
++ +net = knn(nin, nout, k, tr_in, tr_targets) ++ + +
net = knn(nin, nout, k, tr_in, tr_targets) creates a KNN model net
+with input dimension nin, output dimension nout and k
+neighbours. The training data is also stored in the data structure and the
+targets are assumed to be using a 1-of-N coding.
+
+The fields in net are
+
+ + type = 'knn' + nin = number of inputs + nout = number of outputs + tr_in = training input data + tr_targets = training target data ++ + +
kmeans, knnfwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/knnfwd.htm b/sourcecodes/bnt-master/nethelp3.3/knnfwd.htm new file mode 100644 index 00000000..6670920f --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/knnfwd.htm @@ -0,0 +1,66 @@ + +
++ +[y, l] = knnfwd(net, x) ++ + +
[y, l] = knnfwd(net, x) takes a matrix x
+of input vectors (one vector per row)
+ and uses the k-nearest-neighbour rule on the training data contained
+in net to
+produce
+a matrix y of outputs and a matrix l of classification
+labels.
+The nearest neighbours are determined using Euclidean distance.
+The ijth entry of y counts the number of occurrences that
+an example from class j is among the k closest training
+examples to example i from x.
+The matrix l contains the predicted class labels
+as an index 1..N, not as 1-of-N coding.
+
++ +net = knn(size(xtrain, 2), size(t_train, 2), 3, xtrain, t_train); +y = knnfwd(net, xtest); +conffig(y, t_test); ++ +Creates a 3 nearest neighbour model
net and then applies it to
+the data xtest. The results are plotted as a confusion matrix with
+conffig.
+
+kmeans, knnCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/linef.htm b/sourcecodes/bnt-master/nethelp3.3/linef.htm new file mode 100644 index 00000000..c6eb9188 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/linef.htm @@ -0,0 +1,43 @@ + +
+linef(lambda, fn, x, d) calculates the value of the function
+fn at the point x+lambda*d. Here x is a row vector
+and lambda is a scalar.
+
+linef(lambda, fn, x, d, p1, p2, ...) allows additional
+arguments to be passed to fn().
+This function is used for convenience in some of the optimisation routines.
+
+
linemin.
+
+gradchek, lineminCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/linemin.htm b/sourcecodes/bnt-master/nethelp3.3/linemin.htm new file mode 100644 index 00000000..e4d7a8b9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/linemin.htm @@ -0,0 +1,74 @@ + +
+[x, options] = linemin(f, pt, dir, fpt, options) uses Brent's
+algorithm to find the minimum of the function f(x) along the
+line dir through the point pt. The function value at the
+starting point is fpt. The point at which f has a local minimum
+is returned as x. The function value at that point is returned
+in options(8).
+
+linemin(f, pt, dir, fpt, options, p1, p2, ...) allows
+additional arguments to be passed to f().
+
+
The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values.
+
+
options(2) is a measure of the absolute precision required for the value
+of x at the solution.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. Both this and the previous condition must be
+satisfied for termination.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
f in the direction sd can be found in conjgrad
++ +x = linemin(f, xold, sd, fold, lineoptions); ++ + +
minbrack). This is adapted
+to minimize a function along a line.
+This implementation
+is based on that in Numerical Recipes.
+
+conjgrad, minbrack, quasinewCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/maxitmess.htm b/sourcecodes/bnt-master/nethelp3.3/maxitmess.htm new file mode 100644 index 00000000..a42b1f8b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/maxitmess.htm @@ -0,0 +1,41 @@ + +
++s = maxitmess ++ + +
s = maxitmess returns a standard string that it used by training
+algorithms when the maximum number of iterations (as specified in
+options(14) is reached.
+
+conjgrad, glmtrain, gmmem, graddesc, gtmem, kmeans, olgd, quasinew, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdn.htm b/sourcecodes/bnt-master/nethelp3.3/mdn.htm new file mode 100644 index 00000000..5e5b83a8 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdn.htm @@ -0,0 +1,84 @@ + +
++net = mdn(nin, nhidden, ncentres, dimtarget) +net = mdn(nin, nhidden, ncentres, dimtarget, mixtype, ... + prior, beta) ++ + +
net = mdn(nin, nhidden, ncentres, dimtarget) takes the number of
+inputs,
+hidden units for a 2-layer feed-forward
+network and the number of centres and target dimension for the
+mixture model whose parameters are set from the outputs of the neural network.
+The fifth argument mixtype is used to define the type of mixture
+model. (Currently there is only one type supported: a mixture of Gaussians with
+a single covariance parameter for each component.) For this model,
+the mixture coefficients are computed from a group of softmax outputs,
+the centres are equal to a group of linear outputs, and the variances are
+obtained by applying the exponential function to a third group of outputs.
+
+The network is initialised by a call to mlp, and the arguments
+prior, and beta have the same role as for that function.
+Weight initialisation uses the Matlab function randn
+ and so the seed for the random weight initialization can be
+set using randn('state', s) where s is the seed value.
+A specialised data structure (rather than gmm)
+is used for the mixture model outputs to improve
+the efficiency of error and gradient calculations in network training.
+The fields are described in mdnfwd where they are set up.
+
+
The fields in net are
+
+ + type = 'mdn' + nin = number of input variables + nout = dimension of target space (not number of network outputs) + nwts = total number of weights and biases + mdnmixes = data structure for mixture model output + mlp = data structure for MLP network ++ + +
+ +net = mdn(2, 4, 3, 1, 'spherical'); ++ +This creates a Mixture Density Network with 2 inputs and 4 hidden units. +The mixture model has 3 components and the target space has dimension 1. + +
mdnfwd, mdnerr, mdn2gmm, mdngrad, mdnpak, mdnunpak, mlpCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdn2gmm.htm b/sourcecodes/bnt-master/nethelp3.3/mdn2gmm.htm new file mode 100644 index 00000000..51369a50 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdn2gmm.htm @@ -0,0 +1,58 @@ + +
++gmmmixes = mdn2gmm(mdnmixes) ++ + +
gmmmixes = mdn2gmm(mdnmixes) takes an MDN mixture data structure
+mdnmixes
+containing three matrices (for priors, centres and variances) where each
+row represents the corresponding parameter values for a different mixture model
+and creates an array of GMMs. These can then be used with the standard
+Netlab Gaussian mixture model functions.
+
++ +mdnmixes = mdnfwd(net, x); +mixes = mdn2gmm(mdnmixes); +p = gmmprob(mixes(1), y); ++ +This creates an array GMM mixture models (one for each data point in +
x). The vector p is then filled with the conditional
+probabilities of the values y given x(1,:).
+
+gmm, mdn, mdnfwdCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdndist2.htm b/sourcecodes/bnt-master/nethelp3.3/mdndist2.htm new file mode 100644 index 00000000..76713ae4 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdndist2.htm @@ -0,0 +1,50 @@ + +
++n2 = mdndist2(mixparams, t) ++ + +
n2 = mdndist2(mixparams, t) takes takes the centres of the Gaussian
+contained in
+ mixparams and the target data matrix, t, and computes the squared
+Euclidean distance between them. If t has m rows and n
+columns, then the centres field in
+the mixparams structure should have m rows and
+n*mixparams.ncentres columns: the centres in each row relate to
+the corresponding row in t.
+The result has m rows and mixparams.ncentres columns.
+The i, jth entry is the
+squared distance from the ith row of x to the jth
+centre in the ith row of mixparams.centres.
+
+mdnfwd, mdnprobCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnerr.htm b/sourcecodes/bnt-master/nethelp3.3/mdnerr.htm new file mode 100644 index 00000000..41d2f260 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnerr.htm @@ -0,0 +1,46 @@ + +
++e = mdnerr(net, x, t) ++ + +
e = mdnerr(net, x, t) takes a mixture density network data
+structure net, a matrix x of input vectors and a matrix
+t of target vectors, and evaluates the error function
+e. The error function is the negative log likelihood of the
+target data under the conditional density given by the mixture model
+parameterised by the MLP. Each row of x corresponds to one
+input vector and each row of t corresponds to one target vector.
+
+mdn, mdnfwd, mdngradCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnfwd.htm b/sourcecodes/bnt-master/nethelp3.3/mdnfwd.htm new file mode 100644 index 00000000..67c5c5d3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnfwd.htm @@ -0,0 +1,71 @@ + +
++mixparams = mdnfwd(net, x) +[mixparams, y, z] = mdnfwd(net, x) +[mixparams, y, z, a] = mdnfwd(net, x) ++ + +
mixparams = mdnfwd(net, x) takes a mixture density network data
+structure net and a matrix x of input vectors, and forward
+propagates the inputs through the network to generate a structure
+mixparams which contains the parameters of several mixture models.
+Each row of x represents
+one input vector and the corresponding row of the matrices in mixparams
+represents the parameters of a mixture model for the conditional probability
+of target vectors given the input vector. This is not represented as an array
+of gmm structures to improve the efficiency of MDN training.
+
+The fields in mixparams are
+
+ + type = 'mdnmixes' + ncentres = number of mixture components + dimtarget = dimension of target space + mixcoeffs = mixing coefficients + centres = means of Gaussians: stored as one row per pattern + covars = covariances of Gaussians + nparams = number of parameters ++ + +
[mixparams, y, z] = mdnfwd(net, x) also generates a matrix y of
+the outputs of the MLP and a matrix z of the hidden
+unit activations where each row corresponds to one pattern.
+
+
[mixparams, y, z, a] = mlpfwd(net, x) also returns a matrix a
+giving the summed inputs to each output unit, where each row
+corresponds to one pattern.
+
+
mdn, mdn2gmm, mdnerr, mdngrad, mlpfwdCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdngrad.htm b/sourcecodes/bnt-master/nethelp3.3/mdngrad.htm new file mode 100644 index 00000000..b3842991 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdngrad.htm @@ -0,0 +1,47 @@ + +
++ +g = mdngrad(net, x, t) ++ + +
g = mdngrad(net, x, t) takes a mixture density network data
+structure net, a matrix x of input vectors and a matrix
+t of target vectors, and evaluates the gradient g of the
+error function with respect to the network weights. The error function
+is negative log likelihood of the target data. Each row of x
+corresponds to one input vector and each row of t corresponds to
+one target vector.
+
+mdn, mdnfwd, mdnerr, mdnprob, mlpbkpCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdninit.htm b/sourcecodes/bnt-master/nethelp3.3/mdninit.htm new file mode 100644 index 00000000..9e205c51 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdninit.htm @@ -0,0 +1,50 @@ + +
++net = mdninit(net, prior) +net = mdninit(net, prior, t, options) ++ + +
net = mdninit(net, prior) takes a Mixture Density Network
+net and sets the weights and biases by sampling from a Gaussian
+distribution. It calls mlpinit for the MLP component of net.
+
+
net = mdninit(net, prior, t, options) uses the target data t to
+initialise the biases for the output units after initialising the
+other weights as above. It calls gmminit, with t and options
+as arguments, to obtain a model of the unconditional density of t. The
+biases are then set so that net will output the values in the Gaussian
+mixture model.
+
+
mdn, mlp, mlpinit, gmminitCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnpak.htm b/sourcecodes/bnt-master/nethelp3.3/mdnpak.htm new file mode 100644 index 00000000..d5e39033 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnpak.htm @@ -0,0 +1,41 @@ + +
++w = mdnpak(net) ++ + +
w = mdnpak(net) takes a mixture density
+network data structure net and
+combines the network weights into a single row vector w.
+
+mdn, mdnunpak, mdnfwd, mdnerr, mdngradCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnpost.htm b/sourcecodes/bnt-master/nethelp3.3/mdnpost.htm new file mode 100644 index 00000000..3ae071a0 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnpost.htm @@ -0,0 +1,48 @@ + +
++post = mdnpost(mixparams, t) +[post, a] = mdnpost(mixparams, t) ++ + +
post = mdnpost(mixparams, t) computes the posterior
+probability p(j|t) of each
+data vector in t under the Gaussian mixture model represented by the
+corresponding entries in mixparams. Each row of t represents a
+single vector.
+
+[post, a] = mdnpost(mixparams, t) also computes the activations
+a (i.e. the probability p(t|j) of the data conditioned on
+each component density) for a Gaussian mixture model.
+
+
mdngrad, mdnprobCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnprob.htm b/sourcecodes/bnt-master/nethelp3.3/mdnprob.htm new file mode 100644 index 00000000..cffa2517 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnprob.htm @@ -0,0 +1,47 @@ + +
++prob = mdnprob(mixparams, t) +[prob, a] = mdnprob(mixparams, t) ++ + +
prob = mdnprob(mixparams, t) computes the probability p(t) of each
+data vector in t under the Gaussian mixture model represented by the
+corresponding entries in mixparams. Each row of t represents a
+single vector.
+
+[prob, a] = mdnprob(mixparams, t) also computes the activations
+a (i.e. the probability p(t|j) of the data conditioned on
+each component density) for a Gaussian mixture model.
+
+
mdnerr, mdnpostCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mdnunpak.htm b/sourcecodes/bnt-master/nethelp3.3/mdnunpak.htm new file mode 100644 index 00000000..de6d2610 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mdnunpak.htm @@ -0,0 +1,42 @@ + +
++net = mdnunpak(net, w) ++ + +
net = mdnunpak(net, w) takes an mdn network data structure net and
+a weight vector w, and returns a network data structure identical to
+the input network, except that the weights in the MLP sub-structure are
+set to the corresponding elements of w.
+
+mdn, mdnpak, mdnfwd, mdnerr, mdngradCopyright (c) Ian T Nabney (1996-9) +
David J Evans (1998) + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/metrop.htm b/sourcecodes/bnt-master/nethelp3.3/metrop.htm new file mode 100644 index 00000000..93c3ed49 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/metrop.htm @@ -0,0 +1,108 @@ + +
+
+
+samples = metrop(f, x, options)
+samples = metrop(f, x, options, [], P1, P2, ...)
+[samples, energies, diagn] = metrop(f, x, options)
+s = metrop('state')
+metrop('state', s)
+
+
+
+samples = metrop(f, x, options) uses
+the Metropolis algorithm to sample from the distribution
+p ~ exp(-f), where f is the first argument to metrop.
+The Markov chain starts at the point x and each
+candidate state is picked from a Gaussian proposal distribution and
+accepted or rejected according to the Metropolis criterion.
+
+samples = metrop(f, x, options, [], p1, p2, ...) allows
+additional arguments to be passed to f(). The fourth argument is
+ignored, but is included for compatibility with hmc and the
+optimisers.
+
+
[samples, energies, diagn] = metrop(f, x, options) also returns
+a log of the energy values (i.e. negative log probabilities) for the
+samples in energies and diagn, a structure containing
+diagnostic information (position and
+acceptance threshold) for each step of the chain in diagn.pos and
+diagn.acc respectively. All candidate states (including rejected
+ones) are stored in diagn.pos.
+
+
s = metrop('state') returns a state structure that contains the
+state of the two random number generators rand and randn.
+These are contained in fields
+randstate,
+randnstate.
+
+
metrop('state', s) resets the state to s. If s is an integer,
+then it is passed to rand and randn.
+If s is a structure returned by metrop('state') then
+it resets the generator to exactly the same state.
+
+
The optional parameters in the options vector have the following
+interpretations.
+
+
options(1) is set to 1 to display the energy values and rejection
+threshold at each step of the Markov chain. If the value is 2, then the
+position vectors at each step are also displayed.
+
+
options(14) is the number of samples retained from the Markov chain;
+default 100.
+
+
options(15) is the number of samples omitted from the start of the
+chain; default 0.
+
+
options(18) is the variance of the proposal distribution; default 1.
+
+
+
+w = mlppak(net);
+[samples, energies] = metrop('neterr', w, options, 'netgrad', net, x, t);
+
+
+
+hmcCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/minbrack.htm b/sourcecodes/bnt-master/nethelp3.3/minbrack.htm new file mode 100644 index 00000000..f49d18f5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/minbrack.htm @@ -0,0 +1,65 @@ + +
+brmin, brmid, brmax, numevals] = minbrack(f, a, b, fa)
+finds a bracket of three points around a local minimum of
+f. The function f must have a one dimensional domain.
+a < b is an initial guess at the minimum and maximum points
+of a bracket, but minbrack will search outside this interval if
+necessary. The bracket consists of three points (in increasing order)
+such that f(brmid) < f(brmin) and f(brmid) < f(brmax).
+fa is the value of the function at a: it is included to
+avoid unnecessary function evaluations in the optimization routines.
+The return value numevals is the number of function evaluations
+in minbrack.
+
+minbrack(f, a, b, fa, p1, p2, ...) allows additional
+arguments to be passed to f
+
+
f in the direction sd can be found in linemin
+
+
+[min, mid, max, nevals]] = minbrack('linef', 0.0, 1.0, fa, f, pt, dir);
+
+
+where the function linef is used to turn a general function f
+into a one dimensional one.
+
+linemin, linefCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlp.htm b/sourcecodes/bnt-master/nethelp3.3/mlp.htm new file mode 100644 index 00000000..b8237991 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlp.htm @@ -0,0 +1,94 @@ + +
++net = mlp(nin, nhidden, nout, func) +net = mlp(nin, nhidden, nout, func, prior) +net = mlp(nin, nhidden, nout, func, prior, beta) ++ + +
net = mlp(nin, nhidden, nout, func) takes the number of inputs,
+hidden units and output units for a 2-layer feed-forward network,
+together with a string func which specifies the output unit
+activation function, and returns a data structure net. The
+weights are drawn from a zero mean, unit variance isotropic Gaussian,
+with varianced scaled by the fan-in of the hidden or output units as
+appropriate. This makes use of the Matlab function
+randn and so the seed for the random weight initialization can be
+set using randn('state', s) where s is the seed value.
+The hidden units use the tanh activation function.
+
+The fields in net are
+
+ + type = 'mlp' + nin = number of inputs + nhidden = number of hidden units + nout = number of outputs + nwts = total number of weights and biases + actfn = string describing the output unit activation function: + 'linear' + 'logistic + 'softmax' + w1 = first-layer weight matrix + b1 = first-layer bias vector + w2 = second-layer weight matrix + b2 = second-layer bias vector ++ +Here
w1 has dimensions nin times nhidden, b1 has
+dimensions 1 times nhidden, w2 has
+dimensions nhidden times nout, and b2 has
+dimensions 1 times nout.
+
+net = mlp(nin, nhidden, nout, func, prior), in which prior is
+a scalar, allows the field net.alpha in the data structure
+net to be set, corresponding to a zero-mean isotropic Gaussian
+prior with inverse variance with value prior. Alternatively,
+prior can consist of a data structure with fields alpha
+and index, allowing individual Gaussian priors to be set over
+groups of weights in the network. Here alpha is a column vector
+in which each element corresponds to a separate group of weights,
+which need not be mutually exclusive. The membership of the groups is
+defined by the matrix indx in which the columns correspond to
+the elements of alpha. Each column has one element for each
+weight in the matrix, in the order defined by the function
+mlppak, and each element is 1 or 0 according to whether the
+weight is a member of the corresponding group or not. A utility
+function mlpprior is provided to help in setting up the
+prior data structure.
+
+
net = mlp(nin, nhidden, nout, func, prior, beta) also sets the
+additional field net.beta in the data structure net, where
+beta corresponds to the inverse noise variance.
+
+
mlpprior, mlppak, mlpunpak, mlpfwd, mlperr, mlpbkp, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpbkp.htm b/sourcecodes/bnt-master/nethelp3.3/mlpbkp.htm new file mode 100644 index 00000000..00fd6846 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpbkp.htm @@ -0,0 +1,50 @@ + +
++g = mlpbkp(net, x, z, deltas)+ + +
g = mlpbkp(net, x, z, deltas) takes a network data structure
+net together with a matrix x of input vectors, a matrix
+z of hidden unit activations, and a matrix deltas of the
+gradient of the error function with respect to the values of the
+output units (i.e. the summed inputs to the output units, before the
+activation function is applied). The return value is the gradient
+g of the error function with respect to the network
+weights. Each row of x corresponds to one input vector.
+
+This function is provided so that the common backpropagation algorithm +can be used by multi-layer perceptron network models to compute +gradients for mixture density networks as well as standard error +functions. + +
mlp, mlpgrad, mlpderiv, mdngradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpderiv.htm b/sourcecodes/bnt-master/nethelp3.3/mlpderiv.htm new file mode 100644 index 00000000..514dafc6 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpderiv.htm @@ -0,0 +1,43 @@ + +
++g = mlpderiv(net, x)+ + +
g = mlpderiv(net, x) takes a network data structure net
+and a matrix of input vectors x and returns a three-index matrix
+g whose i, j, k element contains the
+derivative of network output k with respect to weight or bias
+parameter j for input pattern i. The ordering of the
+weight and bias parameters is defined by mlpunpak.
+
+mlp, mlppak, mlpgrad, mlpbkpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlperr.htm b/sourcecodes/bnt-master/nethelp3.3/mlperr.htm new file mode 100644 index 00000000..33d748da --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlperr.htm @@ -0,0 +1,49 @@ + +
++e = mlperr(net, x, t) ++ + +
e = mlperr(net, x, t) takes a network data structure net together
+with a matrix x of input vectors and a matrix t of target
+vectors, and evaluates the error function e. The choice of error
+function corresponds to the output unit activation function. Each row
+of x corresponds to one input vector and each row of t
+corresponds to one target vector.
+
+[e, edata, eprior] = mlperr(net, x, t) additionally returns the
+data and prior components of the error, assuming a zero mean Gaussian
+prior on the weights with inverse variance parameters alpha and
+beta taken from the network data structure net.
+
+
mlp, mlppak, mlpunpak, mlpfwd, mlpbkp, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/mlpevfwd.htm new file mode 100644 index 00000000..1cf75307 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpevfwd.htm @@ -0,0 +1,50 @@ + +
++ +[y, extra] = mlpevfwd(net, x, t, x_test) +[y, extra, invhess] = mlpevfwd(net, x, t, x_test, invhess) ++ + +
y = mlpevfwd(net, x, t, x_test) takes a network data structure
+net together with the input x and target t training data
+and input test data x_test.
+It returns the normal forward propagation through the network y
+together with a matrix extra which consists of error bars (variance)
+for a regression problem or moderated outputs for a classification problem.
+The optional argument (and return value)
+invhess is the inverse of the network Hessian
+computed on the training data inputs and targets. Passing it in avoids
+recomputing it, which can be a significant saving for large training sets.
+
+fevbayesCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpfwd.htm b/sourcecodes/bnt-master/nethelp3.3/mlpfwd.htm new file mode 100644 index 00000000..2f2be47a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpfwd.htm @@ -0,0 +1,52 @@ + +
++y = mlpfwd(net, x) +[y, z] = mlpfwd(net, x) +[y, z, a] = mlpfwd(net, x) ++ + +
y = mlpfwd(net, x) takes a network data structure net together with
+a matrix x of input vectors, and forward propagates the inputs
+through the network to generate a matrix y of output
+vectors. Each row of x corresponds to one input vector and each
+row of y corresponds to one output vector.
+
+[y, z] = mlpfwd(net, x) also generates a matrix z of the hidden
+unit activations where each row corresponds to one pattern.
+
+
[y, z, a] = mlpfwd(net, x) also returns a matrix a
+giving the summed inputs to each output unit, where each row
+corresponds to one pattern.
+
+
mlp, mlppak, mlpunpak, mlperr, mlpbkp, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpgrad.htm b/sourcecodes/bnt-master/nethelp3.3/mlpgrad.htm new file mode 100644 index 00000000..08943cc9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpgrad.htm @@ -0,0 +1,51 @@ + +
++ +g = mlpgrad(net, x, t) ++ + +
g = mlpgrad(net, x, t) takes a network data structure net
+together with a matrix x of input vectors and a matrix t
+of target vectors, and evaluates the gradient g of the error
+function with respect to the network weights. The error funcion
+corresponds to the choice of output unit activation function. Each row
+of x corresponds to one input vector and each row of t
+corresponds to one target vector.
+
+[g, gdata, gprior] = mlpgrad(net, x, t) also returns separately
+the data and prior contributions to the gradient. In the case of
+multiple groups in the prior, gprior is a matrix with a row
+for each group and a column for each weight parameter.
+
+
mlp, mlppak, mlpunpak, mlpfwd, mlperr, mlpbkpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlphdotv.htm b/sourcecodes/bnt-master/nethelp3.3/mlphdotv.htm new file mode 100644 index 00000000..7ecf2765 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlphdotv.htm @@ -0,0 +1,45 @@ + +
++hdv = mlphdotv(net, x, t, v)+ + +
hdv = mlphdotv(net, x, t, v) takes an MLP network data structure
+net, together with the matrix x of input vectors, the
+matrix t of target vectors and an arbitrary row vector v
+whose length equals the number of parameters in the network, and
+returns the product of the data-dependent contribution to the Hessian
+matrix with v. The implementation is based on the R-propagation
+algorithm of Pearlmutter.
+
+
mlp, mlphess, hesschekCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlphess.htm b/sourcecodes/bnt-master/nethelp3.3/mlphess.htm new file mode 100644 index 00000000..59a6ec91 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlphess.htm @@ -0,0 +1,73 @@ + +
++h = mlphess(net, x, t) +[h, hdata] = mlphess(net, x, t) +h = mlphess(net, x, t, hdata) ++ + +
h = mlphess(net, x, t) takes an MLP network data structure net,
+a matrix x of input values, and a matrix t of target
+values and returns the full Hessian matrix h corresponding to
+the second derivatives of the negative log posterior distribution,
+evaluated for the current weight and bias values as defined by
+net.
+
+[h, hdata] = mlphess(net, x, t) returns both the Hessian matrix
+h and the contribution hdata arising from the data dependent
+term in the Hessian.
+
+
h = mlphess(net, x, t, hdata) takes a network data structure
+net, a matrix x of input values, and a matrix t of
+target values, together with the contribution hdata arising from
+the data dependent term in the Hessian, and returns the full Hessian
+matrix h corresponding to the second derivatives of the negative
+log posterior distribution. This version saves computation time if
+hdata has already been evaluated for the current weight and bias
+values.
+
+
+ + h = beta*hd + alpha*I ++ +where the contribution
hd is evaluated by calls to mlphdotv and
+h is the full Hessian.
+
+mlp, hesschek, mlphdotv, evidenceCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlphint.htm b/sourcecodes/bnt-master/nethelp3.3/mlphint.htm new file mode 100644 index 00000000..82411403 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlphint.htm @@ -0,0 +1,50 @@ + +
++mlphint(net) +[h1, h2] = mlphint(net)+ + +
mlphint(net) takes a network structure net
+and plots the Hinton diagram comprised of two
+figure windows, one displaying the first-layer weights and biases, and
+one displaying the second-layer weights and biases.
+
+
[h1, h2] = mlphint(net) also returns handles h1 and
+h2 to the figures which can be used, for instance, to delete the
+figures when they are no longer needed.
+
+
To print the figure correctly, you should call
+set(h, 'InvertHardCopy', 'on') before printing.
+
+
demhint, hintmat, mlp, mlppak, mlpunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpinit.htm b/sourcecodes/bnt-master/nethelp3.3/mlpinit.htm new file mode 100644 index 00000000..2f7dbd4e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpinit.htm @@ -0,0 +1,48 @@ + +
++net = mlpinit(net, prior) ++ + +
net = mlpinit(net, prior) takes a 2-layer feedforward network
+net and sets the weights and biases by sampling from a Gaussian
+distribution. If prior is a scalar, then all of the parameters
+(weights and biases) are sampled from a single isotropic Gaussian with
+inverse variance equal to prior. If prior is a data
+structure of the kind generated by mlpprior, then the parameters
+are sampled from multiple Gaussians according to their groupings
+(defined by the index field) with corresponding variances
+(defined by the alpha field).
+
+
mlp, mlpprior, mlppak, mlpunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlppak.htm b/sourcecodes/bnt-master/nethelp3.3/mlppak.htm new file mode 100644 index 00000000..aa038f46 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlppak.htm @@ -0,0 +1,55 @@ + +
++w = mlppak(net) ++ + +
w = mlppak(net) takes a network data structure net and
+combines the component weight matrices bias vectors into a single row
+vector w. The facility to switch between these two
+representations for the network parameters is useful, for example, in
+training a network by error function minimization, since a single
+vector of parameters can be handled by general-purpose optimization
+routines.
+
+The ordering of the paramters in w is defined by
+
+ + w = [net.w1(:)', net.b1, net.w2(:)', net.b2]; ++ +where
w1 is the first-layer weight matrix, b1 is the
+first-layer bias vector, w2 is the second-layer weight matrix,
+and b2 is the second-layer bias vector.
+
+mlp, mlpunpak, mlpfwd, mlperr, mlpbkp, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpprior.htm b/sourcecodes/bnt-master/nethelp3.3/mlpprior.htm new file mode 100644 index 00000000..1821e919 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpprior.htm @@ -0,0 +1,58 @@ + +
++prior = mlpprior(nin, nhidden, nout, aw1, ab1, aw2, ab2)+ + +
prior = mlpprior(nin, nhidden, nout, aw1, ab1, aw2, ab2)
+generates a data structure
+prior, with fields prior.alpha and prior.index, which
+specifies a Gaussian prior distribution for the network weights in a
+two-layer feedforward network. Two different cases are possible. In
+the first case, aw1, ab1, aw2 and ab2 are all
+scalars and represent the regularization coefficients for four groups
+of parameters in the network corresponding to first-layer weights,
+first-layer biases, second-layer weights, and second-layer biases
+respectively. Then prior.alpha represents a column vector of
+length 4 containing the parameters, and prior.index is a matrix
+specifying which weights belong in each group. Each column has one
+element for each weight in the matrix, using the standard ordering as
+defined in mlppak, and each element is 1 or 0 according to
+whether the weight is a member of the corresponding group or not. In
+the second case the parameter aw1 is a vector of length equal to
+the number of inputs in the network, and the corresponding matrix
+prior.index now partitions the first-layer weights into groups
+corresponding to the weights fanning out of each input unit. This
+prior is appropriate for the technique of automatic relevance
+determination.
+
+mlp, mlperr, mlpgrad, evidenceCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlptrain.htm b/sourcecodes/bnt-master/nethelp3.3/mlptrain.htm new file mode 100644 index 00000000..ad89786b --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlptrain.htm @@ -0,0 +1,35 @@ + +
+[net, error] = mlptrain(net, x, t, its) trains a network data
+structure net using the scaled conjugate gradient algorithm
+for its cycles with
+input data x, target data t.
+
+
demtrain, scg, netoptCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/mlpunpak.htm b/sourcecodes/bnt-master/nethelp3.3/mlpunpak.htm new file mode 100644 index 00000000..3c6bba21 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/mlpunpak.htm @@ -0,0 +1,44 @@ + +
++net = mlpunpak(net, w) ++ + +
net = mlpunpak(net, w) takes an mlp network data structure net and
+a weight vector w, and returns a network data structure identical to
+the input network, except that the first-layer weight matrix
+w1, the first-layer bias vector b1, the second-layer
+weight matrix w2 and the second-layer bias vector b2 have all
+been set to the corresponding elements of w.
+
+mlp, mlppak, mlpfwd, mlperr, mlpbkp, mlpgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netderiv.htm b/sourcecodes/bnt-master/nethelp3.3/netderiv.htm new file mode 100644 index 00000000..3275eced --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netderiv.htm @@ -0,0 +1,42 @@ + +
++g = netderiv(w, net, x)+ + +
g = netderiv(w, net, x) takes a weight vector w and a network
+data structure net, together with the matrix x of input
+vectors, and returns the
+gradient of the outputs with respect to the weights evaluated at w.
+
+
netevfwd, netoptCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/neterr.htm b/sourcecodes/bnt-master/nethelp3.3/neterr.htm new file mode 100644 index 00000000..2ec53e61 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/neterr.htm @@ -0,0 +1,47 @@ + +
++e = neterr(w, net, x, t) +[e, varargout] = neterr(w, net, x, t) ++ + +
e = neterr(w, net, x, t) takes a weight vector w and a network
+data structure net, together with the matrix x of input
+vectors and the matrix t of target vectors, and returns the
+value of the error function evaluated at w.
+
+
[e, varargout] = neterr(w, net, x, t) also returns any additional
+return values from the error function.
+
+
netgrad, nethess, netoptCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/netevfwd.htm new file mode 100644 index 00000000..ea876bb2 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netevfwd.htm @@ -0,0 +1,52 @@ + +
++ +[y, extra] = netevfwd(w, net, x, t, x_test) +[y, extra, invhess] = netevfwd(w, net, x, t, x_test, invhess) ++ + +
[y, extra] = netevfwd(w, net, x, t, x_test) takes a network data
+structure
+net together with the input x and target t training data
+and input test data x_test.
+It returns the normal forward propagation through the network y
+together with a matrix extra which consists of error bars (variance)
+for a regression problem or moderated outputs for a classification problem.
+
+The optional argument (and return value)
+invhess is the inverse of the network Hessian
+computed on the training data inputs and targets. Passing it in avoids
+recomputing it, which can be a significant saving for large training sets.
+
+
mlpevfwd, rbfevfwd, glmevfwd, fevbayesCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netgrad.htm b/sourcecodes/bnt-master/nethelp3.3/netgrad.htm new file mode 100644 index 00000000..a9ce1571 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netgrad.htm @@ -0,0 +1,42 @@ + +
++g = netgrad(w, net, x, t)+ + +
g = netgrad(w, net, x, t) takes a weight vector w and a network
+data structure net, together with the matrix x of input
+vectors and the matrix t of target vectors, and returns the
+gradient of the error function evaluated at w.
+
+
mlp, neterr, netoptCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/nethelp3.3.zip b/sourcecodes/bnt-master/nethelp3.3/nethelp3.3.zip new file mode 100644 index 00000000..5c316a1d Binary files /dev/null and b/sourcecodes/bnt-master/nethelp3.3/nethelp3.3.zip differ diff --git a/sourcecodes/bnt-master/nethelp3.3/nethess.htm b/sourcecodes/bnt-master/nethelp3.3/nethess.htm new file mode 100644 index 00000000..b918fe0a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/nethess.htm @@ -0,0 +1,66 @@ + +
++h = neterr(w, net, x, t) +[h, varargout] = neterr(w, net, x, t, varargin) ++ + +
h = nethess(w, net, x, t) takes a weight vector w and a network
+data structure net, together with the matrix x of input
+vectors and the matrix t of target vectors, and returns the
+value of the Hessian evaluated at w.
+
+
[e, varargout] = nethess(w, net, x, t, varargin) also returns any additional
+return values from the network Hessian function, and passes additional arguments
+to that function.
+
+
In evidence, this function is called once to compute the
+data contribution to the Hessian
+
+ +[h, dh] = nethess(w, net, x, t, dh); ++ +and again to update the Hessian for new values of the hyper-parameters +
+ +h = nethess(w, net, x, t, dh); ++ + +
neterr, netgrad, netoptCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netinit.htm b/sourcecodes/bnt-master/nethelp3.3/netinit.htm new file mode 100644 index 00000000..3612b574 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netinit.htm @@ -0,0 +1,48 @@ + +
++net = netinit(net, prior) ++ + +
net = netinit(net, prior) takes a network data structure
+net and sets the weights and biases by sampling from a Gaussian
+distribution. If prior is a scalar, then all of the parameters
+(weights and biases) are sampled from a single isotropic Gaussian with
+inverse variance equal to prior. If prior is a data
+structure of the kind generated by mlpprior, then the parameters
+are sampled from multiple Gaussians according to their groupings
+(defined by the index field) with corresponding variances
+(defined by the alpha field).
+
+
mlpprior, netunpak, rbfpriorCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netopt.htm b/sourcecodes/bnt-master/nethelp3.3/netopt.htm new file mode 100644 index 00000000..369c959c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netopt.htm @@ -0,0 +1,75 @@ + +
++[net, options] = netopt(net, options, x, t, alg) +[net, options, varargout] = netopt(net, options, x, t, alg) ++ + +
netopt is a helper function which facilitates the training of
+networks using the general purpose optimizers as well as sampling from the
+posterior distribution of parameters using general purpose Markov chain
+Monte Carlo sampling algorithms. It can be used with any function that
+searches in parameter space using error and gradient functions.
+
+
[net, options] = netopt(net, options, x, t, alg) takes a network
+data structure net, together with a vector options of
+parameters governing the behaviour of the optimization algorithm, a
+matrix x of input vectors and a matrix t of target
+vectors, and returns the trained network as well as an updated
+options vector. The string alg determines which optimization
+algorithm (conjgrad, quasinew, scg, etc.) or Monte
+Carlo algorithm (such as hmc) will be used.
+
+
[net, options, varargout] = netopt(net, options, x, t, alg)
+also returns any additional return values from the optimisation algorithm.
+
+
net = mlp(4, 3, 2, 'linear'). We can then train
+the network with the scaled conjugate gradient algorithm by using
+net = netopt(net, options, x, t, 'scg') where x and
+t are the input and target data matrices respectively, and the
+options vector is set appropriately for scg.
+
+If we also wish to plot the learning curve, we can use the additional
+return value errlog given by scg:
+
+ +[net, options, errlog] = netopt(net, options, x, t, 'scg'); ++ + +
netgrad, bfgs, conjgrad, graddesc, hmc, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netpak.htm b/sourcecodes/bnt-master/nethelp3.3/netpak.htm new file mode 100644 index 00000000..29286b7d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netpak.htm @@ -0,0 +1,47 @@ + +
++w = netpak(net) ++ + +
w = netpak(net) takes a network data structure net and
+combines the component weight matrices into a single row
+vector w. The facility to switch between these two
+representations for the network parameters is useful, for example, in
+training a network by error function minimization, since a single
+vector of parameters can be handled by general-purpose optimization
+routines. This function also takes into account a mask defined
+as a field in net by removing any weights that correspond to
+entries of 0 in the mask.
+
+net, netunpak, netfwd, neterr, netgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/netunpak.htm b/sourcecodes/bnt-master/nethelp3.3/netunpak.htm new file mode 100644 index 00000000..fef34ed7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/netunpak.htm @@ -0,0 +1,46 @@ + +
++net = netunpak(net, w) ++ + +
net = netunpak(net, w) takes an net network data structure net and
+a weight vector w, and returns a network data structure identical to
+the input network, except that the componenet weight matrices have all
+been set to the corresponding elements of w. If there is
+a mask field in the net data structure, then the weights in
+w are placed in locations corresponding to non-zero entries in the
+mask (so w should have the same length as the number of non-zero
+entries in the mask).
+
+netpak, netfwd, neterr, netgradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/olgd.htm b/sourcecodes/bnt-master/nethelp3.3/olgd.htm new file mode 100644 index 00000000..15417b70 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/olgd.htm @@ -0,0 +1,101 @@ + +
+[net, options, errlog, pointlog] = olgd(net, options, x, t) uses
+on-line gradient descent to find a local minimum of the error function for the
+network
+net computed on the input data x and target values
+t. A log of the error values
+after each cycle is (optionally) returned in errlog, and a log
+of the points visited is (optionally) returned in pointlog.
+Because the gradient is computed on-line (i.e. after each pattern)
+this can be quite inefficient in Matlab.
+
+The error function value at final weight vector is returned
+in options(8).
+
+
The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog, and the points visited
+in the return argument pointslog. If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is the precision required for the value
+of x at the solution. If the absolute difference between
+the values of x between two successive steps is less than
+options(2), then this condition is satisfied.
+
+
options(3) is the precision required of the objective
+function at the solution. If the absolute difference between the
+error functions between two successive steps is less than
+options(3), then this condition is satisfied.
+Both this and the previous condition must be
+satisfied for termination. Note that testing the function value at each
+iteration roughly halves the speed of the algorithm.
+
+
options(5) determines whether the patterns are sampled randomly
+with replacement. If it is 0 (the default), then patterns are sampled
+in order.
+
+
options(6) determines if the learning rate decays. If it is 1
+then the learning rate decays at a rate of 1/t. If it is 0
+(the default) then the learning rate is constant.
+
+
options(9) should be set to 1 to check the user defined gradient
+function.
+
+
options(10) returns the total number of function evaluations (including
+those in any line searches).
+
+
options(11) returns the total number of gradient evaluations.
+
+
options(14) is the maximum number of iterations (passes through
+the complete pattern set); default 100.
+
+
options(17) is the momentum; default 0.5.
+
+
options(18) is the learning rate; default 0.01.
+
+
+ +net = mlp(5, 3, 1, 'linear'); +options = foptions; +options(18) = 0.01; +options(5) = 1; +net = olgd(net, options, x, t); ++ + +
graddescCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/pca.htm b/sourcecodes/bnt-master/nethelp3.3/pca.htm new file mode 100644 index 00000000..84fe3e38 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/pca.htm @@ -0,0 +1,51 @@ + +
++PCcoeff = pca(data) +PCcoeff = pca(data, N) +[PCcoeff, PCvec] = pca(data) ++ + +
PCcoeff = pca(data) computes the eigenvalues of the covariance
+matrix of the dataset data and returns them as PCcoeff. These
+coefficients give the variance of data along the corresponding
+principal components.
+
+PCcoeff = pca(data, N) returns the largest N eigenvalues.
+
+
[PCcoeff, PCvec] = pca(data) returns the principal components as
+well as the coefficients. This is considerably more computationally
+demanding than just computing the eigenvalues.
+
+
eigdec, gtminit, ppcaCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/plotmat.htm b/sourcecodes/bnt-master/nethelp3.3/plotmat.htm new file mode 100644 index 00000000..a9a38cc9 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/plotmat.htm @@ -0,0 +1,42 @@ + +
++plotmat(matrix, textcolour, gridcolour, fontsize)+ + +
plotmat(matrix, textcolour, gridcolour, fontsize) displays the matrix
+matrix on the current figure. The textcolour and gridcolour
+arguments control the colours of the numbers and grid labels respectively and
+should follow the usual Matlab specification.
+The parameter fontsize should be an integer.
+
+conffig, demmlp2Copyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/ppca.htm b/sourcecodes/bnt-master/nethelp3.3/ppca.htm new file mode 100644 index 00000000..b55a61c4 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/ppca.htm @@ -0,0 +1,46 @@ + +
++[var, U, lambda] = pca(x, ppca_dim) ++ + +
[var, U, lambda] = ppca(x, ppca_dim) computes the principal component
+subspace U of dimension ppca_dim using a centred
+covariance matrix x. The variable var contains
+the off-subspace variance (which is assumed to be spherical), while the
+vector lambda contains the variances of each of the principal
+components. This is computed using the eigenvalue and eigenvector
+decomposition of x.
+
+eigdec, pcaCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/quasinew.htm b/sourcecodes/bnt-master/nethelp3.3/quasinew.htm new file mode 100644 index 00000000..528ecb1d --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/quasinew.htm @@ -0,0 +1,99 @@ + +
+[x, options, flog, pointlog] = quasinew(f, x, options, gradf)
+uses a quasi-Newton
+algorithm to find a local minimum of the function f(x) whose
+gradient is given by gradf(x). Here x is a row vector
+and f returns a scalar value.
+The point at which f has a local minimum
+is returned as x. The function value at that point is returned
+in options(8). A log of the function values
+after each cycle is (optionally) returned in flog, and a log
+of the points visited is (optionally) returned in pointlog.
+
+quasinew(f, x, options, gradf, p1, p2, ...) allows
+additional arguments to be passed to f() and gradf().
+
+
The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog, and the points visited
+in the return argument pointslog. If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is a measure of the absolute precision required for the value
+of x at the solution. If the absolute difference between
+the values of x between two successive steps is less than
+options(2), then this condition is satisfied.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. If the absolute difference between the
+objective function values between two successive steps is less than
+options(3), then this condition is satisfied.
+Both this and the previous condition must be
+satisfied for termination.
+
+
options(9) should be set to 1 to check the user defined gradient
+function.
+
+
options(10) returns the total number of function evaluations (including
+those in any line searches).
+
+
options(11) returns the total number of gradient evaluations.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
options(15) is the precision in parameter space of the line search;
+default 1e-2.
+
+
+
+w = quasinew('neterr', w, options, 'netgrad', net, x, t);
+
+
+
+conjgrad, graddesc, linemin, minbrack, scgCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbf.htm b/sourcecodes/bnt-master/nethelp3.3/rbf.htm new file mode 100644 index 00000000..97a7f925 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbf.htm @@ -0,0 +1,114 @@ + +
++ +net = rbf(nin, nhidden, nout, rbfunc) +net = rbf(nin, nhidden, nout, rbfunc, outfunc) +net = rbf(nin, nhidden, nout, rbfunc, outfunc, prior, beta) ++ + +
net = rbf(nin, nhidden, nout, rbfunc) constructs and initialises
+a radial basis function network returning a data structure net.
+The weights are all initialised with a zero mean, unit variance normal
+distribution, with the exception of the variances, which are set to one.
+This makes use of the Matlab function
+randn and so the seed for the random weight initialization can be
+set using randn('state', s) where s is the seed value. The
+activation functions are defined in terms of the distance between
+the data point and the corresponding centre. Note that the functions are
+computed to a convenient constant multiple: for example, the Gaussian
+is not normalised. (Normalisation is not needed as the function outputs
+are linearly combined in the next layer.)
+
+The fields in net are
+
+ + type = 'rbf' + nin = number of inputs + nhidden = number of hidden units + nout = number of outputs + nwts = total number of weights and biases + actfn = string defining hidden unit activation function: + 'gaussian' for a radially symmetric Gaussian function. + 'tps' for r^2 log r, the thin plate spline function. + 'r4logr' for r^4 log r. + outfn = string defining output error function: + 'linear' for linear outputs (default) and SoS error. + 'neuroscale' for Sammon stress measure. + c = centres + wi = squared widths (null for rlogr and tps) + w2 = second layer weight matrix + b2 = second layer bias vector ++ + +
net = rbf(nin, nhidden, nout, rbfund, outfunc) allows the user to
+specify the type of error function to be used. The field outfn
+is set to the value of this string. Linear outputs (for regression problems)
+and Neuroscale outputs (for topographic mappings) are supported.
+
+
net = rbf(nin, nhidden, nout, rbfunc, outfunc, prior, beta),
+in which prior is
+a scalar, allows the field net.alpha in the data structure
+net to be set, corresponding to a zero-mean isotropic Gaussian
+prior with inverse variance with value prior. Alternatively,
+prior can consist of a data structure with fields alpha
+and index, allowing individual Gaussian priors to be set over
+groups of weights in the network. Here alpha is a column vector
+in which each element corresponds to a separate group of weights,
+which need not be mutually exclusive. The membership of the groups is
+defined by the matrix indx in which the columns correspond to
+the elements of alpha. Each column has one element for each
+weight in the matrix, in the order defined by the function
+rbfpak, and each element is 1 or 0 according to whether the
+weight is a member of the corresponding group or not. A utility
+function rbfprior is provided to help in setting up the
+prior data structure.
+
+
net = rbf(nin, nhidden, nout, func, prior, beta) also sets the
+additional field net.beta in the data structure net, where
+beta corresponds to the inverse noise variance.
+
+
x through it.
++ +net = rbf(1, 5, 1, 'tps'); +[y, act] = rbffwd(net, x); ++ + +
rbferr, rbffwd, rbfgrad, rbfpak, rbftrain, rbfunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfbkp.htm b/sourcecodes/bnt-master/nethelp3.3/rbfbkp.htm new file mode 100644 index 00000000..cf3a9d52 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfbkp.htm @@ -0,0 +1,51 @@ + +
++g = rbfbkp(net, x, z, n2, deltas)+ + +
g = rbfbkp(net, x, z, n2, deltas) takes a network data structure
+net together with a matrix x of input vectors, a matrix
+z of hidden unit activations, a matrix n2 of the squared
+distances between centres and inputs, and a matrix deltas of the
+gradient of the error function with respect to the values of the
+output units (i.e. the summed inputs to the output units, before the
+activation function is applied). The return value is the gradient
+g of the error function with respect to the network
+weights. Each row of x corresponds to one input vector.
+
+This function is provided so that the common backpropagation algorithm
+can be used by RBF network models to compute
+gradients for the output values (in rbfderiv) as well as standard error
+functions.
+
+
rbf, rbfgrad, rbfderivCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfderiv.htm b/sourcecodes/bnt-master/nethelp3.3/rbfderiv.htm new file mode 100644 index 00000000..f9c0438a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfderiv.htm @@ -0,0 +1,44 @@ + +
++g = rbfderiv(net, x)+ + +
g = rbfderiv(net, x) takes a network data structure net
+and a matrix of input vectors x and returns a three-index matrix
+g whose i, j, k element contains the
+derivative of network output k with respect to weight or bias
+parameter j for input pattern i. The ordering of the
+weight and bias parameters is defined by rbfunpak. This
+function also takes into account any mask in the network data structure.
+
+rbf, rbfpak, rbfgrad, rbfbkpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbferr.htm b/sourcecodes/bnt-master/nethelp3.3/rbferr.htm new file mode 100644 index 00000000..5a97de46 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbferr.htm @@ -0,0 +1,50 @@ + +
++e = rbferr(net, x, t) +[e, edata, eprior] = rbferr(net, x, t) ++ + +
e = rbferr(net, x, t) takes a network data structure net together
+with a matrix x of input
+vectors and a matrix t of target vectors, and evaluates the
+appropriate error function e depending on net.outfn.
+Each row of x corresponds to one
+input vector and each row of t contains the corresponding target vector.
+
+[e, edata, eprior] = rbferr(net, x, t) additionally returns the
+data and prior components of the error, assuming a zero mean Gaussian
+prior on the weights with inverse variance parameters alpha and
+beta taken from the network data structure net.
+
+
rbf, rbffwd, rbfgrad, rbfpak, rbftrain, rbfunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfevfwd.htm b/sourcecodes/bnt-master/nethelp3.3/rbfevfwd.htm new file mode 100644 index 00000000..21513c59 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfevfwd.htm @@ -0,0 +1,51 @@ + +
++ +[y, extra] = rbfevfwd(net, x, t, x_test) +[y, extra, invhess] = rbfevfwd(net, x, t, x_test, invhess) ++ + +
y = rbfevfwd(net, x, t, x_test) takes a network data structure
+net together with the input x and target t training data
+and input test data x_test.
+It returns the normal forward propagation through the network y
+together with a matrix extra which consists of error bars (variance)
+for a regression problem or moderated outputs for a classification problem.
+
+The optional argument (and return value)
+invhess is the inverse of the network Hessian
+computed on the training data inputs and targets. Passing it in avoids
+recomputing it, which can be a significant saving for large training sets.
+
+
fevbayesCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbffwd.htm b/sourcecodes/bnt-master/nethelp3.3/rbffwd.htm new file mode 100644 index 00000000..02e56931 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbffwd.htm @@ -0,0 +1,67 @@ + +
++a = rbffwd(net, x) +function [a, z, n2] = rbffwd(net, x) ++ + +
a = rbffwd(net, x) takes a network data structure
+net and a matrix x of input
+vectors and forward propagates the inputs through the network to generate
+a matrix a of output vectors. Each row of x corresponds to one
+input vector and each row of a contains the corresponding output vector.
+The activation function that is used is determined by net.actfn.
+
+[a, z, n2] = rbffwd(net, x) also generates a matrix z of
+the hidden unit activations where each row corresponds to one pattern.
+These hidden unit activations represent the design matrix for
+the RBF. The matrix n2 is the squared distances between each
+basis function centre and each pattern in which each row corresponds
+to a data point.
+
+
+ +[a, z] = rbffwd(net, x); + ++ +Heretemp = pinv([z ones(size(x, 1), 1)]) * t; +net.w2 = temp(1: nd(2), :); +net.b2 = temp(size(x, nd(2)) + 1, :); +
x is the input data, t are the target values, and we use the
+pseudo-inverse to find the output weights and biases.
+
+rbf, rbferr, rbfgrad, rbfpak, rbftrain, rbfunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfgrad.htm b/sourcecodes/bnt-master/nethelp3.3/rbfgrad.htm new file mode 100644 index 00000000..30759dfe --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfgrad.htm @@ -0,0 +1,55 @@ + +
++ +g = rbfgrad(net, x, t) +[g, gdata, gprior] = rbfgrad(net, x, t) ++ + +
g = rbfgrad(net, x, t) takes a network data structure net
+together with a matrix x of input
+vectors and a matrix t of target vectors, and evaluates the gradient
+g of the error function with respect to the network weights (i.e.
+including the hidden unit parameters). The error
+function is sum of squares.
+Each row of x corresponds to one
+input vector and each row of t contains the corresponding target vector.
+If the output function is 'neuroscale' then the gradient is only
+computed for the output layer weights and biases.
+
+[g, gdata, gprior] = rbfgrad(net, x, t) also returns separately
+the data and prior contributions to the gradient. In the case of
+multiple groups in the prior, gprior is a matrix with a row
+for each group and a column for each weight parameter.
+
+
rbf, rbffwd, rbferr, rbfpak, rbfunpak, rbfbkpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfhess.htm b/sourcecodes/bnt-master/nethelp3.3/rbfhess.htm new file mode 100644 index 00000000..a9c6be79 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfhess.htm @@ -0,0 +1,72 @@ + +
++h = rbfhess(net, x, t) +[h, hdata] = rbfhess(net, x, t) +h = rbfhess(net, x, t, hdata) ++ + +
h = rbfhess(net, x, t) takes an RBF network data structure net,
+a matrix x of input values, and a matrix t of target
+values and returns the full Hessian matrix h corresponding to
+the second derivatives of the negative log posterior distribution,
+evaluated for the current weight and bias values as defined by
+net. Currently, the implementation only computes the
+Hessian for the output layer weights.
+
+[h, hdata] = rbfhess(net, x, t) returns both the Hessian matrix
+h and the contribution hdata arising from the data dependent
+term in the Hessian.
+
+
h = rbfhess(net, x, t, hdata) takes a network data structure
+net, a matrix x of input values, and a matrix t of
+target values, together with the contribution hdata arising from
+the data dependent term in the Hessian, and returns the full Hessian
+matrix h corresponding to the second derivatives of the negative
+log posterior distribution. This version saves computation time if
+hdata has already been evaluated for the current weight and bias
+values.
+
+
+ + h = beta*hdata + alpha*I ++ + +
mlphess, hesschek, evidenceCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfjacob.htm b/sourcecodes/bnt-master/nethelp3.3/rbfjacob.htm new file mode 100644 index 00000000..853e59a3 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfjacob.htm @@ -0,0 +1,42 @@ + +
++g = rbfjacob(net, x)+ + +
g = rbfjacob(net, x) takes a network data structure net
+and a matrix of input vectors x and returns a three-index matrix
+g whose i, j, k element contains the
+derivative of network output k with respect to input
+parameter j for input pattern i.
+
+rbf, rbfgrad, rbfbkpCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfpak.htm b/sourcecodes/bnt-master/nethelp3.3/rbfpak.htm new file mode 100644 index 00000000..08769b16 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfpak.htm @@ -0,0 +1,40 @@ + +
++w = rbfpak(net) ++ + +
w = rbfpak(net) takes a network data structure net and combines
+the component parameter matrices into a single row vector w.
+
+rbfunpak, rbfCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfprior.htm b/sourcecodes/bnt-master/nethelp3.3/rbfprior.htm new file mode 100644 index 00000000..202bdde7 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfprior.htm @@ -0,0 +1,59 @@ + +
++[mask, prior] = rbfprior(rbfunc, nin, nhidden, nout, aw2, ab2)+ + +
[mask, prior] = rbfprior(rbfunc, nin, nhidden, nout, aw2, ab2)
+generates a vector
+mask that selects only the output
+layer weights. This is because most uses of RBF networks in a Bayesian
+context have fixed basis functions with the output layer as the only
+adjustable parameters. In particular, the Neuroscale output error function
+is designed to work only with this mask.
+
+The return value
+prior is a data structure,
+with fields prior.alpha and prior.index, which
+specifies a Gaussian prior distribution for the network weights in an
+RBF network. The parameters aw2 and ab2 are all
+scalars and represent the regularization coefficients for two groups
+of parameters in the network corresponding to
+ second-layer weights, and second-layer biases
+respectively. Then prior.alpha represents a column vector of
+length 2 containing the parameters, and prior.index is a matrix
+specifying which weights belong in each group. Each column has one
+element for each weight in the matrix, using the standard ordering as
+defined in rbfpak, and each element is 1 or 0 according to
+whether the weight is a member of the corresponding group or not.
+
+
rbf, rbferr, rbfgrad, evidenceCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfsetbf.htm b/sourcecodes/bnt-master/nethelp3.3/rbfsetbf.htm new file mode 100644 index 00000000..c4fbf443 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfsetbf.htm @@ -0,0 +1,43 @@ + +
++net = rbfsetbf(net, options, x) ++ + +
net = rbfsetbf(net, options, x) sets the basis functions of the
+RBF network net so that they model the unconditional density of the
+dataset x. This is done by training a GMM with spherical covariances
+using gmmem. The options vector is passed to gmmem.
+The widths of the functions are set by a call to rbfsetfw.
+
+rbftrain, rbfsetfw, gmmemCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfsetfw.htm b/sourcecodes/bnt-master/nethelp3.3/rbfsetfw.htm new file mode 100644 index 00000000..ae3fab56 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfsetfw.htm @@ -0,0 +1,46 @@ + +
++net = rbfsetfw(net, scale) ++ + +
net = rbfsetfw(net, scale) sets the widths of
+the basis functions of the
+RBF network net.
+If Gaussian basis functions are used, then the variances are set to
+the largest squared distance between centres if scale is non-positive
+and scale times the mean distance of each centre to its nearest
+neighbour if scale is positive. Non-Gaussian basis functions do
+not have a width.
+
+rbftrain, rbfsetbf, gmmemCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbftrain.htm b/sourcecodes/bnt-master/nethelp3.3/rbftrain.htm new file mode 100644 index 00000000..04c2bef5 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbftrain.htm @@ -0,0 +1,90 @@ + +
+net = rbftrain(net, options, x, t) uses a
+two stage training
+algorithm to set the weights in the RBF model structure net.
+Each row of x corresponds to one
+input vector and each row of t contains the corresponding target vector.
+The centres are determined by fitting a Gaussian mixture model
+with circular covariances using the EM algorithm through a call to
+rbfsetbf. (The mixture model is
+initialised using a small number of iterations of the K-means algorithm.)
+If the activation functions are Gaussians, then the basis function widths
+are then set to the maximum inter-centre squared distance.
+
+For linear outputs, +the hidden to output +weights that give rise to the least squares solution +can then be determined using the pseudo-inverse. For neuroscale outputs, +the hidden to output weights are determined using the iterative shadow +targets algorithm. + Although this two stage +procedure may not give solutions with as low an error as using general +purpose non-linear optimisers, it is much faster. + +
The options vector may have two rows: if this is the case, then the second row
+is passed to rbfsetbf, which allows the user to specify a different
+number iterations for RBF and GMM training.
+The optional parameters to rbftrain have the following interpretations.
+
+
options(1) is set to 1 to display error values during EM training.
+
+
options(2) is a measure of the precision required for the value
+of the weights w at the solution.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. Both this and the previous condition must be
+satisfied for termination.
+
+
options(5) is set to 1 if the basis functions parameters should remain
+unchanged; default 0.
+
+
options(6) is set to 1 if the output layer weights should be should
+set using PCA. This is only relevant for Neuroscale outputs; default 0.
+
+
options(14) is the maximum number of iterations for the shadow
+targets algorithm;
+default 100.
+
+
+ +net = rbf(1, 4, 1, 'gaussian'); +options(1, :) = foptions; +options(2, :) = foptions; +options(2, 14) = 10; % 10 iterations of EM +options(2, 5) = 1; % Check for covariance collapse in EM +net = rbftrain(net, options, x, t); ++ + +
rbf, rbferr, rbffwd, rbfgrad, rbfpak, rbfunpak, rbfsetbfCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rbfunpak.htm b/sourcecodes/bnt-master/nethelp3.3/rbfunpak.htm new file mode 100644 index 00000000..20464916 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rbfunpak.htm @@ -0,0 +1,44 @@ + +
++net = rbfunpak(net, w) ++ + +
net = rbfunpak(net, w) takes an RBF network data structure net and
+a weight vector w, and returns a network data structure identical to
+the input network, except that the centres
+c, the widths wi, the second-layer
+weight matrix w2 and the second-layer bias vector b2 have all
+been set to the corresponding elements of w.
+
+rbfpak, rbfCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rosegrad.htm b/sourcecodes/bnt-master/nethelp3.3/rosegrad.htm new file mode 100644 index 00000000..fc4fa9d6 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rosegrad.htm @@ -0,0 +1,40 @@ + +
++g = rosegrad(x) ++ + +
g = rosegrad(x) computes the gradient of Rosenbrock's function
+at each row of x, which should have two columns.
+
+demopt1, rosenCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/rosen.htm b/sourcecodes/bnt-master/nethelp3.3/rosen.htm new file mode 100644 index 00000000..2b8f37aa --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/rosen.htm @@ -0,0 +1,40 @@ + +
++y = rosen(x) ++ + +
y = rosen(x) computes the value of Rosenbrock's function
+at each row of x, which should have two columns.
+
+demopt1, rosegradCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/scg.htm b/sourcecodes/bnt-master/nethelp3.3/scg.htm new file mode 100644 index 00000000..e2e512e2 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/scg.htm @@ -0,0 +1,97 @@ + +
+[x, options] = scg(f, x, options, gradf) uses a scaled conjugate
+gradients
+algorithm to find a local minimum of the function f(x) whose
+gradient is given by gradf(x). Here x is a row vector
+and f returns a scalar value.
+The point at which f has a local minimum
+is returned as x. The function value at that point is returned
+in options(8).
+
+[x, options, flog, pointlog, scalelog] = scg(f, x, options, gradf)
+also returns (optionally) a log of the function values
+after each cycle in flog, a log
+of the points visited in pointlog, and a log of the scale values
+in the algorithm in scalelog.
+
+
scg(f, x, options, gradf, p1, p2, ...) allows
+additional arguments to be passed to f() and gradf().
+
+The optional parameters have the following interpretations.
+
+
options(1) is set to 1 to display error values; also logs error
+values in the return argument errlog, and the points visited
+in the return argument pointslog. If options(1) is set to 0,
+then only warning messages are displayed. If options(1) is -1,
+then nothing is displayed.
+
+
options(2) is a measure of the absolute precision required for the value
+of x at the solution. If the absolute difference between
+the values of x between two successive steps is less than
+options(2), then this condition is satisfied.
+
+
options(3) is a measure of the precision required of the objective
+function at the solution. If the absolute difference between the
+objective function values between two successive steps is less than
+options(3), then this condition is satisfied.
+Both this and the previous condition must be
+satisfied for termination.
+
+
options(9) is set to 1 to check the user defined gradient function.
+
+
options(10) returns the total number of function evaluations (including
+those in any line searches).
+
+
options(11) returns the total number of gradient evaluations.
+
+
options(14) is the maximum number of iterations; default 100.
+
+
+
+w = scg('neterr', w, options, 'netgrad', net, x, t);
+
+
+
+nparams
+successful weight updates where nparams is the total number of
+parameters in x. The algorithm is based on that given by Williams
+(1991), with a simplified procedure for updating lambda when
+rho < 0.25.
+
+conjgrad, quasinewCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/som.htm b/sourcecodes/bnt-master/nethelp3.3/som.htm new file mode 100644 index 00000000..637e752a --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/som.htm @@ -0,0 +1,58 @@ + +
++ +net = som(nin, map_size) ++ + +
net = som(nin, map_size) creates a SOM net
+with input dimension (i.e. data dimension) nin and map dimensions
+map_size. Only two-dimensional maps are currently implemented.
+
+The fields in net are
+
+ + type = 'som' + nin = number of inputs + map_dim = dimension of map (constrained to be 2) + map_size = grid size: number of nodes in each dimension + num_nodes = number of nodes: the product of values in map_size + map = map_dim+1 dimensional array containing nodes + inode_dist = map of inter-node distances using Manhatten metric ++ + +
The map contains the node vectors arranged column-wise in the first +dimension of the array. + +
kmeans, somfwd, somtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/somfwd.htm b/sourcecodes/bnt-master/nethelp3.3/somfwd.htm new file mode 100644 index 00000000..82102b4c --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/somfwd.htm @@ -0,0 +1,59 @@ + +
++ +d2 = somfwd(net, x) ++ + +
d2 = somfwd(net, x) propagates the data matrix x through
+ a SOM net, returning the squared distance matrix d2 with
+dimension nin by num_nodes. The $i$th row represents the
+squared Euclidean distance to each of the nodes of the SOM.
+
+[d2, win_nodes] = somfwd(net, x) also returns the indices of the
+winning nodes for each pattern.
+
+
The following code fragment creates a SOM with a $5times 5$ map for an +8-dimensional data space. It then applies the test data to the map. +
+ +net = som(8, [5, 5]); +[d2, wn] = somfwd(net, test_data); ++ + +
som, somtrainCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/sompak.htm b/sourcecodes/bnt-master/nethelp3.3/sompak.htm new file mode 100644 index 00000000..902ef52e --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/sompak.htm @@ -0,0 +1,44 @@ + +
++c = sompak(net) ++ + +
c = sompak(net) takes a SOM data structure net and
+combines the node weights into a matrix of centres
+c where each row represents the node vector.
+
+The ordering of the parameters in w is defined by the indexing of the
+multi-dimensional array net.map.
+
+
som, somunpakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/somtrain.htm b/sourcecodes/bnt-master/nethelp3.3/somtrain.htm new file mode 100644 index 00000000..ecf4e2cb --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/somtrain.htm @@ -0,0 +1,104 @@ + +
+
+
+net = somtrain{net, options, x)
+
+
+
+net = somtrain{net, options, x) uses Kohonen's algorithm to
+train a SOM. Both on-line and batch algorithms are implemented.
+The learning rate (for on-line) and neighbourhood size decay linearly.
+There is no error function minimised during training (so there is
+no termination criterion other than the number of epochs), but the
+sum-of-squares is computed and returned in options(8).
+
+The optional parameters have the following interpretations. + +
options(1) is set to 1 to display error values; also logs learning
+rate alpha and neighbourhood size nsize.
+Otherwise nothing is displayed.
+
+
options(5) determines whether the patterns are sampled randomly
+with replacement. If it is 0 (the default), then patterns are sampled
+in order. This is only relevant to the on-line algorithm.
+
+
options(6) determines if the on-line or batch algorithm is
+used. If it is 1
+then the batch algorithm is used. If it is 0
+(the default) then the on-line algorithm is used.
+
+
options(14) is the maximum number of iterations (passes through
+the complete pattern set); default 100.
+
+
options(15) is the final neighbourhood size; default value is the
+same as the initial neighbourhood size.
+
+
options(16) is the final learning rate; default value is the same
+as the initial learning rate.
+
+
options(17) is the initial neighbourhood size; default 0.5*maximum
+map size.
+
+
options(18) is the initial learning rate; default 0.9. This parameter
+must be positive.
+
+
+ +net = som(nin, [8, 7]); +options = foptions; + ++ + +% Ordering phase +options(1) = 1; +options(14) = 50; +options(18) = 0.9; % Initial learning rate +options(16) = 0.05; % Final learning rate +options(17) = 8; % Initial neighbourhood size +options(15) = 1; % Final neighbourhood size +net2 = somtrain(net, options, x); + +
% Convergence phase +options(14) = 400; +options(18) = 0.05; +options(16) = 0.01; +options(17) = 0; +options(15) = 0; +net3 = somtrain(net2, options, x); +
kmeans, som, somfwdCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file diff --git a/sourcecodes/bnt-master/nethelp3.3/somunpak.htm b/sourcecodes/bnt-master/nethelp3.3/somunpak.htm new file mode 100644 index 00000000..68032339 --- /dev/null +++ b/sourcecodes/bnt-master/nethelp3.3/somunpak.htm @@ -0,0 +1,44 @@ + +
++net = somunpak(net, w) ++ + +
net = somunpak(net, w) takes a SOM data structure net and
+weight matrix w (each node represented by a row) and
+puts the nodes back into the multi-dimensional array net.map.
+
+The ordering of the parameters in w is defined by the indexing of the
+multi-dimensional array net.map.
+
+
som, sompakCopyright (c) Ian T Nabney (1996-9) + + + + \ No newline at end of file -- cgit 1.4.1