1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
|
<html>
<head>
<title>
Netlab Reference Manual glmhess
</title>
</head>
<body>
<H1> glmhess
</H1>
<h2>
Purpose
</h2>
Evaluate the Hessian matrix for a generalised linear model.
<p><h2>
Synopsis
</h2>
<PRE>
h = glmhess(net, x, t)
[h, hdata] = glmhess(net, x, t)
h = glmhess(net, x, t, hdata)
</PRE>
<p><h2>
Description
</h2>
<CODE>h = glmhess(net, x, t)</CODE> takes a GLM network data structure <CODE>net</CODE>,
a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of target
values and returns the full Hessian matrix <CODE>h</CODE> corresponding to
the second derivatives of the negative log posterior distribution,
evaluated for the current weight and bias values as defined by
<CODE>net</CODE>. Note that the target data is not required in the calculation,
but is included to make the interface uniform with <CODE>nethess</CODE>. For
linear and logistic outputs, the computation is very simple and is
done (in effect) in one line in <CODE>glmtrain</CODE>.
<p><CODE>[h, hdata] = glmhess(net, x, t)</CODE> returns both the Hessian matrix
<CODE>h</CODE> and the contribution <CODE>hdata</CODE> arising from the data dependent
term in the Hessian.
<p><CODE>h = glmhess(net, x, t, hdata)</CODE> takes a network data structure
<CODE>net</CODE>, a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of
target values, together with the contribution <CODE>hdata</CODE> arising from
the data dependent term in the Hessian, and returns the full Hessian
matrix <CODE>h</CODE> corresponding to the second derivatives of the negative
log posterior distribution. This version saves computation time if
<CODE>hdata</CODE> has already been evaluated for the current weight and bias
values.
<p><h2>
Example
</h2>
The Hessian matrix is used by <CODE>glmtrain</CODE> to take a Newton step for
softmax outputs.
<PRE>
Hessian = glmhess(net, x, t);
deltaw = -gradient*pinv(Hessian);
</PRE>
<p><h2>
See Also
</h2>
<CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE>, <CODE><a href="nethess.htm">nethess</a></CODE><hr>
<b>Pages:</b>
<a href="index.htm">Index</a>
<hr>
<p>Copyright (c) Ian T Nabney (1996-9)
</body>
</html>
|
