1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
|
<html>
<head>
<title>
Netlab Reference Manual quasinew
</title>
</head>
<body>
<H1> quasinew
</H1>
<h2>
Purpose
</h2>
Quasi-Newton optimization.
<p><h2>
Description
</h2>
<CODE>[x, options, flog, pointlog] = quasinew(f, x, options, gradf)</CODE>
uses a quasi-Newton
algorithm to find a local minimum of the function <CODE>f(x)</CODE> whose
gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector
and <CODE>f</CODE> returns a scalar value.
The point at which <CODE>f</CODE> has a local minimum
is returned as <CODE>x</CODE>. The function value at that point is returned
in <CODE>options(8)</CODE>. A log of the function values
after each cycle is (optionally) returned in <CODE>flog</CODE>, and a log
of the points visited is (optionally) returned in <CODE>pointlog</CODE>.
<p><CODE>quasinew(f, x, options, gradf, p1, p2, ...)</CODE> allows
additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>.
<p>The optional parameters have the following interpretations.
<p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error
values in the return argument <CODE>errlog</CODE>, and the points visited
in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0,
then only warning messages are displayed. If <CODE>options(1)</CODE> is -1,
then nothing is displayed.
<p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value
of <CODE>x</CODE> at the solution. If the absolute difference between
the values of <CODE>x</CODE> between two successive steps is less than
<CODE>options(2)</CODE>, then this condition is satisfied.
<p><CODE>options(3)</CODE> is a measure of the precision required of the objective
function at the solution. If the absolute difference between the
objective function values between two successive steps is less than
<CODE>options(3)</CODE>, then this condition is satisfied.
Both this and the previous condition must be
satisfied for termination.
<p><CODE>options(9)</CODE> should be set to 1 to check the user defined gradient
function.
<p><CODE>options(10)</CODE> returns the total number of function evaluations (including
those in any line searches).
<p><CODE>options(11)</CODE> returns the total number of gradient evaluations.
<p><CODE>options(14)</CODE> is the maximum number of iterations; default 100.
<p><CODE>options(15)</CODE> is the precision in parameter space of the line search;
default <CODE>1e-2</CODE>.
<p><h2>
Examples
</h2>
An example of
the use of the additional arguments is the minimization of an error
function for a neural network:
<PRE>
w = quasinew('neterr', w, options, 'netgrad', net, x, t);
</PRE>
<p><h2>
Algorithm
</h2>
The quasi-Newton algorithm builds up an
approximation to the inverse Hessian over a number of steps. The
method requires order W squared storage, where W is the number of function
parameters. The Broyden-Fletcher-Goldfarb-Shanno formula for the
inverse Hessian updates is used. The line searches are carried out to
a relatively low precision (1.0e-2).
<p><h2>
See Also
</h2>
<CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr>
<b>Pages:</b>
<a href="index.htm">Index</a>
<hr>
<p>Copyright (c) Ian T Nabney (1996-9)
</body>
</html>
|
