Line search methods for optimization and root-finding
This package provides an interface to line search algorithms implemented in Julia. The code was originally written as part of Optim, but has now been separated out to its own package.
In the docs we show how to choose between the line search algorithms
in Optim
.
HagerZhang
(Taken from the Conjugate Gradient implementation
by Hager and Zhang, 2006)MoreThuente
(From the algorithm in More and Thuente, 1994)BackTracking
(Described in Nocedal and Wright, 2006)StrongWolfe
(Nocedal and Wright)Static
(Takes the proposed initial step length.)The package provides some procedures to calculate the initial step
length that is passed to the line search algorithm. See the docs for
its usage in Optim
.
InitialPrevious
(Use the step length from the previous
optimization iteration)InitialStatic
(Use the same initial step length each time)InitialHagerZhang
(Taken from Hager and Zhang, 2006)InitialQuadratic
(Propose initial step length based on a quadratic
interpolation)InitialConstantChange
(Propose initial step length assuming
constant change in step length)For more details and options, see the documentation
Here is how to get a simple linesearch for a one-dimensional function working:
using LineSearches
ϕ(x) = (x - π)^4
dϕ(x) = 4*(x-π)^3
ϕdϕ(x) = ϕ(x),dϕ(x)
α0 = 9.0
ϕ0 = ϕ(0.0)
dϕ0 = dϕ(0.0)
for ls in (Static,BackTracking,HagerZhang,MoreThuente,StrongWolfe)
res = (ls())(ϕ, dϕ, ϕdϕ, α0, ϕ0,dϕ0)
println(ls, ": ", res)
end
For more examples, see the documentation.