A compact updating formula for quasi-Newton minimization algorithms

Abstract

Quasi-Newton algorithms minimize a functionF(x),x ∈R ⁿ, searching at any iterationk along the directions _k=−H _kg_k, whereg _k=∇F(x _k) andH _k approximates in some sense the inverse Hessian ofF(x) atx _k.

When the matrixH is updated according to the formulas in Broyden's family and when an exact line search is performed at any iteration, a compact algorithm (free from the Broyden's family parameter) can be conceived in terms of the followingn ×n matrix:

$$H{_R} = H - Hgg{^T} H/g{^T} Hg,$$

which can be viewed as an approximating reduced inverse Hessian.

In this paper, a new algorithm is proposed which uses at any iteration an (n−1)×(n−1) matrixK related toH _R by

$$H_R = Q\left[ {\begin{array}{*{20}c} 0 & 0 \\ 0 & K \\ \end{array} } \right]Q$$

whereQ is a suitable orthogonaln×n matrix. The updating formula in terms of the matrixK incorporated in this algorithm is only moderately more complicated than the standard updating formulas for variable-metric methods, but, at the same time, it updates at any iteration a positive definite matrixK, instead of a singular matrixH _R.

Other than the compactness with respect to the algorithms with updating formulas in Broyden's class, a further noticeable feature of the reduced Hessian algorithm is that the downhill condition can be stated in a simple way, and thus efficient line searches may be implemented.