Abstract
The numerical solution of nonlinear equation systems is often achieved by so-called quasi-Newton methods. They preserve the rapid local convergence of Newton’s method at a significantly reduced cost per step by successively approximating the system Jacobian though low-rank updates. We analyze two variants of the recently proposed adjoint Broyden update, which for the first time combines the classical least change property with heredity on affine systems. However, the new update does require, the evaluation of so-called adjoint vectors, namely products of the transposed Jacobian with certain dual direction vectors. The resulting quasi-Newton method is linear contravariant in the sense of Deuflhard (Newton methods for nonlinear equations. Springer, Heidelberg, 2006) and it is shown here to be locally and q-superlinearly convergent. Our numerical results on a range of test problems demonstrate that the new method usually outperforms Newton’s and Broyden’s method in terms of runtime and iterations count, respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bücker, H.M., Corliss, G.F., Hovland, P., Naumann, U., Norris, B. (eds.): Automatic Differentiation: Applications, Theory, and Implementations. Lecture Notes in Computational Science and Engineering, vol 50. Springer, New York (2005)
Broyden C.G., Dennis J.E. Jr, Moré J.J.: On the local and superlinear convergence of quasi-Newton methods. JIMA 12, 223–246 (1973)
Bennett J.M.: Triangular factors of modified matrices. Numer. Math. 7, 217–221 (1965)
Broyden C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comp. 19, 577–593 (1965)
Broyden C.G.: Quasi-Newton methods and their application to function minimization. Math. Comp. 21, 368–381 (1967)
Brown K.M.: A quadratic convergent Newton-like method based upon Gaussian elimination. J. Numer. Anal. 6, 560–569 (1969)
Broyden C.G.: The convergence of an algorithm for solving sparse nonlinear systems. Math. Comp. 25, 285–294 (1971)
Conn A.R., Gould N.I.M., Toint Ph.L.: Convergence of quasi-Newton matrices generated by the symmetric rank one update. Math. Progr. 50, 177–195 (1991)
Davidon W.C.: Variance algorithms for minimization. Comp. J. 10, 406–410 (1968)
Deuflhard P.: Newton Methods for Nonlinear Equations. Springer, Heidelberg (2006)
Dennis J.E. Jr, Moré J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comp. 28, 549–560 (1974)
Dennis J.E. Jr, Schnabel R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, New Jersey (1996)
Fiacco A.V., McCormick G.P.: Nonlinear Programming. Wiley, New York (1968)
Gay D.M.: Some convergence properties of Broyden’s method. SIAM J. Numer. Anal. 16, 623–630 (1979)
Griewank A., Juedes D., Utke J.: ADOL-C: A package for automatic differentiation of algorithms written in C/C++. TOMS 22, 131–167 (1996)
Griewank A.: The “global” convergence of Broyden-like methods with a suitable line search. J. Aust. Math. Soc. Ser. B 28, 75–92 (1986)
Griewank, A.: Evaluating derivatives: principles and techniques of algorithmic differentiation. Number 19 in Frontiers in Appl. Math. SIAM, 2000
Gay D.M., Schnabel R.B.: Solving systems of nonlinear equations by Broyden’s method with projected updates. In: Mangasarian, O., Meyer, R., Robinson, S.(eds) Nonlinear Programming, vol 3, pp. 245–281. Academic Press, NY (1978)
Griewank A., Schlenkrich S., Walther A.: A quasi-Newton method with optimal R-order without independence assumption. Opt. Meth. Soft. 23(2), 215–225 (2008)
Griewank, A., Schlenkrich, S., Walther, A.: Adjoint Broyden a la GMRES. MATHEON Preprint ??? (2007)
Griewank A., Walther A.: On constrained optimization by adjoint based quasi-Newton methods. Opt. Meth. Soft. 17, 869–889 (2002)
Haber E.: Quasi-Newton methods for large scale electromagnetic inverse problems. Inverse Problems 21, 305–317 (2004)
Kielbasinski A., Schwetlick H.: Numerische Lineare Algebra. VEB Deutscher Verlag der Wissenschaften, Berlin (1988)
Moré J.J., Cosnard M.Y.: Numerical solution of nonlinear equations. TOMS 5, 64–85 (1979)
Moré J.J., Garbow B.S., Hillstrom K.E.: Testing unconstrained optimization software. TOMS 7, 17–41 (1981)
Murtagh B.A., Sargent R.W.H. : A constrained minimization method with quadratic convergence. In: Fletcher, R. Optimization, Academic Press, London (1969)
Nocedal J., Wright S.J.: Numerical Optimization. Springer, Heidelberg (1999)
Saad Y.: Iterative Methods for Sparse Linear Systems. SIAM, Philadelphia (2003)
Schwetlick H.: Numerische Lösung nichtlinearer Gleichungen. VEB Deutscher Verlag der Wissenschaften, Berlin (1979)
Schlenkrich S.: Adjoint-based Quasi-Newton Methods for Nonlinear Equations. Sierke Verlag, Gottingen (2007)
Stange, P., Griewank, A., Bollhöfer, M.: On the efficient update of rectangular LU factorizations subject to low rank modifications. TU Berlin, Preprint 2005/27 (2005)
Spedicato, E.: Computational experience with quasi-Newton algorithms for minimization problems of moderately large size. Rep. CISE-N-175, Segrate (Milano) (1975)
Schlenkrich, S., Walther, A.: Global convergence of quasi-Newton methods based on Adjoint Tangent Rank-1 updates. TU Dresden Preprint MATH-WR-02-2006. Appl. Numer. Math. (2008), doi:10.1016/j.apnum.2008.05.007
Schlenkrich, S., Walther, A., Griewank, A.: AD-based quasi-Newton methods for the integration of stiff ODEs. In Bücker et al. [1], pp. 89–98
Wolfe, P.: Another variable metric method. working paper (1968)
Author information
Authors and Affiliations
Corresponding author
Additional information
Partially supported by the DFG Research Center Matheon “Mathematics for Key Technologies”, Berlin and the DFG grant WA 1607/2-1.
Rights and permissions
About this article
Cite this article
Schlenkrich, S., Griewank, A. & Walther, A. On the local convergence of adjoint Broyden methods. Math. Program. 121, 221–247 (2010). https://doi.org/10.1007/s10107-008-0232-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-008-0232-y