Mathematical Programming

, Volume 35, Issue 1, pp 71–82 | Cite as

Broyden's method in Hilbert space

  • Ekkehard W. Sachs


Broyden's method is formulated for the solution of nonlinear operator equations in Hilbert spaces. The algorithm is proven to be well defined and a linear rate of convergence is shown. Under an additional assumption on the initial approximation for the derivative we prove the superlinear rate of convergence.

Key words

Broyden's method quasi-Newton methods superlinear convergence rate 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    S. Agmon,Lectures on elliptic boundary value problems (Van Nostrand, Princeton, 1965).Google Scholar
  2. [2]
    P.M. Anselone,Collectively compact operator approximation theory and applications to integral equations (Prentice-Hall, Englewood Cliffs, 1971).Google Scholar
  3. [3]
    C.G. Broyden, “A class of methods for solving nonlinear simultaneous equations“,Mathematics of Computation 19(1965) 577–593.Google Scholar
  4. [4]
    C.G. Broyden, “The convergence of single-rank quasi-Newton methods“,Mathematics of Computation 24(1970) 365–382.Google Scholar
  5. [5]
    C.G. Broyden, J.E. Dennis, Jr., and J.J. Moré, “On the local and superlinear convergence of quasi-Newton methods“,Journal of Institute of Mathematics and Applications 12(1973) 223–246.Google Scholar
  6. [6]
    D.W. Decker and C.T. Kelley, “Broyden's method for a class of problems having singular Jacobian at the root“,SIAM Journal Control and Optimization 22(1985) 566–574.Google Scholar
  7. [7]
    J.E. Dennis, Jr., “Toward a unified convergence theory for Newton-like methods“, in: L.B. Rall, ed.,Nonlinear functional analysis and applications (Academic Press, New York, London, 1971) pp. 425–472.Google Scholar
  8. [8]
    J.E. Dennis, Jr. and J.J. Moré, “A characterization of superlinear convergence and its application to quasi-Newton methods“,Mathematics of Computation 28(1974) 543–560.Google Scholar
  9. [9]
    J. Dieudonné,Foundations of modern analysis (Academic Press, New York and London, 1960).Google Scholar
  10. [10]
    Z. Fortuna, “Some convergence properties of the conjugate gradient method in Hilbert space“,SIAM Journal on Numerical Analysis 16(1979) 380–384.Google Scholar
  11. [11]
    A Griewank, “The superlinear convergence of secant method on mildly non-linear problems in Hilbert space”, preprint, Southern Methodist University (Dallas, August 1984).Google Scholar
  12. [12]
    W.A. Gruver and E. Sachs,Algorithmic methods in optimal control (Pitman, Boston, London, Melbourne, 1981).Google Scholar
  13. [13]
    L.B. Horwitz and P.E. Sarachik, “Davidon's methods in Hilbert space“,SIAM Journal on Applied Mathematics 16(1968) 676–695.Google Scholar
  14. [14]
    L.V. Kantorovich, “Functional analysis and applied mathematics“,Upspekhi Matematiki Nauk 3(1948) 89–185.Google Scholar
  15. [15]
    L.V. Kantorovich and G.P. Akilov,Functional analysis in normed spaces (MacMillan, New York, 1964).Google Scholar
  16. [16]
    T. Kato,Perturbation theory for linear operators (Springer-Verlag, New York, 1966).Google Scholar
  17. [17]
    R.H. Martin,Nonlinear operators and differential equations in Banach spaces (Wiley, New York-Chichester-Brisbane-Toronto, 1976).Google Scholar
  18. [18]
    K. Madsen, “Minimax solutions of nonlinear equations without calculating derivatives“,Mathematical Programming Study 3(1975) 110–126.Google Scholar
  19. [19]
    R.V. Mayorga and V.H. Quintana, “A family of variable metric methods in function space, without exact line searches“,Journal of Optimization Theory and Applications 31 (1980) 303–329.Google Scholar
  20. [20]
    R.H. Moore, “Approximations to nonlinear operator equations and Newton'smethods“,Numerische Mathematik 12(1968) 23–34.Google Scholar
  21. [21]
    J.J. Moré and J.A. Trangenstein, “On the global convergence of Broyden's method“,Mathematics of Computation 30(1976) 523–540.Google Scholar
  22. [22]
    E. Sachs, “Global convergence of quasi-Newton-type algorithms for some nonsmooth optimization problems“,Journal of Optimization Theory and Applications 40(1983) 201–219.Google Scholar
  23. [23]
    E. Sachs, “Convergence rates of quasi-Newton algorithms for some nonsmooth optimization problems“,SIAM Journal on Control and Optimization 23(1985) 401–418.Google Scholar
  24. [24]
    M.J. Todd, “Quasi-Newton updates in abstract vector spaces“,SIAM Review 26(1984) 367–377.Google Scholar
  25. [25]
    H. Tokumaru, N. Adachi, and K. Goto, “Davidon's method for minimization problems in Hilbert space with an application to control problems“,SIAM Journal on Control and Optimization 8(1970) 163–178.Google Scholar
  26. [26]
    P.R. Turner and E. Huntley, “Variable metric methods in Hilbert space with applications to control problems“,Journal of Optimization Theory and Applications 19(1976) 381–400.Google Scholar
  27. [27]
    R. Winther, “A Numerical Galerkin Method for a Parabolic Problem”, Ph.D. Thesis, Cornell University (Ithaca, NY, 1977).Google Scholar

Copyright information

© The Mathematical Programming Society, Inc. 1986

Authors and Affiliations

  • Ekkehard W. Sachs
    • 1
  1. 1.Department of MathematicsNorth Carolina State UniversityRaleighUSA

Personalised recommendations