Journal of Global Optimization

, Volume 49, Issue 3, pp 365–379 | Cite as

Quasi-Newton methods in infinite-dimensional spaces and application to matrix equations

  • Boubakeur BenahmedEmail author
  • Hocine Mokhtar-Kharroubi
  • Bruno de Malafosse
  • Adnan Yassine


In the first part of this paper, we give a survey on convergence rates analysis of quasi-Newton methods in infinite Hilbert spaces for nonlinear equations. Then, in the second part we apply quasi-Newton methods in their Hilbert formulation to solve matrix equations. So, we prove, under natural assumptions, that quasi-Newton methods converge locally and superlinearly; the global convergence is also studied. For numerical calculations, we propose new formulations of these methods based on the matrix representation of the dyadic operator and the vectorization of matrices. Finally, we apply our results to algebraic Riccati equations.


Nonlinear equations Optimization problems Quasi-Newton methods Rate of convergence Linear convergence Superlinear convergence Hilbert space Matrix equations Algebraic Riccati equation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abou-Kandil H., Freiling G., Ionescu V., Jank G.: Matrix Riccati Equations in Control and Systems Theory. Birkhauser Verlag, Berlin (2003)CrossRefGoogle Scholar
  2. 2.
    Bartels R.T., Stewart G.W.: Solution of the matrix equation AX+BX=C. Comm. ACM 15, 820–826 (1972)CrossRefGoogle Scholar
  3. 3.
    Benahmed, B.: Méthodes du second ordre et problèmes extrémaux. Thèse de doctorat d’état, Univesité d’Oran, Algérie (2007)Google Scholar
  4. 4.
    Benahmed, B., de Malafosse, B., Yassine, A.: Matrix transformation and quasi-Newton methods, IJMMS, 2007, Article ID25704, 17 doi: 10.1155/2007/25705
  5. 5.
    Benahmed B., Fares A., de Malafosse B., Yassine A.: The quasi-Newton method and the infinite matrix theory applied to the continued fractions, Afr Diaspora J Math, Special Issue in Memory of Prof. Ibn Oumar Mahamat Salah 8(2), 1–16 (2009)Google Scholar
  6. 6.
    Broyden C.G., Dennis J.E. Jr., Moré J.J.: On the local and superlinear of convergence quasi-Newton methods. Math. Comput. 12, 223–246 (1973)Google Scholar
  7. 7.
    Broyden C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19, 577–593 (1965)CrossRefGoogle Scholar
  8. 8.
    Broyden C.G.: Quasi-Newton methods and their applications to function minimization. Math. Comp. 21, 368–381 (1967)CrossRefGoogle Scholar
  9. 9.
    Dennis J.E. Jr., Moré J.J.: A characterization of superlinear convergence and its applications to quasi-Newton methods. Math. Comput. 28, 549–560 (1974)CrossRefGoogle Scholar
  10. 10.
    Gay D.M.: Some convergence properties of Broyden’s method. SIAM J. Numer. Anal. 16, 623–630 (1979)CrossRefGoogle Scholar
  11. 11.
    Griewank A.: The local convergence of Broyden-like methods on lipschitzian problems in Hilbert spaces. SIAM J. Numer. Anal. 24, 684–705 (1987)CrossRefGoogle Scholar
  12. 12.
    Gruver W.A., Sachs E.W.: Algorithmic Methods in Optimal Control. Pitman Publishing limited, London (1980)Google Scholar
  13. 13.
    Guo C.-H.: Newton’s method for discrete algebraic Riccati equation when the closed-loop matrix has eigenvalues on the unit circle. SIAM J. Matrix Anal. Appl. 20(2), 279–294 (1998)CrossRefGoogle Scholar
  14. 14.
    Horwitz L.B., Sarachik P.E.: Davidon’s method in Hilbert space. SIAM J. Appl. Math. 16(4), 676–695 (1968)CrossRefGoogle Scholar
  15. 15.
    Juang J.: Existence of algebraic matrix Riccati equations arising in transport theory. Linear Algebra Appl. 230, 89–100 (1999)CrossRefGoogle Scholar
  16. 16.
    Juang J., Lin W.W.: Nonsymmetric algebraic Riccati equations and Hamiltonian-like matrices. SIAM J. Matrix Anal. Appl. 20, 228–243 (1999)CrossRefGoogle Scholar
  17. 17.
    Kantorovich L.V.: On Newton’s method for functional equations. Dokl. Akad. Nauk SSSR 59, 1237–1240 (1948) (in Russian)Google Scholar
  18. 18.
    Lancaster, P., Rodman, L.: Algebraic Riccati Equations. Oxford University Press, (1995). xvii+480 pp. ISBN 0-19-853795-6Google Scholar
  19. 19.
    Kelley C.T., Sachs E.W.: A quasi-Newton method for elliptic boundary value problems. SIAM J. Numer. Anal. 24, 516–531 (1987)CrossRefGoogle Scholar
  20. 20.
    Kelley C.T., Sachs E.W.: Quasi-Newton methods and unconstrained optimal control problems. SIAM J. Control Optim. 23, 1503–1516 (1987)CrossRefGoogle Scholar
  21. 21.
    Kelley C.T., Sachs E.W.: A new proof of superlinear convergence for Broyden’s method in Hilbert space. SIAM J. Optim. 1(1), 146–150 (1991)CrossRefGoogle Scholar
  22. 22.
    Laub A.J.: A Schur method for solving algebraic Riccati equations. IEEE Trans. Automat. Control. AC-24, 913–921 (1979)CrossRefGoogle Scholar
  23. 23.
    Lûtkepohl H.: Handbook of Matrices. John Wiley & Sons edition, New York (1996)Google Scholar
  24. 24.
    Man F.T.: The Davidon method of solution of the algebraic matrix Riccati equation. Int. J. Control 10(6), 713–719 (1969)CrossRefGoogle Scholar
  25. 25.
    Martinez J.M.: Practical quasi-Newton methods for solving nonlinear systems. J. Comput. Appl. Math. 124, 97–121 (2000)CrossRefGoogle Scholar
  26. 26.
    Ortega J.M., Rheinboldt W.C.: Iterative Solution of Nonlinear Variable Equations in Several Variables. Academic Press, San Diego (1970)Google Scholar
  27. 27.
    Sachs E.W.: Broyden’s method in Hilbert space. Math. Program. 35, 71–82 (1986)CrossRefGoogle Scholar
  28. 28.
    Sherman J., Morrison W.J.: Ajustment of an inverse matrix corresponding to changes in the elements of a given comumn or a given row of the original matrix. Ann. Math. Stat. 20, 621 (1949)Google Scholar
  29. 29.
    Stoer J.: Two examples on the convergence of certain rank-2 minimization methods for quadratic functionals in Hilbert space. Linear Algebra Appl. 20, 217–222 (1979)CrossRefGoogle Scholar
  30. 30.
    Tokumaru H., Adachi N., Goto K.: Davidon’s method for minimization problems in Hilbert space with application to control problems. SIAM J. Control 8(N°2), 163–178 (1970)CrossRefGoogle Scholar
  31. 31.
    Wen-huan Y.: A quasi-Newton method in infinite-dimentional spaces and its application for solving a parabolic inverse problem. J. Comput. Math. 16(4), 305–318 (1998)Google Scholar
  32. 32.
    Wen-huan Y.: A quasi-Newton approach to identification of a parabolic system. J. Austral. Math. Soc. Ser. B 40, 1–22 (1998)CrossRefGoogle Scholar
  33. 33.
    Yamamoto T.: Historical developments in convergence analysis for Newton’s and Newton-like methods. J. Comput. Appl. Math. 124, 1–23 (2000)CrossRefGoogle Scholar
  34. 34.
    Zeidler, E.: Nonlinear Functional Analysis and its Applications, Part I, Fixed-Point Theorems. Springer-Verlag (1986)Google Scholar

Copyright information

© Springer Science+Business Media, LLC. 2010

Authors and Affiliations

  • Boubakeur Benahmed
    • 1
    Email author
  • Hocine Mokhtar-Kharroubi
    • 2
  • Bruno de Malafosse
    • 3
  • Adnan Yassine
    • 3
  1. 1.Département de Mathématiques et InformatiqueENSET d’OranEl m’naouerAlgérie
  2. 2.Département de Mathématiques, Faculté des sciencesUniversité d’OranEl m’naouerAlgérie
  3. 3.Laboratoire de Mathématiques Appliquées du HavreUniversité du HavreLe Havre CedexFrance

Personalised recommendations