Skip to main content
SpringerLink
Log in

A diagonal quasi-Newton updating method for unconstrained optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

A diagonal quasi-Newton updating algorithm is presented. The elements of the diagonal matrix approximating the Hessian are determined by minimizing both the size of the change from the previous estimate and the trace of the update, subject to the weak secant equation. Under mild classical assumptions, the convergence of the algorithm is proved to be linear. The diagonal quasi-Newton update satisfies the bounded deterioration property. Numerical experiments with 80 unconstrained optimization test problems of different structures and complexities prove that the suggested algorithm is more efficient and more robust than the steepest descent, Cauchy with Oren and Luenberger scaling algorithm in its complementary form and classical Broyden-Fletcher-Goldfarb-Shanno algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Davidon, W.C.: Variable metric methods for minimization. Argonne national laboratory report, argonne il (1959)

  2. Wolfe, P.: Convergence conditions for ascent methods. SIAM Rev. 11, 226–235 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  3. Wolfe, P.: Convergence conditions for ascent methods. II: Some corrections. SIAM Rev. 13, 185–188 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  4. Dennis, J.E. Jr, Moré, J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28, 549–560 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  5. Dennis, J.E. Jr, Moré, J.J.: Quasi-newton methods, motivation and theory. SIAM Rev. 19, 46–89 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  6. Nocedal, J.: Theory of algorithms for unconstrained optimization. Acta Numerica 1, 1–37 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  7. Dennis, J.E. Jr, Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations. Prentice-hall, inc. Englewood cliffs NJ (1983)

  8. Fletcher, R.: Practical methods of optimization 1, Unconstrained optimization. Wiley, New York (1987)

    MATH  Google Scholar 

  9. Gill, P.E., Murray, W., Wright, M.H.: Practical optimization. Academic Press, London (1981)

    MATH  Google Scholar 

  10. Nocedal, J., Wright, S.J.: Numerical optimization (second edition). Springer Science + Business Media (2006)

  11. Bartholomew-Biggs, M.: Nonlinear Optimization with Engineering Applications. Springer Optimization and its Applications, vol. 19. Springer Science Business Media (2008)

  12. Andrei, N.: Continuous nonlinear optimization for engineering applications in GAMS technology. Springer Optimization and its Applications, vol. 121. Springer Science Business Media (2017)

  13. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  14. Nash, S.G.: Preconditioning of truncated-Newton methods. SIAM J. Sci. Stat. Comput. 6, 599–616 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  15. Nazareth, J.L.: If quasi-Newton then why not quasi-Cauchy? SIAG/Opt Views-and-news 6, 11–14 (1995)

    Google Scholar 

  16. Dennis, J., Wolkowicz, H.: Sizing and least-change secant methods. SIAM. J. Numer. Anal. 30, 1291–1314 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhu, M., Nazareth, J.L., Wolkowicz, H.: The quasi-Cauchy relation and diagonal updating. SIAM J. Optim. 9(4), 1192–1204 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  18. Leong, W.J., Farid, M., Hassan, M.A.: Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method. AMO–Adv Model. Optim. 12(1), 37–44 (2010)

    MathSciNet  MATH  Google Scholar 

  19. Farid, M., Leong, W.J., Zheng, L.: A new diagonal gradient-type method for large scale unconstrained optimization. U.P.B. Sci. Bull. Series A 75(1), 57–64 (2013)

    MathSciNet  MATH  Google Scholar 

  20. Luenberger, D.G.: Linear and nonlinear programming, 2nd edn., Addison-Wesley, Reading, MA (1994)

  21. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim.—An Electron. Int. J. 10, 147–161 (2008)

    MathSciNet  MATH  Google Scholar 

  22. Broyden, C.G., Dennis, J.E., Moré, J.J.: On the local and superlinear convergence of quasi-Newton methods. J. Inst. Math. Appl. 12, 223–246 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  23. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  24. Amini, K., Rizi, A.G.: A new structured quasi-Newton algorithm using partial information on Hessian. J. Comput. Appl. Math. 234, 805–811 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Institute for Informatics, Center for Advanced Modeling and Optimization, 8-10 Averescu Avenue, Bucharest 1, Romania

    Neculai Andrei

Authors
  1. Neculai Andrei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Neculai Andrei.

Additional information

Dr. Neculai Andrei is afull member of Academy of Romanian Scientists, Splaiul Independenţei nr. 54, Sector 5, Bucharest, Romania.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Andrei, N. A diagonal quasi-Newton updating method for unconstrained optimization. Numer Algor 81, 575–590 (2019). https://doi.org/10.1007/s11075-018-0562-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-018-0562-7

Keywords

Mathematics Subject Classification (2010)

Search

Navigation