Skip to main content
SpringerLink
Log in

Spectral Scaling BFGS Method

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we scale the quasiNewton equation and propose a spectral scaling BFGS method. The method has a good selfcorrecting property and can improve the behavior of the BFGS method. Compared with the standard BFGS method, the single-step convergence rate of the spectral scaling BFGS method will not be inferior to that of the steepest descent method when minimizing an n-dimensional quadratic function. In addition, when the method with exact line search is applied to minimize an n-dimensional strictly convex function, it terminates within n steps. Under appropriate conditions, we show that the spectral scaling BFGS method with Wolfe line search is globally and R-linear convergent for uniformly convex optimization problems. The reported numerical results show that the spectral scaling BFGS method outperforms the standard BFGS method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gill, P.E., Leonard, M.W.: Reduced-Hessian quasiNewton methods for unconstrained optimization. SIAM J. Optim. 12, 209–237 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  2. Oren, S.S., Luenberger, D.G.: Self-scaling variables metric (SSVM) algorithms, part I: criteria and sufficient conditions for scaling a class of algorithms. Manage. Sci. 20, 845–862 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  3. Oren, S.S.: Self-scaling variable metric algorithm, part II: implementation and experiments. Manage. Sci. 20, 863–874 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  4. Nocedal, J., Yuan, Y.: Analysis of a selfscaling quasiNewton method. Math. Program. 61, 19–37 (1993)

    Article  MathSciNet  Google Scholar 

  5. Al-Baali, M.: Analysis of a family of selfscaling quasiNewton methods. Technical Report, Department of Mathematics and Computer Science, United Arab Emirates University (1993)

  6. Al-Baali, M.: Global and superlinear convergence of a class of selfscaling methods with inexact line searches. Comput. Optim. Appl. 9, 191–203 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  7. Al-Baali, M.: Numerical experience with a class of selfscaling quasiNewton algorithms. J. Optim. Theory Appl. 96, 533–553 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  8. Yuan, Y.: A modified BFGS algorithm for unconstrained optimization. IMA J. Numer. Anal. 11, 325–332 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bard, Y.: On a numerical instability of Davidon-like methods. Math. Comput. 22, 665–666 (1968)

    MathSciNet  MATH  Google Scholar 

  10. Barzilai, J., Borwein, J.M.: Two-point step size gradient method. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  11. Bryden, C.G.: Quasi-Newton methods and their application to function minimization. Math. Comput. 21, 368–381 (1967)

    Google Scholar 

  12. Loewner, C.: Advanced matrix theory. Lecture Notes, Standford University, Winter (1957)

  13. Luenberger, D.G.: Introduction to Linear and Nonlinear Programming. Addison-Wesley, Reading (1973)

    MATH  Google Scholar 

  14. Dixon, L.C.W.: Variable metric algorithms: necessary and sufficient conditions for identical behavior of nonquadratic functions. J. Optim. Theory Appl. 10, 34–40 (1972)

    Article  MATH  Google Scholar 

  15. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasiNewton methods with application to unconstrained optimization. SIAM J. Numer. Anal. 26, 727–739 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  16. Byrd, R.H., Liu, D.C., Nocedal, J.: Global convergence of a class of quasiNewton methods on convex problems. SIAM J. Numer. Anal. 24, 1171–1190 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  17. Powell, M.J.D.: Updating conjugate directions by the BFGS formula. Math. Program. 38, 693–726 (1987)

    Article  Google Scholar 

  18. Zhang, J.Z., Xu, C.X.: Properties and numerical performance of quasiNewton methods with modified quasiNewton equations. J. Comput. Appl. Math. 137, 269–278 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  19. Xu, C.X., Zhang, J.Z.: A survey of quasiNewton equations and quasiNewton methods for optimization. Ann. Oper. Res. 103, 213–234 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  20. Moré, J.J., Thuente, D.J.: Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw. 20, 286–307 (1994)

    Article  MATH  Google Scholar 

  21. Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.I.: CUTE: constrained and unconstrained testing environments. ACM Trans. Math. Softw. 21, 123–160 (1995)

    Article  MATH  Google Scholar 

  22. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Software, Dongguan University of Technology, Dongguan, 523808, China

    W. Y. Cheng

  2. School of Mathematical Sciences, South China Normal University, Guangzhou, 510631, China

    D. H. Li

Authors
  1. W. Y. Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. H. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. H. Li.

Additional information

Communicated by X.Q. Yang.

Supported by the NSF of China Grant 10771057 and by the Key Project of Chinese Ministry of Education Grant 309023.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheng, W.Y., Li, D.H. Spectral Scaling BFGS Method. J Optim Theory Appl 146, 305–319 (2010). https://doi.org/10.1007/s10957-010-9652-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-010-9652-y

Keywords

Search

Navigation