Advertisement

Properties of the block BFGS update and its application to the limited-memory block BNS method for unconstrained minimization

  • Jan Vlček
  • Ladislav Lukšan
Original Paper
  • 32 Downloads

Abstract

A block version of the Broyden–Fletcher–Goldfarb–Shanno (BFGS) variable metric update formula and its modifications are investigated. In spite of the fact that this formula satisfies the quasi-Newton conditions with all used difference vectors and that the improvement of convergence is the best one in some sense for quadratic objective functions, for general functions, it does not guarantee that the corresponding direction vectors are descent directions. To overcome this difficulty, but at the same time utilize the advantageous properties of the block BFGS update, a block version of the limited-memory variable metric BNS method for large-scale unconstrained optimization is proposed. The global convergence of the algorithm is established for convex sufficiently smooth functions. Numerical experiments demonstrate the efficiency of the new method.

Keywords

Unconstrained minimization Block variable metric methods Limited-memory methods BFGS update Global convergence Numerical results 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

We thank the anonymous referee for careful reading of the paper and for constructive suggestions.

References

  1. 1.
    Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representation of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bongartz, I., Conn, A.R., Gould, N., Toint, P.L.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21, 123–160 (1995)CrossRefzbMATHGoogle Scholar
  3. 3.
    Dennis, J.E. Jr, Schnabel, R.B.: Least change secant updates for quasi-Newton methods. SIAM Rev. 21, 443–459 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Fiedler, M.: Special Matrices and Their Applications in Numerical Mathematics, 2nd edn. Dover Publications, Mineola (2008)zbMATHGoogle Scholar
  6. 6.
    Fletcher, R.: Practical Methods of Optimization. Wiley, Chichester (1987)zbMATHGoogle Scholar
  7. 7.
    Hu, Y.F., Storey, C.: Motivating quasi-Newton Updates by Preconditioned Conjugate Gradient Methods, Math. Report A 150, Department of Mathematical Sciences. Loughborough University of Technology, England (1991)Google Scholar
  8. 8.
    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Prog. 45, 503–528 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Lukšan, L., Matonoha, C., Vlček, J.: Algorithm 896: LSA—algorithms for large-scale optimization. ACM Trans. Math. Softw. 36, 16:1-16:29 (2009)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Luksan, L., Matonoha, C., Vlcek, J.: Sparse test problems for unconstrained optimization, report V-1064. Prague, ICS AS CR (2010)Google Scholar
  11. 11.
    Luksan, L., Matonoha, C., Vlcek, J.: Modified CUTE problems for sparse unconstrained optimization, Report V-1081. Prague, ICS AS CR (2010)Google Scholar
  12. 12.
    Lukšan, L., Spedicato, E.: Variable metric methods for unconstrained optimization and nonlinear least squares. J. Comput. Appl. Math. 124, 61–95 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Lukšan, L., Tuma, M., Matonoha, C., Vlcek, J., Ramesová, N., Siska, M., Hartman, J.: UFO 2014. Interactive system for universal functional optimization, Report V-1218. Prague, ICS AS CR (2014)Google Scholar
  14. 14.
    Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comp. 35, 773–782 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer-Verlag, New York (1999)CrossRefzbMATHGoogle Scholar
  16. 16.
    Schnabel, R.B.: Quasi-Newton methods using multiple secant equations, Technical Report CU-CS-247-83, Department of Computer Science, University of Colorado at Boulder USA (1983)Google Scholar
  17. 17.
    Vlcek, J., Luksan, L.: A conjugate directions approach to improve the limited-memory BFGS method. Appl Math. Comput. 219, 800–809 (2012)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Vlcek, J., Luksan, L.: A modified limited-memory BNS method for unconstrained minimization based on conjugate directions idea. Optim. Methods Softw. 30, 616–633 (2015)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Computer ScienceCzech Academy of SciencesPrague 8Czech Republic
  2. 2.Technical University of LiberecLiberecCzech Republic

Personalised recommendations