Abstract
This study aims to present a limited memory BFGS algorithm to solve non-convex minimization problems, and then use it to find the largest eigenvalue of a real symmetric positive definite matrix. The proposed algorithm is based on the modified secant equation, which is used to the limited memory BFGS method without more storage or arithmetic operations. The proposed method uses an Armijo line search and converges to a critical point without convexity assumption on the objective function. More importantly, we do extensive experiments to compute the largest eigenvalue of the symmetric positive definite matrix of order up to 54,929 from the UF sparse matrix collection, and do performance comparisons with EIGS (a Matlab implementation for computing the first finite number of eigenvalues with largest magnitude). Although the proposed algorithm converges to a critical point, not a global minimum theoretically, the compared results demonstrate that it works well, and usually finds the largest eigenvalue of medium accuracy.
Similar content being viewed by others
References
Absil P-A, Mahony R, Andrews B (2005) Convergence of the iterates of descent methods for analytic cost functions. SIAM J Optim 16:531–547
Auchmuty G (1983) Duality for non-convex variational principles. J Differ Equ 50:80–145
Auchmuty G (1989) Unconstrained variational principles for eigenvalues of real symmetric matrices. SIAM J Math Anal 20:1186–1207
Bai Z, Dongarra J, Ruhe A, van der Vorst H (2000) Templates for the solution of algebraic eigenvalue problems: a practical guide. SIAM, Philadelphia
Byrd RH, Nocedal J (1989) A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J Numer Anal 26:727–739
Byrd RH, Nocedal J, Yuan Y (1987) Global convergence of a class of quasi-Newton methods on convex problems. SIAM J Numer Anal 24:1171–1189
Conn AR, Gould NIM, Toint PhL (1995) CUTE: constrained and unconstrained testing environment. ACM Trans Math Softw 21:123–160
Dai YH (2002) Convergence properties of the BFGS algorithm. SIAM J Optim 13:693–701
Davis TA, Hu Y (2011) The university of Florida sparse matrix collection. ACM Trans Math Softw 38:1–25
Demmel J, Veselić K (1992) Jacobis method is more accurate than QR. SIAM J Matrix Anal Appl 13:1204C1245
Demmel J, Marques OA, Parlett BN, Vömel C (2008) Performance and accuracy of LAPACK’s symmetric tridiagonal eigensolvers. SIAM J Sci Comput 30:1508–1526
Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91:201–213
Gao H, Dai YH, Tong X (2015) Barzilai–Borwein-like methods for the extreme eigenvalue problem. J Ind Manag Optim 11:999–1019
Golub GH, Van Loan CF (1996) Matrix computation, 3rd edn. John Hopkins University Press, Baltimore
Jiang B, Dai YH (2013) Feasible Barzilai–Borwein-like methods for extreme symmetric eigenvalue problems. Optim Methods Softw 28:756–784
Li DH, Fukushima M (2001a) A modified BFGS method and its global convergence in nonconvex minimization. J Comput Appl Math 129:15–35
Li DH, Fukushima M (2001b) On the global convergence of the BFGS method for nonconvex unconstrained optimization problems. SIAM J Optim 11:1054–1064
Liu D, Nocedal J (1989) On the limited memory BFGS method for large-scale optimization. Math Program 45:503–528
Mascarenhas WF (2004) The BFGS method with exact line searchs fails for non-convex objective functions. Math Program 99:49–61
Mongeau M, Torki M (2004) Computing eigenelements of real symmetric Matrices via optimization. Comput Optim Appl 29:263–287
Nocedal J (1980) Updating quasi-Newton matrices with limited storage. Math Comput 35:773–782
Sameh AH, Wisniewski JA (1982) A trace minimization algorithm for the generalized eigenvalue problem compuations. SIAM J Numer Anal 19:1243–1259
Wei Z, Yu G, Yuan G, Lian Z (2004) The superlinear convergence of a modified BFGS-type method for uconstrained optimization. Comput Optim Appl 29:315–332
Wei Z, Li G, Qi LQ (2006) New quasi-Newton methods for unconstrained optimization problems. Appl Math Comput 175:1156–1188
Wei Z, Li G, Qi LQ (2008) Global convergence of the Polak–Ribiere–Polyak conjugate gradient method with inexact line searches for nonconvex unconstrained optimization problems. Math Comput 77:2173–2193
Xiao Y, Li T, Wei Z (2013) Global convergence of a modified limited memory BFGS method for non-convex minimization. Acta Math Appl Sin E 29:555–566
Xiao Y, Wang Q, Wang D (2010) Notes on the Dai–Yuan–Yuan modified spectral gradient method. J Comput Appl Math 234:2986–2992
Xiao Y, Wei Z, Wang Z (2008) A limited memory BFGS-type method for large-scale unconstrained optimization. Comput Math Appl 56:1001–1009
Xu CX, Zhang JZ (2001) A survey of quasi-Newton equations and quasi-Newton methods for optimization. Optimization and numerical algebra. Ann Oper Res 103:213–234
Zhang L (2005) A globally convergent BFGS method for nonconvex minimization without line searches. Optim Method Softw 20:737–747
Zhang JZ, Deng NY, Chen LH (1999) New quasi-Newton equation and related methods for unconstrained optimization. J Optim Theory Appl 102:147–167
Zhang J, Xiao Y, Wei Z (2009) Nonlinear conjugate gradient methods with sufficient descent condition for large-scale unconstrained optimization. Math Prob Eng Article ID 243290:16
Zhou W, Zhang L (2009) Global convergence of the nonmonotone MBFGS method for nonconvex unconstrained minimization. J Comput Appl Math 223:40–47
Acknowledgments
We are grateful to the associate editor and two anonymous referees for their valuable comments and suggestions which have helped us improve the presentation of this paper greatly. The work was supported by the Major State Basic Research Development Program of China (973 Program) (Grant No. 2015CB856003), the National Natural Science Foundation of China (Grant No. 11471101), and the Program for Science and Technology Innovation Talents in Universities of Henan Province (Grant No. 13HASTIT050).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shi, Z., Yang, G. & Xiao, Y. A limited memory BFGS algorithm for non-convex minimization with applications in matrix largest eigenvalue problem. Math Meth Oper Res 83, 243–264 (2016). https://doi.org/10.1007/s00186-015-0527-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00186-015-0527-8
Keywords
- Unconstrained optimization
- Limited memory BFGS method
- Armijo line search
- Global convergence
- Largest eigenvalue problem
- Critical point