Abstract
Three conjugate gradient methods based on the spectral equations are proposed. One is a conjugate gradient method based on the spectral scaling secant equation proposed by Cheng and Li (J Optim Thoery Appl 146:305–319, 2010), which gives the most efficient Dai–Kou conjugate gradient method with sufficient descent in Dai and Kou (SIAM J Optim 23:296–320, 2013) family of conjugate gradient methods. The other is a special case in Dai and Liao (Appl Math Optim 43:87–101, 2001) family of conjugate gradient methods, which adopts a special parameter, an approximation to the spectral of the Hessian of the objective function. Another is a sufficient descent conjugate gradient method, which is based on the second one and can be viewed as a three-term conjugate gradient method based on a spectral scaling secant equation. Convergence properties of the three methods are discussed, and numerical results imply that conjugate gradient methods based on the spectral secant equations perform well, especially when the sufficient descent condition is satisfied.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrei N (2008) An unconstrained optimization test functions collection. Adv Model Optim 10:147–161
Babaie-Kafaki S, Ghanbari R (2014) Two modified three-term conjugate gradient methods with sufficient property. Optim Lett. doi:10.1007/s11590-014-0736-8
Barzilai J, Borwein JM (1988) Two-point step size gradient method. IMA J Numer Anal 8:141–148
Bongartz I, Conn AR, Gould NIM, Toint PhL (1995) CUTE: constrained and unconstrained testing environment. ACM trans Math Softw 20:123–160
Cheng WY, Li DH (2010) Spectral scaling BFGS method. J Optim Thoery Appl 146:305–319
Cheng WY, Liu QF (2010) Sufficient descent nonlinear conjugate gradient methods with conjugacy condition. Numer Algorithms 53:113–131
Dai YH (2011) Nonlinear gradient methods. Wiley Encycl Oper Res Mana Sci. Published online Feb. doi:10.1002/9780470400531.eorms0183/pdf
Dai YH, Yuan YX (1999) A nonlinear conjugate gradient method with a strong global convergence property. SIAM J Optim 10:177–182
Dai YH, Yuan YX (2001) Nonlinear conjugate gradient methods. Shang Hai Science and Technolgy Press, Shanghai (in Chinese)
Dai YH, Liao LZ (2001) New conjugacy conditions and related conjugate gradient methods. Appl Math Optim 43:87–101
Dai YH, Ni Q (2003) Testing different conjugate gradient methods for large-scale unconstrained optimization. J Comput Math 21:311–320
Dai YH, Kou CX (2013) A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J Optim 23:296–320
Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance files. Math Program 91:201–213
Fletcher R, Reeves CM (1964) Function minimization by conjugate gradient methods. Comput J 7:149–154
Gilbert JC, Nocedal J (1992) Global convergence properties of conjugate gradient methods for optimization. SIAM J Optim 2:21–42
Hager WW, Zhang HC (2005) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J Optim 16:170–192
Hager WW, Zhang HC (2006) A survey of nonlinear gradient methods. Pac J Optim 2:35–58
Hestenes MR, Stiefel E (1952) Method of conjugate gradient for solving linear systems. J Res Natl Bur Stand 49:409–436
Kobayashi M, Narushima Y, Yabe H (2010) Nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems. J Comput Appl Math 234:375–397
Li DH, Fukushima M (2001) A modified BFGS method and its global convergence in nonconvex minimization. J Comput Appl Math 129:15–35
Li GY, Tang CM, Wei ZX (2007) New conjugacy condition and related new conjugate gradient methods for unconstrained optimization. J Comput Appl Math 202:523–539
Narushima Y, Yabe H (2012) Conjugate gradient methods on secant conditions that generate descent search directions for unconstrained optimization. J Comput Appl Math 236:4303–4317
Necodal J, Wright SJ (1999) Numerical optimization, Springer Series in Operations Research. Springer-Verlag, New York
Nocedal J (1980) Updating quasi-Newton matrixes with limited storage. Math Comput 35:773–782
Oren SS, Luenberger DG (1974a) Self-scaling variable metric (SSVM) algorithms, part I: criteria and sufficient conditions for scaling a class of algorithms. Manag Sci 20:845–862
Oren SS, Luenberger DG (1974b) Self-scaling variable metric (SSVM) algorithms, part II: implementation and experiments. Manag Sci 20:863–874
Perry JM (1977) A class of conjugate gradient with a two-step variable-metric memory, Discussion Paper 269. Northwestern University, Evanston, Center for Mathematical studies in Economics and Management Sciences
Perry JM (1978) A modified conjugate gradient algorithm. Oper Res 26:1073–1078
Polak E, Ribière G (1969) Note sur la convergence de méthods de directions conjugées. Revueé Franccaise d’Informatique et de Recherche Opiérationnelle 16:35–43
Polyak BT (1969) The conjugate gradient method in extreme problems. USSR Comput Math Math Phys 9:94–112
Powell MJD (1984) Nonconvex minimization calculations and the conjugate gradient method. In: Lecture notes in mathematics, vol 1066. Springer, Berlin, pp 122–141
Shanno DF (1978) Conjugate gradient methods with inexact line searches. Math Oper Res 3:244–256
Sugiki K, Narushima Y, Yabe H (2012) Globally convergent three-term conjugate gradient methods that use secant conditions and generate descent search directions for unconstrained optimization. J Optim Theory Appl 153:733–757
Wei ZX, Li GY, Qi LQ (2006) New quasi-Newton methods for unconstrained optimization problems. Appl Math Comput 175:1156–1188
Yabe H, Takano M (2004) Globla convergence properties of nonlinear conjugate gradient methods with modified secant equation. Comput Optim Appl 28:203–225
Yu GH, Guan LT, Chen WF (2006) Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization. Optim Methods Softw 23:561–572
Zhang JZ, Xu CX (2001) Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations. J Comput Appl Math 137:269–278
Zhang L, Zhou WJ, Li DH (2006a) A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence. IMA J Numer Anal 26:629–640
Zhang L, Zhou WJ, Li DH (2006b) Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search. Numer Math 104:561–572
Zhou WJ, Zhang L (2006) A nonlinear conjugate gradient method based on the MBFGS secant equation. Optim Methods Softw 21:707–714
Acknowledgments
The first author is supported by the National Natural Science Foundation of China (71071075), the National Natural Science Foundation of Jiangsu Province (BK20141409), the Natural Science Fundation of the Jiangsu Higher Education Institutions of China (12KJB110006) and the funding of Jiangsu Overseas Research & Training Program for University Prominent Young & Middle-aged Teachers and Presidents. The second author is supported by China Postdoctoral Science Foundation (2013M541697) and the Postdoctoral Foundation of Jiangsu Province (1302044C). This work is done while the first author has been visiting Professor William Ward Hager at University of Florida. He would like to thank Professor Hager for providing good conditions. Thanks also go to the reviewer and the associate editor for useful suggestions, which greatly improve the quality of this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Natasa Krejic.
Appendix
Rights and permissions
About this article
Cite this article
Liu, H., Yao, Y., Qian, X. et al. Some nonlinear conjugate gradient methods based on spectral scaling secant equations. Comp. Appl. Math. 35, 639–651 (2016). https://doi.org/10.1007/s40314-014-0212-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40314-014-0212-1