Abstract
This paper deals with the choice of the scaling parameter in the spectral conjugate gradient (SCG) method proposed by Birgin and Martínez (in Appl Math Optim 43:117–128, 2001). Theoretical analyses show that the scaling parameter selection not only influences the numerical stability, but also plays an important role in ensuring the descent property of the SCG method. Based on these analyses, an adaptive scaling parameter is proposed to overcome the drawback of the original choice. Global convergence of the SCG method with our new parameter is established for both convex and nonconvex objective functions. Numerical results on CUTEr problems indicate that the proposed scaling parameter is very efficient and promising.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrei, N.: Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl. 38, 401–416 (2007)
Babaie-Kafaki, S., Reza, G.: A descent family of Dai-Liao conjugate gradient methods. Optim. Methods Softw. 29, 583–591 (2014)
Barzilai, J., Borwein, J.M.: Two point step size gradient method. IMA J. Numer. Anal. 8, 141–148 (1988)
Birgin, E.G., Martínez, J.M.: A spectral conjugate gradient method for unconstrained optimization. Appl. Math. Optim. 43, 117–128 (2001)
Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43, 87–101 (2001)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Progr. 91(2), 201–213 (2002)
Gould, N.I.M., Orban, D., Toint, PhL: CUTEr: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)
Hager, W.W., Zhang, H.C.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)
Hager, W.W., Zhang, H.: Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32, 113–137 (2006)
Perry, A.: A modified conjugate gradient algorithm. Oper. Res. 26, 1073–1078 (1978)
Powell, M.J.D.: Restart procedures of the conjugate gradient method. Math. Program. 12, 241–254 (1977)
Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)
Shanno, D.F., Phua, K.H.: Minimization of unconstrained multivariate functions. ACM Trans. Math. Softw. 2, 87–94 (1976)
Yu, G.H., Guan, L.T., Chen, W.F.: Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization. Optim. Methods Softw. 23, 275–293 (2008)
Zoutendijk, G.: Nonlinear programming, computational Methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)
Acknowledgments
This research was supported by the Natural Science Foundation of China (Grant No. 71272086). The authors are grateful to Prof. E.G. Birgin for the SCG code and Profs. W.W. Hager and H. Zhang for the CG_DESCENT code. We also thank the two anonymous referees for their valuable comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, Y., Dan, B. An efficient adaptive scaling parameter for the spectral conjugate gradient method. Optim Lett 10, 119–136 (2016). https://doi.org/10.1007/s11590-015-0865-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-015-0865-8