It is well-known that conjugate gradient algorithms are widely applied in many practical fields, for instance, engineering problems and finance models, as they are straightforward and characterized by a simple structure and low storage. However, challenging problems remain, such as the convergence of the PRP algorithms for nonconvexity under an inexact line search, obtaining a sufficient descent for all conjugate gradient methods, and other theory properties regarding global convergence and the trust region feature for nonconvex functions. This paper studies family conjugate gradient formulas based on the six classic formulas, PRP, HS, CD, FR, LS, and DY, where the family conjugate gradient algorithms have better theory properties than those of the formulas by themselves. Furthermore, this technique of the presented conjugate gradient formulas can be extended to any two-term conjugate gradient formula. This paper designs family conjugate gradient algorithms for nonconvex functions, which have the following features without other conditions: (i) the sufficient descent property holds, (ii) the trust region feature is true, and (iii) the global convergence holds under normal assumptions. Numerical results show that the given conjugate gradient algorithms are competitive with those of normal methods.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)
Dai, Y.: Analysis of Conjugate Gradient Methods, Ph.D. Thesis, Institute of Computational Mathe- matics and Scientific/Engineering Computing, Chese Academy of Sciences (1997)
Dai, Y.: Convergence properties of the BFGS algoritm. SIAM J. Optim. 13, 693–701 (2003)
Dai, Y., Liao, L.: New conjugacy conditions and related nonlinear conjugate gradient methods. App. Math. Optim. 43, 87–101 (2001)
Dai, Y., Yuan, Y.: A nonlinear conjugate gradient with a strong global convergence properties. SIAM J. Optim. 10, 177–182 (2000)
Dai, Y., Yuan, Y.: Nonlinear conjugate gradient methods, Shanghai Scientific and Technical Publishers (1998)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Fan, J., Yuan, Y.: A new trust region algorithm with trust region radius converging to zero. In: Li, D. (ed.) Proceedings of the 5th International Conference on Optimization: Techniques and Applications (December 2001, Hongkong), pp 786–794 (2001)
Fletcher, R.: Practical methods of optimization, 2nd. Wiley, New York (1987)
Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)
Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2, 21–42 (1992)
Grippo, L., Lucidi, S.: A globally convergent version of the Polak-Ribière-Polyak conjugate gradient method. Math. Program. 78, 375–391 (1997)
Hager, W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)
Hager, W., Zhang, H.: Algorithm 851: a conjugate gradient method with guaranteed descent. ACM Trans. Math. Soft. 32, 113–137 (2006)
Hestenes, M.R., Stiefel, E.: Method of conjugate gradient for solving linear equations. J. Res. Nation. Bur. Stand. 49, 409–436 (1952)
Levenberg, K.: A method for the solution of certain nonlinear problem in least squares. Quart. Appl. Math. 2, 164–168 (1944)
Li, D.H., Tian, B.S.: N-step quadratic convergence of the MPRP method with a restart strategy. J. Comput. Appl. Math. 235, 4978–4990 (2011)
Li, X., Wang, S., Jin, Z., Pham, H.: A conjugate gradient algorithm under Yuan-Wei-Lu line search technique for large-scale minimization optimization models, vol. 2018 (2018)
Liu, Y., Storey, C.: Effcient generalized conjugate gradient algorithms part 1: theory. J. Optim. Theo. Appl. 69, 129–137 (1991)
Martinet, B.: Régularisation d’inéquations variationelles par approxiamations succcessives. Rev. Fr. Inform. Rech. Oper. 4, 154–158 (1970)
Nocedal, J., Wright, S.J.: Numerical optimization. Springer, New York (1999)
Polak, E.: The conjugate gradient method in extreme problems. Comput. Math. Mathem. Phy. 9, 94–112 (1969)
Polak, E., Ribière, G.: Note sur la convergence de directions conjugees. Rev. Fran. Inf. Rech. Opérat. 3, 35–43 (1969)
Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, Q.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear programming, vol. 2, pp 1–27. Academic Press, New York (1974)
Powell, M.J.D.: Nonconvex minimization calculations and the conjugate gradient method, Lecture Notes in Mathematics, vol. 1066, pp 122–141. Springer, Berlin (1984)
Powell, M.J.D.: Convergence properties of algorithm for nonlinear optimization. SIAM Rev. 28, 487–500 (1986)
Sheng, Z., Ouyang, A., Liu, L., et al.: A novel parameter estimation method for Muskingum model using new Newton-type trust region algorithm. Math. Probl. Eng. 2014, 1–7 (2014). Art. ID 634852
Sheng, Z., Yuan, G.: An effective adaptive trust region algorithm for nonsmooth minimization. Comput. Optim. Appl. 71, 251–271 (2018)
Sheng, Z., Yuan, G., Cui, Z.: A new adaptive trust region algorithm for optimization problems. Acta Math. Scientia. 38B(2), 479–496 (2018)
Sheng, Z., Yuan, G., Cui, Z., et al.: An adaptive trust region algorithm for large-residual nonsmooth least squares problems. J. Ind. Manage. Optim. 14, 707–718 (2018)
Wei, Z., Yao, S., Liu, L.: The convergence properties of some new conjugate gradient methods. Appl. Math. Comput. 183, 1341–1350 (2006)
Yuan, G.: Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems. Optim. Let. 3, 11–21 (2009)
Yuan, G., Lu, X.: A modified PRP conjugate gradient method. Anna. Operat. Res. 166, 73–90 (2009)
Yuan, G., Lu, S., Wei, Z.: A new trust-region method with line search for solving symmetric nonlinear equations. Intern. J. Comput. Math. 88, 2109–2123 (2011)
Yuan, G., Lu, X., Wei, Z.: A conjugate gradient method with descent direction for unconstrained optimization. J. Comput. Appl. Math. 233, 519–530 (2009)
Yuan, G., Lu, X., Wei, Z.: BFGS trust-region method for symmetric nonlinear equations. J. Compu. Appl. Math. 230, 44–58 (2009)
Yuan, G., Meng, Z., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. J. Optim. Theory. Appl. 168, 129–152 (2016)
Yuan, G., Sheng, Z., Wang, B., et al.: The global convergence of a modified BFGS method for nonconvex functions. J. Comput. Appl. Math. 327, 274–294 (2018)
Yuan, G., Wei, Z., Li, G.: A modified Polak-Ribière-Polyak conjugate gradient algorithm for nonsmooth convex programs. J. Comput. Appl. Math. 255, 86–96 (2014)
Yuan, G., Wei, Z., Lu, X.: Global convergence of the BFGS method and the PRP method for general functions under a modified weak Wolfe-Powell line search. Appl. Math. Model. 47, 811–825 (2017)
Yuan, G., Wei, Z., Yang, Y.: The global convergence of the Polak-Ribière-Polyak conjugate gradient algorithm under inexact line search for nonconvex functions. J. Comput. Appl. Math. 362, 262–275 (2019)
Yuan, G., Zhang, M.: A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations. J. Comput. Appl. Math. 286, 186–195 (2015)
Yuan, Y.: Analysis on the conjugate gradient method. Optim. Meth. Soft. 2, 19–29 (1993)
Zoutendijk, G.: Nonlinear programming computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, Northholland, Amsterdam, pp 37–86 (1970)
The authors would like to thank the editor and the referee for their valuable comments which greatly improve this manuscript.
This work was supported by the National Natural Science Fund of China (Grant No. 11661009), the Guangxi Natural Science Fund for Distinguished Young Scholars (No. 2015GXNSFGA139001), and the Guangxi Natural Science Key Fund (No. 2017GXNSFDA198046).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Yuan, G., Wang, X. & Sheng, Z. Family weak conjugate gradient algorithms and their convergence analysis for nonconvex functions. Numer Algor 84, 935–956 (2020). https://doi.org/10.1007/s11075-019-00787-7
- Nonconvex functions
- Sufficient descent
- Trust region
- Inexact line search
- Global convergence
Mathematics Subject Classification (2010)