Advertisement

A subspace minimization conjugate gradient method based on conic model for unconstrained optimization

  • Yufei Li
  • Zexian LiuEmail author
  • Hongwei Liu
Article
  • 103 Downloads

Abstract

In this paper, we present a new conjugate gradient method, in which the search direction is computed by minimizing a selected approximate model in a two-dimensional subspace. That is, if the objective function is not close to a quadratic, the search direction is generated by a conic model. Otherwise, a quadratic model is considered. The direction of the proposed method is proved to possess the sufficient descent property. With the modified nonmonotone line search, we establish a global convergence of the proposed method under appropriate assumptions. R-linear convergence of the proposed method is also analyzed. Numerical results using two different test function collections show that the proposed algorithm is efficient.

Keywords

Conjugate gradient method Conic model Subspace minimization Nonmonotone line search Global convergence 

Mathematics Subject Classification

90C30 90C06 65K05 

Notes

Acknowledgements

We would like to thank the anonymous referees for their useful suggestions and comments. We also would like to thank Professor Dai, Y. H. and Dr. Kou, C. X. for their CGOPT code, and thank Professor Hager, W. W. and Zhang, H. C. for their CG DESCENT (5.3) code. This research is supported by National Science Foundation of China (No. 11461021) and Shaanxi Science Foundation (No. 2017JM1014).

References

  1. Andrei N (2008) An unconstrained optimization test functions collection. Adv Model Optim 10(1):147–161MathSciNetzbMATHGoogle Scholar
  2. Andrei N (2014) An accelerated subspace minimization three-term conjugate gradient algorithm for unconstrained optimization. Numer Algorithms 65(4):859–874MathSciNetzbMATHGoogle Scholar
  3. Andrei N (2017) Accelerated adaptive Perry conjugate gradient algorithms based on the self-scaling memoryless BFGS update. J Comput Appl Math 325:149–164MathSciNetzbMATHGoogle Scholar
  4. Ariyawansa KA (1990) Deriving collinear scaling algorithms as extension of quasi-Newton methods and the local convergence of DFP and BFGS-related collinear scaling algorithm. Math Program 49(1):23–48MathSciNetzbMATHGoogle Scholar
  5. Barzilai J, Borwein JM (1988) Two-point step size gradient methods. IMA J Numer Anal 8:141–148MathSciNetzbMATHGoogle Scholar
  6. Branch MA, Coleman TF, Li Y (1999) A subspace, interior, and conjugate gradient method for large scale bound-constrained minimization problems. SIAM J Sci Comput 21(1):1–23MathSciNetzbMATHGoogle Scholar
  7. Dai YH, Kou CX (2013) A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J Optim 23(1):296–320MathSciNetzbMATHGoogle Scholar
  8. Dai YH, Kou CX (2016) A Barzilai–Borwein conjugate gradient method. Sci China Math 59(8):1511–1524MathSciNetzbMATHGoogle Scholar
  9. Dai YH, Liao LZ (2001) New conjugacy conditions and related nonlinear conjugate gradient methods. Appl Math Optim 43(1):87–101MathSciNetzbMATHGoogle Scholar
  10. Dai YH, Yuan Y (1999) A nonlinear conjugate gradient method with a strong global convergence property. SIAM J Optim 10(1):177–182MathSciNetzbMATHGoogle Scholar
  11. Dai YH, Yuan YX (2000) Nonlinear conjugate gradient methods. Shanghai Scientific and Technical Publishers, ShanghaiGoogle Scholar
  12. Davidon WC (1980) Conic approximations and collinear scalings for optimizers. SIAM J Numer Anal 17(2):268–281MathSciNetzbMATHGoogle Scholar
  13. Di S, Sun WY (1996) A trust region method for conic model to solve unconstrained optimization. Optim Methods Softw 6(4):237–263Google Scholar
  14. Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91:201–213MathSciNetzbMATHGoogle Scholar
  15. Du XW, Zhang P, Ma WY (2016) Some modified conjugate gradient methods for unconstrained optimization. J Comput Appl Math 305:92–114MathSciNetzbMATHGoogle Scholar
  16. Facchinei F, Pang JS (2003) Finite-dimensional variational inequalities and complementarity problems, vol 2. Springer, BerlinzbMATHGoogle Scholar
  17. Fletcher R, Reeves CM (1964) Function minimization by conjugate gradients. Comput J 7:149–154MathSciNetzbMATHGoogle Scholar
  18. Gould NIM, Orban D, Toint PhL (2003) CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans Math Softw 29(4):373–394zbMATHGoogle Scholar
  19. Hager WW, Zhang H (2005) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J Optim 16(1):170–192MathSciNetzbMATHGoogle Scholar
  20. Hager WW, Zhang H (2006a) A survey of nonlinear conjugate gradient methods. Pac J Optim 2(1):35–58MathSciNetzbMATHGoogle Scholar
  21. Hager WW, Zhang H (2006b) Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans Math Softw 32(1):113–137Google Scholar
  22. Hager WW, Zhang H (2013) The limited memory conjugate gradient method. SIAM J Optim 23(4):2150–2168MathSciNetzbMATHGoogle Scholar
  23. Hestenes MR, Stiefel E (1952) Methods of conjugate gradients for solving linear systems. J Res Natl Bur Stand 49:409–436MathSciNetzbMATHGoogle Scholar
  24. Li M, Liu HW, Liu ZX (2018) A new subspace minimization conjugate gradient method with nonmonotone line search for unconstrained optimization. Numer Algorithms 79(1):195–219MathSciNetzbMATHGoogle Scholar
  25. Liu HW, Liu ZX (2018a) An efficient Barzilai–Borwein conjugate gradient method for unconstrained optimization. J Optim Theory Appl.  https://doi.org/10.1007/s10957-018-1393-3
  26. Liu ZX, Liu HW (2018b) An efficient gradient method with approximate optimal stepsize for large-scale unconstrained optimization. Numer. Algorithms 78(1):21–39MathSciNetzbMATHGoogle Scholar
  27. Liu ZX, Liu HW (2018c) Several efficient gradient methods with approximate optimal stepsizes for large scale unconstrained optimization. J Comput Appl Math 328:400–413MathSciNetzbMATHGoogle Scholar
  28. Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, BerlinzbMATHGoogle Scholar
  29. Polak E, Ribière G (1969) Note sur la convergence de méthodes de directions conjuguées. Rev. Franaise Informat. Rech. Opérationnelle. 3(16):35–43zbMATHGoogle Scholar
  30. Polyak BT (1969) The conjugate gradient method in extremal problems. Ussr Comput Math Math Phys 9(4):94–112zbMATHGoogle Scholar
  31. Raydan M (1997) The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J Optim 7(1):26–33MathSciNetzbMATHGoogle Scholar
  32. Schnabel RB (1982) Conic methods for unconstrained minimization and tensor methods for nonlinear equation. In: Mathamatical Program, pp 417–438Google Scholar
  33. Sheng S (1995) Interpolation by conic model for unconstrained optimization. Computing 54(1):83–98MathSciNetzbMATHGoogle Scholar
  34. Sorensen DC (1980) The Q-Superlinear convergence of a collinear scaling algorithm for unconstrained optimization. SIAM J Numer Anal 17(1):84–114MathSciNetzbMATHGoogle Scholar
  35. Sun WY (1996) On nonquadratic model optimization methods. Asia Pac J Oper Res 13:43–63Google Scholar
  36. Sun WY, Yuan YX (2001) A conic trust-region method for nonlinearly constrained optimization. Ann. Oper. Res. 103:175–191MathSciNetzbMATHGoogle Scholar
  37. Sun W, Yuan J, Yuan Y (2003) Trust region method of conic model for linearly constrained optimization. J Comput Math 21:295–304MathSciNetzbMATHGoogle Scholar
  38. Yang YT, Chen YT, Lu YL (2017) A subspace conjugate gradient algorithm for large-scale unconstrained optimization. Numer Algorithms 76(3):813–828MathSciNetzbMATHGoogle Scholar
  39. Yuan YX (1991) A modified BFGS algorithm for unconstrained optimization. IMA J Numer Anal 11(3):325–332MathSciNetzbMATHGoogle Scholar
  40. Yuan YX (2009) Subspace methods for large scale nonlinear equations and nonlinear least squares. Optim Eng 10(2):207–218MathSciNetzbMATHGoogle Scholar
  41. Yuan YX (2014) A review on subspace methods for nonlinear optimization. In: Proceedings of the international congress of mathematics. Korea, pp 807–827Google Scholar
  42. Yuan YX, Stoer J (1995) A subspace study on conjugate gradient algorithms. Z Angew Math Mech 75(1):69–77MathSciNetzbMATHGoogle Scholar
  43. Yuan YX, Sun WY (1997) Optimization theory and methods. Science Press, BeijingGoogle Scholar
  44. Zhang H, Hager WW (2004) A nonmonotone line search technique and its application to unconstrained optimization. SIAM J Optim 14(4):1043–1056MathSciNetzbMATHGoogle Scholar

Copyright information

© SBMAC - Sociedade Brasileira de Matemática Aplicada e Computacional 2019

Authors and Affiliations

  1. 1.School of Mathematics and StatisticsXidian UniversityXi’anChina
  2. 2.School of Mathematics and Computer ScienceHezhou UniversityHezhouChina

Personalised recommendations