Advertisement

Numerical Algorithms

, Volume 68, Issue 4, pp 671–689 | Cite as

A new nonmonotone line search technique for unconstrained optimization

  • Shuai Huang
  • Zhong WanEmail author
  • Xiaohong Chen
Original Paper

Abstract

In this paper, a new nonmonotone line search rule is proposed,which is verified to be an improved version of the nonmonotone line search technique proposed by Zhang and Hager. Unlike the Zhang and Hager’s method, our nonmonotone line search is proved to own a nice property similar to the standard Armijo line search. In virtue of such a property, global convergence is established for the developed algorithm, where the search direction is supposed to satisfy some mild conditions and the stepsize is chosen by the new line search rule. R-linear convergence of the developed algorithm is proved for strongly convex objective functions. The developed algorithm is used to solve the test problems available in the CUTEr, the numerical results demonstrate that the new line search strategy outperforms the other similar ones.

Keywords

Nonmonotone line search Armijo line search Global convergence R-linear convergence 

Mathematics Subject Classifications (2010)

90C30 62K05 68T37 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Andrei, N.: New accelerated conjugate gradient algorithms as a modification of Dai-Yuan’s computational scheme for unconstrained optimization. J. Comput. Appl. Math. 234, 3397–3410 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Andrei, N.: Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl. 38, 401–416 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16, 1–3 (1966)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Deng, S., Wan, Z., Chen, X.: An improved spectral conjugate gradient algorithm for nonconvex unconstrained optimization problems. J. Optim. Theory Appl. (2013). doi: 10.1007/s10957-012-0239-7
  5. 5.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profile. Math. Prog. 91, 201–213 (2002)CrossRefzbMATHGoogle Scholar
  6. 6.
    Dai, Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315–330 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Goldstein, A.A.: On steepest descen. SIAM J. Control 3, 147–151 (1965)zbMATHGoogle Scholar
  8. 8.
    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Grippo, L., Lampariello, F., Lucidi, S.: A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60, 401–419 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Grippo, L., Lampariello, F., Lucidi, S.: A class of nonmonotone stabilization methods in unconstrained optimization. Numer. Math. 59, 779–805 (1991)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Gould, N.I.M., Orban, D., Toint, P.H.: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Hager, W.W., Zhang, H.: Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32, 113–137 (2006)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Hu, S.L., Huang, Z.H., Lu, N.: A non-monotone line search algorithm for unconstrained optimization. J. Sci. Comput. 42, 38–53 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Prog. 45, 503–528 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)CrossRefzbMATHGoogle Scholar
  16. 16.
    Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (1999)CrossRefzbMATHGoogle Scholar
  18. 18.
    Panier, E.R., Tits, A.L.: Avoiding the Maratos effect by means of a nonmonotone line search I. General constrained problems. SIAM J. Numer. Anal. 28, 1183–1195 (1991)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Toint, P.L.: An assessment of nonmonotone linesearch techniques for unconstrained optimization. SIAM J. Sci. Comput. 17, 725–739 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Wolfe, P.: Convergence conditions for ascent methods. SIAM Rev. 11, 226–235 (1968)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Wan, Z., Chen, Y., Huang, S., Feng, D.: A modified nonmonotone BFGS algorithm for solving smooth nonlinear equations. Optim. Lett. (2014). doi: 10.1007/s11590-013-0678-6
  23. 23.
    Wan, Z., Feng, D.: Investigation on a class of nonmonotone cautious BFGS algorithms. Math. Numer. Sin. 33, 387–396 (2011)zbMATHMathSciNetGoogle Scholar
  24. 24.
    Wan, Z., Hu, C.M., Yang, Z.L.: A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discret. Contin. Dyn. Syst. Ser. B 16, 1157–1169 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Wan, Z., Huang, S., Zheng, X.D.: New cautious BFGS algorithm based on modified Armijo-type line search. J. Inequal. Appl. (2012). doi: 10.1186/1029-242X-2012-241
  26. 26.
    Wan, Z., Teo, K.L., Shen, X.L., Hu, C.M.: New BFGS method for unconstrained optimization problem based on modified Armijo line search. Optimization. (2012). doi: 10.1080/02331934.2011.644284
  27. 27.
    Zoutendijk, G.: Nonlinear programming, computational methods. Integer Nonlinear Program. 143, 37–86 (1970)MathSciNetGoogle Scholar
  28. 28.
    Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043–1056 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Zhou, J.L, Tits, A.L.: Nonmonotone line search for minimax problems. J. Optim. Theory Appl. 76, 455–476 (1993)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.School of Mathematics and StatisticsCentral South UniversityChangshaChina
  2. 2.School of BusinessCentral South UniversityChangshaChina

Personalised recommendations