Skip to main content
SpringerLink
Log in

A new nonmonotone line search technique for unconstrained optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, a new nonmonotone line search rule is proposed,which is verified to be an improved version of the nonmonotone line search technique proposed by Zhang and Hager. Unlike the Zhang and Hager’s method, our nonmonotone line search is proved to own a nice property similar to the standard Armijo line search. In virtue of such a property, global convergence is established for the developed algorithm, where the search direction is supposed to satisfy some mild conditions and the stepsize is chosen by the new line search rule. R-linear convergence of the developed algorithm is proved for strongly convex objective functions. The developed algorithm is used to solve the test problems available in the CUTEr, the numerical results demonstrate that the new line search strategy outperforms the other similar ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andrei, N.: New accelerated conjugate gradient algorithms as a modification of Dai-Yuan’s computational scheme for unconstrained optimization. J. Comput. Appl. Math. 234, 3397–3410 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  2. Andrei, N.: Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl. 38, 401–416 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  3. Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16, 1–3 (1966)

    Article  MATH  MathSciNet  Google Scholar 

  4. Deng, S., Wan, Z., Chen, X.: An improved spectral conjugate gradient algorithm for nonconvex unconstrained optimization problems. J. Optim. Theory Appl. (2013). doi:10.1007/s10957-012-0239-7

  5. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profile. Math. Prog. 91, 201–213 (2002)

    Article  MATH  Google Scholar 

  6. Dai, Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315–330 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  7. Goldstein, A.A.: On steepest descen. SIAM J. Control 3, 147–151 (1965)

    MATH  Google Scholar 

  8. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  9. Grippo, L., Lampariello, F., Lucidi, S.: A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60, 401–419 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  10. Grippo, L., Lampariello, F., Lucidi, S.: A class of nonmonotone stabilization methods in unconstrained optimization. Numer. Math. 59, 779–805 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  11. Gould, N.I.M., Orban, D., Toint, P.H.: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  12. Hager, W.W., Zhang, H.: Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32, 113–137 (2006)

    Article  MathSciNet  Google Scholar 

  13. Hu, S.L., Huang, Z.H., Lu, N.: A non-monotone line search algorithm for unconstrained optimization. J. Sci. Comput. 42, 38–53 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  14. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Prog. 45, 503–528 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  15. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    Article  MATH  Google Scholar 

  16. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  17. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (1999)

    Book  MATH  Google Scholar 

  18. Panier, E.R., Tits, A.L.: Avoiding the Maratos effect by means of a nonmonotone line search I. General constrained problems. SIAM J. Numer. Anal. 28, 1183–1195 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  19. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  20. Toint, P.L.: An assessment of nonmonotone linesearch techniques for unconstrained optimization. SIAM J. Sci. Comput. 17, 725–739 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  21. Wolfe, P.: Convergence conditions for ascent methods. SIAM Rev. 11, 226–235 (1968)

    Article  MathSciNet  Google Scholar 

  22. Wan, Z., Chen, Y., Huang, S., Feng, D.: A modified nonmonotone BFGS algorithm for solving smooth nonlinear equations. Optim. Lett. (2014). doi:10.1007/s11590-013-0678-6

  23. Wan, Z., Feng, D.: Investigation on a class of nonmonotone cautious BFGS algorithms. Math. Numer. Sin. 33, 387–396 (2011)

    MATH  MathSciNet  Google Scholar 

  24. Wan, Z., Hu, C.M., Yang, Z.L.: A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discret. Contin. Dyn. Syst. Ser. B 16, 1157–1169 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  25. Wan, Z., Huang, S., Zheng, X.D.: New cautious BFGS algorithm based on modified Armijo-type line search. J. Inequal. Appl. (2012). doi:10.1186/1029-242X-2012-241

  26. Wan, Z., Teo, K.L., Shen, X.L., Hu, C.M.: New BFGS method for unconstrained optimization problem based on modified Armijo line search. Optimization. (2012). doi:10.1080/02331934.2011.644284

  27. Zoutendijk, G.: Nonlinear programming, computational methods. Integer Nonlinear Program. 143, 37–86 (1970)

    MathSciNet  Google Scholar 

  28. Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043–1056 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  29. Zhou, J.L, Tits, A.L.: Nonmonotone line search for minimax problems. J. Optim. Theory Appl. 76, 455–476 (1993)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Central South University, Changsha, Hunan, China

    Shuai Huang & Zhong Wan

  2. School of Business, Central South University, Changsha, Hunan, China

    Xiaohong Chen

Authors
  1. Shuai Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhong Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaohong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhong Wan.

Additional information

This research is supported by the National Natural Science Foundation of China (Grant No. 71210003, 71221061), the Natural Science Foundation of Hunan Province(13JJ3002) and the Fundamental Research Funds for the Central Universities of Central South University

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, S., Wan, Z. & Chen, X. A new nonmonotone line search technique for unconstrained optimization. Numer Algor 68, 671–689 (2015). https://doi.org/10.1007/s11075-014-9866-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-014-9866-4

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation