Skip to main content
SpringerLink
Log in

The global convergence of the BFGS method with a modified WWP line search for nonconvex functions

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The BFGS method, which has great numerical stability, is one of the quasi-Newton line search methods. However, the global convergence of the BFGS method with a Wolfe line search is still an open problem for general functions. Recently, Yuan et al. (Appl. Math. Mode. 47:811–825, 2017) presented a modified weak Wolfe-Powell (WWP) line search that globally converges for general functions; however, they only partially solved this open problem. In this paper, a further modified WWP line search is proposed, and the global convergence of the BFGS method is established under suitable conditions for general functions. The numerical results show that the new line search produces more interesting results than the normal line search. Moreover, a fact engineering problem is studied with the Muskingum model to show the performance of the presented algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Andrei, N.: A double parameter scaled BFGS method for unconstrained optimization. J. Comput. Appl. Math. 332, 26–44 (2018)

    Article  MathSciNet  Google Scholar 

  2. Broyden, C.G.: The convergence of a class of double-rank minimization algorithms. J. Inst. Math. Appl. 6, 222–231 (1970)

    Article  MathSciNet  Google Scholar 

  3. Broyden, C.G., Dennis, J. Jr, Moré, J. J.: On the local and superlinear convergence of quasi-Newton methods. IMA J. Appl. Math. 12, 223–245 (1973)

    Article  MathSciNet  Google Scholar 

  4. Bongartz, I., Conn, A.R., Gould, N.I., Toint, P.L.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21, 123–160 (1995)

    Article  Google Scholar 

  5. Byrd, R., Nocedal, J.: A tool for the analysis of quasi-newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)

    Article  MathSciNet  Google Scholar 

  6. Byrd, R., Nocedal, J., Yuan, Y.X.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24, 1171–1189 (1987)

    Article  MathSciNet  Google Scholar 

  7. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28, 549–560 (1974)

    Article  MathSciNet  Google Scholar 

  8. Dai, Y.H.: Convergence properties of the BFGS algorithm. SIAM J. Optim. 13, 693–701 (2006)

    Article  MathSciNet  Google Scholar 

  9. Fletcher, R.: A new approach to variable metric algorithms. Comput. J. 13, 317–322 (1970)

    Article  Google Scholar 

  10. Geem, Z.W.: Parameter estimation for the nonlinear Muskingum model using the BFGS technique. J. Hydrol. Eng. 132, 474–478 (2006)

    Article  Google Scholar 

  11. Griewank, A.: The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients. Math. Program. 50, 141–175 (1991)

    Article  MathSciNet  Google Scholar 

  12. Griewank, A., Toint, P.L.: Local convergence analysis for partitioned quasi-Newton updates. Numer. Math. 39, 429–448 (1982)

    Article  MathSciNet  Google Scholar 

  13. Goldfarb, D.: A family of variable-metric methods derived by variational means. Math. Comput. 24, 23–26 (1970)

    Article  MathSciNet  Google Scholar 

  14. Kimiaei, M., Ghaderi, S.: A new restarting adaptive trust-region method for unconstrained optimization. J. Oper. Res. Soc. China 5, 487–507 (2017)

    Article  MathSciNet  Google Scholar 

  15. Li, X.R., Wang, S.H., Jin, Z.Z., Pham, H.: A conjugate gradient algorithm under Yuan-Wei-Lu line search technique for large-scale minimization optimization models. Math. Probl. Eng. 2018, 1–11 (2018)

    MathSciNet  MATH  Google Scholar 

  16. Moré, J.J., Garbow, B.S., Hillstrome, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    Article  MathSciNet  Google Scholar 

  17. Nocedal, J., Wright, S.J.: Numerical optimization. Springer Series in Operations Research and Financial Engineering (2006)

  18. Ouyang, A., Liu, L., Sheng, Z., Wu, F.: A class of parameter estimation methods for nonlinear Muskingum model using hybrid invasive weed optimization algorithm. Math. Probl. Eng. 2015, 1–15 (2015)

    MathSciNet  MATH  Google Scholar 

  19. Ouyang, A., Tang, Z., Li, K., Sallam, A., Sha, E.: Estimating parameters of Muskingum model using an adaptive hybrid PSO algorithm. Int. J. Pattern Recognit. Artif. Intell. 28, 1–29 (2014)

    Article  Google Scholar 

  20. Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. SIAM-AMS Proceedings (1976)

  21. Schropp, J.: One-step and multistep procedures for constrained minimization problems. IMA J. Numer. Anal. 20, 135–152 (2000)

    Article  MathSciNet  Google Scholar 

  22. Shanno, D.F.: Conditioning of quasi-newton methods for function minimization. Math. Comput. 24, 647–647 (1970)

    Article  MathSciNet  Google Scholar 

  23. Toint, P.L.: Global convergence of the partitioned BFGS algorithm for convex partially separable opertimization. Math. Program. 36, 290–306 (1986)

    Article  Google Scholar 

  24. Vrahatis, M.N., Androulakis, G.S., Lambrinos, J.N., et al: A class of gradient unconstrained minimization algorithms with adaptive stepsize. J. Comput. Appl. Math. 114, 367–386 (2000)

    Article  MathSciNet  Google Scholar 

  25. Wan, Z., Huang, S., Zheng, X.: New cautious BFGS Algorithm based on modified Armijo-type line search. J. Inequalities Appl. 241, 1–10 (2012)

    MathSciNet  MATH  Google Scholar 

  26. Wan, Z., Teo, K., Shen, X., Hu, C.: New, BFGS method for unconstrained optimization problem based on modified Armijo line search. Optimization 63, 285–304 (2014)

    Article  MathSciNet  Google Scholar 

  27. Yuan, G.L.: Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems. Optim. Lett. 3, 11–21 (2009)

    Article  MathSciNet  Google Scholar 

  28. Yuan, G.L., Wei, Z.X.: New line search methods for unconstrained optimization. J. Korean Stat. Soc. 38, 29–39 (2009)

    Article  MathSciNet  Google Scholar 

  29. Yuan, G.L., Wei, Z.X.: Convergence analysis of a modified BFGS method on convex minimizations. Comput. Optim. Appl. 47, 237–255 (2010)

    Article  MathSciNet  Google Scholar 

  30. Yuan, G.L., Wei, Z.X.: The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions. Acta Mathematics Sinica. English Series 24, 35–42 (2008)

    Google Scholar 

  31. Yuan, G.L., Wei, Z.X., Lu, X.W.: Global convergence of BFGS and PRP methods under a modified weak Wolfe-Powell line search. Appl. Math. Model. 47, 811–825 (2017)

    Article  MathSciNet  Google Scholar 

  32. Yuan, G.L., Sheng, Z., Wang, B.P., Hu, W.J., Li, C.N.: The global convergence of a modified BFGS method for nonconvex functions. J. Comput. Appl. Math. 327, 274–294 (2018)

    Article  MathSciNet  Google Scholar 

  33. Yuan, Y.X., Sun, W.: Theory and Methods of Optimization. Science Press of China, China (1999)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editor and the referees for their valuable comments which greatly improve this paper.

Funding

This work is supported by the High Level Innovation Teams and Excellent Scholars Program in Guangxi institutions of higher education (Grant No. [2019]52), the National Natural Science Fund of China (Grant No. 11661009), the Guangxi Natural Science Key Fund (No. 2017GXNSFDA198046), and the Special Funds for Local Science and Technology Development Guided by the Central Government (No. ZY20198003).

Author information

Authors and Affiliations

  1. College of Mathematics and Information Science, Center for Applied Mathematical of Guangxi, Guangxi University, Nanning, Guangxi, People’s Republic of China

    Gonglin Yuan, Pengyuan Li & Junyu Lu

Authors
  1. Gonglin Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gonglin Yuan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, G., Li, P. & Lu, J. The global convergence of the BFGS method with a modified WWP line search for nonconvex functions. Numer Algor 91, 353–365 (2022). https://doi.org/10.1007/s11075-022-01265-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-022-01265-3

Keywords

Search

Navigation