Skip to main content
SpringerLink
Log in

An adaptive projection BFGS method for nonconvex unconstrained optimization problems

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The BFGS method is a common and effective method for solving unconstrained optimization problems in quasi-Newton algorithm. However, many scholars have proven that the algorithm may fail in some cases for nonconvex problems under Wolfe conditions. In this paper, an adaptive projection BFGS algorithm is proposed naturally which can solve nonconvex problems, and the following properties are shown in this algorithm: ➀ The generation of the step size \(\alpha _j\) satisfies the popular Wolfe conditions; ➁ a specific condition is proposed which has sufficient descent property, and if the current point satisfies this condition, the ordinary BFGS iteration process proceeds as usual; ➂ otherwise, the next iteration point \(x_{j+1}\) is generated by the proposed adaptive projection method. This algorithm is globally convergent for nonconvex problems under the weak-Wolfe-Powell (WWP) conditions and has a superlinear convergence rate, which can be regarded as an extension of projection BFGS method proposed by Yuan et al. (J. Comput. Appl. Math. 327:274-294, 2018). Furthermore, the final numerical results and the application of the algorithm to the Muskingum model demonstrate the feasibility and competitiveness of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data that support the findings of this study are available on request from the corresponding author, upon reasonable request.

References

  1. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)

    MathSciNet  Google Scholar 

  2. Andrei, N.: An adaptive scaled BFGS method for unconstrained optimization. Num. Algo. 77(2), 413–432 (2018)

    MathSciNet  Google Scholar 

  3. Broyden, C.G.: The convergence of a class of double-rank minimization algorithms 1.general considerations. IMA J Appl. Math. 6(1), 76–90 (1970)

    Google Scholar 

  4. Broyden, C.G., Dennis, J.E., Jr., Moré, J.J.: On the local and superlinear convergence of quasi-Newton methods. IMA J Appl. Math. 12(3), 223–245 (1973)

    MathSciNet  Google Scholar 

  5. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J Numer. Anal. 26(3), 727–739 (1989)

    MathSciNet  Google Scholar 

  6. Byrd, R.H., Nocedal, J., Yuan, Y.-X.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J Numer. Anal. 24(5), 1171–1190 (1987)

    MathSciNet  Google Scholar 

  7. Dai, Y.: A perfect example for the BFGS method. Math. Prog. 138(1), 501–530 (2013)

    MathSciNet  Google Scholar 

  8. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28(126), 549–560 (1974)

    MathSciNet  Google Scholar 

  9. Dennis, J.E., Jr., Moré, J.J.: Quasi-Newton methods, motivation and theory. SIAM Rev. 19(1), 46–89 (1977)

    MathSciNet  Google Scholar 

  10. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Prog. 91(2), 201–213 (2002)

    MathSciNet  Google Scholar 

  11. Fletcher, R.: A new approach to variable metric algorithms. Comp. J 13(3), 317–322 (1970)

    Google Scholar 

  12. Fletcher, R. An overview of unconstrained optimization. Algorithms for Continuous Optimization, pp. 109–143 (1994)

  13. Fu, Z., Ren, K., Shu, J., Sun, X., Huang, F.: Enabling personalized search over encrypted outsourced data with efficiency improvement. IEEE Trans. Parallel Distrib. Syst. 27(9), 2546–2559 (2015)

    Google Scholar 

  14. Fu, Z., Wu, X., Guan, C., Sun, X., Ren, K.: Toward efficient multi-keyword fuzzy search over encrypted outsourced data with accuracy improvement. IEEE Trans. Inform. Forensics Secur. 11(12), 2706–2716 (2016)

    Google Scholar 

  15. Geem, Z.W.: Parameter estimation for the nonlinear Muskingum model using the BFGS technique. J Irrigation Drainage Eng. 132(5), 474–478 (2006)

    Google Scholar 

  16. Goldfarb, D.: A family of variable-metric methods derived by variational means. Math. Comput. 24(109), 23–26 (1970)

    MathSciNet  Google Scholar 

  17. Griewank, A.: The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients. Math. Prog. 50(1), 141–175 (1991)

    MathSciNet  Google Scholar 

  18. Griewank, A., Toint, P.L.: Local convergence analysis for partitioned quasi-Newton updates. Numerische Mathematik 39(3), 429–448 (1982)

    MathSciNet  Google Scholar 

  19. Grippo, L., Lucidi, S.: A globally convergent version of the Polak-Ribiére conjugate gradient method. Math. Prog. 78(3), 375–391 (1997)

    Google Scholar 

  20. Gu, B., Sun, X., Sheng, V.S.: Structural minimax probability machine. IEEE Trans. Neural Netw. Learn. Syst. 28(7), 1646–1656 (2016)

    MathSciNet  PubMed  Google Scholar 

  21. Li, D., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J Comput. Appl. Math. 129(1–2), 15–35 (2001)

    ADS  MathSciNet  Google Scholar 

  22. Li, D., Fukushima, M.: On the global convergence of the BFGS method for nonconvex unconstrained optimization problems. SIAM J Optim. 11(4), 1054–1064 (2001)

    MathSciNet  Google Scholar 

  23. Liu, J., Li, S.: A projection method for convex constrained monotone nonlinear equations with applications. Comput. Math. Appl. 70(10), 2442–2453 (2015)

    MathSciNet  Google Scholar 

  24. Moscariello, A., Takx, R.A., Schoepf, U.J., Renker, M., Zwerner, P.L., O’Brien, T.X., Allmendinger, T., Vogt, S., Schmidt, B., Savino, G., et al.: Coronary CT angiography: image quality, diagnostic accuracy, and potential for radiation dose reduction using a novel iterative image reconstruction technique–comparison with traditional filtered back projection. European Radiology 21(10), 2130–2138 (2011)

    PubMed  Google Scholar 

  25. Ouyang, A., Liu, L.-B., Sheng, Z., Wu, F. A class of parameter estimation methods for nonlinear Muskingum model using hybrid invasive weed optimization algorithm. Mathematical Problems in Engineering, 2015, (2015)

  26. Ouyang, A., Tang, Z., Li, K., Sallam, A., Sha, E.: Estimating parameters of Muskingum model using an adaptive hybrid PSO algorithm. Int. J Pattern Recogn. Artif. Intell. 28(01), 1459003 (2014)

    Google Scholar 

  27. Pan, Z., Lei, J., Zhang, Y., Sun, X., Kwong, S.: Fast motion estimation based on content property for low-complexity H. 265/HEVC encoder. IEEE Trans. Broadcasting 62(3), 675–684 (2016)

    Google Scholar 

  28. Pan, Z., Zhang, Y., Kwong, S.: Efficient motion and disparity estimation optimization for low complexity multiview video coding. IEEE Trans. Broadcasting 61(2), 166–176 (2015)

    Google Scholar 

  29. Powell, M. J. A new algorithm for unconstrained optimization. In Nonlinear Programming, pp. 31–65. Elsevier (1970)

  30. Powell, M. J. Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In Nonlinear Programming, SIAM-AMS Proceedings, vol. 9 (1976)

  31. Shanno, D.F.: Conditioning of quasi-Newton methods for function minimization. Math. Comput. 24(111), 647–656 (1970)

    MathSciNet  Google Scholar 

  32. Toint, P.L.: Global convergence of the partitioned BFGS algorithm for convex partially separable optimization. Math. Prog. 36(3), 290–306 (1986)

    MathSciNet  Google Scholar 

  33. Wang, C., Wang, Y., Xu, C.: A projection method for a system of nonlinear monotone equations with convex constraints. Math. Method Oper. Res. 66(1), 33–46 (2007)

    MathSciNet  Google Scholar 

  34. Wei, Z., Li, G., Qi, L.: New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems. Appl. Math. Comput. 179(2), 407–430 (2006)

    MathSciNet  Google Scholar 

  35. Wei, Z., Yu, G., Yuan, G., Lian, Z.: The superlinear convergence of a modified BFGS-type method for unconstrained optimization. Comput. Optim. Appl. 29(3), 315–332 (2004)

    MathSciNet  Google Scholar 

  36. Xia, Z., Wang, X., Sun, X., Liu, Q., Xiong, N.: Steganalysis of LSB matching using differences between nonadjacent pixels. Multimed. Tool Appl. 75(4), 1947–1962 (2016)

    Google Scholar 

  37. Xia, Z., Wang, X., Zhang, L., Qin, Z., Sun, X., Ren, K.: A privacy-preserving and copy-deterrence content-based image retrieval scheme in cloud computing. IEEE Trans. Inform. Forensics Secur. 11(11), 2594–2608 (2016)

    Google Scholar 

  38. Yuan, G., Li, P., Lu, J. The global convergence of the BFGS method with a modified WWP line search for nonconvex functions. Numerical Algorithms, pp. 1–13 (2022)

  39. Yuan, G., Sheng, Z., Wang, B., Hu, W., Li, C.: The global convergence of a modified BFGS method for nonconvex functions. J Comput. Appl. Math. 327, 274–294 (2018)

    MathSciNet  Google Scholar 

  40. Yuan, G., Wang, X., Sheng, Z.: The projection technique for two open problems of unconstrained optimization problems. J Optim. Theory Appl. 186(2), 590–619 (2020)

    MathSciNet  Google Scholar 

  41. Yuan, G., Wei, Z.: Convergence analysis of a modified BFGS method on convex minimizations. Comput. Optim. Appl. 47(2), 237–255 (2010)

    MathSciNet  Google Scholar 

  42. Yuan, G., Wei, Z., Lu, X.: Global convergence of BFGS and PRP methods under a modified weak Wolfe-Powell line search. Appl. Math. Model. 47, 811–825 (2017)

    MathSciNet  Google Scholar 

  43. Yuan, G., Wei, Z., Yang, Y.: The global convergence of the Polak-Ribiére-Polyak conjugate gradient algorithm under inexact line search for nonconvex functions. J Comput. Appl. Math. 362, 262–275 (2019)

    MathSciNet  Google Scholar 

  44. Yuan, G., Zhang, M., Zhou, Y.: Adaptive scaling damped BFGS method without gradient Lipschitz continuity. Appl. Math. Lett. 124, 107634 (2022)

    MathSciNet  Google Scholar 

  45. Zhang, L., Zhou, W.: Spectral gradient projection method for solving nonlinear monotone equations. J Comput. Appl. Math. 196(2), 478–484 (2006)

    ADS  MathSciNet  Google Scholar 

  46. Zhang, S.: Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Opt. Lasers Eng. 48(2), 149–158 (2010)

    ADS  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the editor and anonymous reviewers for their suggestions, which improve this paper greatly.

Funding

This work is supported by Guangxi Science and Technology base and Talent Project (Grant No. AD22080047), the Special Funds for Local Science and Technology Development Guided by the Central Government (No. ZY20198003), the Innovation Funds of Chinese University (Grant No. 2021BCF03001), and the special foundation for Guangxi Ba Gui Scholars.

Author information

Authors and Affiliations

  1. School of Mathematics and Information Science, Center for Applied Mathematics of Guangxi, Guangxi University, Nanning, Guangxi, 530004, China

    Gonglin Yuan, Xiong Zhao, Kejun Liu & Xiaoxuan Chen

Authors
  1. Gonglin Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kejun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoxuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The contributions can be divided into the following parts: Gonglin Yuan: conceptualization, methodology, software, supervision. Xiong Zhao: data curation, writing—original draft preparation, writing—reviewing and editing, software, validation, visualization, investigation, formal analysis. Kejun Liu: writing—original draft preparation, data curation. Xiaoxuan Chen: writing—review and editing.

Corresponding author

Correspondence to Xiong Zhao.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, G., Zhao, X., Liu, K. et al. An adaptive projection BFGS method for nonconvex unconstrained optimization problems. Numer Algor 95, 1747–1767 (2024). https://doi.org/10.1007/s11075-023-01626-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-023-01626-6

Keywords

Search

Navigation