Skip to main content
SpringerLink
Log in

An Inertial Spectral CG Projection Method Based on the Memoryless BFGS Update

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

Combining the derivative-free projection with inertial technique, we propose a hybrid inertial spectral conjugate gradient projection method for solving constrained nonlinear monotone equations. The conjugate parameter is a hybrid modification based on the memoryless BFGS update. The spectral parameter is obtained from quasi-Newton equations and double-truncated to ensure the sufficient descent. The search direction with a restart procedure satisfies sufficient descent condition and the trust region property at each iteration, independent of the choice of line search. We also investigate the theoretical properties, such as the global convergence and linear convergence rate, of the inertial projection method under normal assumptions. Numerical performances indicate the superiority of the proposed method in solving large-scale equations and restoring the blurred images contaminated by the Gaussian noise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Abubakar, A.B., Kumam, P., Ibrahim, A.H.: Inertial derivative-free projection method for nonlinear monotone operator equations with convex constraints. IEEE Access 9, 92157–92167 (2021)

    Google Scholar 

  2. Amini, K., Faramarzi, P., Bahrami, S.: A spectral conjugate gradient projection algorithm to solve the large-scale system of monotone nonlinear equations with application to compressed sensing. Int. J. Comput. Math. 99(11), 2290–230 (2022)

    MathSciNet  MATH  Google Scholar 

  3. Awwal, A.M., Kumam, P., Mohammad, H., Watthayu, W., Abubakar, A.B.: A Perry-type derivative-free algorithm for solving nonlinear system of equations and minimizing \(l_1\) regularized problem. Optimization 70(5–6), 1231–1259 (2021)

    MathSciNet  MATH  Google Scholar 

  4. Awwal, A.M., Wang, L., Kumam, P., Mohammad, H.: A two-step spectral gradient projection method for system of nonlinear monotone equations and image deblurring problems. Symmetry 12(6), 874 (2020)

    Google Scholar 

  5. Bovik, A.: Handbook of Image and Video Processing. Academic Press, San Diego (2000)

    MATH  Google Scholar 

  6. Cruz, W.L., Raydan, M.: Nonmonotone spectral methods for large-scale nonlinear systems. Optim. Methods Softw. 18(5), 583–599 (2003)

    MathSciNet  MATH  Google Scholar 

  7. Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)

    MathSciNet  MATH  Google Scholar 

  8. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    MathSciNet  MATH  Google Scholar 

  9. Dong, X.L., Liu, H.W., He, Y.B.: A self-adjusting conjugate gradient method with sufficient descent condition and conjugacy condition. J. Optim. Theory Appl. 165, 225–241 (2015)

    MathSciNet  MATH  Google Scholar 

  10. Faramarzi, P., Amini, K.: A modified spectral conjugate gradient method with global convergence. J. Optim. Theory Appl. 182, 667–690 (2019)

    MathSciNet  MATH  Google Scholar 

  11. Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction, application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Sign. Process. 1(4), 586–597 (2008)

    Google Scholar 

  12. Gao, P.T., He, C.J.: An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints. Calcolo 55(4), 53 (2018)

    MathSciNet  MATH  Google Scholar 

  13. Gao, P.T., He, C.J., Liu, Y.: An adaptive family of projection methods for constrained monotone nonlinear equations with applications. Appl. Math. Comput. 359, 1–16 (2019)

    MathSciNet  MATH  Google Scholar 

  14. Huang, L.H.: Limited memory technique using trust regions for nonlinear equations. Appl. Math. Model. 39(19), 5969–5981 (2015)

    MathSciNet  MATH  Google Scholar 

  15. Ibrahim, A.H., Kumam, P., Rapajić, S., Papp, Z., Abubakar, A.B.: Approximation methods with inertial term for large-scale nonlinear monotone equations. Appl. Numer. Math. 181, 417–435 (2022)

    MathSciNet  MATH  Google Scholar 

  16. Ibrahim, A.H., Kumam, P., Sun, M., Chaipunya, P.: Projection method with inertial step for nonlinear equations: Application to signal recovery. J. Ind. Manag. Optim. 19(1), 30–55 (2023)

    MathSciNet  MATH  Google Scholar 

  17. Jian, J.B., Chen, Q., Jiang, X.Z., Zeng, Y.F., Yin, J.H.: A new spectral conjugate gradient method for large-scale unconstrained optimization. Optim. Methods Softw. 32(3), 503–515 (2017)

    MathSciNet  MATH  Google Scholar 

  18. Jian, J.B., Yin, J.H., Tang, C.M., Han, D.L.: A family of inertial derivative-free projection methods for constrained nonlinear pseudo-monotone equations with applications. Comput. Appl. Math. 41, 309 (2022)

    MathSciNet  MATH  Google Scholar 

  19. Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints. J. Comput. Appl. Math. 172, 375–397 (2004)

    MathSciNet  MATH  Google Scholar 

  20. Koorapetse, M., Kaelo, P., Lekoko, S., Diphofu, T.: A derivative-free RMIL conjugate gradient projection method for convex constrained nonlinear monotone equations with applications in compressive sensing. Appl. Numer. Math. 165, 431–441 (2021)

    MathSciNet  MATH  Google Scholar 

  21. Kou, C.X., Dai, Y.H.: A modified self-scaling memoryless Broyden-Fletcher-Goldfarb-Shanno method for unconstrained optimization. J. Optim. Theory Appl. 165, 209–224 (2015)

    MathSciNet  MATH  Google Scholar 

  22. La Cruz, W., Martinez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations: Theory and experiments. Math. Comp. 75(225), 1429–1448 (2006)

    MathSciNet  MATH  Google Scholar 

  23. Li, M., Liu, H.W., Liu, Z.X.: A new family of conjugate gradient methods for unconstrained optimization. J. Appl. Math. Comput. 58, 219–234 (2018)

    MathSciNet  MATH  Google Scholar 

  24. Li, Y., Yuan, G.L., Wei, Z.X.: A limited-memory BFGS algorithm based on a trust-region quadratic model for large-scale nonlinear equations. PLoS One 10(5), e0120993 (2015)

    Google Scholar 

  25. Liu, J.K., Feng, Y.M.: A derivative-free iterative method for nonlinear monotone equations with convex constraints. Numer. Algorithms 82, 245–262 (2019)

    MathSciNet  MATH  Google Scholar 

  26. Liu, J.K., Feng, Y.M., Zou, L.M.: A spectral conjugate gradient method for solving large-scale unconstrained optimization. Comput. Math. Appl. 77(3), 731–739 (2019)

    MathSciNet  MATH  Google Scholar 

  27. Liu, P.J., Shao, H., Wang, Y., Wu, X.Y.: A three-term CGPM-based algorithm without Lipschitz continuity for constrained nonlinear monotone equations with applications. Appl. Numer. Math. 175, 98–107 (2022)

    MathSciNet  MATH  Google Scholar 

  28. Liu, P.J., Shao, H., Yuan, Z.H., Wu, X.Y., Zheng, T.L.: A family of three-term conjugate gradient projection methods with a restart procedure and their relaxed-inertial extensions for the constrained nonlinear pseudo-monotone equations with applications. Numer. Algorithms (2023). https://doi.org/10.1007/s11075-023-01527-8

    Article  Google Scholar 

  29. Liu, P.J., Wu, X.Y., Shao, H., Zhang, Y., Cao, S.H.: Three adaptive hybrid derivative-free projection methods for constrained monotone nonlinear equations and their applications. Numer. Linear Algebr. Appl. 30(2), e2471 (2023)

    MathSciNet  MATH  Google Scholar 

  30. Liu, Z.X., Liu, H.W., Dai, Y.H.: An improved Dai-Kou conjugate gradient algorithm for unconstrained optimization. Comput. Optim. Appl. 75, 145–167 (2020)

    MathSciNet  MATH  Google Scholar 

  31. Luo, D., Li, Y., Lu, J.Y., Yuan, G.L.: A conjugate gradient algorithm based on double parameter scaled Broyden-Fletcher-Goldfarb-Shanno update for optimization problems and image restoration. Neural Comput. Appl. 34(1), 535–553 (2022)

    Google Scholar 

  32. Ma, G.D., Jin, J.C., Jian, J.B., Yin, J.H., Han, D.L.: A modified inertial three-term conjugate gradient projection method for constrained nonlinear equations with applications in compressed sensing. Numer. Algorithms 92, 1621–1653 (2022)

    MathSciNet  MATH  Google Scholar 

  33. Oren, S.S., Luenberger, D.G.: Self-scaling variable metric (SSVM) algorithms, part I: criteria and sufficient conditions for scaling a class of algorithms. Manag. Sci. 20(5), 845–862 (1974)

    MATH  Google Scholar 

  34. Pang, J.S.: Inexact Newton methods for the nonlinear complementary problem. Math. Program. 36(1), 54–71 (1986)

    MathSciNet  MATH  Google Scholar 

  35. Papp, Z., Rapajić, S.: FR type methods for systems of large-scale nonlinear monotone equations. Appl. Math. Comput. 269, 816–823 (2015)

    MathSciNet  MATH  Google Scholar 

  36. Perry, J.M.: A class of conjugate gradient algorithms with a two-step variable-metric memory, Discussion Paper 269, Center for Mathematical Studies in Economics and Management Sciences, Northwestern University, Evanston, Illinois (1977)

  37. Polyak, B.T.: Introduction to optimization, optimization software, p. 49. Publications Division, New York (1987)

    Google Scholar 

  38. Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)

    Google Scholar 

  39. Rui, S.P., Zhang, J.: An inexact algorithm for a class of CP with non-Lipschitzian function. J. Huaibei Norm. Univ. Nat. Sci. 33(1), 16–18 (2012)

    Google Scholar 

  40. Shanno, D.F.: On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15, 1247–1257 (1978)

    MathSciNet  MATH  Google Scholar 

  41. Solodov, M.V., Svaiter, B.F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L. (eds.) Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, pp. 355–369. Kluwer, Dordrecht (1998)

    Google Scholar 

  42. Stanimirović, P.S., Ivanov, B., Djordjević, S., Brajević, I.: New hybrid conjugate gradient and Broyden-Fletcher-Goldfarb-Shanno conjugate gradient methods. J. Optim. Theory Appl. 178, 860–884 (2018)

    MathSciNet  MATH  Google Scholar 

  43. Wang, C.W., Wang, Y.J., Xu, C.L.: A projection method for a system of nonlinear monotone equations with convex constraints. Math. Methods Oper. Res. 66(1), 33–46 (2007)

    MathSciNet  MATH  Google Scholar 

  44. Wu, X.Y., Shao, H., Liu, P.J., Zhang, Y., Zhuo, Y.: An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with application in signal recovery and image denoising problems. J. Comput. Appl. Math. 422, 114879 (2023)

    MathSciNet  MATH  Google Scholar 

  45. Xiao, Y.H., Wang, Q.Y., Hu, Q.J.: Non-smooth equations based method for \(l_1\)-norm problems with applications to compressed sensing. Nonlinear Anal. Theor. 74(11), 3570–3577 (2011)

    MATH  Google Scholar 

  46. Xiao, Y.H., Zhu, H.: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405, 310–319 (2013)

    MathSciNet  MATH  Google Scholar 

  47. Yin, J.H., Jian, J.B., Jiang, X.Z., Liu, M.X., Wang, L.Z.: A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer. Algorithms 88, 389–418 (2021)

    MathSciNet  MATH  Google Scholar 

  48. Yin, J.H., Jian, J.B., Jiang, X.Z., Wu, X.D.: A family of inertial-relaxed DFPM-based algorithms for solving large-scale monotone nonlinear equations with application to sparse signal restoration. J. Comput. Appl. Math. 419, 114674 (2023)

    MathSciNet  MATH  Google Scholar 

  49. Yuan, G.L., Li, T.T., Hu, W.J.: A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. 147, 129–141 (2020)

    MathSciNet  MATH  Google Scholar 

  50. Yuan, G.L., Meng, Z.H., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. J. Optim. Theory Appl. 168, 129–152 (2016)

    MathSciNet  MATH  Google Scholar 

  51. Zhou, G., Toh, K.C.: Superline convergence of a Newton-type algorithm for monotone equations. J. Optim. Theory Appl. 125, 205–221 (2005)

    MathSciNet  MATH  Google Scholar 

  52. Zhou, W.J., Li, D.H.: A globally convergent BFGS method for nonlinear monotone equations without any merit functions. Math. Comput. 77, 2231–2240 (2008)

    MathSciNet  MATH  Google Scholar 

  53. Zhou, W.J., Li, D.H.: Limited memory BFGS method for nonlinear monotone equations. J. Comput. Math. 25, 89–96 (2007)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (72071202), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX23_2651) and the Graduate Innovation Program of China University of Mining and Technology (2023WLJCRCZL139).

Author information

Authors and Affiliations

  1. School of Mathematics, China University of Mining and Technology, Xuzhou, 221116, Jiangsu, China

    Xiaoyu Wu, Hu Shao, Pengjie Liu & Yue Zhuo

Authors
  1. Xiaoyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hu Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengjie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yue Zhuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hu Shao.

Additional information

Communicated by Sébastien Le Digabel.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Shao, H., Liu, P. et al. An Inertial Spectral CG Projection Method Based on the Memoryless BFGS Update. J Optim Theory Appl 198, 1130–1155 (2023). https://doi.org/10.1007/s10957-023-02265-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-023-02265-6

Keywords

Mathematics Subject Classification

Search

Navigation