Skip to main content
SpringerLink
Log in

An inertial Fletcher–Reeves-type conjugate gradient projection-based method and its spectral extension for constrained nonlinear equations

  • Original Research
  • Published:
Journal of Applied Mathematics and Computing Aims and scope Submit manuscript

Abstract

In this paper, we initially enhance the Fletcher–Reeves (FR) conjugate parameter through a shrinkage multiplier, leading to a derivative-free two-term search direction and its extended spectral version. The results indicate that both search directions demonstrate sufficient descent and trust-region properties, irrespective of the line search method utilized. Then, by combining the hyperplane projection-based approach and inertia technique, we present two inertial FR-type conjugate gradient projection-based methods for solving constrained nonlinear equations. The global convergence of our methods is theoretically established, without requiring the monotonicity or pseudo-monotonicity of the underlying mapping, nor the Lipschitz continuity condition. Numerical experiments conducted on constrained nonlinear equations, as well as applications in regularized decentralized logistic regression problems and sparse signal restoration problems, have demonstrated the numerical efficacy of our methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availibility statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Solodov, M.V., Svaiter, B.F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L. (eds.) Reformulation: piecewise smooth, semismooth and smoothing methods. In: Applied Optimization, vol. 22, pp. 355–369. Springer, Boston (1998)

  2. Zhou, G., Toh, K.C.: Superlinear convergence of a Newton-type algorithm for monotone equations. J. Optim. Theory Appl. 125(1), 205–221 (2005)

    Article  MathSciNet  Google Scholar 

  3. Chen, Z., Cheng, W., Li, X.: A global convergent quasi-Newton method for systems of monotone equations. J. Appl. Math. Comput. 44(1), 455–465 (2014)

    Article  MathSciNet  Google Scholar 

  4. Zhou, W.J., Li, D.H.: A globally convergent BFGS method for nonlinear monotone equations without any merit functions. Math. Comput. 77(264), 2231–2240 (2008)

    Article  MathSciNet  Google Scholar 

  5. Fasano, G., Lampariello, F., Sciandrone, M.: A truncated nonmonotone Gauss-Newton method for large-scale nonlinear least-squares problems. Comput. Optim. Appl. 34, 343–358 (2006)

    Article  MathSciNet  Google Scholar 

  6. Li, D.H., Fukushima, M.: A globally and superlinearly convergent Gauss–Newton-based BFGS method for symmetric nonlinear equations. SIAM J. Numer. Anal. 37(1), 152–172 (2000)

    Article  MathSciNet  Google Scholar 

  7. Xiao, X., Li, Y., Wen, Z., Zhang, L.: A regularized semi-smooth Newton method with projection steps for composite convex programs. J. Sci. Comput. 76(1), 364–389 (2018)

    Article  MathSciNet  Google Scholar 

  8. Yin, J.H., Jian, J.B., Ma, G.D.: A modified inexact Levenberg–Marquardt method with the descent property for solving nonlinear equations. Comput. Optim. Appl. 87, 289–322 (2024)

    Article  MathSciNet  Google Scholar 

  9. Cheng, W.Y.: A PRP type method for systems of monotone equations. Math. Comput. Model. 50, 15–20 (2009)

    Article  MathSciNet  Google Scholar 

  10. Polak, B., Ribi\(\grave{{\rm r}}\)re, G.: Note surla convergence des mèthodes de directions conjugèes. Rev. Fr. Inf. Rech. Oper. 3(1), 35–43 (1969)

  11. Papp, Z., Rapajić, S.: FR type methods for systems of large-scale nonlinear monotone equations. Appl. Math. Comput. 269, 816–823 (2015)

    MathSciNet  Google Scholar 

  12. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)

    Article  MathSciNet  Google Scholar 

  13. Ibrahim, A.H., Kumama, P., Rapajić, S., Papp, Z., Abubakar, A.B.: Approximation methods with inertial term for large-scale nonlinear monotone equations. Appl. Numer. Math. 181, 417–435 (2022)

    Article  MathSciNet  Google Scholar 

  14. Abubakar, A.B., Muangchoo, K., Ibrahim, A.H., Abubakar, J., Rano, S.A.: FR-type algorithm for finding approximate solutions to nonlinear monotone operator equations. Arabian J. Math. 10, 261–270 (2021)

    Article  MathSciNet  Google Scholar 

  15. Abubakar, A.B., Kumama, P., Ibrahim, A.H., Chaipunya, P., Rano, S.A.: New hybrid three-term spectral-conjugate gradient method for finding solutions of nonlinear monotone operator equations with applications. Math. Comput. Simul. 201, 670–683 (2022)

    Article  MathSciNet  Google Scholar 

  16. Ibrahim, A.H., Kumama, P., Abubakar, A.B., Jirakitpuwapat, W., Abubakar, J.: A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing. Heliyon 6, e03466 (2020)

    Article  Google Scholar 

  17. Danmalam, K.U., Mohammad, H., Abubakar, A.B., Awwal, A.M.: Hybrid algorithm for system of nonlinear monotone equations based on the convex combination of Fletcher–Reeves and a new conjugate residual parameters. Thai J. Math. 18(4), 2093–2106 (2020)

    MathSciNet  Google Scholar 

  18. Yin, J.H., Jian, J.B., Jiang, X.Z., Liu, M.X., Wang, L.Z.: A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer. Algorithms 88, 389–418 (2021)

    Article  MathSciNet  Google Scholar 

  19. Yin, J.H., Jian, J.B., Jiang, X.Z.: A generalized hybrid CGPM-based algorithm for solving large-scale convex constrained equations with applications to image restoration. J. Comput. Appl. Math. 391, 113423 (2021)

    Article  MathSciNet  Google Scholar 

  20. Ma, G.D., Liu, L.Q., Jian, J.B., Yan, X.H.: A new hybrid CGPM-based algorithm for constrained nonlinear monotone equations with applications. J. Appl. Math. Comput. 70, 103–147 (2024)

    Article  MathSciNet  Google Scholar 

  21. Liu, P.J., Shao, H., Wang, Y., Wu, X.Y.: A three-term CGPM-based algorithm without Lipschitz continuity for constrained nonlinear monotone equations with applications. Appl. Numer. Math. 175, 98–107 (2022)

    Article  MathSciNet  Google Scholar 

  22. Li, D.D., Wu, J.Q., Li, Y., Wang, S.H.: A modified spectral gradient projection-based algorithm for large-scale constrained nonlinear equations with applications in compressive sensing. J. Comput. Appl. Math. 424, 115006 (2023)

    Article  MathSciNet  Google Scholar 

  23. Jian, J.B., Yin, J.H., Tang, C.M., Han, D.L.: A family of inertial derivative-free projection methods for constrained nonlinear pseudo-monotone equations with applications. Comput. Appl. Math. 41(7), 309 (2022)

    Article  MathSciNet  Google Scholar 

  24. Jian, J.B., Ren, Z.W., Yin, J.H., Han, D.L., Wu, X.D.: An effective inertial-relaxed CGPM for nonlinear monotone equations. J. Appl. Math. Comput. 70, 689–710 (2024)

    Article  MathSciNet  Google Scholar 

  25. Ma, G.D., Jin, J.C., Jian, J.B., Yin, J.H., Han, D.L.: A modified inertial three-term conjugate gradient projection method for constrained nonlinear equations with applications in compressed sensing. Numer. Algorithms 92(3), 1621–1653 (2023)

    Article  MathSciNet  Google Scholar 

  26. Liu, P.J., Shao, H., Yuan, Z.H., Wu, X.Y., Zheng, T.L.: A family of three-term conjugate gradient projection methods with a restart procedure and their relaxed-inertial extensions for the constrained nonlinear pseudo-monotone equations with applications. Numer. Algorithms 94, 1055–1083 (2023)

    Article  MathSciNet  Google Scholar 

  27. Wu, X.Y., Shao, H., Liu, P.J., Zhuo, Y.: An inertial spectral CG projection method based on the memoryless BFGS update. J. Optim. Theory Appl. 198(3), 1130–1155 (2023)

    Article  MathSciNet  Google Scholar 

  28. Liu, W.L., Jian, J.B., Yin, J.H.: An inertial spectral conjugate gradient projection method for constrained nonlinear pseudo-monotone equations. Numer. Algorithms (2024) https://doi.org/10.1007/s11075-023-01736-1

  29. Babaie-Kafaki, S., Mirhoseini, N., Aminifard, Z.: A class of CG algorithms overcoming jamming of the iterative solving process and its application in image restoration. J. Comput. Appl. Math. 442, 115727 (2024)

    Article  MathSciNet  Google Scholar 

  30. Barzilai, J., Borwein, J.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  Google Scholar 

  31. Solodov, M.V., Svaiter, B.F.: A new projection method for variational inequality problems. SIAM J. Control. Optim. 37(3), 765–776 (1999)

    Article  MathSciNet  Google Scholar 

  32. Ibrahim, A.H., Kumam, P., Hassan, B.A., Abubakar, A.B., Abubakar, J.: A derivative-free three-term Hestenes–Stiefel type method for constrained nonlinear equations and image restoration. Inter. J. Comput. Math. 99(5), 1041–1065 (2022)

    Article  MathSciNet  Google Scholar 

  33. Ou, Y.G., Xu, W.J.: A unified derivative-free projection method model for large-scale nonlinear equations with convex constraints. J. Ind. Manag. Optim. 18(5), 3539–3560 (2022)

    Article  MathSciNet  Google Scholar 

  34. Zheng, L.: A new projection algorithm for solving a system of nonlinear equations with convex constraints. Bull. Korean Math. Soc. 50, 823–832 (2013)

    Article  MathSciNet  Google Scholar 

  35. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I. Springer, Berlin (2003)

    Google Scholar 

  36. Waziri, M.Y., Ahmed, K.: Two descent Dai–Yuan conjugate gradient methods for systems of monotone nonlinear equations. J. Sci. Comput. 90, 1–53 (2022)

    Article  MathSciNet  Google Scholar 

  37. Wu, X.Y., Shao, H., Liu, P.J., Zhang, Y., Zhuo, Y.: An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with applications in signal recovery and image denoising problems. J. Comput. Appl. Math. 422, 114879 (2023)

    Article  MathSciNet  Google Scholar 

  38. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    Article  MathSciNet  Google Scholar 

  39. Luo, H.: Accelerated primal-dual methods for linearly constrained convex optimization problems. arXiv preprint arXiv:2109.12604 (2021)

  40. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

  41. Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction, application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)

    Article  Google Scholar 

  42. Pang, J.S.: Inexact Newton methods for the nonlinear complementary problem. Math. Program. 36(1), 54–71 (1986)

    Article  Google Scholar 

  43. Xiao, Y.H., Wang, Q.Y., Hu, Q.J.: Non-smooth equations based method for \(\ell _1\)-norm problems with applications to compressed sensing. Nonlinear Anal. Theory Methods Appl. 74(11), 3570–3577 (2011)

    Article  Google Scholar 

  44. Waziri, M.Y., Ahmed, K., Halilu, A.S., Awwal, A.M.: Modified Dai–Yuan iterative scheme for nonlinear systems and its application. Numer. Algebra, Control. Optim. 13(1), 53–80 (2023)

  45. Liu, J.K., Feng, Y.M.: A derivative-free iterative method for nonlinear monotone equations with convex constraints. Numer. Algorithms 82, 245–262 (2019)

    Article  MathSciNet  Google Scholar 

  46. Waziri, M.Y., Kiri, A.I., Kiri, A.A., Halilu, A.S., Ahmed, K.: A modified conjugate gradient parameter via hybridization approach for solving large-scale systems of nonlinear equations. SeMA J. 80(3), 479–501 (2023)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Mathematics and Information Science, Guangxi University, Nanning, 530004, China

    Haiyan Zheng

  2. School of Mathematics, China University of Mining and Technology, Xuzhou, 221116, China

    Jiayi Li & Pengjie Liu

  3. School of Computational Science and Electronics, Hunan Institute of Engineering, Xiangtan, 411104, China

    Xianglin Rong

Authors
  1. Haiyan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiayi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengjie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianglin Rong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Pengjie Liu or Xianglin Rong.

Ethics declarations

Conflicts of interest

The authors declare that they have no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China (No. 71861002) and the Guangxi Natural Science Foundation (No. 2017GXNSFBA198238).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, H., Li, J., Liu, P. et al. An inertial Fletcher–Reeves-type conjugate gradient projection-based method and its spectral extension for constrained nonlinear equations. J. Appl. Math. Comput. (2024). https://doi.org/10.1007/s12190-024-02062-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12190-024-02062-y

Keywords

Mathematics Subject Classification

Search

Navigation