Skip to main content
SpringerLink
Log in

Two Descent Dai-Yuan Conjugate Gradient Methods for Systems of Monotone Nonlinear Equations

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, we present two Dai-Yuan type iterative methods for solving large-scale systems of nonlinear monotone equations. The methods can be considered as extensions of the classical Dai-Yuan conjugate gradient method for unconstrained optimization. By employing two different approaches, the Dai-Yuan method is modified to develop two different search directions, which are combined with the hyperplane projection technique of Solodov and Svaiter. The first search direction was obtained by carrying out eigenvalue study of the search direction matrix of an adaptive DY scheme, while the second is obtained by minimizing the distance between two adaptive versions of the DY method. Global convergence of the methods are established under mild conditions and preliminary numerical results show that the proposed methods are promising and more effective compared to some existing methods in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Ahmed, H.I., Al-Naemi, G.M.: A modified Dai-Yuan conjugate gradient method and its global convergence. Iraqi J. Sci. 53(3), 620–628 (2012)

    Google Scholar 

  2. Andrei, N.: Open problems in conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. 34(2), 319–330 (2011)

    MathSciNet  MATH  Google Scholar 

  3. Andrei, N.: A simple three-term conjugate gradient algorithm for unconstrained optimization. J. Comput. Appl. Math. 241, 19–29 (2013)

    MathSciNet  MATH  Google Scholar 

  4. Arzuka, I., Abu Bakar, M., Leon, W.J.: A scaled three-term conjugate gradient method for unconstrained optimization. J. Ineq. Appl. 2016, 325 (2016). https://doi.org/10.1186/s13660-016-1239-1

    Article  MathSciNet  MATH  Google Scholar 

  5. Babaie-Kafaki, S., Ghanbari, R.: A descent family of Dai-Liao conjugate gradient methods. Optim. Methods Softw. 29(3), 583–591 (2013)

    MathSciNet  MATH  Google Scholar 

  6. Babaie-Kafaki, S., Ghanbari, R.: The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234, 625–630 (2014)

    MathSciNet  MATH  Google Scholar 

  7. Babaie-Kafaki, S., Ghanbari, R.: A descent extension of of the Polak-Ribier\(\acute{e}\)-Polyak conjugate gradient method. Comput. Math. Appl. 68, 2005–2011 (2014)

    MathSciNet  MATH  Google Scholar 

  8. Babaie-Kafaki, S., Ghanbari, R.: Two optimal Dai-Liao conjugate gradient methods. Optimization 64, 2277–2287 (2015)

    MathSciNet  MATH  Google Scholar 

  9. Bouaricha, A., Schnabel, R.B.: Tensor methods for large sparse systems of nonlinear equations. Math. Program. 82, 377–400 (1998)

    MathSciNet  MATH  Google Scholar 

  10. Cheng, W.: A PRP type method for systems of monotone equations. Math. Comput. Model. 50, 15–20 (2009)

    MathSciNet  MATH  Google Scholar 

  11. Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23, 296–320 (2013)

    MathSciNet  MATH  Google Scholar 

  12. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)

    MathSciNet  MATH  Google Scholar 

  13. Dai, Y.H., Yuan, Y.: A Nonlinear conjugate gradient method with a strong global convergence property. Soc. Ind. Appl. Math. 10(1), 177–182 (1999)

    MathSciNet  MATH  Google Scholar 

  14. Dai, Y.H., Yuan, Y.X.: Nonlinear Conjugate Gradient Methods. Shanghai Scientific and Technical Publishers, Shanghai (2000)

    Google Scholar 

  15. Dai, Z., Chen, X., Wen, F.: A modified Perry’s conjugate gradient method-based derivative-free method for solving large-scale nonlinear monotone equation. Appl. Math. Comput. 270, 378–386 (2015)

    MathSciNet  MATH  Google Scholar 

  16. Dauda, M.K., Mamat, M., Mohamed, M.A., Waziri, M.Y.: Improved quasi-Newton method via SR1 update for solving symmetric systems of nonlinear equations. Malay. J. Fund. Appl. Sci. 15(1), 117–120 (2019)

    Google Scholar 

  17. Dolan, E.D., Mor, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2, Ser. A), 201–2013 (2002)

    MathSciNet  MATH  Google Scholar 

  18. Dong, X., He, Y.: Global convergence of a modified DY conjugate gradient method. J. Numer. Methods Comput. Appl. 31(1), 1 (2010)

    MathSciNet  MATH  Google Scholar 

  19. Fasano, G., Lampariello, F., Sciandrone, M.: A truncated nonmonotone Gauss–Newton method for large-scale nonlinear least-squares problems. Comput. Optim. Appl. 34, 343–358 (2006)

    MathSciNet  MATH  Google Scholar 

  20. Fatemi, M.: An optimal parameter for Dai-Liao family of conjugate gradient methods. J. Optim. Theory Appl. 169(2), 587–605 (2016)

    MathSciNet  MATH  Google Scholar 

  21. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone linesearch technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)

    MathSciNet  MATH  Google Scholar 

  22. Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 35–58 (2006)

    MathSciNet  MATH  Google Scholar 

  23. Halilu, A.S., Majumder, A., Waziri, M.Y., Ahmed, K.: Signal recovery with convex constrained nonlinear monotone equations through conjugate gradient hybrid approach. Math. Comput. Simul. (2021). https://doi.org/10.1016/j.matcom.2021.03.020

    Article  MathSciNet  MATH  Google Scholar 

  24. Halilu, A.S., Majumder, A., Waziri, M.Y., Awwal, A.M., Ahmed, K.: On solving double direction methods for convex constrained monotone nonlinear equations with image restoration. Comput. Appl. Math. 40, 239 (2021)

    MathSciNet  MATH  Google Scholar 

  25. Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–436 (1952)

    MathSciNet  MATH  Google Scholar 

  26. Hively, G.A.: On a class of nonlinear integral equations arising in transport theory. SIAM J. Math. Anal. 9(5), 787–792 (1978)

    MathSciNet  MATH  Google Scholar 

  27. Jinkui, L., Shengjie, L.: Multivariate spectral projection method for convex constrained nonlinear monotone equations. J. Ind. Manag. Optim. 13(1), 283–297 (2017)

    MathSciNet  MATH  Google Scholar 

  28. Jinkui, L., Xianglin, D., Kairong, W.: A mixed spectral conjugate descent-Dai-Yuan conjugate gradient method. J. Appl. Math. 2012, 569795 (2012). https://doi.org/10.1155/2012/569795

    Article  MATH  Google Scholar 

  29. Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties. J. Comput. Appl. Math. 172, 375–397 (2004)

    MathSciNet  MATH  Google Scholar 

  30. Keyvan, A., Ahmad, K.: A new line search strategy for finding separating hyperplane in projection-based methods. Numer. Algorithms 70(3), 559–570 (2015)

    MathSciNet  MATH  Google Scholar 

  31. Kincaid, D., Cheney, W.: Numerical Analysis. Brooks/Cole Publishing Company, California (1991)

    MATH  Google Scholar 

  32. Kobayashi, H., Narushima, Y., Yabe, H.: Descent three-term conjugate gradient methods based on secant conditions for unconstrained optimization. Optim. Methods Softw. (2017). https://doi.org/10.1080/10556788.2017.1338288

    Article  MathSciNet  MATH  Google Scholar 

  33. Koorapetse, M., Kaelo, P.: Self adaptive spectral conjugate gradient method for solving nonlinear monotone equations. J. Egypt. Math. Soc. 28(4), 1–21 (2020)

    MathSciNet  MATH  Google Scholar 

  34. Koorapetse, M., Kaelo, P., Offen, E.R.: A scaled derivative-free projection method for solving nonlinear monotone equations. Bull. Iran. Math. Soc. 45(3), 755–770 (2018)

    MathSciNet  MATH  Google Scholar 

  35. La Cruz, W. Martínez, J.M. Raydan, M.: Spectral residual method without gradient information for Solving large-scale nonlinear systems: Theory and Experiments, Citeseer, Technical Report RT-04-08 (2004)

  36. La Cruz, W.: A spectral algorithm for large-scale systems of nonlinear monotone equations. Numer. Algorithms 76, 1109–1130 (2017)

    MathSciNet  MATH  Google Scholar 

  37. Li, D.H., Fukushima, M.: A globally and superlinearly convergent Gauss–Newton-based BFGS method for symmetric nonlinear equations. SIAM J. Numer. Anal. 37(1), 152–172 (2000)

    MathSciNet  MATH  Google Scholar 

  38. Li, D.H., Fukushima, M.: A derivative-free linesearch and global convergence of Broyden-like method for nonlinear equations. Optim. Methods Softw. 13, 583–599 (2000)

    Google Scholar 

  39. Li, G., Tang, C., Wei, Z.: New conjugacy condition and related new conjugate gradient methods for unconstrained optimization. J. Comput. Appl. Math. 202(2), 523–539 (2007)

    MathSciNet  MATH  Google Scholar 

  40. Liu, J., Du, S.: Modified three-term conjugate gradient method and its applications. Hindawi Mathematical Problems in Engineering 2019, 5976595 (2019). https://doi.org/10.1155/2019/5976595

    Article  MathSciNet  MATH  Google Scholar 

  41. Liu, J., Wu, X.: New three-term conjugate gradient method for solving unconstrained optimization problems. ScienceAsia. 40, 295–300 (2014)

    Google Scholar 

  42. Livieris, I.E. Pintelas, P.: A descent Dai-Liao conjugate gradient method based on a modified secant equation and its global convergence. ISRN Comput. Math. 2012 (2012)

  43. Livieris, I.E., Pintelas, P.: Globally convergent modified Perrys conjugate gradient method. Appl. Math. Comput. 218, 9197–9207 (2012)

    MathSciNet  MATH  Google Scholar 

  44. Marquardt, D.W.: An algorithm for least\(-\)squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)

    MathSciNet  MATH  Google Scholar 

  45. Mompati, S., Koorapetse, M., Kaelo, P.: Globally convergent three-term conjugate gradient projection methods for solving nonlinear monotone equations. Arab. J. Math. 7(4), 289–301 (2018)

    MathSciNet  MATH  Google Scholar 

  46. Narushima, Y., Yabe, H., Ford, J.A.: A three-term conjugate gradient method with sufficient descent property for unconstrained optimization. SIAM J. Optim. 21, 212–230 (2011)

    MathSciNet  MATH  Google Scholar 

  47. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)

    MATH  Google Scholar 

  48. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)

    MATH  Google Scholar 

  49. Peiting, G., Chuanjiang, H.: A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations. Optimization 67, 1631–1648 (2018)

    MathSciNet  MATH  Google Scholar 

  50. Perry, A.: A modified conjugate gradient algorithm. Oper. Res. Tech. Notes. 26(6), 1073–1078 (1978)

    MathSciNet  MATH  Google Scholar 

  51. Polak, B.T.: The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 4, 94–112 (1969)

    Google Scholar 

  52. Polak, E., Ribiìre, G.: Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inform. Rech. Oper. 16, 35–43 (1969)

    MATH  Google Scholar 

  53. Sabi’u, J., Shah, A., Waziri, M.Y., Ahmed, K.: Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving monotone nonlinear equations with convex constraint. Int. J. Comput. Methods. (2020). https://doi.org/10.1142/S0219876220500437

    Article  MATH  Google Scholar 

  54. Shiker, A.K., Amini, K.: A new projection-based algorithm for solving a large-scale nonlinear system of nonlinear equations. Croat. Oper. Res. Rev. 9, 63–73 (2018)

    MATH  Google Scholar 

  55. Solodov, M.V., Iusem, A.N.: Newton-type methods with generalized distances for constrained optimization. Optimization 41(3), 257–27 (1997)

    MathSciNet  MATH  Google Scholar 

  56. Solodov, M.V., Svaiter, B.F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L. (eds.) Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, pp. 355–369. Kluwer, Dordrecht (1998)

    Google Scholar 

  57. Sun, W., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer, NewYork (2006)

    MATH  Google Scholar 

  58. Wang, C.W., Wang, Y.J., Xu, C.L.: A projection method for a system of nonlinear monotone equations with convex constraints. Math. Methods Oper. Res. 66, 33–46 (2007)

    MathSciNet  MATH  Google Scholar 

  59. Waziri, M.Y., Sabiu, J.: A Derivative-free conjugate gradient method and its global convergence for solving symmetric nonlinear equations. Hindawi Publishing Corporation International Journal of Mathematics and Mathematical Sciences (2015)

  60. Waziri, M.Y., Leong, W.J., Hassan, M.A.: Jacobian free-diagonal Newton’s method for nonlinear systems with singular Jacobian. Malays. J. Math. Sci. 5(2), 241–255 (2011)

    MathSciNet  MATH  Google Scholar 

  61. Waziri, M.Y., Ahmed, K., Sabi’u, J.: A family of Hager-Zhang conjugate gradient methods for system of monotone nonlinear equations. Appl. Math. Comput. 361, 645–660 (2019)

    MathSciNet  MATH  Google Scholar 

  62. Waziri, M.Y., Ahmed, K., Sabi’u, J.: A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arab. J. Math. 9, 443–457 (2020)

    MathSciNet  MATH  Google Scholar 

  63. Waziri, M.Y., Ahmed, K., Sabi’u, J.: Descent Perry conjugate gradient methods for systems of monotone nonlinear equations. Numer. Algorithms 85, 763–785 (2020)

    MathSciNet  MATH  Google Scholar 

  64. Waziri, M.Y., Usman, H., Halilu, A.S., Ahmed, K.: Modified matrix-free methods for solving systems of nonlinear equations. Optimization (2020). https://doi.org/10.1080/02331934.2020.1778689

    Article  MATH  Google Scholar 

  65. Waziri, M.Y., Ahmed, K., Sabi’u, J., Halilu, A.S.: Enhanced Dai-Liao conjugate gradient methods for systems of monotone nonlinear equations. SeMA J. 78, 15–51 (2021). https://doi.org/10.1007/s40324-020-00228-9

    Article  MathSciNet  MATH  Google Scholar 

  66. Wu, G., Li, Y., Yuan, G.: A three-term conjugate gradient algorithm with quadratic convergence for Unconstrained optimization problems. Hindawi Math. Probl. Eng. 2018, 4813030 (2018). https://doi.org/10.1155/2018/4813030

    Article  MathSciNet  MATH  Google Scholar 

  67. Xiao, Y., Zhu, H.: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405, 310–319 (2013)

    MathSciNet  MATH  Google Scholar 

  68. Yan, Q.R., Peng, X.Z., Li, D.H.: A globally convergent derivative-free method for solving large-scale nonlinear monotone equations. J. Comput. Appl. Math. 234, 649–657 (2010)

    MathSciNet  MATH  Google Scholar 

  69. Yu, G.: A derivative-free method for solving large-scale nonlinear systems of equations. J. Ind. Manag. Optim. 6, 149–160 (2010)

    MathSciNet  MATH  Google Scholar 

  70. Yu, G.: Nonmonotone spectral gradient-type methods for large-scale unconstrained optimization and nonlinear systems of equations. Pac. J. Optim. 7, 387–404 (2011)

    MathSciNet  MATH  Google Scholar 

  71. Yu, G.H., Niu, S.Z., Ma, J.H.: Multivariate spectral gradient projection method for non-linear monotone equations with convex constraints. J. Ind. Manag. Optim. 9, 117–129 (2013)

    MathSciNet  MATH  Google Scholar 

  72. Yuan, Y.: Subspace methods for large scale nonlinear equations and nonlinear least squares. Optim. Eng. 10, 207–218 (2009)

    MathSciNet  MATH  Google Scholar 

  73. Yuan, G., Zhang, M.: A three term Polak-Ribiere-Polyak conjugate gradient algorithm for large-scale nonlinear equations. J. Comput. Appl. Math. 286, 186–195 (2015)

    MathSciNet  MATH  Google Scholar 

  74. Yuan, G., Wang, B., Sheng, Z.: The Hager-Zhang conjugate gradient algorithm for large-scale nonlinear equations. Int. J. Comput Math. 96(8), 1533–1547 (2019)

    MathSciNet  Google Scholar 

  75. Zhang, J., Wang, Y.: A new trust region method for nonlinear equations. Math. Methods Oper. Res. 58, 283–298 (2003)

    MathSciNet  MATH  Google Scholar 

  76. Zhang, L., Zhou, W.J.: Spectral gradient projection method for solving nonlinear monotone equations. J. Comput. Appl. Math. 196, 478–484 (2006)

    MathSciNet  MATH  Google Scholar 

  77. Zhang, J.Z., Deng, N.Y., Chen, L.H.: New quasi-Newton equation and related methods for unconstrained optimization. J. Optim. Theory Appl. 102(1), 147–157 (1999)

    MathSciNet  MATH  Google Scholar 

  78. Zhang, K., Liu, H., Liu, Z.: A New Dai-Liao conjugate gradient method with optimal parameter choice. Numer. Funct. Anal. Optim. (2018). https://doi.org/10.1080/01630563.2018.1535506

    Article  MATH  Google Scholar 

  79. Zhao, Y.B., Li, D.: Monotonicity of fixed point and normal mappings associated with variational inequality and its application. SIAM J. Optim. 11, 962–973 (2001)

    MathSciNet  MATH  Google Scholar 

  80. Zhou, W., Shen, D.: Convergence properties of an iterative method for solving symmetric non-linear equations. J. Optim. Theory Appl. 164(1), 277–289 (2015)

    MathSciNet  MATH  Google Scholar 

  81. Zhou, W., Wang, F.: A PRP-based residual method for large-scale monotone nonlinear equations. Appl. Math. Comput. 261, 1–7 (2015)

    MathSciNet  MATH  Google Scholar 

  82. Zhou, W., Zhang, L.: A nonlinear conjugate gradient method based on the MBFGS secant condition. Optim. Methods Softw. 21(5), 707–714 (2006)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the entire members of the numerical optimization research group, Bayero University, Kano for their comments and encouragement in the course of this work.

Author information

Authors and Affiliations

  1. Numerical Optimization Research Group, Bayero University, Kano, Nigeria

    Mohammed Yusuf Waziri & Kabiru Ahmed

Authors
  1. Mohammed Yusuf Waziri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kabiru Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed Yusuf Waziri.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Waziri, M.Y., Ahmed, K. Two Descent Dai-Yuan Conjugate Gradient Methods for Systems of Monotone Nonlinear Equations. J Sci Comput 90, 36 (2022). https://doi.org/10.1007/s10915-021-01713-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-021-01713-7

Keywords

Mathematics Subject Classification

Search

Navigation