Skip to main content
SpringerLink
Log in

Accelerated Dai-Liao projection method for solving systems of monotone nonlinear equations with application to image deblurring

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

A modified Dai-Liao type conjugate gradient method for solving large-scale nonlinear systems of monotone equations is introduced and investigated in actual research. The starting point is the Dai-Liao type conjugate gradient method which is based on the descent Dai-Liao method and the hyperplane projection technique, known as the Dai-Liao projection method (DLPM). Our algorithm, termed MSMDLPM, proposes a novel search direction for the DLPM, which arises from appropriate acceleration parameters obtained after hybridizing the accelerated gradient-descent method MSM with the DLPM method. The main goal of the proposed MSMDLPM method is to correlate the MSM and the DLPM. The global convergence and the convergence rate of the MSMDLPM method are investigated theoretically. Numerical results show the efficiency of the proposed method in solving large-scale nonlinear systems of monotone equations. The effectiveness of the method in image restoration is verified based on performed numerical experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abdullah, H., Waziri, M.Y., Yusuf, S.O.: A double direction conjugate gradient method for solving largE-scale system of nonlinear equations. J. Math. Comput. Sci. 7, 606–624 (2017)

    Google Scholar 

  2. Abubakar, A.B., Kumam, P.: A descent Dai-Liao conjugate gradient method for nonlinear equations. Numer. Algorithms 81, 197–210 (2019)

    Article  MATH  Google Scholar 

  3. Abubakar, A.B., Kumam, P., Mohammad, H.: A note on the spectral gradient projection method for nonlinear monotone equations with applications. Computational and Applied Mathematics 39 (2020), Article number: 129

  4. Abubakar, A.B., Muangchoo, K., Ibrahim, A.H., Muhammad, A.B., Jolaoso, L.O., Aremu, K.O.: A new three-term Hestenes-Stiefel type method for nonlinear monotone operator equations and image restoration. IEEE Access 9, 18262–18277 (2021)

    Article  Google Scholar 

  5. Ahookhosh, M., Amini, K., Bahrami, S.: Two derivative-free projection approaches for systems of large-scale nonlinear monotone equations. Numer. Algor. 64(1), 21–42 (2013)

    Article  MATH  Google Scholar 

  6. Al-Baali, M., Spedicato, E., Maggioni, F.: Broyden’s Quasi-Newton methods for a nonlinear system of equations and unconstrained optimization: A review and open problems. Optim. Methods Software 29(5), 937–954 (2014)

    Article  MATH  Google Scholar 

  7. Aminifard, Z., Babaie-Kafaki, S.: An optimal parameter choice for the Dai-Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix. A Quarterly Journal of Operations Research 17, 317–330 (2019)

    MATH  Google Scholar 

  8. Andrei, N.: A Dai-Liao conjugate gradient algorithm with clustering of eigenvalues. Numer. Algorithms 77(4), 1273–1282 (2018)

    Article  MATH  Google Scholar 

  9. Argyros, I.K.: On a class of Newton-like methods for solving nonlinear equations. J. Comput. Appl. Math. 228(1), 115–122 (2009)

    Article  MATH  Google Scholar 

  10. Awwal, A.M., Wang, L., Kumam, P., Mohammad, H.: A two-step spectral gradient projection method for system of nonlinear monotone equations and image deblurring problems. Symmetry 12(6) (2020), Article number: 874

  11. Babaie-Kafaki, S., Ghanbari, R.: The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. European J. Oper. Res. 234(3), 625–630 (2014)

    Article  MATH  Google Scholar 

  12. Babaie-Kafaki, S., Gambari, R.: A descent family of Dai-Liao conjugate gradient methods. Optim. Meth. Soft. 29(3), 583–591 (2014)

    Article  MATH  Google Scholar 

  13. Banham, M.R., Katsaggelos, A.K.: Digital image restoration. IEEE Signal Process. Mag. 14(2), 24–41 (1997)

    Article  Google Scholar 

  14. Bovik, A.C.: Handbook of Image and Video Processing. Academic, New York, NY, USA (2010)

    MATH  Google Scholar 

  15. Chan, C.L., Katsaggelos, A.K., Sahakian, A.V.: Image sequence filtering in quantum-limited noise with applications to low-dose fluoroscopy. IEEE Trans. Med. Imaging 12(3), 610–621 (1993)

    Article  Google Scholar 

  16. Cheng, W.: A PRP type method for systems of monotone equations. Math. Comput. Model. 50, 15–20 (2009)

    Article  MATH  Google Scholar 

  17. Cordero, A., Hueso, J.L., Martınez, E., Torregrosa, J.R.: Increasing the convergence order of an iterative method for nonlinear systems. Appl. Math. Lett. 25(12), 2369–2374 (2012)

    Article  MATH  Google Scholar 

  18. Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)

    Article  MATH  Google Scholar 

  19. Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)

    Article  MATH  Google Scholar 

  20. Dauda, M.K., Magaji, A.S., Abdullah, H., Sabi’u, J., Halilu, A.S.: A new search direction via hybrid conjugate gradient coefficient for solving nonlinear system of equations. Malaysian Journal of Computing and Applied Mathematics 2, 8–15 (2019)

    Google Scholar 

  21. González-Lima, D.M., de Oca, F.M.: A Newton-like method for nonlinear system of equations. Numer. Algorithms 52(3), 479–506 (2009)

    Article  MATH  Google Scholar 

  22. Dauda, M.K., Usman, S., Ubale, H., Mamat, M.: An alternative modified conjugate gradient coefficient for solving nonlinear system of equations. Open Journal of Science and Technology 2, 5–8 (2019)

    Article  Google Scholar 

  23. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MATH  Google Scholar 

  24. Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)

    Article  MATH  Google Scholar 

  25. Halilu, A.S., Waziri, M.Y.: An enhanced matrix-free method via double step length approach for solving systems of nonlinear equations, International Journal of Applied. Math. Res. 6, 147–156 (2017)

    Google Scholar 

  26. Halilu, A.S., Waziri, M.Y.: A transformed double step length method for solving large-scale systems of nonlinear equations. J. Numer. Math. Stoch. 9(1), 20–32 (2017)

    MATH  Google Scholar 

  27. Hirsch, M.J., Pardalos, P.M., Resende, M.G.C.: Solving systems of nonlinear equations with continuous GRASP. Nonlinear Anal. Real World Appl. 10, 2000–2006 (2009)

    Article  MATH  Google Scholar 

  28. Ibrahim, A.H., Kumam, P., Abubakar, A.B., Jirakitpuwapat, W., Abubakar, J.: A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing. Heliyon 6(3), e03466 (2020)

    Article  Google Scholar 

  29. Ibrahim, A.H., Kumam, P., Kumam, W.: A family of derivative-free conjugate gradient methods for constrained nonlinear equations and image restoration. IEEE Access 8, 162714–162729 (2020)

    Article  Google Scholar 

  30. Ivanov, B., Stanimirović, P. S., Shaini, B. I., Ahmad, H., Wang, M. K.: A Novel Value for the Parameter in the Dai-Liao-Type Conjugate Gradient Method. Journal of Function Spaces 2021 (2021), Article ID 6693401, 10 pages

  31. Ivanov, B., Stanimirović, P. S., Milovanović, G. V., Djordjević, S., Brajević, I.: Accelerated multiple step-size methods for solving unconstrained optimization problems. Optimization Methods and Software 2019

  32. Koorapetse, M.S., Kaelo, P.: Globally convergent three-term conjugate gradient projection methods for solving nonlinear monotone equations. Arab. J. Math. (Springer) 7, 289–301 (2018)

    Article  MATH  Google Scholar 

  33. Koorapetse, M., Kaelo, P.: A new three-term conjugate gradient-based projection method for solving large-cale nonlinear monotone equations. Math. Model. Anal. 24(4), 550–563 (2019)

    Article  Google Scholar 

  34. Koorapetse, M., Kaelo, P.: Self adaptive spectral conjugate gradient method for solving nonlinear monotone equations. J. Egyptian Math. Soc. 28(1) (2020), Paper No. 4, 21 pp

  35. La Cruz, W., Martinez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations: theory and experiments. Math. Comp. 75(225), 1429–1448 (2006)

    Article  MATH  Google Scholar 

  36. Leong, W.J., Hassan, M.A., Yusuf, M.W.: A matrix-free quasi-Newton method for solving large-scale nonlinear systems. Comput. Math. Appl. 62, 2354–2363 (2011)

    Article  MATH  Google Scholar 

  37. Li, Q., Li, D.H.: A class of derivative-free methods for large-scale nonlinear monotone equations. IMA J. Numer. Anal. 31, 1625–1635 (2011)

    Article  MATH  Google Scholar 

  38. Liu, J., Li, S.: Multivariate spectral DY-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization 13(1), 283–295 (2017)

    Article  MATH  Google Scholar 

  39. Liu, J.K., Li, S.J.: A three-term derivative-free projection method for nonlinear monotone system of equations. Calcolo 53, 427–450 (2016)

    Article  MATH  Google Scholar 

  40. Lotfi, M., Hosseini, S.M.: An efficient Dai–Liao type conjugate gradient method by reformulating the CG parameter in the search direction equation. J. Comput. Appl. Math. 371 (2020), Article 112708

  41. Luo, Y.Z., Tang, G.J., Zhou, L.N.: Hybrid approach for solving systems of nonlinear equations using chaos optimization and quasi-Newton method. Appl. Soft Comput. 8(2), 1068–1073 (2008)

    Article  Google Scholar 

  42. Mo, Y., Liu, H., Wang, Q.: Conjugate direction particle swarm optimization solving systems of nonlinear equations. Comput. Math. Appl. 57(11), 1877–1882 (2009)

    Article  MATH  Google Scholar 

  43. Muhammad, K., Mamat, M., Waziri, M.Y.: A Broyden’s-like method for solving systems of nonlinear equations. World Appl. Sci. J. 21, 168–173 (2013)

    Google Scholar 

  44. Osinuga, I.A., Yusuff, S.O.: Quadrature based Broyden-like method for systems of nonlinear equations. Stat. Optim. Inf. Comput. 6, 130–138 (2018)

    Article  Google Scholar 

  45. Pei, J., Dražić, Z., Dražić, M., Mladenović, N., Pardalos, P.M.: Continuous Variable Neighborhood Search (C-VNS) for solving systems of nonlinear equations. INFORMS Journal on Computing, Articles in advance, pp. 1–16

  46. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7(1), 26–33 (1997)

    Article  MATH  Google Scholar 

  47. Sabi’u, J.: Enhanced derivative-free conjugate gradient method for solving symmetric nonlinear equations. International Journal of Advances in Applied Sciences 5, 50–57 (2016)

    Article  Google Scholar 

  48. Sabi’u, J., Shah, A., Waziri, M.Y.: Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations. Appl. Numer. Math. 153, 217–233 (2020)

    Article  MATH  Google Scholar 

  49. Sabi’u, J., Gadu, A.M.: A Projected hybrid conjugate gradient method for solving large-scale system of nonlinear equations. Malaysian Journal of Computing and Applied Mathematics 1, 10–20 (2018)

    Google Scholar 

  50. Sabi’u, J., Sanusi, U.: An efficient new conjugate gradient approach for solving symmetric nonlinear equations. Asian Journal of Mathematics and Computer Research 12, 34–43 (2016)

    Google Scholar 

  51. Sabi’u, J., Waziri, M.Y.: Effective modified hybrid conjugate gradient method for large-scale symmetric nonlinear equations. Appl. Appl. Math. 12, 1036–1056 (2017)

    MATH  Google Scholar 

  52. Sabi’u, J., Waziri, M.Y., Idris, A.: A new hybrid Dai-Yuan and Hestenes-Stiefel conjugate gradient parameter for solving system of nonlinear equations, MAYFEB. Journal of Mathematics 1, 44–55 (2017)

    Google Scholar 

  53. Sharma, J.R., Guha, R.K.: Simple yet efficient Newton-like method for systems of nonlinear equations. Calcolo 53(3), 451–473 (2016)

    Article  MATH  Google Scholar 

  54. Solodov, M.V., Svaiter, B.F.: A globally convergent inexact Newton method for systems of monotone equations, Mathematical programming: Reformulation; nonsmooth, piecewise smooth, semismooth and smoothing methods. Appl. Optim. 22, 355–369 (1998)

    Article  Google Scholar 

  55. Stanimirović, P.S., Ivanov, B., Ma, H., Mosić, D.: A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive 28(4), 1573–1624 (2020)

    Article  MATH  Google Scholar 

  56. Stanimirović, P.S., Miladinović, M.B.: Accelerated gradient descent methods with line search. Numer. Algorithms 54, 503–520 (2010)

    Article  MATH  Google Scholar 

  57. Uba, L.Y., Waziri, M.Y.: Three-step derivative-free diagonal updating method for solving large-scale systems of nonlinear equations. J. Numer. Math. Stoch. 6, 73–83 (2014)

    MATH  Google Scholar 

  58. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  59. Wang, S., Guan, H.: A scaled conjugate gradient method for solving monotone nonlinear equations with convex constraints. J. Appl. Math. 2013 (2013), Article ID 286486

  60. Waziri, M.Y., Ahmed, K., Sabi’u, J.: A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arab. J. Math. (Springer) 9, 443–457 (2020)

    Article  MATH  Google Scholar 

  61. Waziri, M.Y., Ahmed, K., Sabi’u, J.: A family of Hager-Zhang conjugate gradient methods for system of monotone nonlinear equations. Appl. Math. Comput. 361, 645–660 (2019)

    MATH  Google Scholar 

  62. Waziri, M.Y., Aisha, H.A.: A diagonal quasi-Newton method for system of nonlinear equations. Appl. Math. Comput. Sci. 6, 21–30 (2014)

    MATH  Google Scholar 

  63. Waziri, M.Y., Leong, W.J., Hassan, M.A.: Diagonal Broyden-like method for large-scale systems of nonlinear equations. Malays. J. Math. Sci. 6, 59–73 (2012)

    Google Scholar 

  64. Waziri, M.Y., Leong, W. J., Hassan, M. A., Mamat, M.: A two-step matrix-free secant method for solving large-scale systems of nonlinear equations. J. Appl. Math. 2012, Art. ID 348654, 9 pp

  65. Waziri, M.Y., Leong, W.J., Hassan, M.A., Monsi, M.: Jacobian computation-free Newton’s method for systems of nonlinear equations. J. Numer. Math. Stoch. 2(1), 54–63 (2010)

    MATH  Google Scholar 

  66. Waziri, M.Y., Leong, W.J., Mamat, M.: An efficient solver for systems of nonlinear equations with singular Jacobian via diagonal updating. Appl. Math. Sci. (Ruse) 4(69–72), 3403–3412 (2010)

    MATH  Google Scholar 

  67. Waziri, M.Y., Leong, W.J., Hassan, M.A., Monsi, M.: A new Newton’s Method with diagonal Jacobian approximation for systems of nonlinear equations. J. Math. Stat. 6, 246–252 (2010)

    Article  MATH  Google Scholar 

  68. Waziri, M.Y., Leong, W.J., Mamat, M., Moyi, A.U.: Two-step derivative-free diagonally Newton’s method for large-scale nonlinear equations. World Appl. Sci. J. 21, 86–94 (2013)

    Google Scholar 

  69. Waziri, M.Y., Majid, Z.A.: An improved diagonal Jacobian approximation via a new quasi-Cauchy condition for solving large-scale systems of nonlinear equations. J. Appl. Math. 2013, Art. ID 875935, 6 pp

  70. Waziri, M.Y., Sabi’u, J.: A derivative-free conjugate gradient method and its global convergence for solving symmetric nonlinear equations, Int. J. Math. Math. Sci. 2015, Art. ID 961487, 8 pp

  71. Waziri, M.Y., Sabi’u, J.: An alternative conjugate gradient approach for large-scale symmetric nonlinear equations. J. Math. Comput. Sci. 6, 855–874 (2016)

    Google Scholar 

  72. Yakubu, U.A., Mamat, M., Mohamad, M.A., Rivaie, M., Sabi’u, J.: A recent modification on Dai-Liao conjugate gradient method for solving symmetric nonlinear equations. Far East J. Math. Sci. (FJMS) 103, 1961–1974 (2018)

    Article  Google Scholar 

  73. Yan, Q.-R., Peng, X.-Z., Li, D.-H.: A globally convergent derivative-free method for solving large-scale nonlinear monotone equations. J. Comput. Appl. Math. 234, 649–657 (2010)

    Article  MATH  Google Scholar 

  74. Xiao, Y.H., Wang, Q.Y., Hu, Q.J.: Non-smooth equations based method for \(\ell _1\)-norm problems with applications to compressed sensing. Nonlinear Anal. 74(11), 3570–3577 (2011)

    Article  MATH  Google Scholar 

  75. Xiao, Y., Zhu, H.: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405(1), 310–319 (2013)

    Article  MATH  Google Scholar 

  76. Yana, Q.R., Peng, X.Z., Li, D.H.: A globally convergent derivative-free method for solving large-scale nonlinear monotone equations. J. Comput. Appl. Math. 234, 649–657 (2010)

    Article  MATH  Google Scholar 

  77. Yuan, G., Li, T., Hu, W.: A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. 147, 129–141 (2020)

    Article  MATH  Google Scholar 

  78. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Transactions on Computational Imaging 3(1), 47–57 (2017)

    Article  Google Scholar 

  79. Zheng, L., Yang, L., Liang, Y.: A conjugate gradient projection method for solving equations with convex constraints. J. Comput. Appl. Math. 375 (2020), https://doi.org/10.1016/j.cam.2020.112781.

  80. Zhou, W., Li, D.: Limited memory BFGS method for nonlinear monotone equations. J. Comput. Math. 25(1), 89–96 (2007)

    Google Scholar 

  81. Zhou, W.J., Li, D.H.: A globally convergent bfgs method for nonlinear monotone equations without any merit functions. Math. Comp. 77(264), 2231–2240 (2008)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

The work of the second author was supported in part by the Serbian Academy of Sciences and Arts (\(\Phi \)-96). Predrag Stanimirović is supported by the Ministry of Education, Science and Technological Development, Republic of Serbia (No. 451-03-68/2022-14/200124), as well as by the Science Fund of the Republic of Serbia, (No. 7750185, Quantitative Automata Models: Fundamental Problems and Applications - QUAM).

Author information

Author notes

  1. Branislav Ivanov, Gradimir V. Milovanović and Predrag S. Stanimirović contributed equally to this work.

Authors and Affiliations

  1. Technical Faculty in Bor, University of Belgrade, Vojske Jugoslavije 12, Bor, 19210, Serbia

    Branislav Ivanov

  2. Mathematical Institute, Serbian Academy of Sciences and Arts, Kneza Mihaila 35, Belgrade, 11000, Serbia

    Gradimir V. Milovanović

  3. Faculty of Sciences and Mathematics, University of Niš, Višegradska 33, Niš, 18000, Serbia

    Gradimir V. Milovanović & Predrag S. Stanimirović

Authors
  1. Branislav Ivanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gradimir V. Milovanović
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Predrag S. Stanimirović
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gradimir V. Milovanović.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ivanov, B., Milovanović, G.V. & Stanimirović, P.S. Accelerated Dai-Liao projection method for solving systems of monotone nonlinear equations with application to image deblurring. J Glob Optim 85, 377–420 (2023). https://doi.org/10.1007/s10898-022-01213-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-022-01213-4

Keywords

Mathematics Subject Classification

Search

Navigation