Abstract
Combining the derivative-free projection with inertial technique, we propose a hybrid inertial spectral conjugate gradient projection method for solving constrained nonlinear monotone equations. The conjugate parameter is a hybrid modification based on the memoryless BFGS update. The spectral parameter is obtained from quasi-Newton equations and double-truncated to ensure the sufficient descent. The search direction with a restart procedure satisfies sufficient descent condition and the trust region property at each iteration, independent of the choice of line search. We also investigate the theoretical properties, such as the global convergence and linear convergence rate, of the inertial projection method under normal assumptions. Numerical performances indicate the superiority of the proposed method in solving large-scale equations and restoring the blurred images contaminated by the Gaussian noise.
Similar content being viewed by others
References
Abubakar, A.B., Kumam, P., Ibrahim, A.H.: Inertial derivative-free projection method for nonlinear monotone operator equations with convex constraints. IEEE Access 9, 92157–92167 (2021)
Amini, K., Faramarzi, P., Bahrami, S.: A spectral conjugate gradient projection algorithm to solve the large-scale system of monotone nonlinear equations with application to compressed sensing. Int. J. Comput. Math. 99(11), 2290–230 (2022)
Awwal, A.M., Kumam, P., Mohammad, H., Watthayu, W., Abubakar, A.B.: A Perry-type derivative-free algorithm for solving nonlinear system of equations and minimizing \(l_1\) regularized problem. Optimization 70(5–6), 1231–1259 (2021)
Awwal, A.M., Wang, L., Kumam, P., Mohammad, H.: A two-step spectral gradient projection method for system of nonlinear monotone equations and image deblurring problems. Symmetry 12(6), 874 (2020)
Bovik, A.: Handbook of Image and Video Processing. Academic Press, San Diego (2000)
Cruz, W.L., Raydan, M.: Nonmonotone spectral methods for large-scale nonlinear systems. Optim. Methods Softw. 18(5), 583–599 (2003)
Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)
Dong, X.L., Liu, H.W., He, Y.B.: A self-adjusting conjugate gradient method with sufficient descent condition and conjugacy condition. J. Optim. Theory Appl. 165, 225–241 (2015)
Faramarzi, P., Amini, K.: A modified spectral conjugate gradient method with global convergence. J. Optim. Theory Appl. 182, 667–690 (2019)
Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction, application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Sign. Process. 1(4), 586–597 (2008)
Gao, P.T., He, C.J.: An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints. Calcolo 55(4), 53 (2018)
Gao, P.T., He, C.J., Liu, Y.: An adaptive family of projection methods for constrained monotone nonlinear equations with applications. Appl. Math. Comput. 359, 1–16 (2019)
Huang, L.H.: Limited memory technique using trust regions for nonlinear equations. Appl. Math. Model. 39(19), 5969–5981 (2015)
Ibrahim, A.H., Kumam, P., Rapajić, S., Papp, Z., Abubakar, A.B.: Approximation methods with inertial term for large-scale nonlinear monotone equations. Appl. Numer. Math. 181, 417–435 (2022)
Ibrahim, A.H., Kumam, P., Sun, M., Chaipunya, P.: Projection method with inertial step for nonlinear equations: Application to signal recovery. J. Ind. Manag. Optim. 19(1), 30–55 (2023)
Jian, J.B., Chen, Q., Jiang, X.Z., Zeng, Y.F., Yin, J.H.: A new spectral conjugate gradient method for large-scale unconstrained optimization. Optim. Methods Softw. 32(3), 503–515 (2017)
Jian, J.B., Yin, J.H., Tang, C.M., Han, D.L.: A family of inertial derivative-free projection methods for constrained nonlinear pseudo-monotone equations with applications. Comput. Appl. Math. 41, 309 (2022)
Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints. J. Comput. Appl. Math. 172, 375–397 (2004)
Koorapetse, M., Kaelo, P., Lekoko, S., Diphofu, T.: A derivative-free RMIL conjugate gradient projection method for convex constrained nonlinear monotone equations with applications in compressive sensing. Appl. Numer. Math. 165, 431–441 (2021)
Kou, C.X., Dai, Y.H.: A modified self-scaling memoryless Broyden-Fletcher-Goldfarb-Shanno method for unconstrained optimization. J. Optim. Theory Appl. 165, 209–224 (2015)
La Cruz, W., Martinez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations: Theory and experiments. Math. Comp. 75(225), 1429–1448 (2006)
Li, M., Liu, H.W., Liu, Z.X.: A new family of conjugate gradient methods for unconstrained optimization. J. Appl. Math. Comput. 58, 219–234 (2018)
Li, Y., Yuan, G.L., Wei, Z.X.: A limited-memory BFGS algorithm based on a trust-region quadratic model for large-scale nonlinear equations. PLoS One 10(5), e0120993 (2015)
Liu, J.K., Feng, Y.M.: A derivative-free iterative method for nonlinear monotone equations with convex constraints. Numer. Algorithms 82, 245–262 (2019)
Liu, J.K., Feng, Y.M., Zou, L.M.: A spectral conjugate gradient method for solving large-scale unconstrained optimization. Comput. Math. Appl. 77(3), 731–739 (2019)
Liu, P.J., Shao, H., Wang, Y., Wu, X.Y.: A three-term CGPM-based algorithm without Lipschitz continuity for constrained nonlinear monotone equations with applications. Appl. Numer. Math. 175, 98–107 (2022)
Liu, P.J., Shao, H., Yuan, Z.H., Wu, X.Y., Zheng, T.L.: A family of three-term conjugate gradient projection methods with a restart procedure and their relaxed-inertial extensions for the constrained nonlinear pseudo-monotone equations with applications. Numer. Algorithms (2023). https://doi.org/10.1007/s11075-023-01527-8
Liu, P.J., Wu, X.Y., Shao, H., Zhang, Y., Cao, S.H.: Three adaptive hybrid derivative-free projection methods for constrained monotone nonlinear equations and their applications. Numer. Linear Algebr. Appl. 30(2), e2471 (2023)
Liu, Z.X., Liu, H.W., Dai, Y.H.: An improved Dai-Kou conjugate gradient algorithm for unconstrained optimization. Comput. Optim. Appl. 75, 145–167 (2020)
Luo, D., Li, Y., Lu, J.Y., Yuan, G.L.: A conjugate gradient algorithm based on double parameter scaled Broyden-Fletcher-Goldfarb-Shanno update for optimization problems and image restoration. Neural Comput. Appl. 34(1), 535–553 (2022)
Ma, G.D., Jin, J.C., Jian, J.B., Yin, J.H., Han, D.L.: A modified inertial three-term conjugate gradient projection method for constrained nonlinear equations with applications in compressed sensing. Numer. Algorithms 92, 1621–1653 (2022)
Oren, S.S., Luenberger, D.G.: Self-scaling variable metric (SSVM) algorithms, part I: criteria and sufficient conditions for scaling a class of algorithms. Manag. Sci. 20(5), 845–862 (1974)
Pang, J.S.: Inexact Newton methods for the nonlinear complementary problem. Math. Program. 36(1), 54–71 (1986)
Papp, Z., Rapajić, S.: FR type methods for systems of large-scale nonlinear monotone equations. Appl. Math. Comput. 269, 816–823 (2015)
Perry, J.M.: A class of conjugate gradient algorithms with a two-step variable-metric memory, Discussion Paper 269, Center for Mathematical Studies in Economics and Management Sciences, Northwestern University, Evanston, Illinois (1977)
Polyak, B.T.: Introduction to optimization, optimization software, p. 49. Publications Division, New York (1987)
Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)
Rui, S.P., Zhang, J.: An inexact algorithm for a class of CP with non-Lipschitzian function. J. Huaibei Norm. Univ. Nat. Sci. 33(1), 16–18 (2012)
Shanno, D.F.: On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15, 1247–1257 (1978)
Solodov, M.V., Svaiter, B.F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L. (eds.) Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, pp. 355–369. Kluwer, Dordrecht (1998)
Stanimirović, P.S., Ivanov, B., Djordjević, S., Brajević, I.: New hybrid conjugate gradient and Broyden-Fletcher-Goldfarb-Shanno conjugate gradient methods. J. Optim. Theory Appl. 178, 860–884 (2018)
Wang, C.W., Wang, Y.J., Xu, C.L.: A projection method for a system of nonlinear monotone equations with convex constraints. Math. Methods Oper. Res. 66(1), 33–46 (2007)
Wu, X.Y., Shao, H., Liu, P.J., Zhang, Y., Zhuo, Y.: An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with application in signal recovery and image denoising problems. J. Comput. Appl. Math. 422, 114879 (2023)
Xiao, Y.H., Wang, Q.Y., Hu, Q.J.: Non-smooth equations based method for \(l_1\)-norm problems with applications to compressed sensing. Nonlinear Anal. Theor. 74(11), 3570–3577 (2011)
Xiao, Y.H., Zhu, H.: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405, 310–319 (2013)
Yin, J.H., Jian, J.B., Jiang, X.Z., Liu, M.X., Wang, L.Z.: A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer. Algorithms 88, 389–418 (2021)
Yin, J.H., Jian, J.B., Jiang, X.Z., Wu, X.D.: A family of inertial-relaxed DFPM-based algorithms for solving large-scale monotone nonlinear equations with application to sparse signal restoration. J. Comput. Appl. Math. 419, 114674 (2023)
Yuan, G.L., Li, T.T., Hu, W.J.: A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Appl. Numer. Math. 147, 129–141 (2020)
Yuan, G.L., Meng, Z.H., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. J. Optim. Theory Appl. 168, 129–152 (2016)
Zhou, G., Toh, K.C.: Superline convergence of a Newton-type algorithm for monotone equations. J. Optim. Theory Appl. 125, 205–221 (2005)
Zhou, W.J., Li, D.H.: A globally convergent BFGS method for nonlinear monotone equations without any merit functions. Math. Comput. 77, 2231–2240 (2008)
Zhou, W.J., Li, D.H.: Limited memory BFGS method for nonlinear monotone equations. J. Comput. Math. 25, 89–96 (2007)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (72071202), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX23_2651) and the Graduate Innovation Program of China University of Mining and Technology (2023WLJCRCZL139).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Sébastien Le Digabel.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wu, X., Shao, H., Liu, P. et al. An Inertial Spectral CG Projection Method Based on the Memoryless BFGS Update. J Optim Theory Appl 198, 1130–1155 (2023). https://doi.org/10.1007/s10957-023-02265-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-023-02265-6
Keywords
- Nonlinear monotone equations
- Conjugate gradient projection method
- Inertial step
- Global convergence
- Image restoration