Abstract
Regularization of certain linear discrete ill-posed problems, as well as of certain regression problems, can be formulated as large-scale, possibly nonconvex, minimization problems, whose objective function is the sum of the p th power of the ℓp-norm of a fidelity term and the q th power of the ℓq-norm of a regularization term, with 0 < p,q ≤ 2. We describe new restarted iterative solution methods that require less computer storage and execution time than the methods described by Huang et al. (BIT Numer. Math. 57,351–378, 14). The reduction in computer storage and execution time is achieved by periodic restarts of the method. Computed examples illustrate that restarting does not reduce the quality of the computed solutions.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)
Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. SIAM, Philadelphia (1998)
Buccini, A., De la Cruz Cabrera, O. , Donatelli, M., Martinelli, A., Reichel, L.: Large-scale regression with non-convex loss and penalty. Applied Numerical Mathematics 157, 590–601 (2020)
Buccini, A., Donatelli, M., Ramlau, R.: A semiblind regularization algorithm for inverse problems with application to image deblurring. SIAM J. Sci. Comput. 40(1), 452–483 (2018)
Belge, M., Kilmer, M.E., Miller, E.L.: Efficient determination of multiple regularization parameters in a generalized L-curve. Inverse Prob. 18, 1161–1183 (2002)
Fornasier, M., Naumova, V., Pereverzyev, S.V.: Parameter choice strategies for multipenalty regularization. SIAM J. Numer. Anal. 52, 1770–1794 (2014)
Gazzola, S., Reichel, L.: A new framework for multi-parameter regularization. BIT Numer. Math. 56, 919–949 (2016)
Ito, K., Jin, B., Takeuchi, T.: Multi-parameter Tikhonov regularization. Methods Appl. Anal. 18, 31–46 (2011)
Buccini, A., Reichel, L.: Generalized cross validation for ℓp-ℓq minimization. Numer. Algo. 88, 1595–1616 (2021)
Buccini, A., Reichel, L.: An ℓ2-ℓq regularization method for large discrete ill-posed problems. J. Sci. Comput. 78, 1526–1549 (2019)
Lanza, A., Morigi, S., Reichel, L., Sgallari, F.: A generalized Krylov subspace method for ℓp-ℓq minimization. SIAM J. Sci. Comput. 37, 30–50 (2015)
Buccini, A., Pasha, M., Reichel, L.: Modulus-based iterative methods for constrained ℓp-ℓq minimization. Inverse Prob. 36(8), 084001 (2020)
Buccini, A., De la Cruz Cabrera, O., Koukouvinos, C., Mitrouli, M., Reichel, L.: Variable selection in saturated and supersaturated designs via minimization. Communications in Statistics - Simulation and Computation, In Press (2021)
Huang, G., Lanza, A., Morigi, S., Reichel, L., Sgallari, F.: Majorization-minimization generalized Krylov subspace methods for ℓp-ℓq optimization applied to image restoration. BIT Numer. Math. 57, 351–378 (2017)
Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd. edn SIAM (2003)
Chan, R.H., Liang, H.X.: Half-quadratic algorithm for ℓp-ℓq problems with applications to TV-ℓ1 image restoration and compressive sensing. In: Efficient Algorithms for Global Optimization Methods in Computer Vision, Lecture Notes in Computer Science # 8293, pp 78–103. Springer (2014)
Rodríguez, P., Wohlberg, B.: Efficient minimization method for a generalized total variation functional. IEEE Trans. Image Process. 18(2), 322–332 (2009)
Daniel, J.W., Gragg, W.B., Kaufman, L., Stewart, G.W.: Reorthogonalization and stable algorithms for updating the Gram–Schmidt QR factorization. Math. Comput. 30, 772–795 (1976)
Buccini, A., Pragliola, M., Reichel, L., Sgallari, F.: A comparison of parameter choice rules for ℓp-ℓq minimization. Annali dell Università, di Ferrara 68, 441–463 (2022)
Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)
Kindermann, S., Raik, K.: A simplified L-curve as error estimator. Electron. Trans. Numer. Anal. 53, 213–238 (2020)
Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numer Algo 63, 65–87 (2013)
Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra, and Filtering. SIAM (2006)
Donatelli, M., Reichel, L.: Square smoothing regularization matrices with accurate boundary conditions. J. Comput. Appl. Math. 272, 334–349 (2014)
Golub, G.H., Van Loan, C.F. In: 4 (ed.) : Matrix Computations. Johns Hopkins University Press, Baltimore (2013)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)
Natterer, F.: The Mathematics of Computerized Tomorgraphy. SIAM, Philadelphia (2001)
Gazzola, S., Hansen, P.C., Nagy, J.G.: IR Tools: A MATLAB package of iterative regularization methods and large-scale test problems. Numer. Algo. 81(3), 773–811 (2019)
Acknowledgements
The authors would like to thank the referees for comments. A.B. is a member of the GNCS-INdAM group.
Funding
Open access funding provided by Universitá degli Studi di Cagliari within the CRUI-CARE Agreement. A.B. research is partially founded by the GNCS-INdAM project “Regularization Methods and Models for large scale inverse ill-posed problems.” Moreover, his research is partially supported by the Regione Autonoma della Sardegna research project “Algorithms and Models for Imaging Science [AMIS]” (RASSR57257, intervento finanziato con risorse FSC 2014–2020 - Patto per lo Sviluppo della Regione Sardegna) and by Fondazione di Sardegna, Progetto biennale bando 2021, “Computational Methods and Networks in Civil Engineering (COMANCHE).”
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Communicated by: Raymond H. Chan
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Alessandro Buccini and Lothar Reichel contributed equally to this work.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Buccini, A., Reichel, L. Limited memory restarted ℓp-ℓq minimization methods using generalized Krylov subspaces. Adv Comput Math 49, 26 (2023). https://doi.org/10.1007/s10444-023-10020-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10444-023-10020-8