Abstract
We consider a stochastic gradient descent (SGD) algorithm for solving linear inverse problems (e.g., CT image reconstruction) in the Banach space framework of variable exponent Lebesgue spaces \(\ell ^{(p_n)}(\mathbb {R})\). Such non-standard spaces have been recently proved to be the appropriate functional framework to enforce pixel-adaptive regularisation in signal and image processing applications. Compared to its use in Hilbert settings, however, the application of SGD in the Banach setting of \(\ell ^{(p_n)}(\mathbb {R})\) is not straightforward, due, in particular to the lack of a closed-form expression and the non-separability property of the underlying norm. In this manuscript, we show that SGD iterations can effectively be performed using the associated modular function. Numerical validation on both simulated and real CT data show significant improvements in comparison to SGD solutions both in Hilbert and other Banach settings, in particular when non-Gaussian or mixed noise is observed in the data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For \(\mathbf {{SGD}_2}\) \(\mu _0\) is set as \(0.95/\max _{i}\Vert A_i\Vert ^2\) and \(\gamma =0.51\). For \(\mathbf {{SGD}_p}\) and \(\mathbf {{SGD}_{p_n,q_n}}\), we use \(\mu _0=0.015\) with \(\gamma =(p-1)/p+0.01\) and \(\gamma =(p_--1)/p_-+0.01\) respectively.
- 2.
For \(\textbf{SGD}_2\), \(\mu _0=0.95/\max _{i}\Vert A_i\Vert ^2\), \(\gamma =0.51\). For \(\textbf{SGD}_{p_n,q_n}\) we \(\mu _0=0.001\), \(\gamma =0.58\).
References
Alparone, M., Nunziata, F., Estatico, C., Lenti, F., Migliaccio, M.: An adaptive \(l^{p}\) -penalization method to enhance the spatial resolution of microwave radiometer measurements. IEEE Trans. Geosci. Remote Sens. 57(9), 6782–6791 (2019)
Bonino, B., Estatico, C., Lazzaretti, M.: Dual descent regularization algorithms in variable exponent Lebesgue spaces for imaging. Numer. Algorithms 92(6) (2023)
Cioranescu, I.: Geometry of Banach Spaces, Duality Mappings and Nonlinear Problems. Springer, Dordrecht (1990). https://doi.org/10.1007/978-94-009-2121-4
Cruz-Uribe, D.V., Fiorenza, A.: Variable Lebesgue Spaces. Springer Birkhäuser, Basel (2013). https://doi.org/10.1007/978-3-0348-0548-3
Diening, L., Harjulehto, P., Hästö, P., Ruzicka, M.: Lebesgue and Sobolev Spaces with Variable Exponents. Lecture Notes in Math, Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18363-8
Eicke, B.: Iteration methods for convexly constrained ill-posed problems in hilbert space. Numer. Funct. Anal. Optim. 13(5–6), 413–429 (1992)
Engl, H.W., Hanke, A., Neubauer, M.: Regularization of Inverse Problems. Mathematics and Its Applications, Springer, Dordrecht (2000)
Guan, W.-B., Song, W.: The generalized forward-backward splitting method for the minimization of the sum of two functions in banach spaces. Numer. Funct. Anal. Optim. 36(7), 867–886 (2015)
Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient (positron emission tomography application). IEEE Trans. Med. Imaging 12(3), 600–609 (1993)
Jin, Q., Lu, X., Zhang, L.: Stochastic mirror descent method for linear ill-posed problems in Banach spaces (2022). arXiv preprint https://arxiv.org/abs/2207.06584
Jin, Q., Stals, L.: Nonstationary iterated Tikhonov regularization for ill-posed problems in Banach spaces. Inverse Probl. 28(10), 104011 (2012)
Jørgensen, J.S., et al.: Core imaging library - Part I: a versatile python framework for tomographic imaging. Phil. Trans. R. Soc. A 379(2204), 20200192 (2021)
Kereta, Z., Jin, B.: On the convergence of stochastic gradient descent for linear inverse problems in Banach spaces. SIAM J. Imaging Sci. (2023, in press). arXiv preprint https://arxiv.org/abs/2302.05197
Lazzaretti, M., Calatroni, L., Estatico, C.: Modular-proximal gradient algorithms in variable exponent Lebesgue spaces. SIAM J. Sci. Comput. 44(6), A3463–A3489 (2022)
Meaney, A.: X-ray dataset of walnut (2020-11-11), November 2020
Natterer, F.: The Mathematics of Computerized Tomography. Wiley, Hoboken (1986)
Needell, D., Zhao, R., Zouzias, A.: Randomized block Kaczmarz method with projection for solving least squares. Linear Algebra Appl. 484, 322–343 (2015)
Neubauer, A.: Tikhonov-regularization of ill-posed linear operator equations on closed convex sets. J. Approx. Theory 53(3), 304–320 (1988)
Piana, M., Bertero, M.: Projected Landweber method and preconditioning. Inverse Probl. 13(2), 441–463 (1997)
Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22(3), 400–407 (1951)
Schuster, T., Kaltenbacher, B., Hofmann, B., Kazimierski, K.S.: Regularization Methods in Banach Spaces. De Gruyter (2012)
Schöpfer, F., Louis, A.K., Schuster, T.: Nonlinear iterative methods for linear ill-posed problems in Banach spaces. Inverse Probl. 22(1), 311–329 (2006)
Twyman, R., Arridge, S., et al.: An investigation of stochastic variance reduction algorithms for relative difference penalized 3D PET image reconstruction. IEEE Trans. Med. Imaging 42(1), 29–41 (2023)
Acknowledgement
CE and ML acknowledge the support of the Italian INdAM group on scientific calculus GNCS. LC acknowledges the support received by the ANR projects TASKABILE (ANR-22-CE48-0010) and MICROBLIND (ANR-21-CE48-0008), the H2020 RISE projects NoMADS (GA. 777826) and the GdR ISIS project SPLIN. ZK acknowledges support from EPSRC grants EP/T000864/1 and EP/X010740/1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lazzaretti, M., Kereta, Z., Estatico, C., Calatroni, L. (2023). Stochastic Gradient Descent for Linear Inverse Problems in Variable Exponent Lebesgue Spaces. In: Calatroni, L., Donatelli, M., Morigi, S., Prato, M., Santacesaria, M. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2023. Lecture Notes in Computer Science, vol 14009. Springer, Cham. https://doi.org/10.1007/978-3-031-31975-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-031-31975-4_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31974-7
Online ISBN: 978-3-031-31975-4
eBook Packages: Computer ScienceComputer Science (R0)