Abstract
The sparsity-based signal reconstruction is typically formulated as an \(\ell _1\) regularization (i.e., lasso problem in statistics), and the reconstruction accuracy strongly depends on the regularization parameter. In this paper, we develop two data-driven optimization schemes, based on minimization of Stein’s unbiased risk estimate (SURE). First, a recursive evaluation of SURE is proposed to estimate the mean squared error during the reconstruction iterations, which enables us to optimize the regularization parameter. Second, for fast optimization, we perform the alternating updates between regularization parameter and solution. In particular, we perform the convergence analysis of the recursive SURE and the accompanied Monte Carlo simulation, based on the Jacobian recursion, support identification and proximal point framework. Numerical experiments show that the proposed methods lead to highly accurate estimate of regularization parameter and nearly optimal reconstruction.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The last unknown term \(\Vert \mathbf {x}_0\Vert ^2/N\) in (4) is a constant that is irrelevant to the optimization of the solution \(\widehat{\mathbf {x}}_\lambda \).
In numerical tests, we use the true value of \(\Vert \mathbf {x}_0\Vert ^2\) to compute the SURE, noting that adding any constant is irrelevant to the minimization of SURE.
For a \(256 \times 256\) image, the size of \(\mathbf {A}\) and Jacobian matrices is \(65536 \times 65536\). If the 3 levels of Haar redundant transform are used, the size of \(\mathbf {D}\) is \(655360 \times 65536\). These matrices are difficult to store and compute in personal computer.
References
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics, Springer (2011)
Bredies, K., Sun, H.: A proximal point analysis of the preconditioned alternating direction method of multipliers. J. Optim. Theory Appl. 173, 878–907 (2017)
Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging. Vis. 40(1), 120–145 (2011)
Eldar, Y.: Generalized SURE for exponential families: applications to regularization. IEEE Trans. Signal Process. 57(2), 471–481 (2009)
Fergus, R., Singh, B., Hertzmann, A., Roweis, S., Freeman, W.: Removing camera shake from a single photograph. ACM Trans. Graph. 25, 787–794 (2006)
Giryes, R., Elad, M., Eldar, Y.: The projected GSURE for automatic parameter tuning in iterative shrinkage methods. Appl. Comput. Harmon. Anal. 30(3), 407–422 (2011)
Golub, G., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2), 215–223 (1979)
Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34(4), 561–580 (1992)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, vol. 2. Springer, Berlin (2009)
He, B., Yuan, X.: On the \(\cal{O}(1/n)\) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)
Kim, B.C., et al.: Dependence modeling for large-scale project cost and time risk assessment: additive risk factor approaches. IEEE Trans. Eng. Manag. (2021, to appear)
Li, J.: A blur-SURE-based approach to kernel estimation for motion deblurring. Pattern Recognit. Image Anal. 29(1), 56–67 (2019)
Loris, I., Verhoeven, C.: On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty. Inverse Prob. 27, 125007 (2011)
Morozov, V.: Methods for Solving Incorrectly Posed Problems. Springer, New York (1984)
Stein, C.M.: Estimation of the mean of a multivariate normal distribution. Ann. Stat. 9(6), 1135–1151 (1981)
Vaiter, S., Peyré, G., Dossal, C., Fadili, J.: Robust sparse analysis regularization. IEEE Trans. Inf. Theory 59(4), 2001–2016 (2013)
Vonesch, C., Ramani, S., Unser, M.: Recursive risk estimation for non-linear image deconvolution with a wavelet-domain sparsity constraint. In: Proceedings of the 15th IEEE Int. Conf. Image Process., San Diego CA, USA, October 12–15, 2008, pp. 665–668
Xue, F., Blu, T.: A novel SURE-based criterion for parametric PSF estimation. IEEE Trans. Image Process. 24(2), 595–607 (2015)
Xue, F., Liu, J., Ai, X.: Parametric PSF estimation based on predicted-SURE with \(\ell _1\)-penalized sparse deconvolution. Signal Image Video Process. 13(4), 635–642 (2019)
Xue, F., Liu, J., Ai, X.: Recursive SURE for image recovery via total variation minimization. Signal Image Video Process. 13(4), 795–803 (2019)
Xue, F., Luisier, F., Blu, T.: Multi-Wiener SURE-LET deconvolution. IEEE Trans. Image Process. 22(5), 1954–1968 (2013)
Xue, F., Yagola, A.G.: Analysis of point-target detection performance based on ATF and TSF. Infrared Phys. Technol. 52(5), 166–173 (2009)
Xue, F., Yagola, A.G., Liu, J., Meng, G.: Recursive SURE for iterative reweighted least square algorithms. Inverse Probl. Sci. Eng. 24(4), 625–646 (2016)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was financially supported by Renmin University of China (Grant Number 2017030130).
Rights and permissions
About this article
Cite this article
Li, J. A recursive unbiased risk estimate for the analysis-based \(\ell _1\) minimization. SIViP 15, 1917–1925 (2021). https://doi.org/10.1007/s11760-021-01945-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-021-01945-y