Adaptive shrinkage of singular values
- 765 Downloads
To recover a low-rank structure from a noisy matrix, truncated singular value decomposition has been extensively used and studied. Recent studies suggested that the signal can be better estimated by shrinking the singular values as well. We pursue this line of research and propose a new estimator offering a continuum of thresholding and shrinking functions. To avoid an unstable and costly cross-validation search, we propose new rules to select two thresholding and shrinking parameters from the data. In particular we propose a generalized Stein unbiased risk estimation criterion that does not require knowledge of the variance of the noise and that is computationally fast. A Monte Carlo simulation reveals that our estimator outperforms the tested methods in terms of mean squared error on both low-rank and general signal matrices across different signal-to-noise ratio regimes. In addition, it accurately estimates the rank of the signal when it is detectable.
KeywordsDenoising Singular values shrinking and thresholding Stein’s unbiased risk estimate Adaptive trace norm Rank estimation
The authors are grateful to the editors and for the helpful comments of the reviewers. J. J. is supported by an AgreenSkills fellowship of the European Union Marie-Curie FP7 COFUND People Programme. S. S. is supported by the Swiss National Science Foundation. This work started while both authors were visiting Stanford University and the authors would like to thank the Department of Statistics for hosting them and for its stimulating seminars.
- Caussinus, H.: Models and uses of principal component analysis (with discussion). In: de Leeuw, J. (ed.) Multidimensional Data Analysis. DSWO, Leiden, The Netherlands (1986)Google Scholar
- Chatterjee, S.: Matrix estimation by universal singular value thresholding. arXiv:1212.1247 (2013)
- de Leeuw, J.D., Mooijaart, A., Leeden, M.: Fixed Factor Score Models with Linear Restrictions. University of Leiden, Leiden, The Netherlands (1985)Google Scholar
- Donoho, D.L., Gavish, M.: Minimax risk of matrix denoising by singular value thresholding. Ann. Statist. 42(6), 2413–2440 (2014b)Google Scholar
- Gaiffas, S., Lecue, G.: Weighted algorithms for compressed sensing and matrix completion. arXiv:1107.1638 (2011)
- Gavish, M., Donoho, D.L.: Optimal shrinkage of singular values. arXiv:1405.7511v2 (2014)
- Hoff, P.D.: Equivariant and scale-free tucker decomposition models. arXiv:1312.6397 (2013)
- Rao, N.R.: Optshrink-low-rank signal matrix denoising via optimal, data-driven singular value shrinkage. arXiv:1306.6042 (2013)
- Sardy, S.: Blockwise and coordinatewise thresholding to combine tests of different natures in modern anova. arXiv:1302.6073 (2013)
- Verbanck, M., Josse, J., Husson, F.: Regularized PCA to denoise and visualize data. Statist. Comput. 25(2), 471–486 (2015)Google Scholar
- Zanella, A., Chiani, M., Win, M.Z.: On the marginal distribution of the eigenvalues of Wishart matrices. IEEE Trans. Commun. 57(4), 1050–1060 (2009)Google Scholar