Skip to main content
SpringerLink
Log in

Rapid evaluation of the spectral signal detection threshold and Stieltjes transform

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

Accurate detection of signal components is a frequently-encountered challenge in statistical applications with a low signal-to-noise ratio. This problem is particularly challenging in settings with heteroscedastic noise. In certain signal-plus-noise models of data, such as the classical spiked covariance model and its variants, there are closed formulas for the spectral signal detection threshold (the largest sample eigenvalue attributable solely to noise) for isotropic noise in the limit of infinitely large data matrices. However, more general noise models currently lack provably fast and accurate methods for numerically evaluating the threshold. In this work, we introduce a rapid algorithm for evaluating the spectral signal detection threshold in the limit of infinitely large data matrices. We consider noise matrices with a separable variance profile (whose variance matrix is rank 1), as these arise often in applications. The solution is based on nested applications of Newton’s method. We also devise a new algorithm for evaluating the Stieltjes transform of the spectral distribution at real values exceeding the threshold. The Stieltjes transform on this domain is known to be a key quantity in parameter estimation for spectral denoising methods. The correctness of both algorithms is proven from a detailed analysis of the master equations characterizing the Stieltjes transform, and their performance is demonstrated in numerical experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bai, Z., Silverstein, J.W: Spectral Analysis of Large Dimensional Random Matrices, Springer Series in Statistics. Springer, Berlin (2009)

    Google Scholar 

  2. Bao, Z., Ding, X., Wang, K.: Singular vector and singular subspace distribution for the matrix denoising model. Ann. Stat. 49(1), 370–392 (2021)

    Article  MathSciNet  Google Scholar 

  3. Benaych-Georges, F., Nadakuditi, R.R.: The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivar. Anal. 111, 120–135 (2012)

    Article  MathSciNet  Google Scholar 

  4. Bhamre, T., Zhang, T., Singer, A.: Denoising and covariance estimation of single particle cryo-EM images. J. Struct. Biol. 195(1), 72–81 (2016)

    Article  Google Scholar 

  5. Biglieri, E., Calderbank, R., Constantinides, A., Goldsmith, A., Paulraj, A., Vincent Poor, H.: MIMO Wireless Communications. Cambridge University Press, Cambridge (2007)

    Book  Google Scholar 

  6. Buja, A., Eyuboglu, N.: Remarks on parallel analysis. Multivar. Behav. Res. 27(4), 509–540 (1992)

    Article  Google Scholar 

  7. Cordero-Grande, L.: MIXANDMIX: Numerical techniques for the computation of empirical spectral distributions of population mixtures. Comput. Stat. Data Ana. 141, 1–11 (2020)

    Article  MathSciNet  Google Scholar 

  8. Couillet, R., Debbah, M.: Random matrix methods for wireless communications. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  9. Couillet, R., Hachem, W.: Analysis of the limiting spectral measure of large random matrices of the separable covariance type. Random Matrices: Theory and Applications, 3(4) (2014)

  10. Dahlquist, G., Björck, Å: Numerical Methods. Prentice Hall Inc, Hoboken (1974)

    MATH  Google Scholar 

  11. Ding, X., Yang, F.: Spiked separable covariance matrices and principal components. Ann. Stat. 49(2), 1113–1138 (2021)

    Article  MathSciNet  Google Scholar 

  12. Dobriban, E.: Efficient computation of limit spectra of sample covariance matrices. Random Matrices: Theory and Applications, 4(4) (2015)

  13. Dobriban, E.: Permutation methods for factor analysis and PCA. Ann. Stat. 48(5), 2824–2847 (2020)

    Article  MathSciNet  Google Scholar 

  14. Dobriban, E., Leeb, W., Singer, A.: Optimal prediction in the linearly transformed spiked model. Ann. Stat. 48(1), 491–513 (2020)

    Article  MathSciNet  Google Scholar 

  15. Dobriban, E., Owen, A.B.: Deterministic parallel analysis: An Improved Method for Selecting Factors and Principal Components. Journal of the Royal Statistical Society Series B (Statistical Methodology) (2018)

  16. Donoho, D.L, Gavish, M., Johnstone, I.M: Optimal shrinkage of eigenvalues in the spiked covariance model. Annals of Statistics, 46(6) (2018)

  17. El Karoui, N.: Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Stat. 36(6), 2757–2790 (2008)

    MathSciNet  MATH  Google Scholar 

  18. Gavish, M., Donoho, D.L.: Optimal shrinkage of singular values. IEEE Trans. Inf. Theory 63(4), 2137–2152 (2017)

    Article  MathSciNet  Google Scholar 

  19. Geman, S.: A limit theorem for the norm of random matrices. Ann. Probab. 8(2), 252–261 (1980)

    Article  MathSciNet  Google Scholar 

  20. Hong, D., Balzano, L., Fessler, J.A.: Towards a Theoretical Analysis of PCA for Heteroscedastic Data. In: 54Th Annual Allerton Conference on Communication, Control, and Computing, pp 496–503. IEEE (2016)

  21. Hong, D., Balzano, L., Fessler, J.A.: Asymptotic performance of PCA for high-dimensional heteroscedastic data. J. Multivar. Anal. 167, 435–452 (2018)

    Article  MathSciNet  Google Scholar 

  22. Hong, D., Balzano, L., Fessler, J.A: Optimally weighted PCA for high-dimensional heteroscedastic data. arXiv:1810.12862 (2018)

  23. Horn, J.L.: A rationale and test for the number of factors in factor analysis. Psychometrika 30(2), 179–185 (1965)

    Article  Google Scholar 

  24. Iain, M.: Johnstone, on the distribution of the largest eigenvalue in principal components analysis. Ann. Stat. 29(2), 295–327 (2001)

    MATH  Google Scholar 

  25. Kritchman, S., Nadler, B.: Determining the number of components in a factor model from limited noisy data. Chemom. Intell. Lab. Syst. 94(1), 19–32 (2008)

    Article  Google Scholar 

  26. Kritchman, S., Nadler, B.: Non-parametric detection of the number of signals: Hypothesis testing and random matrix theory. IEEE Trans. Signal Process. 57(10), 3930–3941 (2009)

    Article  MathSciNet  Google Scholar 

  27. Ledoit, O., Wolf, M.: Spectrum estimation: A unified framework for covariance matrix estimation and PCA in large dimensions. J. Multivar. Anal. 139, 360–384 (2015)

    Article  MathSciNet  Google Scholar 

  28. Ledoit, O., Wolf, M.: Numerical implementation of the quEST function. Comput. Stat. Data Anal. 115, 199–223 (2017)

    Article  MathSciNet  Google Scholar 

  29. Leeb, W.: Optimal singular value shrinkage for operator norm loss. arXiv:2005.11807 (2020)

  30. Leeb, W.: Matrix Denoising for Weighted Loss Functions and Heterogeneous Signals. SIAM Journal on Mathematics of Data Science Accepted (2021)

  31. Leeb, W., Romanov, E.: Optimal spectral shrinkage and PCA with heteroscedastic noise. IEEE Trans. Inf. Theory 67(5), 3009–3037 (2021)

    Article  Google Scholar 

  32. Marchenko, V.A, Pastur, L.A: Distribution of eigenvalues for some sets of random matrices. Mat. Sb. 114(4), 507–536 (1967)

    MATH  Google Scholar 

  33. Nadakuditi, R.R.: Optshrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage. IEEE Trans. Inf. Theory 60(5), 3002–3018 (2014)

    Article  MathSciNet  Google Scholar 

  34. Nesterov, Y.: Lectures on Convex Optimization, volume 137 of Springer Optimization and Its Applications, 2nd edn. Springer, Berlin (2018)

    Google Scholar 

  35. Onatski, A.: Determining the number of factors from empirical distribution of eigenvalues. Rev. Econ. Stat. 92(4), 1004–1016 (2010)

    Article  Google Scholar 

  36. Onatski, A., Moreira, M.J, Hallin, M.: Asymptotic power of sphericity tests for high-dimensional data. Ann. Stat. 41(3), 1204–1231 (2013)

    Article  MathSciNet  Google Scholar 

  37. Passemier, D., Yao, Jian-Feng: On determining the number of spikes in a high-dimensional spiked population model. Random Matrices: Theor. Appl. 1(01), 1150002 (2012)

    Article  MathSciNet  Google Scholar 

  38. Paul, D.: Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Stat. Sin. 17(4), 1617–1642 (2007)

    MathSciNet  MATH  Google Scholar 

  39. Paul, D., Silverstein, J.W.: No eigenvalues outside the support of the limiting empirical spectral distribution of a separable covariance matrix. J. Multivar. Anal. 100, 37–57 (2009)

    Article  MathSciNet  Google Scholar 

  40. Raj Rao, N., Edelman, A.: The polynomial method for random matrices. Found. Comput. Math. 8, 649–702 (2008)

    Article  MathSciNet  Google Scholar 

  41. Shabalin, A.A., Nobel, A.B.: Reconstruction of a low-rank matrix in the presence of Gaussian noise. J. Multivar. Anal. 118, 67–76 (2013)

    Article  MathSciNet  Google Scholar 

  42. Silverstein, J.W.: Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices. J. Multivar. Anal. 55, 331–339 (1995)

    Article  MathSciNet  Google Scholar 

  43. Silverstein, J.W., Bai, Z.: On the empirical distribution of eigenvalues of a class of large dimensional random matrices. J. Multivar. Anal. 54, 175–192 (1995)

    Article  MathSciNet  Google Scholar 

  44. Tandra, R., Sahai, A.: SNR Walls for signal detection. IEEE J. Select. Top. Signal Process. 2(1), 4–17 (2008)

    Article  Google Scholar 

  45. Tao, T.: Topics in random matrix theory american mathematical society (2012)

  46. Yücek, T., Arslan, H.: A survey of spectrum sensing algorithms for cognitive radio applications. IEEE Commun. Surv. Tutor. 11(1), 116–130 (2009)

    Article  Google Scholar 

  47. Zeng, Y., Liang, Y.-C.: Covariance based signal detections for cognitive radio. In: 2nd IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, pp 202–207. IEEE (2007)

  48. Zeng, Y., Liang, Y.-C.: Eigenvalue-based spectrum sensing algorithms for cognitive radio. IEEE Trans. Commun. 57(6), 1784–1793 (2009)

    Article  Google Scholar 

  49. Anru Zhang, T., Cai, T., Wu, Y., Heteroskedastic PCA: Algorithm, optimality, and applications. arXiv:1810.08316 (2018)

Download references

Acknowledgements

I thank Edgar Dobriban for helpful discussions and for pointing out the method from [15]. I also thank the reviewers for their helpful comments on the manuscript.

Funding

This work was supported by NSF BIGDATA award IIS 1837992 and BSF award 2018230.

Author information

Authors and Affiliations

  1. School of Mathematics, University of Minnesota, 206 Church Street SE, Minneapolis, MN, 55455, USA

    William Leeb

Authors
  1. William Leeb
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William Leeb.

Additional information

Communicated by: Holger Rauhut

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Leeb, W. Rapid evaluation of the spectral signal detection threshold and Stieltjes transform. Adv Comput Math 47, 60 (2021). https://doi.org/10.1007/s10444-021-09890-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10444-021-09890-7

Keywords

Mathematics Subject Classification (2010)

Search

Navigation