Skip to main content
SpringerLink
Log in

On the convergence of the extremal eigenvalues of empirical covariance matrices with dependence

  • Published:
Probability Theory and Related Fields Aims and scope Submit manuscript

Abstract

Consider a sample of a centered random vector with unit covariance matrix. We show that under certain regularity assumptions, and up to a natural scaling, the smallest and the largest eigenvalues of the empirical covariance matrix converge, when the dimension and the sample size both tend to infinity, to the left and right edges of the Marchenko–Pastur distribution. The assumptions are related to tails of norms of orthogonal projections. They cover isotropic log-concave random vectors as well as random vectors with i.i.d. coordinates with almost optimal moment conditions. The method is a refinement of the rank one update approach used by Srivastava and Vershynin to produce non-asymptotic quantitative estimates. In other words we provide a new proof of the Bai and Yin theorem using basic tools from probability theory and linear algebra, together with a new extension of this theorem to random matrices with dependent entries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adamczak, R.: On the Marchenko–Pastur and circular laws for some classes of random matrices with dependent entries. Electron. J. Probab. 16(37), 1068–1095 (2011)

    MathSciNet  MATH  Google Scholar 

  2. Adamczak, R.: Some remarks on the Dozier–Silverstein theorem for random matrices with dependent entries. Random Matrices Theory Appl. 2(2), 1250017 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Adamczak, R., Chafaï, D.: Circular law for random matrices with unconditional log-concave distribution. Commun. Contemp. Math. 17(4), 1550020 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  4. Adamczak, R., Latała, R., Litvak, A.E., Oleszkiewicz, K., Pajor, A., Tomczak-Jaegermann, N.: A short proof of Paouris’ inequality. Can. Math. Bull. 57(1), 3–8 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  5. Adamczak, R., Litvak, A.E., Pajor, A., Tomczak-Jaegermann, N.: Quantitative estimates of the convergence of the empirical covariance matrix in log-concave ensembles. J. Am. Math. Soc. 23(2), 535–561 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  6. Adamczak, R., Litvak, A.E., Pajor, A., Tomczak-Jaegermann, N.: Sharp bounds on the rate of convergence of the empirical covariance matrix. C. R. Math. Acad. Sci. Paris 349(3–4), 195–200 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  7. Anttila, M., Ball, K., Perissinaki, I.: The central limit problem for convex bodies. Trans. Am. Math. Soc. 355(12), 4723–4735 (2003). (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bai, Z.-D., Silverstein, J.W.: No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26(1), 316–345 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bai, Z.-D., Silverstein, J.W.: Spectral Analysis of Large Dimensional Random Matrices. Springer Series in Statistics, 2nd edn. Springer, New York (2010)

    MATH  Google Scholar 

  10. Bai, Z.-D., Silverstein, J.W.: No eigenvalues outside the support of the limiting spectral distribution of information-plus-noise type matrices. Random Matrices Theory Appl. 1(1), 1150004, 44 (2012)

  11. Bai, Z.-D., Yin, Y.-Q.: Necessary and sufficient conditions for almost sure convergence of the largest eigenvalue of a Wigner matrix. Ann. Probab. 16(4), 1729–1741 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bai, Z.-D., Yin, Y.-Q.: Limit of the smallest eigenvalue of a large-dimensional sample covariance matrix. Ann. Probab. 21(3), 1275–1294 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  13. Bai, Z.-D., Zhou, W.: Large sample covariance matrices without independence structures in columns. Stat. Sin. 18(2), 425–442 (2008)

    MathSciNet  MATH  Google Scholar 

  14. Batson, J., Spielman, D.A., Srivastava, N.: Twice-Ramanujan sparsifiers. SIAM J. Comput. 41(6), 1704–1721 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  15. Batson, J.D., Spielman, D.A., Srivastava, N.: Twice-Ramanujan sparsifiers. In: STOC’09—Proceedings of the 2009 ACM International Symposium on Theory of Computing, pp. 255–262. ACM, New York (2009)

  16. Benaych-Georges, F., Nadakuditi, R.R.: The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivar. Anal. 111, 120–135 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Bordenave, C.: Notes on random matrices (2014) (Available on the author webpage)

  18. Borell, C.: Convex measures on locally convex spaces. Ark. Mat. 12, 239–252 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  19. Borodin, A., Forrester, P.J.: Increasing subsequences and the hard-to-soft edge transition in matrix ensembles. J. Phys. A 36(12), 2963–2981 (2003). Random matrix theory

    Article  MathSciNet  MATH  Google Scholar 

  20. Brazitikos, S., Giannopoulos, A., Valettas, P., Vritsiou, B.-H.: Geometry of Isotropic Convex Bodies, Mathematical Surveys and Monographs, vol. 196. American Mathematical Society, Providence (2014)

    MATH  Google Scholar 

  21. Capitaine, M., Donati-Martin, C.: Strong asymptotic freeness for Wigner and Wishart matrices. Indiana Univ. Math. J. 56(2), 767–803 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. de la Peña, V.H., Giné, E.: Decoupling. From Dependence to Independence, Randomly Stopped Processes. \(U\)-Statistics and Processes Martingales and Beyond. Probability and its Applications (New York). Springer, New York (1999)

    Google Scholar 

  23. Feldheim, O.N., Sodin, S.: A universality result for the smallest eigenvalues of certain sample covariance matrices. Geom. Funct. Anal. 20(1), 88–123 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  24. Feller, W.: An Introduction to Probability Theory and its Applications, vol. II, 2nd edn. Wiley, New York, London, Sydney (1971)

    MATH  Google Scholar 

  25. Fleury, B.: Concentration in a thin Euclidean shell for log-concave measures. J. Funct. Anal. 259(4), 832–841 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Guédon, O.: Concentration phenomena in high dimensional geometry. In: Journées MAS 2012, ESAIM Proceedings, vol. 44, pp. 47–60. EDP Sciences, Les Ulis (2014)

  27. Guédon, O., Milman, E.: Interpolating thin-shell and sharp large-deviation estimates for isotropic log-concave measures. Geom. Funct. Anal. 21(5), 1043–1068 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Haagerup, U., Thorbjørnsen, S.: A new application of random matrices: \({\rm Ext}(C^*_{\rm red}(F_2))\) is not a group. Ann. of Math. (2) 162(2), 711–775 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  29. Johansson, K.: Shape fluctuations and random matrices. Commun. Math. Phys. 209(2), 437–476 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  30. Johnstone, I.M.: On the distribution of the largest eigenvalue in principal components analysis. Ann. Stat. 29(2), 295–327 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  31. Klartag, B.: Power-law estimates for the central limit theorem for convex sets. J. Funct. Anal. 245(1), 284–310 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  32. Koltchinskii, V., Mendelson, S.: Bounding the smallest singular value of a random matrix without concentration, preprint arXiv:1312.3580 to appear in International Mathematics Research Notices (2015)

  33. Lee, J.O., Yin, J.: A necessary and sufficient condition for edge universality of Wigner matrices. Duke Math. J. 163(1), 117–173 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  34. Litvak, A.E., Pajor, A., Rudelson, M., Tomczak-Jaegermann, N.: Smallest singular value of random matrices and geometry of random polytopes. Adv. Math. 195(2), 491–523 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  35. Marchenko, V.O., Pastur, L.A.: Distribution of eigenvalues in certain sets of random matrices. Mat. Sb. (N.S.) 72(114), 507–536 (1967)

    MathSciNet  MATH  Google Scholar 

  36. Pajor, A., Pastur, L.A.: On the limiting empirical measure of eigenvalues of the sum of rank one matrices with log-concave distribution. Stud. Math. 195(1), 11–29 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  37. Paouris, G.: Concentration of mass on convex bodies. Geom. Funct. Anal. 16(5), 1021–1049 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  38. Pastur, L.A., Shcherbina, M.: Eigenvalue Distribution of Large Random Matrices, Mathematical Surveys and Monographs (2011). ISBN:978-0-8218-5285-9, http://www.ams.org/mathscinet-getitem?mr=2808038

  39. Péché, S.: Universality results for the largest eigenvalues of some sample covariance matrix ensembles. Probab. Theory Relat. Fields 143(3–4), 481–516 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  40. Petrov, V.V.: Limit Theorems of Probability Theory, Oxford Studies in Probability. Sequences of Independent Random Variables, Oxford Science Publications, vol. 4. The Clarendon Press, Oxford University Press, New York (1995)

    Google Scholar 

  41. Pillai, N.S., Yin, J.: Universality of covariance matrices. Ann. Appl. Probab. 24(3), 935–1001 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Richard, K., Guionnet, A.: Central limit theorem and convergence of the support for Wishart matrices with correlated entries, preprint arXiv:1401.7367 (2014)

  43. Rudelson, M., Vershynin, R.: The Littlewood–Offord problem and invertibility of random matrices. Adv. Math. 218(2), 600–633 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  44. Rudelson, M., Vershynin, R.: Non-asymptotic theory of random matrices: extreme singular values. In: Proceedings of the International Congress of Mathematicians, vol. III,, pp. 1576–1602. Hindustan Book Agency, New Delhi (2010)

  45. Schultz, H.: Non-commutative polynomials of independent Gaussian random matrices. The real and symplectic cases. Probab. Theory Relat. Fields 131(2), 261–309 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  46. Shiryaev, A.N.: Probability. 1. Graduate Texts in Mathematics, 3rd ed., vol. 95. Springer, New York (2016). Translated from the fourth (2007) Russian edition by R. P. Boas and D. M. Chibisov

  47. Soshnikov, A.: A note on universality of the distribution of the largest eigenvalues in certain sample covariance matrices. J. Stat. Phys. 108(5–6), 1033–1056 (2002). Dedicated to David Ruelle and Yasha Sinai on the occasion of their 65th birthdays

    Article  MathSciNet  MATH  Google Scholar 

  48. Srivastava, N., Vershynin, R.: Covariance estimation for distributions with \(2+\varepsilon \) moments. Ann. Probab. 41(5), 3081–3111 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  49. Tikhomirov, K.: The limit of the smallest singular value of random matrices with i.i.d. entries. Adv. Math. 284, 1–20 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  50. Vershynin, R.: A simple decoupling inequality in probability theory, unpublished, available on the Internet (2011)

  51. Yaskov, P.: Lower bounds on the smallest eigenvalue of a sample covariance matrix. Electron. Commun. Probab. 19(83), 1–10 (2014)

    MathSciNet  MATH  Google Scholar 

  52. Yaskov, P.: Sharp lower bounds on the least singular value of a random matrix without the fourth moment condition. Electron. Commun. Probab. 20(44), 1–9 (2015)

    MathSciNet  MATH  Google Scholar 

  53. Yin, Y.-Q., Bai, Z.-D., Krishnaiah, P.R.: On the limit of the largest eigenvalue of the large-dimensional sample covariance matrix. Probab. Theory Relat. Fields 78(4), 509–521 (1988)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

D.C. would like to warmly thank Radosław Adamczak, Charles Bordenave, and Alain Pajor for discussions on this topic in Warsaw, Toulouse, and Paris. K.T. would like to thank Nicole Tomczak-Jaegermann for her support, Pierre Youssef for introducing him to the result of Batson–Spielman–Srivastava, and especially thank Alain Pajor for the invitation to visit Université Paris-Est Marne-la-Vallée in November–December, 2014. A significant part of the present work was done during that period. Both authors are grateful to the anonymous referees for valuable suggestions.

Author information

Authors and Affiliations

  1. CEREMADE, PSL, IUF, Université Paris-Dauphine, Paris, France

    Djalil Chafaï

  2. Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, Canada

    Konstantin Tikhomirov

Authors
  1. Djalil Chafaï
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantin Tikhomirov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Djalil Chafaï.

Additional information

Dedicated to Alain Pajor & Nicole Tomczak-Jaegermann.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chafaï, D., Tikhomirov, K. On the convergence of the extremal eigenvalues of empirical covariance matrices with dependence. Probab. Theory Relat. Fields 170, 847–889 (2018). https://doi.org/10.1007/s00440-017-0778-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00440-017-0778-9

Keywords

Mathematics Subject Classification

Search

Navigation