Skip to main content
SpringerLink
Log in
Book cover

High Dimensional Probability VII pp 173–202Cite as

Birkhäuser

The Expected Norm of a Sum of Independent Random Matrices: An Elementary Approach

  • Conference paper
  • First Online:

Part of the book series: Progress in Probability ((PRPR,volume 71))

Abstract

In contemporary applied and computational mathematics, a frequent challenge is to bound the expectation of the spectral norm of a sum of independent random matrices. This quantity is controlled by the norm of the expected square of the random matrix and the expectation of the maximum squared norm achieved by one of the summands; there is also a weak dependence on the dimension of the random matrix. The purpose of this paper is to give a complete, elementary proof of this important inequality.

Mathematics Subject Classification (2010). 60B20; 60F10, 60G50, 60G42

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. A. Bandeira, R.V. Handel, Sharp nonasymptotic bounds on the norm of random matrices with independent entries (2014). Available at http://arXiv.org/abs/1408.6185

  2. R.F. Bass, K. Gröchenig, Relevant sampling of band-limited functions. Ill. J. Math. 57 (1), 43–58 (2013)

    MathSciNet  MATH  Google Scholar 

  3. R. Bhatia, Matrix Analysis. Graduate Texts in Mathematics, vol. 169 (Springer, New York, 1997)

    Google Scholar 

  4. S. Boucheron, G. Lugosi, P. Massart, Concentration Inequalities (Oxford University Press, Oxford, 2013). A nonasymptotic theory of independence, With a foreword by Michel Ledoux

    Google Scholar 

  5. A. Buchholz, Operator Khintchine inequality in non-commutative probability. Math. Ann. 319 (1), 1–16 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  6. X. Chen, T.M. Christensen, Optimal uniform convergence rates for sieve nonparametric instrumental variables regression (2013). Available at http://arXiv.org/abs/1311.0412

  7. R.Y. Chen, A. Gittens, J.A. Tropp, The masked sample covariance estimator: an analysis using matrix concentration inequalities. Inf. Inference 1 (1), 2–20 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Y. Chen, S. Bhojanapalli, S. Sanghavi, R. Ward, Coherent matrix completion, in Proceedings of the 31st International Conference on Machine Learning, Beijing (2014)

    Google Scholar 

  9. Y. Chen, L. Guibas, Q. Huang, Near-optimal joint object matching via convex relaxation, in Proceedings of the 31st International Conference on Machine Learning, Beijing (2014)

    Google Scholar 

  10. S.-S. Cheung, A.M.-C. So, K. Wang, Linear matrix inequalities with stochastically dependent perturbations and applications to chance-constrained semidefinite optimization. SIAM J. Optim. 22 (4), 1394–1430 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. A. Cohen, M.A. Davenport, D. Leviatan, On the stability and accuracy of least squares approximations. Found. Comput. Math. 13 (5), 819–834 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  12. M.B. Cohen, R. Kyng, G.L. Miller, J.W. Pachocki, R. Peng, A.B. Rao, S.C. Xu, Solving SDD linear systems in nearly mlog1∕2(n) time, in Proceedings of the 46th Annual ACM Symposium on Theory of Computing, STOC ’14 (ACM, New York, 2014), pp. 343–352

    Google Scholar 

  13. P. Constantine, D. Gleich, Computing active subspaces (2014). Available at http://arXiv.org/abs/1408.0545

  14. S. Dirksen, Itô isomorphisms for L p-valued Poisson stochastic integrals. Ann. Probab. 42 (6), 2595–2643 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  15. J. Djolonga, A. Krause, V. Cevher, High-dimensional Gaussian process bandits, in Advances in Neural Information Processing Systems, vol. 26, ed. by C. Burges, L. Bottou, M. Welling, Z. Ghahramani, K. Weinberger (Curran Associates, Inc., Red Hook, 2013), pp. 1025–1033

    Google Scholar 

  16. M. Fornasier, K. Schnass, J. Vybiral, Learning functions of few arbitrary linear parameters in high dimensions. Found. Comput. Math. 12 (2), 229–262 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. N.J.A. Harvey, N. Olver, Pipage rounding, pessimistic estimators and matrix concentration, in Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’14 (SIAM, Philadelphia, 2014), pp. 926–945

    Book  Google Scholar 

  18. A.S. Holevo, Quantum systems, channels, information, in De Gruyter Studies in Mathematical Physics, vol. 16 (De Gruyter, Berlin, 2012). A Mathematical Introduction

    Google Scholar 

  19. R.A. Horn, C.R. Johnson, Matrix Analysis, 2nd edn. (Cambridge University Press, Cambridge, 2013)

    MATH  Google Scholar 

  20. M. Junge, Q. Xu, Noncommutative Burkholder/Rosenthal inequalities. Ann. Probab. 31 (2), 948–995 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. M. Junge, Q. Xu, Noncommutative Burkholder/Rosenthal inequalities. II. Applications. Isr. J. Math. 167, 227–282 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  22. M. Junge, Q. Zeng, Noncommutative Bennett and Rosenthal inequalities. Ann. Probab. 41 (6), 4287–4316 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  23. V. Koltchinskii, Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Mathematics, vol. 2033 (Springer, Heidelberg, 2011). Lectures from the 38th Probability Summer School held in Saint-Flour, 2008, École d’Été de Probabilités de Saint-Flour. [Saint-Flour Probability Summer School]

    Google Scholar 

  24. R. Latała, K. Oleszkiewicz, On the best constant in the Khinchin-Kahane inequality. Stud. Math., 109 (1), 101–104 (1994)

    MathSciNet  MATH  Google Scholar 

  25. M. Ledoux, M. Talagrand, Probability in Banach Spaces. Classics in Mathematics (Springer, Berlin, 2011). Isoperimetry and Processes, reprint of the 1991 edition

    Google Scholar 

  26. D. Lopez-Paz, S. Sra, A. Smola, Z. Ghahramani, B. Schölkopf, Randomized nonlinear component analysis, in Proceedings of the 31st International Conference on Machine Learning, Beijing (2014)

    Google Scholar 

  27. F. Lust-Piquard, Inégalités de Khintchine dans C p  (1 < p < ∞). C. R. Acad. Sci. Paris Sér. I Math. 303 (7), 289–292 (1986)

    MathSciNet  MATH  Google Scholar 

  28. L. Mackey, M.I. Jordan, R.Y. Chen, B. Farrell, J.A. Tropp, Matrix concentration inequalities via the method of exchangeable pairs. Ann. Probab. 42 (3), 906–945 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  29. A. Magen, A. Zouzias, Low rank matrix-valued Chernoff bounds and approximate matrix multiplication, in Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms (SIAM, Philadelphia, PA, 2011), pp. 1422–1436

    Google Scholar 

  30. W.B. March, G. Biros, Far-field compression for fast kernel summation methods in high dimensions (2014). Available at http://arXiv.org/abs/1409.2802

  31. E. Morvant, S. Koço, L. Ralaivola, PAC-Bayesian generalization bound on confusion matrix for multi-class classification, in Proceedings of the 29th International Conference on Machine Learning, Edinburgh (2012)

    Google Scholar 

  32. S.V. Nagaev, I.F. Pinelis, Some inequalities for the distributions of sums of independent random variables. Teor. Verojatnost. i Primenen. 22 (2), 254–263 (1977)

    MathSciNet  MATH  Google Scholar 

  33. R.I. Oliveira, The spectrum of random k-lifts of large graphs (with possibly large k). J. Comb. 1 (3–4), 285–306 (2010)

    MathSciNet  MATH  Google Scholar 

  34. R.I. Oliveira, The lower tail of random quadratic forms, with applications to ordinary least squares and restricted eigenvalue properties (2013). Available at http://arXiv.org/abs/1312.2903

    Google Scholar 

  35. D. Paulin, L. Mackey, J.A. Tropp, Efron–Stein inequalities for random matrices. Ann. Probab. (2015, to appear). Available at http://arXiv.org/abs/1408.3470

  36. G. Pisier, Non-commutative vector valued L p -spaces and completely p-summing maps. Astérisque 247, vi+131 (1998)

    Google Scholar 

  37. B. Rosén, On bounds on the central moments of even order of a sum of independent random variables. Ann. Math. Stat. 41, 1074–1077 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  38. H.P. Rosenthal, On the subspaces of L p ( p > 2) spanned by sequences of independent random variables. Isr. J. Math. 8, 273–303 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  39. M. Rudelson, Random vectors in the isotropic position. J. Funct. Anal. 164 (1), 60–72 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  40. M. Rudelson, R. Vershynin, Sampling from large matrices: an approach through geometric functional analysis. J. ACM 54 (4), Art. 21, 19 pp. (electronic) (2007)

    Google Scholar 

  41. J.A. Tropp, On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25 (1), 1–24 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  42. J.A. Tropp, Improved analysis of the subsampled randomized Hadamard transform. Adv. Adapt. Data Anal. 3 (1–2), 115–126 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  43. J.A. Tropp, The expected norm of a sum of independent random matrices: an elementary approach (2015). Available at http://arXiv.org/abs/1506.04711

    Google Scholar 

  44. J.A. Tropp, An introduction to matrix concentration inequalities. Found. Trends Mach. Learn. 8 (1–2), 1–230 (2015)

    Article  MATH  Google Scholar 

  45. J.A. Tropp, Second-order matrix concentration inequalities (2015). Available at http://arXiv.org/abs/1504.05919

    MATH  Google Scholar 

Download references

Acknowledgements

The author wishes to thank Ryan Lee for a careful reading of the manuscript. The author gratefully acknowledges support from ONR award N00014-11-1002 and the Gordon & Betty Moore Foundation.

Author information

Authors and Affiliations

  1. Annenberg Center, MC 305-16, Pasadena, CA, USA

    Joel A. Tropp

  2. California Institute of Technology, 1200 E. California Blvd., Pasadena, CA, 91125-5000, USA

    Joel A. Tropp

Authors
  1. Joel A. Tropp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joel A. Tropp .

Editor information

Editors and Affiliations

  1. Georgia Institute of Technology, Atlanta, Georgia, USA

    Christian Houdré

  2. University of Delaware, Department of Applied Economics and Stat University of Delaware, Newark, Delaware, USA

    David M. Mason

  3. Centre national de la recherche scientifique, Laboratoire J.A. Dieudonné, Université Côte d’Azur, Nice, France

    Patricia Reynaud-Bouret

  4. Department of Mathematics, University of Tennessee Department of Mathematics, Knoxville, Tennessee, USA

    Jan Rosiński

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Tropp, J.A. (2016). The Expected Norm of a Sum of Independent Random Matrices: An Elementary Approach. In: Houdré, C., Mason, D., Reynaud-Bouret, P., Rosiński, J. (eds) High Dimensional Probability VII. Progress in Probability, vol 71. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-40519-3_8

Download citation

Publish with us

Policies and ethics

Search

Navigation