Skip to main content
SpringerLink
Log in

Concentration of Measure Without Independence: A Unified Approach Via the Martingale Method

  • Conference paper
  • First Online:
Convexity and Concentration

Part of the book series: The IMA Volumes in Mathematics and its Applications ((IMA,volume 161))

Abstract

The concentration of measure phenomenon may be summarized as follows: a function of many weakly dependent random variables that is not too sensitive to any of its individual arguments will tend to take values very close to its expectation. This phenomenon is most completely understood when the arguments are mutually independent random variables, and there exist several powerful complementary methods for proving concentration inequalities, such as the martingale method, the entropy method, and the method of transportation inequalities. The setting of dependent arguments is much less well understood. This chapter focuses on the martingale method for deriving concentration inequalities without independence assumptions. In particular, we use the machinery of so-called Wasserstein matrices to show that the Azuma-Hoeffding concentration inequality for martingales with almost surely bounded differences, when applied in a sufficiently abstract setting, is powerful enough to recover and sharpen several known concentration results for nonproduct measures. Wasserstein matrices provide a natural formalism for capturing the interplay between the metric and the probabilistic structures, which is fundamental to the concentration phenomenon.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 119.00
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Though commonly referred to as Pinsker’s inequality, (11) as given here (with the optimal constant \(\frac{1} {2}\)) was proven by Csiszár [7] and Kullback [18] in 1967.

References

  1. Sergey G. Bobkov and Friedrich Götze. Exponential integrability and transportation cost related to logarithmic Sobolev inequalities. J. Funct. Anal., 163:1–28, 1999.

    Article  MathSciNet  MATH  Google Scholar 

  2. Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013.

    Book  MATH  Google Scholar 

  3. Pietro Caputo, Georg Menz, and Prasad Tetali. Approximate tensorization of entropy at high temperature. Annales de la Faculté des Sciences de Toulouse Sér. 6, 24(4):691–716, 2015.

    Google Scholar 

  4. Sourav Chatterjee. Concentration inequalities with exchangeable pairs. PhD thesis, Stanford University, 2005.

    Google Scholar 

  5. Sourav Chatterjee. Stein’s method for concentration inequalities. Probability Theory and Related Fields, 138:305–321, 2007.

    Article  MathSciNet  MATH  Google Scholar 

  6. Jean-René Chazottes, Pierre Collet, Christof Külske, and Frank Redig. Concentration inequalities for random fields via coupling. Probability Theory and Related Fields, 137 (1-2):201–225, 2007.

    Article  MathSciNet  MATH  Google Scholar 

  7. Imre Csiszár. Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar., 2:299–318, 1967.

    MathSciNet  MATH  Google Scholar 

  8. H. Djellout, A. Guillin, and L. Wu. Transportation cost-information inequalities and applications to random dynamical systems and diffusions. Ann. Probab., 32(3B):2702–2732, 2004.

    Article  MathSciNet  MATH  Google Scholar 

  9. Roland L. Dobrushin. Prescribing a system of random variables by conditional distributions. Theory of Probability and Its Applications, 15(3):458–486, 1970. Translated from Russian.

    Article  MATH  Google Scholar 

  10. Hans Föllmer. Tail structure of Markov chains on infinite product spaces. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 50:273–285, 1979.

    Article  MathSciNet  MATH  Google Scholar 

  11. Hans Föllmer. A covariance estimate for Gibbs measures. Journal of Functional Analsysi, 46:387–395, 1982.

    Article  MathSciNet  MATH  Google Scholar 

  12. Hans-Otto Georgii. Gibbs Measures and Phase Transitions, volume 9 of de Gruyter Studies in Mathematics. Walter de Gruyter & Co., 2nd edition, 2011.

    Google Scholar 

  13. Sheldon Goldstein. Maximal coupling. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 46(193–204), 1979.

    Google Scholar 

  14. Aryeh Kontorovich. Obtaining measure concentration from Markov contraction. Markov Processes and Related Fields, 4:613–638, 2012.

    MathSciNet  MATH  Google Scholar 

  15. Aryeh Kontorovich and Roi Weiss. Uniform Chernoff and Dvoretzky-Kiefer-Wolfowitz-type inequalities for Markov chains and related processes. Journal of Applied Probability, 51:1–14, 2014.

    Article  MathSciNet  MATH  Google Scholar 

  16. Aryeh (Leonid) Kontorovich. Measure Concentration of Strongly Mixing Processes with Applications. PhD thesis, Carnegie Mellon University, 2007.

    Google Scholar 

  17. Leonid (Aryeh) Kontorovich and Kavita Ramanan. Concentration Inequalities for Dependent Random Variables via the Martingale Method. Ann. Probab., 36(6):2126–2158, 2008.

    Google Scholar 

  18. Solomon Kullback. A lower bound for discrimination information in terms of variation. IEEE Trans. Inform. Theory, 13:126–127, 1967. Correction, volume 16, p. 652, 1970.

    Google Scholar 

  19. Christof Külske. Concentration inequalities for functions of Gibbs fields with application to diffraction and random Gibbs measures. Commun. Math. Phys., 239:29–51, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  20. Paul Lévy. Problèmes concrets d’analyse fonctionnelle. Gauthier-Villars, Paris, 1951. 2d ed.

    Google Scholar 

  21. Andrei A. Markov. Extension of the law of large numbers to dependent quantities. Izvestiia Fiz.-Matem. Obsch. Kazan Univ., 15:135–156, 1906.

    Google Scholar 

  22. Katalin Marton. Bounding \(\bar{d}\)-distance by informational divergence: a method to prove measure concentration. Ann. Probab., 24(2):857–866, 1996.

    Article  MathSciNet  MATH  Google Scholar 

  23. Katalin Marton. Measure concentration for a class of random processes. Probability Theory and Related Fields, 110(3):427–439, 1998.

    Article  MathSciNet  MATH  Google Scholar 

  24. Katalin Marton. Measure concentration and strong mixing. Studia Scientiarum Mathematicarum Hungarica, 19(1-2):95–113, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  25. Katalin Marton. Measure concentration for Euclidean distance in the case of dependent random variables. Ann. Probab., 32(3):2526–2544, 2004.

    Article  MathSciNet  MATH  Google Scholar 

  26. Katalin Marton. An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces. Journal of Functional Analysis, 264(34–61), 2013.

    Google Scholar 

  27. Katalin Marton. Logarithmic Sobolev inequalities in discrete product spaces: a proof by a transportation cost distance. arXiv.org preprint 1507.02803, July 2015.

    Google Scholar 

  28. Bernard Maurey. Construction de suites symétriques. C. R. Acad. Sci. Paris Sér. A-B 288, (14):A679–A681, 1979.

    Google Scholar 

  29. Daniel Paulin. Concentration inequalities for Markov chains by Marton couplings and spectral methods. Electronic Journal of Probability, 20:1–32, 2015.

    Article  MathSciNet  MATH  Google Scholar 

  30. Maxim Raginsky and Igal Sason. Concentration of Measure Inequalities in Information Theory, Communications, and Coding. Foundations and Trends in Communications and Information Theory. Now Publishers, 2nd edition, 2014.

    Google Scholar 

  31. Patrick Rebeschini and Ramon van Handel. Comparison theorems for Gibbs measures. J. Stat. Phys., 157:234–281, 2014.

    Article  MathSciNet  MATH  Google Scholar 

  32. Emmanuel Rio. Inégalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes. C. R. Acad. Sci. Paris Sér. I Math., 330(10):905–908, 2000.

    Article  MathSciNet  MATH  Google Scholar 

  33. Paul-Marie Samson. Concentration of measure inequalities for Markov chains and Φ-mixing processes. Ann. Probab., 28(1):416–461, 2000.

    Article  MathSciNet  MATH  Google Scholar 

  34. Ramon van Handel. Probability in high dimension. ORF 570 Lecture Notes, Princeton University, June 2014.

    Google Scholar 

  35. Olivier Wintenberger. Weak transport inequalities and applications to exponential and oracle inequalities. Electronic Journal of Probability, 20(114):1–27, 2015.

    MathSciNet  MATH  Google Scholar 

  36. Liming Wu. Poincaré and transportation inequalities for Gibbs measures under the Dobrushin uniqueness condition. Ann. Probab., 34(5):1960–1989, 2006.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The second author would like to thank IMA for an invitation to speak at the workshop on Information Theory and Concentration Phenomena in Spring 2015, which was part of the annual program “Discrete Structures: Analysis and Applications.” The authors are grateful to the anonymous referee for several constructive suggestions, and to Dr. Naci Saldi for spotting an error in an earlier version of the manuscript. A. Kontorovich was partially supported by the Israel Science Foundation (grant No. 1141/12) and a Yahoo Faculty award. M. Raginsky would like to acknowledge the support of the U.S. National Science Foundation via CAREER award CCF–1254041.

Author information

Authors and Affiliations

  1. Department of Computer Science, Ben-Gurion University, Beer-Sheva, Israel

    Aryeh Kontorovich

  2. Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois, Urbana, IL, USA

    Maxim Raginsky

Authors
  1. Aryeh Kontorovich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maxim Raginsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maxim Raginsky .

Editor information

Editors and Affiliations

  1. Department of Mathematics, Rutgers University, Piscataway, New Jersey, USA

    Eric Carlen

  2. Department of Mathematical Sciences, University of Delaware, Newark, Delaware, USA

    Mokshay Madiman

  3. Department of Mathematics, Case Western Reserve University, Cleveland, Ohio, USA

    Elisabeth M. Werner

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media LLC

About this paper

Cite this paper

Kontorovich, A., Raginsky, M. (2017). Concentration of Measure Without Independence: A Unified Approach Via the Martingale Method. In: Carlen, E., Madiman, M., Werner, E. (eds) Convexity and Concentration. The IMA Volumes in Mathematics and its Applications, vol 161. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-7005-6_6

Download citation

Publish with us

Policies and ethics

Search

Navigation