Probability Inequalities

  • Roberto Tempo
  • Giuseppe Calafiore
  • Fabrizio Dabbene
Part of the Communications and Control Engineering book series (CCE)


This chapter addresses the issue of finite sample size in probability estimation, that is, the so-called sample complexity. The main objective in this context is to analyze rigorously the reliability of the probabilistic estimates introduced in Chap.  7, for a finite sample size. This issue is crucial in the development of randomized algorithms for uncertain systems and control, and makes a clear distinction from the asymptotic methods which are instead based on the laws of large numbers. Specifically, the chapter includes Markov, Chebychev and Hoeffding inequalities. The additive and multiplicative Chernoff bounds are subsequently derived and the sample complexity for estimation of extrema is also studied.




  1. 10.
    Alamo T, Tempo R, Luque A (2010) On the sample complexity of probabilistic analysis and design methods. In: Hara S, Ohta Y, Willems JC (eds) Perspectives in mathematical system theory, control, and signal processing. Springer, Berlin, pp 39–50 CrossRefGoogle Scholar
  2. 36.
    Bai E-W, Tempo R, Fu M (1998) Worst-case properties of the uniform distribution and randomized algorithms for robustness analysis. Math Control Signals Syst 11:183–196 MathSciNetMATHCrossRefGoogle Scholar
  3. 52.
    Bennett G (1962) Probability inequalities for the sum of independent random variables. J Am Stat Assoc 57:33–45 MATHCrossRefGoogle Scholar
  4. 54.
    Bernoulli J (1713) Ars conjectandi, Paris Google Scholar
  5. 55.
    Bernstein SN (1946) The theory of probabilities. Gostehizdat Publishing House, Moscow (in Russian) Google Scholar
  6. 56.
    Bertsimas D, Sethuraman J (2000) Moment problems and semidefinite optimization. In: Wolkowicz H, Saigal R, Vandenberghe L (eds) Handbook of semidefinite programming. Kluwer Academic Publishers, Boston, pp 469–509 CrossRefGoogle Scholar
  7. 66.
    Boucheron S, Lugosi G, Massart P (2003) Concentration inequalities using the entropy method. Ann Probab 31:1583–1614 MathSciNetMATHCrossRefGoogle Scholar
  8. 100.
    Chebychev P (1874) Sur les valeurs limites des intégrales. J Math Pures Appl 19:157–160 Google Scholar
  9. 103.
    Chen X, Zhou K (1998) Order statistics and probabilistic robust control. Syst Control Lett 35 Google Scholar
  10. 104.
    Chernoff H (1952) A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann Math Stat 23:493–507 MathSciNetMATHCrossRefGoogle Scholar
  11. 111.
    Conover WJ (1980) Practical nonparametric statistics. Wiley, New York Google Scholar
  12. 119.
    Dabbene F, Shcherbakov PS, Polyak BT (2010) A randomized cutting plane method with probabilistic geometric convergence. SIAM J Optim 20:3185–3207 MathSciNetMATHCrossRefGoogle Scholar
  13. 130.
    Dembo A, Zeitouni O (1993) Large deviations techniques and applications. Jones and Bartlett, Boston MATHGoogle Scholar
  14. 155.
    Efron B, Stein C (1981) The jackknife estimate of variance. Ann Stat 9:586–596 MathSciNetMATHCrossRefGoogle Scholar
  15. 205.
    Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J Am Stat Assoc 58:13–30 MathSciNetMATHCrossRefGoogle Scholar
  16. 233.
    Khargonekar P, Tikku A (1996) Randomized algorithms for robust control analysis and synthesis have polynomial complexity. In: Proceedings of the IEEE conference on decision and control Google Scholar
  17. 274.
    Markov A (1884) On certain applications of algebraic continued fractions. PhD Dissertation, St. Petersburg (in Russian) Google Scholar
  18. 276.
    Marshall A, Olkin I (1960) Multivariate Chebyshev inequalities. Ann Math Stat 31:1001–1014 MathSciNetMATHCrossRefGoogle Scholar
  19. 319.
    Papoulis A, Pillai SU (2002) Probability, random variables and stochastic processes. McGraw-Hill, New York Google Scholar
  20. 329.
    Popescu I (1999) Applications of optimization in probability, finance and revenue management. PhD dissertation, Massachusetts Institute of Technology, Cambridge Google Scholar
  21. 332.
    Ray LR, Stengel RF (1993) A Monte Carlo approach to the analysis of control system robustness. Automatica 29:229–236 MathSciNetMATHCrossRefGoogle Scholar
  22. 370.
    Stieltjes TJ (1894) Recherches sur les fractions continues. Ann Fac Sci Toulouse 8:1–122 MathSciNetCrossRefGoogle Scholar
  23. 371.
    Stieltjes TJ (1895) Recherches sur les fractions continues. Ann Fac Sci Toulouse 9:5–47 MathSciNetCrossRefGoogle Scholar
  24. 377.
    Talagrand M (1996) New concentration inequalities in product spaces. Invent Math 126:505–563 MathSciNetMATHCrossRefGoogle Scholar
  25. 382.
    Tempo R, Bai E-W, Dabbene F (1997) Probabilistic robustness analysis: explicit bounds for the minimum number of samples. Syst Control Lett 30:237–242 MathSciNetMATHCrossRefGoogle Scholar
  26. 387.
    Tong YL (1980) Probability inequalities in multivariate distributions. Academic Press, New York MATHGoogle Scholar
  27. 397.
    Uspensky JV (1937) Introduction to mathematical probability. McGraw-Hill, New York MATHGoogle Scholar
  28. 406.
    Vidyasagar M (2002) Learning and generalization: with applications to neural networks, 2nd edn. Springer, New York Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Roberto Tempo
    • 1
  • Giuseppe Calafiore
    • 2
  • Fabrizio Dabbene
    • 1
  1. 1.CNR - IEIITPolitecnico di TorinoTurinItaly
  2. 2.Dip. Automatica e InformaticaPolitecnico di TorinoTurinItaly

Personalised recommendations