Advertisement

Probability Inequalities

  • Roberto Tempo
  • Giuseppe Calafiore
  • Fabrizio Dabbene
Part of the Communications and Control Engineering book series (CCE)

Abstract

This chapter addresses the issue of finite sample size in probability estimation, that is, the so-called sample complexity. The main objective in this context is to analyze rigorously the reliability of the probabilistic estimates introduced in Chap.  7, for a finite sample size. This issue is crucial in the development of randomized algorithms for uncertain systems and control, and makes a clear distinction from the asymptotic methods which are instead based on the laws of large numbers. Specifically, the chapter includes Markov, Chebychev and Hoeffding inequalities. The additive and multiplicative Chernoff bounds are subsequently derived and the sample complexity for estimation of extrema is also studied.

Keywords

Sample Complexity Classical Inequality Nonnegative Random Variable Markov Inequality Probability Inequality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 10.
    Alamo T, Tempo R, Luque A (2010) On the sample complexity of probabilistic analysis and design methods. In: Hara S, Ohta Y, Willems JC (eds) Perspectives in mathematical system theory, control, and signal processing. Springer, Berlin, pp 39–50 CrossRefGoogle Scholar
  2. 36.
    Bai E-W, Tempo R, Fu M (1998) Worst-case properties of the uniform distribution and randomized algorithms for robustness analysis. Math Control Signals Syst 11:183–196 MathSciNetzbMATHCrossRefGoogle Scholar
  3. 52.
    Bennett G (1962) Probability inequalities for the sum of independent random variables. J Am Stat Assoc 57:33–45 zbMATHCrossRefGoogle Scholar
  4. 54.
    Bernoulli J (1713) Ars conjectandi, Paris Google Scholar
  5. 55.
    Bernstein SN (1946) The theory of probabilities. Gostehizdat Publishing House, Moscow (in Russian) Google Scholar
  6. 56.
    Bertsimas D, Sethuraman J (2000) Moment problems and semidefinite optimization. In: Wolkowicz H, Saigal R, Vandenberghe L (eds) Handbook of semidefinite programming. Kluwer Academic Publishers, Boston, pp 469–509 CrossRefGoogle Scholar
  7. 66.
    Boucheron S, Lugosi G, Massart P (2003) Concentration inequalities using the entropy method. Ann Probab 31:1583–1614 MathSciNetzbMATHCrossRefGoogle Scholar
  8. 100.
    Chebychev P (1874) Sur les valeurs limites des intégrales. J Math Pures Appl 19:157–160 Google Scholar
  9. 103.
    Chen X, Zhou K (1998) Order statistics and probabilistic robust control. Syst Control Lett 35 Google Scholar
  10. 104.
    Chernoff H (1952) A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann Math Stat 23:493–507 MathSciNetzbMATHCrossRefGoogle Scholar
  11. 111.
    Conover WJ (1980) Practical nonparametric statistics. Wiley, New York Google Scholar
  12. 119.
    Dabbene F, Shcherbakov PS, Polyak BT (2010) A randomized cutting plane method with probabilistic geometric convergence. SIAM J Optim 20:3185–3207 MathSciNetzbMATHCrossRefGoogle Scholar
  13. 130.
    Dembo A, Zeitouni O (1993) Large deviations techniques and applications. Jones and Bartlett, Boston zbMATHGoogle Scholar
  14. 155.
    Efron B, Stein C (1981) The jackknife estimate of variance. Ann Stat 9:586–596 MathSciNetzbMATHCrossRefGoogle Scholar
  15. 205.
    Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J Am Stat Assoc 58:13–30 MathSciNetzbMATHCrossRefGoogle Scholar
  16. 233.
    Khargonekar P, Tikku A (1996) Randomized algorithms for robust control analysis and synthesis have polynomial complexity. In: Proceedings of the IEEE conference on decision and control Google Scholar
  17. 274.
    Markov A (1884) On certain applications of algebraic continued fractions. PhD Dissertation, St. Petersburg (in Russian) Google Scholar
  18. 276.
    Marshall A, Olkin I (1960) Multivariate Chebyshev inequalities. Ann Math Stat 31:1001–1014 MathSciNetzbMATHCrossRefGoogle Scholar
  19. 319.
    Papoulis A, Pillai SU (2002) Probability, random variables and stochastic processes. McGraw-Hill, New York Google Scholar
  20. 329.
    Popescu I (1999) Applications of optimization in probability, finance and revenue management. PhD dissertation, Massachusetts Institute of Technology, Cambridge Google Scholar
  21. 332.
    Ray LR, Stengel RF (1993) A Monte Carlo approach to the analysis of control system robustness. Automatica 29:229–236 MathSciNetzbMATHCrossRefGoogle Scholar
  22. 370.
    Stieltjes TJ (1894) Recherches sur les fractions continues. Ann Fac Sci Toulouse 8:1–122 MathSciNetCrossRefGoogle Scholar
  23. 371.
    Stieltjes TJ (1895) Recherches sur les fractions continues. Ann Fac Sci Toulouse 9:5–47 MathSciNetCrossRefGoogle Scholar
  24. 377.
    Talagrand M (1996) New concentration inequalities in product spaces. Invent Math 126:505–563 MathSciNetzbMATHCrossRefGoogle Scholar
  25. 382.
    Tempo R, Bai E-W, Dabbene F (1997) Probabilistic robustness analysis: explicit bounds for the minimum number of samples. Syst Control Lett 30:237–242 MathSciNetzbMATHCrossRefGoogle Scholar
  26. 387.
    Tong YL (1980) Probability inequalities in multivariate distributions. Academic Press, New York zbMATHGoogle Scholar
  27. 397.
    Uspensky JV (1937) Introduction to mathematical probability. McGraw-Hill, New York zbMATHGoogle Scholar
  28. 406.
    Vidyasagar M (2002) Learning and generalization: with applications to neural networks, 2nd edn. Springer, New York Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Roberto Tempo
    • 1
  • Giuseppe Calafiore
    • 2
  • Fabrizio Dabbene
    • 1
  1. 1.CNR - IEIITPolitecnico di TorinoTurinItaly
  2. 2.Dip. Automatica e InformaticaPolitecnico di TorinoTurinItaly

Personalised recommendations