Advertisement

Optimal Bounds on Tail Probabilities: A Study of an Approach

  • Aviad Cohen
  • Yuri Rabinovich
  • Assaf Schuster
  • Hadas Shachnai
Part of the Combinatorial Optimization book series (COOP, volume 5)

Abstract

In Computer Science and Statistics it is often desirable to obtain tight bounds on the decay rate of probabilies of the type \(\Pr \left\{ {{S_n} - E\left[ {{S_n}} \right] \geqslant na} \right\},\), where S n is a sum of independent random variables \(\left\{ {{X_i}} \right\}_{i = 1}^n\). This is usually done by means of Chernoff inequality, or the more general Hoeffding inequality. The latter inequality is assymptotically optimal as far as the expectations of X i -s go, but ceases to be so when the variances are also given. The variances are taken into account in the stronger Bennett inequality, which despite its potential usefulness is virtually unknown in CS community.

In this paper we provide a systematic account of the general method (based on Laplace transform) underlying most asymptotically tight extimations of tail probablilities, and show how it can be used in various situations. In particular, we provide new and simple proofs of the Hoeffding and the Bennet bounds, and obtain their natural generalisation, which takes into account the first k moments of X i -s. We discuss also a typical application of the general method to a concrete problem from Computer Science, and obtain estimations superiour to those previously known.

The main goal of this work is to give a clear, coherent exposition of the general method and its various aspects, in the belief that a better acquaintance with this powerful tool might prove beneficial in studies involving estimations of tail probablilities, e.g., in analysis of performances of randomized algorithms.

Keywords

Hoeffding bound Bennett bound Laplace transform tail probabilities moment sequence 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Bentley J. L., McGeogh, Amortized Analyses of Self-Organizing Sequential Search Heuristics, Comm. ACM, 28, pp. 404–411, 1985.MathSciNetGoogle Scholar
  2. [2]
    S. Bernstein, Sur une modification de Vinequalite de Tchebichef, Annals Science Institute Sav. Ukraine, Sect. Math. I, 1924 ( Russian, French summary).Google Scholar
  3. [3]
    H. Chernoff, A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations, Annals of Math. Stat., 23, 1952, 493–507.CrossRefGoogle Scholar
  4. [4]
    G. Bennett, Probability Inequalities for the Sum of Independent Random Variables, J. Am. Stat. Ass., 57, 1962, 33–45.CrossRefGoogle Scholar
  5. [5]
    Hester J. H., Hirschberg D. S., Self-organizing Linear Search, ACM Comput. Surveys, 17, pp. 295–312, 1985.Google Scholar
  6. [6]
    W. Hoeffding, Probability Inequalities for Sums of Bounded Random Variables, J. Am. Stat. Ass., 58, 1963, 13–30.MATHCrossRefGoogle Scholar
  7. [7]
    Hofri M., Shachnai H., Self-Organizing Lists and Independent References - a Statistical Synergy, Jour, of Alg., 12, pp. 533–555, 1991.MATHGoogle Scholar
  8. [8]
    McCabe J., On Serial Files with Relocatable Records, Operations Res., 13, pp. 609–618, 1965.MathSciNetMATHGoogle Scholar
  9. [9]
    T. Hagerup and C. Rüb, A Guided Tour of Chernoff Bounds, Inf. Proc. Lett., 33, 1990, 305–308.MathSciNetMATHCrossRefGoogle Scholar
  10. [10]
    A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications, Springer, April 1998 (to appear).MATHGoogle Scholar
  11. [11]
    M.G. Krein and A.A. Nudelman, The Markov Moment Problem and Extremal Problems, Translations of Math. Monographs (From Russian), Vol. 50, 1977, American Mathematical Society.MathSciNetGoogle Scholar
  12. [12]
    W. Feller, An Introduction to Probability Theory and its Applications, John Wiley and Sons, 1957.MATHGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Aviad Cohen
    • 1
  • Yuri Rabinovich
    • 2
  • Assaf Schuster
    • 3
  • Hadas Shachnai
    • 3
  1. 1.Israel Software LabIntelHaifaIsrael
  2. 2.Dept. of Mat. and Computer ScienceBen-Gurion UniversityBeer-ShevaIsrael
  3. 3.Dept. of Computer ScienceTechnion IITHaifaIsrael

Personalised recommendations