# Optimal Bounds on Tail Probabilities: A Study of an Approach

## Abstract

In Computer Science and Statistics it is often desirable to obtain tight bounds on the decay rate of probabilies of the type \(\Pr \left\{ {{S_n} - E\left[ {{S_n}} \right] \geqslant na} \right\},\), where *S* _{ n } is a sum of independent random variables \(\left\{ {{X_i}} \right\}_{i = 1}^n\). This is usually done by means of Chernoff inequality, or the more general Hoeffding inequality. The latter inequality is assymptotically optimal as far as the expectations of *X* _{ i }-s go, but ceases to be so when the variances are also given. The variances are taken into account in the stronger Bennett inequality, which despite its potential usefulness is virtually unknown in CS community.

In this paper we provide a systematic account of the general method (based on Laplace transform) underlying most asymptotically tight extimations of tail probablilities, and show how it can be used in various situations. In particular, we provide new and simple proofs of the Hoeffding and the Bennet bounds, and obtain their natural generalisation, which takes into account the first *k* moments of *X* _{ i }-s. We discuss also a typical application of the general method to a concrete problem from Computer Science, and obtain estimations superiour to those previously known.

The main goal of this work is to give a clear, coherent exposition of the general method and its various aspects, in the belief that a better acquaintance with this powerful tool might prove beneficial in studies involving estimations of tail probablilities, e.g., in analysis of performances of randomized algorithms.

## Keywords

Hoeffding bound Bennett bound Laplace transform tail probabilities moment sequence## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Bentley J. L., McGeogh,
*Amortized Analyses of Self-Organizing Sequential Search Heuristics, Comm. ACM*,**28**, pp. 404–411, 1985.MathSciNetGoogle Scholar - [2]S. Bernstein,
*Sur une modification de Vinequalite de Tchebichef*, Annals Science Institute Sav. Ukraine, Sect. Math. I, 1924 ( Russian, French summary).Google Scholar - [3]H. Chernoff,
*A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations*, Annals of Math. Stat.,**23**, 1952, 493–507.CrossRefGoogle Scholar - [4]G. Bennett,
*Probability Inequalities for the Sum of Independent Random Variables*, J. Am. Stat. Ass.,**57**, 1962, 33–45.CrossRefGoogle Scholar - [5]Hester J. H., Hirschberg D. S.,
*Self-organizing Linear Search, ACM Comput. Surveys*,**17**, pp. 295–312, 1985.Google Scholar - [6]W. Hoeffding,
*Probability Inequalities for Sums of Bounded Random Variables*, J. Am. Stat. Ass.,**58**, 1963, 13–30.MATHCrossRefGoogle Scholar - [7]Hofri M., Shachnai H.,
*Self-Organizing Lists and Independent References - a Statistical Synergy, Jour, of Alg*.,**12**, pp. 533–555, 1991.MATHGoogle Scholar - [8]McCabe J.,
*On Serial Files with Relocatable Records, Operations Res*.,**13**, pp. 609–618, 1965.MathSciNetMATHGoogle Scholar - [9]T. Hagerup and C. Rüb,
*A Guided Tour of Chernoff Bounds*, Inf. Proc. Lett.,**33**, 1990, 305–308.MathSciNetMATHCrossRefGoogle Scholar - [10]A. Dembo and O. Zeitouni,
*Large Deviations Techniques and Applications*, Springer, April 1998 (to appear).MATHGoogle Scholar - [11]M.G. Krein and A.A. Nudelman,
*The Markov Moment Problem and Extremal Problems*, Translations of Math. Monographs (From Russian), Vol. 50, 1977, American Mathematical Society.MathSciNetGoogle Scholar - [12]W. Feller,
*An Introduction to Probability Theory and its Applications*, John Wiley and Sons, 1957.MATHGoogle Scholar