Optimal Bounds on Tail Probabilities: A Study of an Approach

Abstract

In Computer Science and Statistics it is often desirable to obtain tight bounds on the decay rate of probabilies of the type $\Pr \left\{ {{S_n} - E\left[ {{S_n}} \right] \geqslant na} \right\},$, where S _n is a sum of independent random variables $\left\{ {{X_i}} \right\}_{i = 1}^n$. This is usually done by means of Chernoff inequality, or the more general Hoeffding inequality. The latter inequality is assymptotically optimal as far as the expectations of X _i -s go, but ceases to be so when the variances are also given. The variances are taken into account in the stronger Bennett inequality, which despite its potential usefulness is virtually unknown in CS community.

In this paper we provide a systematic account of the general method (based on Laplace transform) underlying most asymptotically tight extimations of tail probablilities, and show how it can be used in various situations. In particular, we provide new and simple proofs of the Hoeffding and the Bennet bounds, and obtain their natural generalisation, which takes into account the first k moments of X _i -s. We discuss also a typical application of the general method to a concrete problem from Computer Science, and obtain estimations superiour to those previously known.

The main goal of this work is to give a clear, coherent exposition of the general method and its various aspects, in the belief that a better acquaintance with this powerful tool might prove beneficial in studies involving estimations of tail probablilities, e.g., in analysis of performances of randomized algorithms.