# Limit theorems for empirical processes

• David Pollard
Article

## Summary

The empirical measure P n for iid sampling on a distribution P is formed by placing mass n−1 at each of the first n observations. Generalizations of the classical Glivenko-Cantelli theorem for empirical measures have been proved by Vapnik and červonenkis using combinatorial methods. They found simple conditions on a class C to ensure that sup {|P n (C) − P(C)|: CC} converges in probability to zero. They used a randomization device that reduced the problem to finding exponential bounds on the tails of a hypergeometric distribution. In this paper an alternative randomization is proposed. The role of the hypergeometric distribution is thereby taken over by the binomial distribution, for which the elementary Bernstein inequalities provide exponential boundson the tails. This leads to easier proofs of both the basic results of Vapnik-červonenkis and the extensions due to Steele. A similar simplification is made in the proof of Dudley's central limit theorem forn1/2(P P n −P)— a result that generalizes Donsker's functional central limit theorem for empirical distribution functions.

### Keywords

Mathematical Biology Central Limit Theorem Empirical Distribution Combinatorial Method Simple Condition

## Preview

### References

1. Ash, R.B.: Real Analysis and Probability. New York: Academic Press 1972Google Scholar
2. Dudley, R.M.: Weak convergence of probabilities on nonseparable metric spaces and empirical measures on Euclidean spaces. Illinois J. Math. 10, 109–126 (1966)Google Scholar
3. Dudley, R.M.: Measures on nonseparable metric spaces Illinois J. Math. 11, 449–453 (1967)Google Scholar
4. Dudley, R.M.: Central limit theorems for empirical measures. Ann. Probability 6, 899–929 (1978). Correction, ibid 7, 909–911 (1979)Google Scholar
5. Durst, M., Dudley, R.M.: Empirical processes, Vapnik-červonenkis classes and Poisson processes. Prob. Math. Stat. 1 (1980)Google Scholar
6. Gihman, I.I., Skorohod, A.V.: The Theory of Stochastic Processes I. Berlin-Heidelberg-New York: Springer 1974Google Scholar
7. Halmos, P.R.: Measure Theory. New York: Van Nostrand Reinhold 1969Google Scholar
8. Neveu, J.: Mathematical Foundations of the Calculus of Probability. San Francisco: Holden Day 1965Google Scholar
9. Pollard, D.: Beyond the heuristic approach to Kolmogorov-Smirnov theorems. In: Essays in Statistical Science. Festschrift for P.A.P. Moran. Eds. J. Gani and E.J. Hannan. Applied Probability Trust (to appear 1982)Google Scholar
10. Pollard, D.: A central limit Theorem for empirical processes. J. Australian Math. Soc. (to appear 1981)Google Scholar
11. Steele, J.M.: Combinatorial Entropy and Uniform Limit Laws. Ph.D. dissertation. Stanford (1975)Google Scholar
12. Steele, J.M.: Empirical discrepancies and subadditive processes. Ann. Probability 6, 118–127 (1978)Google Scholar
13. Uspensky, J.V.: Introduction to Mathematical Probability. New York: McGraw-Hill 1937Google Scholar
14. Vapnik, V.N., červonenkis, A.Ya.: On the uniform convergence of relative frequencies of events to their probabilities. Theor. Prob. Appl. 16, 264–280 (1971)Google Scholar