Process Metrics and the Ergodic Decomposition

  • Robert M. Gray


Given two probability measures, say p and m, on a common probability space, how different or distant from each other are they? Similarly, given two random processes with distributions p and m, how distant are the processes from each other and what impact does such a distance have on their respective ergodic properties? The goal of this final chapter is to develop two quite distinct notions of the distance d(p,m) between measures or processes and to use these ideas to delve further into the ergodic properties of processes and the ergodic decomposition. One metric, the distributional distance, measures how well the probabilities of certain important events match up for the two probability measures, and hence this metric need not have any relation to any underlying metric on the original sample space. In other words, the metric makes sense even when we are not putting probability measures on metric spaces. The second metric, the ρ̄-distance (rho-bar distance) depends very strongly on a metric on the output space of the process and measures distance not by how different probabilities are, but by how well one process can be made to approximate another. The second metric is primarily useful in applications in information theory and statistics.


Probability Measure Generate Field Prob Ability Ergodic Measure Ergodic Property 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    P. Billingsley, Convergence of Probability Measures, Wiley Series in Probability and Mathematical Statistics, Wiley, New York, 1968.MATHGoogle Scholar
  2. 2.
    P. Billingsley, Weak Convergence of Measures: Applications in Probability, SIAM, Philadelphia, 1971.MATHCrossRefGoogle Scholar
  3. 3.
    R. Dobrushin. private correspondenceGoogle Scholar
  4. 4.
    R. M. Dudley, “Distances of probability measures and random variables,” Ann. Math. Statist., vol. 39, pp. 1563–1572, 1968.MathSciNetMATHGoogle Scholar
  5. 5.
    R. M. Gray and L. D. Davisson, Ergodic and Information Theory, Benchmark Papers in Electrical Engineering and Computer Science, 19, Dowden, Hutchninson, & Ross, Stroudsbug PA, 1977.Google Scholar
  6. 6.
    R. M. Gray, D. L. Neuhoff, and P. C. Shields, “A generalization of Ornstein’s d-bar distance with applications to information theory,” Annals of Probability, vol. 3, pp. 315–328, April 1975.MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    F. R. Hample, “A general qualitative definition of robustness,” Ann. Math. Statist., vol. 42, pp. 1887–1896, 1971.MathSciNetCrossRefGoogle Scholar
  8. 8.
    K. Jacobs, “The ergodic decomposition of the Kolmogorov-Sinai invariant,” in Ergodic Theory, ed. F. B. Wright, Academic Press, New York, 1963.Google Scholar
  9. 9.
    V. I. Levenshtein, “Binary codes capable of correcting deletions, insertions, and reversals,” Sov. Phys. -Dokl, vol. 10, pp. 707–710, 1966.MathSciNetGoogle Scholar
  10. 10.
    J. Moser, E. Phillips, and S. Varadhan, Ergodic Theory: A Seminar, Courant Institute of Math. Sciences, New York, 1975.MATHGoogle Scholar
  11. 11.
    D. Ornstein, “An application of ergodic theory to probability theory,” Ann. Probability, vol. 1, pp. 43–58, 1973.MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    D. Ornstein, Ergodic Theory, Randomness, and Dynamical Systems, Yale University Press, New Haven, 1975.Google Scholar
  13. 13.
    P. Papantoni-Kazakos and R. M. Gray, “Robustness of estimators on stationary observations,” Annals of Probability, vol. 7, pp. 989–1002, Dec. 1979.MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    V. Strassen, “The existence of probability measures with given marginals,” Ann. Math. Statist, vol. 36, pp. 423–429, 1965.MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    S. S. Vallender, “Computing the Wasserstein distance between probability distributions on the line,” Theor. Probability Appl., vol. 18, pp. 824–827, 1973.Google Scholar
  16. 16.
    L. N. Vasershtein, “Markov processes on countable product space describing large systems of automata,” Problemy Peredachi Informatsii, vol. 5, pp. 64–73, 1969.MathSciNetGoogle Scholar
  17. 16.
    L. N. Vasershtein, “Markov processes on countable product space describing large systems of automata,” Problemy Peredachi Informatsii, vol. 5, pp. 64–73, 1969.MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1988

Authors and Affiliations

  • Robert M. Gray
    • 1
  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA

Personalised recommendations