Probability and Random Processes

  • Robert M. Gray


In this chapter we develop basic mathematical models of discrete time random processes. Such processes are also called discrete time stochastic processes, information sources, and time series. Physically a random process is something that produces a succession of symbols called “outputs” in a random or nondeterministic manner. The symbols produced may be real numbers such as produced by voltage measurements from a transducer, binary numbers as in computer data, two-dimensional intensity fields as in a sequence of images, continuous or discontinuous waveforms, and so on. The space containing all of the possible output symbols is called the alphabet of the random process, and a random process is essentially an assignment of a probability measure to events consisting of sets of sequences of symbols from the alphabet. It is useful, however, to treat the notion of time explicitly as a transformation of sequences produced by the random process. Thus in addition to the common random process model we shall also consider modeling random processes by dynamical systems as considered in ergodic theory.


Probability Measure Random Process Probability Space Measurable Space Event Space 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.Google Scholar
  2. 2.
    P. Billingsley, Ergodic Theory and Information, Wiley, New York, 1965.MATHGoogle Scholar
  3. 3.
    L. Breiman, Probability, Addison-Wesley, Menlo Park, Calif., 1968.MATHGoogle Scholar
  4. 4.
    J. C. Candy, “A use of limit cycle oscillations to obtain robust analog-to-digital converters,” IEEE Transactions on Communications, vol. COM-22, pp. 298–305, March 1974.CrossRefGoogle Scholar
  5. 5.
    K. L. Chung, A Course in Probability Theory, Academic Press, New York, 1974.MATHGoogle Scholar
  6. 6.
    H. Cramer, Mathematical Methods of Statistics, Princeton University Press, Princeton, NJ, 1946.MATHGoogle Scholar
  7. 7.
    M. Denker, C. Grillenberger, and K. Sigmund, Ergodic Theory on Compact Spaces, 57, Lecture Notes in Mathematics, Springer-Verlag, New York, 1970.Google Scholar
  8. 8.
    N. A. Friedman, Introduction to Ergodic Theory, Van Nostrand Reinhold Company, New York, 1970.MATHGoogle Scholar
  9. 9.
    R. M. Gray, “Oversampled Sigma-Delta Modulation,” IEEE Transactions on Communications, vol. COM-35, pp. 481–489, April 1987.MATHCrossRefGoogle Scholar
  10. 10.
    P. R. Halmos, Measure Theory, Van Nostrand Reinhold, New York, 1950.MATHGoogle Scholar
  11. 11.
    P. R. Halmos, Lectures on Ergodic Theory, Chelsea, New York, 1956.MATHGoogle Scholar
  12. 12.
    H. Inose and Y. Yasuda, “A unity bit coding method by negative feedback,” Proceedings of the IEEE, vol. 51, pp. 1524–1535, November 1963.CrossRefGoogle Scholar
  13. 13.
    A. N. Kolmogorov, Foundations of the Theory of Probability, Chelsea, New York, 1950.Google Scholar
  14. 14.
    U. Krengel, Ergodic Theorems, De Gruyter Series in Mathematics, De Gruyter, New York, 1985.Google Scholar
  15. 15.
    M. Loève, Probability Theory, D. Van Nostrand, Princeton, New Jersey, 1963. Third Edition.MATHGoogle Scholar
  16. 16.
    L. E. Maistrov, Probability Theory: A Historical Sketch, Academic Press, New York, 1974. Translated by S. Kotz.MATHGoogle Scholar
  17. 17.
    D. Ornstein, Ergodic Theory, Randomness, and Dynamical Systems, Yale University Press, New Haven, 1975.Google Scholar
  18. 18.
    K. Petersen, Ergodic Theory, Cambridge University Press, Cambridge, 1983.MATHCrossRefGoogle Scholar
  19. 19.
    P. C. Shields, The Theory of Bernoulli Shifts, The University of Chicago Press, Chicago, Ill., 1973.MATHGoogle Scholar
  20. 20.
    Ya. G. Sinai, Introduction to Ergodic Theory, Mathematical Notes, Princeton University Press, Princeton, 1976.MATHGoogle Scholar
  21. 21.
    P. Walters, Ergodic Theory-Introductory Lectures, Lecture Notes in Mathematics No. 458, Springer-Verlag, New York, 1975.MATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1988

Authors and Affiliations

  • Robert M. Gray
    • 1
  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA

Personalised recommendations