Ergodicity, Decisions, and Partial Information

Chapter
First Online:
Part of the Lecture Notes in Mathematics book series (LNM, volume 2123)

Abstract

In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision u n is made as a function of past observations \(X_{0},\ldots,X_{n-1}\), and a loss l(u n , X n ) is incurred. In this setting, it is known that one may choose (under a mild integrability assumption) a decision strategy whose pathwise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y, the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system \((\varOmega,\mathcal{B},\mathbf{P},T)\) with partial information \(\mathcal{Y}\subseteq \mathcal{B}\). The existence of pathwise optimal strategies grounded in two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a general sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observations \(\bigvee _{n\geq 0}T^{n}\mathcal{Y}\). If the conditional ergodicity assumption is strengthened, the complexity assumption can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information. Our results also yield a decision-theoretic characterization of weak mixing in ergodic theory, and establish pathwise optimality of ergodic nonlinear filters.

This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was partially supported by NSF grant DMS-1005575.

References

  1. 1.
    P.H. Algoet, The strong law of large numbers for sequential decisions under uncertainty. IEEE Trans. Inform. Theory 40(3), 609–633 (1994)CrossRefMathSciNetMATHGoogle Scholar
  2. 2.
    A. Bellow, V. Losert, The weighted pointwise ergodic theorem and the individual ergodic theorem along subsequences. Trans. Am. Math. Soc. 288(1), 307–345 (1985)CrossRefMathSciNetMATHGoogle Scholar
  3. 3.
    D. Berend, V. Bergelson, Mixing sequences in Hilbert spaces. Proc. Am. Math. Soc. 98(2), 239–246 (1986)CrossRefMathSciNetMATHGoogle Scholar
  4. 4.
    O. Cappé, E. Moulines, T. Rydén, Inference in Hidden Markov Models (Springer, New York, 2005)MATHGoogle Scholar
  5. 5.
    P. Chigansky, R. van Handel, A complete solution to Blackwell’s unique ergodicity problem for hidden Markov chains. Ann. Appl. Probab. 20(6), 2318–2345 (2010)CrossRefMathSciNetMATHGoogle Scholar
  6. 6.
    J.P. Conze, Convergence des moyennes ergodiques pour des sous-suites. In: Contributions au calcul des probabilités. Bull. Soc. Math. France, Mém. No. 35 (Soc. Math. France, Paris, 1973), pp. 7–15Google Scholar
  7. 7.
    D. Crisan, B. Rozovskiĭ (eds.), The Oxford Handbook of Nonlinear Filtering (Oxford University Press, Oxford, 2011)MATHGoogle Scholar
  8. 8.
    P. Del Moral, M. Ledoux, Convergence of empirical processes for interacting particle systems with applications to nonlinear filtering. J. Theor. Probab. 13(1), 225–257 (2000)CrossRefMATHGoogle Scholar
  9. 9.
    C. Dellacherie, P.A. Meyer, Probabilities and Potential. C (North-Holland, Amsterdam, 1988)Google Scholar
  10. 10.
    R.M. Dudley, Uniform Central Limit Theorems (Cambridge University Press, Cambridge, 1999)CrossRefMATHGoogle Scholar
  11. 11.
    N. Etemadi, An elementary proof of the strong law of large numbers. Z. Wahrsch. Verw. Gebiete 55(1), 119–122 (1981)CrossRefMathSciNetMATHGoogle Scholar
  12. 12.
    A. Grothendieck, Produits tensoriels topologiques et espaces nucléaires. Mem. Am. Math. Soc. 1955(16), 140 (1955)Google Scholar
  13. 13.
    P.R. Halmos, In general a measure preserving transformation is mixing. Ann. Math. (2) 45, 786–792 (1944)Google Scholar
  14. 14.
    R. van Handel, The stability of conditional Markov processes and Markov chains in random environments. Ann. Probab. 37(5), 1876–1925 (2009)CrossRefMathSciNetMATHGoogle Scholar
  15. 15.
    R. van Handel, Uniform time average consistency of Monte Carlo particle filters. Stoch. Process. Appl. 119(11), 3835–3861 (2009)CrossRefMATHGoogle Scholar
  16. 16.
    R. van Handel, On the exchange of intersection and supremum of σ-fields in filtering theory. Isr. J. Math. 192, 763–784 (2012)CrossRefMATHGoogle Scholar
  17. 17.
    van Handel, R.: The universal Glivenko-Cantelli property. Probab. Theor. Relat. Fields 155(3–4), 911–934 (2013)CrossRefMATHGoogle Scholar
  18. 18.
    J. Hoffmann-Jørgensen, Uniform Convergence of Martingales. In: Probability in Banach spaces, 7 (Oberwolfach, 1988), Progr. Probab., vol. 21 (Birkhäuser Boston, Boston, 1990), pp. 127–137Google Scholar
  19. 19.
    O. Kallenberg, Foundations of Modern Probability, 2nd edn. (Springer, New York, 2002)CrossRefMATHGoogle Scholar
  20. 20.
    H. Kunita, Asymptotic behavior of the nonlinear filtering errors of Markov processes. J. Multivariate Anal. 1, 365–393 (1971)CrossRefMathSciNetGoogle Scholar
  21. 21.
    T. Lindvall, Lectures on the Coupling Method (Dover Publications, Mineola, 2002). Corrected reprint of the 1992 originalGoogle Scholar
  22. 22.
    S. Meyn, R.L. Tweedie, Markov Chains and Stochastic Stability, 2nd edn. (Cambridge University Press, Cambridge, 2009)CrossRefMATHGoogle Scholar
  23. 23.
    J. Neveu, Discrete-Parameter Martingales (North-Holland, Amsterdam, 1975)MATHGoogle Scholar
  24. 24.
    A.B. Nobel, On optimal sequential prediction for general processes. IEEE Trans. Inform. Theory 49(1), 83–98 (2003)CrossRefMathSciNetMATHGoogle Scholar
  25. 25.
    D. Pollard, A User’S Guide to Measure Theoretic Probability (Cambridge University Press, Cambridge, 2002)MATHGoogle Scholar
  26. 26.
    D.J. Rudolph, Pointwise and L 1 mixing relative to a sub-sigma algebra. Ill. J. Math. 48(2), 505–517 (2004)MathSciNetMATHGoogle Scholar
  27. 27.
    X.T. Tong, R. van Handel, Conditional ergodicity in infinite dimension (2012). PreprintGoogle Scholar
  28. 28.
    H. Totoki, On a class of special flows. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 15, 157–167 (1970)CrossRefMathSciNetGoogle Scholar
  29. 29.
    A.W. van der Vaart, J.A. Wellner, Weak Convergence and Empirical Processes (Springer, New York, 1996)CrossRefMATHGoogle Scholar
  30. 30.
    V.A. Volkonskiĭ, Y.A. Rozanov, Some limit theorems for random functions. I. Theor. Probab. Appl. 4, 178–197 (1959)CrossRefGoogle Scholar
  31. 31.
    P. Walters, An Introduction to Ergodic Theory (Springer, New York, 1982)CrossRefMATHGoogle Scholar
  32. 32.
    T. Weissman, N. Merhav, Universal prediction of random binary sequences in a noisy environment. Ann. Appl. Probab. 14(1), 54–89 (2004)CrossRefMathSciNetMATHGoogle Scholar
  33. 33.
    H. von Weizsäcker, Exchanging the order of taking suprema and countable intersections of σ-algebras. Ann. Inst. H. Poincaré Sect. B (N.S.) 19(1), 91–100 (1983)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Princeton UniversityPrincetonUSA

Personalised recommendations