Skip to main content
SpringerLink
Log in

Ergodicity, Decisions, and Partial Information

  • Chapter
  • First Online:
Séminaire de Probabilités XLVI

Part of the book series: Lecture Notes in Mathematics ((SEMPROBAB,volume 2123))

Abstract

In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision u n is made as a function of past observations \(X_{0},\ldots,X_{n-1}\), and a loss l(u n , X n ) is incurred. In this setting, it is known that one may choose (under a mild integrability assumption) a decision strategy whose pathwise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y, the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system \((\varOmega,\mathcal{B},\mathbf{P},T)\) with partial information \(\mathcal{Y}\subseteq \mathcal{B}\). The existence of pathwise optimal strategies grounded in two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a general sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observations \(\bigvee _{n\geq 0}T^{n}\mathcal{Y}\). If the conditional ergodicity assumption is strengthened, the complexity assumption can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information. Our results also yield a decision-theoretic characterization of weak mixing in ergodic theory, and establish pathwise optimality of ergodic nonlinear filters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    To be precise, our definitions are time-reversed with respect to the textbook definitions; however, T is a K-automorphism if and only if T −1 is a K-automorphism [31, p. 110], and the corresponding statement for weak mixing is trivial. Therefore, our definitions are equivalent to those in [31].

  2. 2.

    Non-dominated loss functions may also be of significant interest, see [24] for example. We will restrict attention to dominated loss functions, however, which suffice in many cases of interest.

  3. 3.

    In [1, Appendix II.B] it is shown that under a continuity assumption on the loss function l, the optimal asymptotic loss in the full information setting is given by E[inf u E[l(u, X 1) | X 0, X −1, …]]. However, a counterexample is given of a discontinuous loss function for which this expression does not yield the optimal asymptotic loss. The key difference with the expression for L ⋆ given in Theorem 2.6 is that in the latter the essential infimum runs over \(u \in \mathbb{U}_{0}\), while it is implicit in [1] that the infimum in the above expression is an essential infimum over \(u \in \mathbb{U}_{-\infty,0}\). As the counterexample in [1] shows, these quantities need not coincide in the absence of continuity assumptions.

  4. 4.

    The pointwise separability assumption in [17, Corollary 1.4(2\(\Rightarrow \) 7)] is not needed here, as the essential supremum can be reduced to a countable supremum as in the proof of Lemma 2.5.

  5. 5.

    Some of the statements in [27] are time-reversed as compared to their counterparts stated here. However, as both the absolute regularity and the nondegeneracy assumptions are invariant under time reversal (cf. [30] for the former; the latter is trivial), the present statements follow immediately.

  6. 6.

    As particle filters employ a random sampling mechanism, the strategy \(\mathbf{\tilde{u}}^{N}\) is technically speaking not admissible in the sense of this paper: Π k N (and therefore \(\tilde{u}_{k}^{N}\)) depends also on auxiliary sampling variables ξ 0, …, ξ k that are independent of Y 0, …, Y k . However, it is easily seen that all our results still hold when such randomized strategies are considered. Indeed, it suffices to condition on (ξ k ) k ≥ 0, so that all our results apply immediately under the conditional distribution.

References

  1. P.H. Algoet, The strong law of large numbers for sequential decisions under uncertainty. IEEE Trans. Inform. Theory 40(3), 609–633 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. A. Bellow, V. Losert, The weighted pointwise ergodic theorem and the individual ergodic theorem along subsequences. Trans. Am. Math. Soc. 288(1), 307–345 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  3. D. Berend, V. Bergelson, Mixing sequences in Hilbert spaces. Proc. Am. Math. Soc. 98(2), 239–246 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  4. O. Cappé, E. Moulines, T. Rydén, Inference in Hidden Markov Models (Springer, New York, 2005)

    MATH  Google Scholar 

  5. P. Chigansky, R. van Handel, A complete solution to Blackwell’s unique ergodicity problem for hidden Markov chains. Ann. Appl. Probab. 20(6), 2318–2345 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  6. J.P. Conze, Convergence des moyennes ergodiques pour des sous-suites. In: Contributions au calcul des probabilités. Bull. Soc. Math. France, Mém. No. 35 (Soc. Math. France, Paris, 1973), pp. 7–15

    Google Scholar 

  7. D. Crisan, B. Rozovskiĭ (eds.), The Oxford Handbook of Nonlinear Filtering (Oxford University Press, Oxford, 2011)

    MATH  Google Scholar 

  8. P. Del Moral, M. Ledoux, Convergence of empirical processes for interacting particle systems with applications to nonlinear filtering. J. Theor. Probab. 13(1), 225–257 (2000)

    Article  MATH  Google Scholar 

  9. C. Dellacherie, P.A. Meyer, Probabilities and Potential. C (North-Holland, Amsterdam, 1988)

    Google Scholar 

  10. R.M. Dudley, Uniform Central Limit Theorems (Cambridge University Press, Cambridge, 1999)

    Book  MATH  Google Scholar 

  11. N. Etemadi, An elementary proof of the strong law of large numbers. Z. Wahrsch. Verw. Gebiete 55(1), 119–122 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  12. A. Grothendieck, Produits tensoriels topologiques et espaces nucléaires. Mem. Am. Math. Soc. 1955(16), 140 (1955)

    Google Scholar 

  13. P.R. Halmos, In general a measure preserving transformation is mixing. Ann. Math. (2) 45, 786–792 (1944)

    Google Scholar 

  14. R. van Handel, The stability of conditional Markov processes and Markov chains in random environments. Ann. Probab. 37(5), 1876–1925 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  15. R. van Handel, Uniform time average consistency of Monte Carlo particle filters. Stoch. Process. Appl. 119(11), 3835–3861 (2009)

    Article  MATH  Google Scholar 

  16. R. van Handel, On the exchange of intersection and supremum of σ-fields in filtering theory. Isr. J. Math. 192, 763–784 (2012)

    Article  MATH  Google Scholar 

  17. van Handel, R.: The universal Glivenko-Cantelli property. Probab. Theor. Relat. Fields 155(3–4), 911–934 (2013)

    Article  MATH  Google Scholar 

  18. J. Hoffmann-Jørgensen, Uniform Convergence of Martingales. In: Probability in Banach spaces, 7 (Oberwolfach, 1988), Progr. Probab., vol. 21 (Birkhäuser Boston, Boston, 1990), pp. 127–137

    Google Scholar 

  19. O. Kallenberg, Foundations of Modern Probability, 2nd edn. (Springer, New York, 2002)

    Book  MATH  Google Scholar 

  20. H. Kunita, Asymptotic behavior of the nonlinear filtering errors of Markov processes. J. Multivariate Anal. 1, 365–393 (1971)

    Article  MathSciNet  Google Scholar 

  21. T. Lindvall, Lectures on the Coupling Method (Dover Publications, Mineola, 2002). Corrected reprint of the 1992 original

    Google Scholar 

  22. S. Meyn, R.L. Tweedie, Markov Chains and Stochastic Stability, 2nd edn. (Cambridge University Press, Cambridge, 2009)

    Book  MATH  Google Scholar 

  23. J. Neveu, Discrete-Parameter Martingales (North-Holland, Amsterdam, 1975)

    MATH  Google Scholar 

  24. A.B. Nobel, On optimal sequential prediction for general processes. IEEE Trans. Inform. Theory 49(1), 83–98 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  25. D. Pollard, A User’S Guide to Measure Theoretic Probability (Cambridge University Press, Cambridge, 2002)

    MATH  Google Scholar 

  26. D.J. Rudolph, Pointwise and L 1 mixing relative to a sub-sigma algebra. Ill. J. Math. 48(2), 505–517 (2004)

    MathSciNet  MATH  Google Scholar 

  27. X.T. Tong, R. van Handel, Conditional ergodicity in infinite dimension (2012). Preprint

    Google Scholar 

  28. H. Totoki, On a class of special flows. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 15, 157–167 (1970)

    Article  MathSciNet  Google Scholar 

  29. A.W. van der Vaart, J.A. Wellner, Weak Convergence and Empirical Processes (Springer, New York, 1996)

    Book  MATH  Google Scholar 

  30. V.A. Volkonskiĭ, Y.A. Rozanov, Some limit theorems for random functions. I. Theor. Probab. Appl. 4, 178–197 (1959)

    Article  Google Scholar 

  31. P. Walters, An Introduction to Ergodic Theory (Springer, New York, 1982)

    Book  MATH  Google Scholar 

  32. T. Weissman, N. Merhav, Universal prediction of random binary sequences in a noisy environment. Ann. Appl. Probab. 14(1), 54–89 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  33. H. von Weizsäcker, Exchanging the order of taking suprema and countable intersections of σ-algebras. Ann. Inst. H. Poincaré Sect. B (N.S.) 19(1), 91–100 (1983)

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by NSF grant DMS-1005575.

Author information

Authors and Affiliations

  1. Princeton University, Sherrerd Hall 227, Princeton, NJ, 08544, USA

    Ramon van Handel

Authors
  1. Ramon van Handel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramon van Handel .

Editor information

Editors and Affiliations

  1. Université de Versailles-St-Quentin Laboratoire de Mathématiques, Versailles, France

    Catherine Donati-Martin

  2. INRIA, Nancy-Université, Vandoeuvre-lès-Nancy, France

    Antoine Lejay

  3. Laboratoire de Mathématiques, Université de Versailles-St-Quentin, Versailles, France

    Alain Rouault

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

van Handel, R. (2014). Ergodicity, Decisions, and Partial Information. In: Donati-Martin, C., Lejay, A., Rouault, A. (eds) Séminaire de Probabilités XLVI. Lecture Notes in Mathematics(), vol 2123. Springer, Cham. https://doi.org/10.1007/978-3-319-11970-0_18

Download citation

Publish with us

Policies and ethics

Search

Navigation