Abstract
This paper studies transition probabilities from a Borel subset of a Polish space to a product of two Borel subsets of Polish spaces. For such transition probabilities it introduces and studies the property of semi-uniform Feller continuity. This paper provides several equivalent definitions of semi-uniform Feller continuity and establishes its preservation under integration. The motivation for this study came from the theory of Markov decision processes with incomplete information, and this paper provides the fundamental results useful for this theory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
References
Aoki, M.: Optimal control of partially observable Markovian systems. J. Frankl. Inst. 280(5), 367–386 (1965)
Åström, K.J.: Optimal control of Markov processes with incomplete state information. J. Math. Anal. Appl. 10, 174–205 (1965)
Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Academic Press, New York (1978)
Bogachev, V.I.: Measure Theory, vol. II. Springer-Verlag, Berlin (2007)
Dynkin, E.B.: Controlled random sequences. Theory Probab. Appl. 10(1), 1–14 (1965)
Dynkin, E.B., Yushkevich, A.A.: Controlled Markov Processes. Springer-Verlag, New York (1979)
Feinberg, E.A., Kasyanov, P.O.: Equivalent conditions for weak continuity of nonlinear filters. arXiv:2207.07544 (2022)
Feinberg, E.A., Kasyanov, P.O., Liang, Y.: Fatou’s lemma in its classical form and Lebesgue’s convergence theorems for varying measures with applications to Markov decision processes. Theory Probab. Appl. 65(2), 270–291 (2020)
Feinberg, E.A., Kasyanov, P.O., Royset, J.O.: Epi-convergence of expectation functions under varying measures and integrands. J. Convex Anal. 30(2). arXiv:2208.03805 (2023) (to appear)
Feinberg, E.A., Kasyanov, P.O., Zgurovsky, M.Z.: Convergence of probability measures and Markov decision models with incomplete information. Proc. Steklov Inst. Math. 287(1), 96–117 (2014)
Feinberg, E.A., Kasyanov, P.O., Zgurovsky, M.Z.: Uniform Fatou’s lemma. J. Math. Anal. Appl. 444(1), 550–567 (2016)
Feinberg, E.A., Kasyanov, P.O., Zgurovsky, M.Z.: Partially observable total-cost Markov decision processes with weakly continuous transition probabilities. Math. Oper. Res. 41(2), 656–681 (2016)
Feinberg, E.A., Kasyanov, P.O., Zgurovsky, M.Z.: Markov decision processes with incomplete information and semi-uniform Feller transition probabilities. SIAM J. Control. Optim. 60, 2488–2513 (2022)
Feinberg, E.A., Kasyanov, P.O., Zgurovsky, M.Z.: Average cost Markov decision processes with semi-uniform Feller transition probabilities. In: Piunovskiy, A., Zhang, Y. (eds.) Modern Trends in Controlled Stochastic Processes: Theory and Applications, pp. 1–18. Springer Nature, Cham (2021)
Hernández-Lerma, O.: Adaptive Markov Control Processes. Springer-Verlag, New York (1989)
Kara, A.D., Saldi, N., Yüksel, S.: Weak Feller property of non-linear filters. Syst. Control Lett. 134, 104512 (2019)
Ma, L.: Sequential convergence on the space of Borel measures. arXiv:2102.05840 (2021)
Papanicolaou, G.C.: Asymptotic analysis of stochastic equations. In: Rosenblatt, M. (ed.) Studies in Probability Theory, pp. 111–179. Mathematical Association of America, Washington, DC (1978)
Parthasarathy, K.R.: Probability Measures on Metric Spaces. Academic Press, New York (1967)
Rhenius, D.: Incomplete information in Markovian decision models. Ann. Stat. 2(6), 1327–1334 (1974)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)
Rudin, W.: Principles of Mathematical Analysis, 2nd edn. McGraw-Hill Inc, New York (1964)
Runggaldier, W.J., Stettner, L.: Approximations of Discrete Time Partially Observed Control Problems. Applied Mathematics Monographs CNR, Giardini Editori, Pisa (1994)
Schäl, M.: On dynamic programming: compactness of the space of policies. Stoch. Process. Appl. 3, 345–364 (1975)
Shiryaev, A.N.: On the theory of decision functions and control by an observation process with incomplete data. Transactions of the Third Prague Conference on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1962), pp. 657–681 (in Russian); Engl. transl. in Select. Transl. Math. Statist. Probab. 6(1966), 162–188 (1964)
Shiryaev, A.N.: Some new results in the theory of controlled random processes. Transactions of the Fourth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1965), pp. 131–201 (in Russian); Engl. transl. in Select. Transl. Math. Statist. Probab. 8(1969), 49–130 (1967)
Smallwood, R.D., Sondik, E.J.: The optimal control of partially observable Markov processes over a finite horizon. Oper. Res. 21(5), 1071–1088 (1973)
Yushkevich, A.A.: Reduction of a controlled Markov model with incomplete data to a problem with complete information in the case of Borel state and control spaces. Theory Probab. Appl. 21(1), 153–158 (1976)
Acknowledgements
We thank Janey (Huizhen) Yu and Yi Zhang for valuable remarks. The second and the third authors were partially supported by the National Research Foundation of Ukraine, Grant No. 2020.01/0283. The second author was partially supported by a U4U non-residential fellowship by UC Berkeley Economics/Haas.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Feinberg, E.A., Kasyanov, P.O. & Zgurovsky, M.Z. Semi-uniform Feller Stochastic Kernels. J Theor Probab 36, 2262–2283 (2023). https://doi.org/10.1007/s10959-022-01230-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10959-022-01230-9