Abstract
We address stability of a class of Markovian discrete-time stochastic hybrid systems. This class of systems is characterized by the state-space of the system being partitioned into a safe or target set and its exterior, and the dynamics of the system being different in each domain. We give conditions for L 1-boundedness of Lyapunov functions based on certain negative drift conditions outside the target set, together with some more minor assumptions. We then apply our results to a wide class of randomly switched systems (or iterated function systems), for which we give conditions for global asymptotic stability almost surely and in L 1. The systems need not be time-homogeneous, and our results apply to certain systems for which functional-analytic or martingale-based estimates are difficult or impossible to get.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, D.F., Craciun, G., Kurtz, T.G.: Product-form stationary distributions for deficiency zero chemical reaction networks. arXiv:0803.3042 (2008)
Barnsley, M.F., Demko, S.G., Elton, J.H., Geronimo, J.S.: Invariant measures for Markov processes arising from iterated function systems with place-dependent probabilities. Ann. Ins. Henri Poincaré Probab. Stat. 24(3), 367–394 (1988). Erratum in ibid. 24(4), 589–590 (1989)
Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Mathematics in Science and Engineering, vol. 139. Academic Press, San Diego (1978)
Borkar, V.S.: Topics in Controlled Markov Chains. Pitman Research Notes in Mathematics Series, vol. 240. Longman, Harlow (1991)
Borodin, A., Kleinberg, J., Raghavan, P., Sudan, M., Williamson, D.P.: Adversarial queuing theory. J. ACM 48(1), 13–38 (2001)
Chatterjee, D., Cinquemani, E., Chaloulos, G., Lygeros, J.: Stochastic control up to a hitting time: optimality and rolling-horizon implementation. arXiv:0806.3008 (2008)
Chatterjee, D., Hokayem, P., Lygeros, J.: Stochastic receding horizon control with bounded control inputs: a vector space approach. arXiv:0903.5444 (2009)
Chow, Y.S., Robbins, H., Siegmund, D.: Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, Boston (1971)
Costa, O.L.V., Fragoso, M.D., Marques, R.P.: Discrete-Time Markov Jump Linear Systems. Probability and Its Applications. Springer, Berlin (2005)
Diaconis, P., Freedman, D.: Iterated random functions. SIAM Rev. 41(1), 45–76 (1999) (electronic)
Douc, R., Fort, G., Moulines, E., Soulier, P.: Practical drift conditions for subgeometric rates of convergence. Ann. Appl. Probab. 14(3), 1353–1377 (2004)
Foss, S., Konstantopoulos, T.: An overview of some stochastic stability methods. J. Oper. Res. Soc. Jpn. 47(4), 275–303 (2004)
Håstad, J., Leighton, T., Rogoff, B.: Analysis of backoff protocols for multiple access channels. SIAM J. Comput. 25(4), 740–774 (1996)
Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Applications of Mathematics, vol. 30. Springer, New York (1996)
Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Applications of Mathematics, vol. 42. Springer, New York (1999)
Hobson, D.G.: Volatility misspecification, option pricing and superreplication via coupling. Ann. Appl. Probab. 8(1), 193–205 (1998)
Jahnke, T., Huisinga, W.: Solving the chemical master equation for monomolecular reaction systems analytically. J. Math. Biol. 54(1), 1–26 (2007)
Jarner, S.F., Tweedie, R.L.: Locally contracting iterated functions and stability of Markov chains. J. Appl. Probab. 38(2), 494–507 (2001)
Jiang, Z.-P., Wang, Y.: Input-to-state stability for discrete-time nonlinear systems. Automatica 37(6), 857–869 (2001)
Karatzas, I., Shreve, S.: Brownian Motion and Stochastic Calculus, 2nd edn. Graduate Texts in Mathematics. Springer, Berlin (1991)
Kifer, Y.: Ergodic Theory of Random Transformations. Progress in Probability and Statistics, vol. 10. Birkhäuser, Boston (1986)
Liberzon, D.: Switching in Systems and Control, Systems & Control: Foundations & Applications. Birkhäuser, Boston (2003)
Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise, 2nd edn. Applied Mathematical Sciences, vol. 97. Springer, New York (1994)
Lasota, A., Myjak, J.: On a dimension of measures. Pol. Acad. Sci. Bull. Math. 50(2), 221–235 (2002)
Lasota, A., Szarek, T.: Dimension of measures invariant with respect to the Ważewska partial differential equation. J. Differ. Equ. 196(2), 448–465 (2004)
Maciejowski, J.M.: Predictive Control with Constraints. Prentice Hall, New York (2001)
Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability, 2nd edn. Cambridge University Press, Cambridge (2009)
Pemantle, R., Rosenthal, J.S.: Moment conditions for a sequence with negative drift to be uniformly bounded in L r. Stoch. Process. Appl. 82(1), 143–155 (1999)
Peskir, G., Shiryaev, A.N.: Optimal Stopping and Free-Boundary Problems. Lectures in Mathematics ETH Zürich. Birkhäuser, Basel (2006)
Szarek, T.: Feller processes on nonlocally compact spaces. Ann. Probab. 34(5), 1849–1863 (2006)
van der Schaft, A., Schumacher, H.: An Introduction to Hybrid Dynamical Systems. Lecture Notes in Control and Information Sciences, vol. 251. Springer, Berlin (2000)
Werner, I.: Contractive Markov systems. J. Lond. Math. Soc. 71(1), 236–258 (2005)
Wilkinson, D.J.: Stochastic Modelling for Systems Biology. Chapman & Hall/CRC Mathematical and Computational Biology Series. Chapman & Hall/CRC, London/Boca Raton (2006)
Author information
Authors and Affiliations
Corresponding author
Additional information
Debasish Chatterjee’s research is partially supported by the Swiss National Science foundation grant 200021-122072.
Rights and permissions
About this article
Cite this article
Chatterjee, D., Pal, S. An Excursion-Theoretic Approach to Stability of Discrete-Time Stochastic Hybrid Systems. Appl Math Optim 63, 217–237 (2011). https://doi.org/10.1007/s00245-010-9117-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-010-9117-6