Abstract
This paper presents an analytic, systematic approach to handle quadratic functionals associated with Markov jump linear systems with general jumping state. The Markov chain is finite state, but otherwise general, possibly reducible and periodic. We study how the second moment dynamics are affected by the additive noise and the asymptotic behaviour, either oscillatory or invariant, of the Markov chain. The paper comprises a series of evaluations that lead to a tight two-sided bound for quadratic cost functionals. A tight two-sided bound for the norm of the second moment of the system is also obtained. These bounds allow us to show that the long-run average cost is well defined for system that are stable in the mean square sense, in spite of the periodic behaviour of the chain and taking into consideration that it may not be unique, as it may depend on the initial distribution. We also address the important question of approximation of the long-run average cost via adherence of finite horizon costs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Costa OLV, Fragoso MD (1993) Stability results for discrete-time linear systems with Markovian jumping parameters. J Math Anal Appl 179: 154–178
Ji Y, Chizeck HJ (1990) Jump linear quadratic Gaussian control: steady-state solution and testable conditions. Control Theory Adv Technol 6(3): 289–319
Costa OLV, Fragoso MD, Marques RP (2005) Discrete-time Markovian jump linear systems. Springer, New York
Zampolli F (2006) Optimal monetary policy in a regime-switching economy: the response to abrupt shifts in exchange rate dynamics. J Econ Dynam Control 30: 1527–1567
Costa OLV, de Paulo WL (2007) Indefinite quadratic with linear costs optimal control of Markov jump with multiplicative noise systems. Automatica 43: 587–597
Khanbaghi M, Malhame RP, Perrier M (2002) Optimal white water and broke recirculation policies in paper mills via jump linear quadratic control. IEEE Trans Automat Control 10(4): 578–588
Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer, New York
Arapostathis A, Borkar VS, Fernández-Gaucherand E, Ghosh MK, Marcus SI (1993) Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J Control Optim 31(2): 282–344
Grimm G, Messina MJ, Tuna SE, Teel AR (2005) Model predictive control: for want of a local control Lyapunov function, all is not lost. IEEE Trans Automat Control 50: 546–558
Costa EF, do Val JBR (2009) Uniform approximation of infinite horizon control problems for nonlinear systems and stability of the approximating controls. IEEE Trans Automat Control 54(4): 881–886
Costa EF, do Val JBR (2006) Obtaining stabilizing stationary controls via finite horizon cost. In: Proc. American Control Conference, Minneapolis, Minnesota, pp 4297–4302
Jadbabaie A, Hauser J (2005) On the stability of receding horizon control with a general terminal cost. IEEE Trans Automat Control 50: 674–678
Costa EF, do Val JBR, Fragoso MD (2005) A new approach to detectability of discrete-time Markov jump linear systems. SIAM J Control Optim 43(6): 2132–2156
Çinlar E (1975) Introduction to stochastic processes. Prentice Hall, New York
Costa OLV, Fragoso MD (1995) Discrete-time LQ-optimal control problems for finite Markov jump parameters systems. IEEE Trans Automat Control 40: 2076–2088
Author information
Authors and Affiliations
Corresponding author
Additional information
Research supported by FAPESP Grants 03/06736-7 and 04/06947-0 and by the CNPq Grants 304429/2007-4, 304856/2007-0 and 306466/2010-4.
Rights and permissions
About this article
Cite this article
Costa, E.F., Vargas, A.N. & do Val, J.B.R. Quadratic costs and second moments of jump linear systems with general Markov chain. Math. Control Signals Syst. 23, 141–157 (2011). https://doi.org/10.1007/s00498-011-0064-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00498-011-0064-9