Skip to main content
SpringerLink
Log in

Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

  • Published:
Applied Mathematics & Optimization Submit manuscript

Abstract

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP’s) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space ℝn. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter ε>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as ε goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as ε goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bensoussan, A.: Perturbation Methods in Optimal Control. Wiley/Gauthier-Villars Series in Modern Applied Mathematics. Wiley, New York (1988). Translated from the French by C. Tomson

    MATH  Google Scholar 

  2. Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete Time Case. Mathematics in Science and Engineering, vol. 139. Academic Press, San Diego (1978)

    MATH  Google Scholar 

  3. Costa, O.L.V., Dufour, F.: Relaxed long run average continuous control of piecewise deterministic Markov processes. In: Proceedings of the European Control Conference, Kos, Greece, July 2007, pp. 5052–5059 (2007)

    Google Scholar 

  4. Costa, O.L.V., Dufour, F.: Average control of piecewise deterministic Markov processes. SIAM J. Control Optim. 48(7), 4262–4291 (2010)

    Article  MathSciNet  Google Scholar 

  5. Davis, M.H.A.: Markov Models and Optimization. Chapman and Hall, London (1993)

    MATH  Google Scholar 

  6. Dempster, M.A.H., Ye, J.J.: Necessary and sufficient optimality conditions for control of piecewise deterministic Markov processes. Stoch. Stoch. Rep. 40(3–4), 125–145 (1992)

    MathSciNet  MATH  Google Scholar 

  7. Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Applications of Mathematics, vol. 30. Springer, New York (1996)

    Google Scholar 

  8. Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Applications of Mathematics, vol. 42. Springer, New York (1999)

    MATH  Google Scholar 

  9. Kokotović, P.V.: Applications of singular perturbation techniques to control problems. SIAM Rev. 26(4), 501–550 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  10. Kokotović, P.V., Bensoussan, A., Blankenship, G.: Singular Perturbations and Asymptotic Analysis in Control Systems. Lecture Notes in Control and Inform. Sci., vol. 90. Springer, Berlin (1987)

    Book  MATH  Google Scholar 

  11. Kokotović, P.V., Khalil, H.K., O’Reilly, J.: Singular Perturbation Methods in Control: Analysis and Design. Academic Press, Harcourt Brace Jovanovich, San Diego (1986)

    MATH  Google Scholar 

  12. Kushner, H.J.: Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems. Systems & Control: Foundations & Applications, vol. 3. Birkhäuser Boston, Cambridge (1990)

    MATH  Google Scholar 

  13. Yin, G., Zhang, Q.: Control of dynamic systems under the influence of singularly perturbed Markov chains. J. Math. Anal. Appl. 216(1), 343–367 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  14. Yin, G.G., Zhang, Q.: Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach. Applications of Mathematics, vol. 37. Springer, New York (1998)

    MATH  Google Scholar 

  15. Yin, G.G., Zhang, Q.: Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications. Applications of Mathematics, vol. 55. Springer, New York (2005)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Engenharia de Telecomunicações e Controle, Escola Politécnica da Universidade de São Paulo, 05508 900, São Paulo, Brazil

    O. L. V. Costa

  2. IMB, Institut Mathématiques de Bordeaux, INRIA Bordeaux Sud Ouest, Team: CQFD, Universite Bordeaux I, 351 cours de la Liberation, 33405, Talence Cedex, France

    F. Dufour

Authors
  1. O. L. V. Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Dufour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to O. L. V. Costa.

Additional information

O.L.V. Costa received financial support from CNPq (Brazilian National Research Council), grant 301067/09-0.

F. Dufour was supported by ARPEGE program of the French National Agency of Research (ANR), project “FAUTOCOES”, number ANR-09-SEGI-004.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Costa, O.L.V., Dufour, F. Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes. Appl Math Optim 63, 357–384 (2011). https://doi.org/10.1007/s00245-010-9124-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-010-9124-7

Keywords

Search

Navigation