Skip to main content
SpringerLink
Log in

Discrete-Time Hybrid Control in Borel Spaces

  • Published:
Applied Mathematics & Optimization Submit manuscript

Abstract

A discrete-time hybrid control model with Borel state and action spaces is introduced. In this type of models, the dynamic of the system is composed by two sub-dynamics affecting the evolution of the state; one is of a standard-type that runs almost every time and another is of a special-type that is active under special circumstances. The controller is able to use two different type of actions, each of them is applied to each of the two sub-dynamics, and the activations of these sub-dynamics are possible according to an activation rule that can be handled by the controller. The aim for the controller is to find a control policy, containing a mix of actions (of either standard- or special-type), with the purpose of minimizing an infinite-horizon discounted cost criterion whose discount factor is dependent on the state-action history and may be equal to one at some stages. Two different sets of conditions are proposed to guarantee (i) the finiteness of the cost criterion, (ii) the characterization of the optimal value function and (iii) the existence of optimal control policies; to do so, we employ the dynamic programming approach. A useful characterization that signalizes the accurate times between changes of sub-dynamics in terms of the so-named contact set is also provided. Finally, we introduce two examples that illustrate our results and also show that control models such as discrete-time impulse control models and discrete-time switching control models become special cases of our present hybrid model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. a.k.a. continuous or regular.

  2. a.k.a. discrete or impulsive.

  3. So-called usual or traditional sub-dynamic.

  4. So-called impulse-type or event-driven sub-dynamic.

  5. Or conventional control model.

References

  1. Abate, A., Amin, S., Prandini, M., Lygeros, J., Sastry, S.: Reachability analysis for controlled discrete-time stochastic hybrid systems. In: Hespanha, J., Tiwari A. (eds.) Lecture Notes in Computer Science, vol. 3927, pp. 49–63. Springer, Berlin (2006)

  2. Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44, 2724–2734 (2008)

    Article  MathSciNet  Google Scholar 

  3. Aliprantis, C.D., Border, K.C.: Infinite Dimensional Analysis. Springer, New York (2006)

    MATH  Google Scholar 

  4. Bensoussan, A., Lions, J.L.: Applications of Variational Inequalities in Stochastic Control. North-Holland, Amsterdam (1982) (pages are referred to the French edition)

  5. Bensoussan, A., Lions, J.L.: Impulse Control and Quasi-Variational Inequalities. Gauthier-Villars, Paris (1984) (pages are referred to the French edition)

  6. Bensoussan, A.: Dynamic Programming and Inventory Control. IOS Press, Amsterdam (2011)

    MATH  Google Scholar 

  7. Bensoussan, A., Menaldi, J.L.: Hybrid control and dynamic programming. Dyn. Contin. Discret. Impuls. Syst. 3, 395–442 (1997)

    MathSciNet  MATH  Google Scholar 

  8. Bensoussan, A., Menaldi, J.L.: Stochastic hybrid control. J. Math. Anal. Appl. 249, 261–288 (2000)

    Article  MathSciNet  Google Scholar 

  9. Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific, Belmont, MA (1996)

    MATH  Google Scholar 

  10. Borkar, V.S., Ghosh, M.K., Sahay, P.: Optimal control of a stochastic hybrid system with discounted cost. J. Optim. Theory Appl. 101, 557–580 (1999)

    Article  MathSciNet  Google Scholar 

  11. Branicky, M.S.: Studies in hybrid systems: Modeling, analysis, and control, Thesis MIT, LIDS-TH-2304 (1995), Cambridge, USA

  12. Branicky, M.S., Borkar, V.S., Mitter, S.K.: A unified framework of hybrid control: model and optimal control theory. IEEE Trans. Autom. Control 43, 31–45 (1998)

    Article  MathSciNet  Google Scholar 

  13. Goebel, R., Sanfelice, R.G., Teel, A.R.: Hybrid Dynamical Systems: Modeling, Stability, and Robustness. Princeton University Press, Princeton, NJ (2012)

    Book  Google Scholar 

  14. Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)

    Book  Google Scholar 

  15. Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999)

    Book  Google Scholar 

  16. Jasso-Fuentes, H., Menaldi, J.L., Prieto-Rumeau, T., Robin, M.: Discrete-time hybrid control in Borel spaces: average cost optimality criterion. J. Math. Anal. Appl. 462, 1695–1713 (2018)

    Article  MathSciNet  Google Scholar 

  17. Kallenberg, O.: Foundations of Modern Probability, 2nd edn. Springer, New York (2002)

    Book  Google Scholar 

  18. Lygeros, J.: An overview of hybrid systems control. In: A. Hristu-Varsakelis, D., Levine W.S. (eds.) Handbook of Networked and Embedded Control Systems, vol. IV, pp. 519–537. Birkhäuser, Boston (2005)

  19. Menaldi, J.L.: Optimal impulse control problems for degenerate diffusions with jumps. Acta Appl. Math. 8, 165–198 (1987)

    Article  MathSciNet  Google Scholar 

  20. Menaldi, J.L., Blankenship, G.L.: Optimal stochastic scheduling of power generation systems with scheduling delays and large cost differentials. SIAM J. Control Optim. 22, 121–132 (1984)

    Article  MathSciNet  Google Scholar 

  21. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (1994)

    Book  Google Scholar 

  22. Quadrat, J.P.: Existence de solution et algorithme de resolutions numériques de problémes stochastiques degenerées ou non. SIAM J. Control Optim. 18, 199–226 (1980)

    Article  MathSciNet  Google Scholar 

  23. Rieder, U.: Measurable selection theorems for optimization problems. Manuscr. Math. 24, 115–131 (1978)

    Article  MathSciNet  Google Scholar 

  24. Robin, M.: Controle impulsionnel des processus de Markov. Thesis INRIA, TE-035, Paris, France (1977)

  25. Robin, M.: Contrle impulsionnel avec retard pour les processus de diffusion. CRAS 282, 463–466 (1976)

    MATH  Google Scholar 

  26. Stettner, L.: Discrete time adaptive impulsive control theory. Stoch. Process. Appl. 23, 177–197 (1986)

    Article  MathSciNet  Google Scholar 

  27. Stettner, L.: On some stopping and impulsive control problems with a general discount rate criterion. Probab. Math. Stat. 10, 223–245 (1989)

    MathSciNet  MATH  Google Scholar 

  28. Summers, S., Lygeros, J.: Verification of discrete time stochastic hybrid systems: a stochastic reach-avoid decision problem. Automatica 46, 1951–1961 (2010)

    Article  MathSciNet  Google Scholar 

  29. Yin, G.G., Zhang, Q.: Discrete-Time Markov Chains. Two-Time-Scale Methods and Applications. Springer, New York (2005)

    MATH  Google Scholar 

  30. Yin, G.G., Zhu, C.: Hybrid Switching Diffusions. Springer, New York (2010)

    Book  Google Scholar 

  31. Zhang, W., Jianghai, H., Abate, A.: Infinite-horizon switched LQR problems in discrete time: a suboptimal algorithm with performance analysis. IEEE Trans. Autom. Control 57, 1815–1821 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departmento de Matemáticas, CINVESTAV-IPN, 07000, Mexico, DF, Mexico

    Héctor Jasso-Fuentes

  2. Department of Mathematics, Wayne State University, Detroit, MI, 48202, USA

    José-Luis Menaldi

  3. Department of Statistics and Operations Research, UNED, Madrid, Spain

    Tomás Prieto-Rumeau

Authors
  1. Héctor Jasso-Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José-Luis Menaldi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomás Prieto-Rumeau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomás Prieto-Rumeau.

Additional information

This research was funded in part by CONACyT fellowship 266252 “Estancias sabáticas en el extranjero 2015”, by CONACyT Grant 238045, and by Grant MTM2016-75497-P from the Spanish Ministerio de Economía y Competitividad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jasso-Fuentes, H., Menaldi, JL. & Prieto-Rumeau, T. Discrete-Time Hybrid Control in Borel Spaces. Appl Math Optim 81, 409–441 (2020). https://doi.org/10.1007/s00245-018-9503-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-018-9503-z

Keywords

Mathematics Subject Classification

Search

Navigation