Reachability Analysis for Controlled Discrete Time Stochastic Hybrid Systems

  • Saurabh Amin
  • Alessandro Abate
  • Maria Prandini
  • John Lygeros
  • Shankar Sastry
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3927)


A model for discrete time stochastic hybrid systems whose evolution can be influenced by some control input is proposed in this paper. With reference to the introduced class of systems, a methodology for probabilistic reachability analysis is developed that is relevant to safety verification. This methodology is based on the interpretation of the safety verification problem as an optimal control problem for a certain controlled Markov process. In particular, this allows to characterize through some optimal cost function the set of initial conditions for the system such that safety is guaranteed with sufficiently high probability. The proposed methodology is applied to the problem of regulating the average temperature in a room by a thermostat controlling a heater.


Optimal Control Problem Control Input Discrete State Reachability Analysis Safe Policy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bujorianu, M.L., Lygeros, J.: Reachability questions in piecewise deterministic Markov processes. In: Maler, O., Pnueli, A. (eds.) HSCC 2003. LNCS, vol. 2623, pp. 126–140. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Hu, J., Prandini, M., Sastry, S.: Probabilistic safety analysis in three-dimensional aircraft flight. In: Proc. of the IEEE Conf. on Decision and Control (2003)Google Scholar
  3. 3.
    Prajna, S., Jadbabaie, A., Pappas, G.: Stochastic safety verification using barrier certificates. In: Proc. of the IEEE Conf. on Decision and Control (2004)Google Scholar
  4. 4.
    Hu, J., Prandini, M., Sastry, S.: Aircraft conflict prediction in the presence of a spatially correlated wind field. IEEE Trans. on Intelligent Transportation Systems 6(3), 326–340 (2005)CrossRefGoogle Scholar
  5. 5.
    Davis, M.H.A.: Markov Models and Optimization. Chapman & Hall, London (1993)CrossRefMATHGoogle Scholar
  6. 6.
    Ghosh, M.K., Araposthasis, A., Marcus, S.I.: Ergodic control of switching diffusions. SIAM Journal of Control and Optimization 35(6), 1952–1988 (1997)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Hu, J., Lygeros, J., Sastry, S.: Towards a theory of stochastic hybrid systems. In: Lynch, N., Krogh, B. (eds.) Hybrid Systems: Computation and Control. LNCS, vol. 1790, pp. 160–173. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Lygeros, J., Watkins, O.: Stochastic reachability for discrete time systems: an application to aircraft collision avoidance. In: Proc. of the IEEE Conf. on Decision and Control (2003)Google Scholar
  9. 9.
    Bujorianu, M., Lygeros, J.: General stochastic hybrid systems: Modelling and optimal control. In: Proc. of the IEEE Conf. on Decision and Control (2004)Google Scholar
  10. 10.
    Puterman, M.: Markov decision processes. John Wiley & Sons, Inc., Chichester (1994)CrossRefMATHGoogle Scholar
  11. 11.
    Bertsekas, D.P., Shreve, S.E.: Stochastic optimal control: the discrete-time case. Athena Scientific (1996)Google Scholar
  12. 12.
    Malhame, R., Chong, C.Y.: Electric load model synthesis by diffusion approximation of a high-order hybrid-state stochastic system. IEEE Transactions on Automatic Control AC-30(9), 854–860 (1985)CrossRefMATHGoogle Scholar
  13. 13.
    Milstein, G.: Numerical Integration of Stochastic Differential Equations. Kluver Academic Press, London (1995)CrossRefMATHGoogle Scholar
  14. 14.
    Kumar, P.R., Varaiya, P.P.: Stochastic Systems: Estimation, Identification, and Adaptive Control. Prentice Hall, Inc., New Jersey (1986)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Saurabh Amin
    • 1
  • Alessandro Abate
    • 1
  • Maria Prandini
    • 2
  • John Lygeros
    • 3
  • Shankar Sastry
    • 1
  1. 1.University of California at BerkeleyBerkeleyUSA
  2. 2.Politecnico di MilanoMilanoItaly
  3. 3.University of PatrasPatrasGreece

Personalised recommendations