Exploiting Robust Optimization for Interval Probabilistic Bisimulation

  • Ernst Moritz Hahn
  • Vahid HashemiEmail author
  • Holger Hermanns
  • Andrea Turrini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9826)


Verification of PCTL properties of MDPs with convex uncertainties has been investigated recently by Puggelli et al. However, model checking algorithms typically suffer from the state space explosion problem. In this paper, we discuss the use of probabilistic bisimulation to reduce the size of such an MDP while preserving the PCTL properties it satisfies. As a core part, we show that deciding bisimilarity of a pair of states can be encoded as adjustable robust counterpart of an uncertain LP. We show that using affine decision rules, probabilistic bisimulation relation can be approximated in polynomial time. We have implemented our approach and demonstrate its effectiveness on several case studies.


Linear Programming Problem Robust Optimization Uncertain Data Robust Counterpart Computational Tractability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We would like to thank Arkadi Nemirovski (Georgia Institute of Technology) and Daniel Kuhn (EPFL) for many invaluable and insightful discussions. This work is supported by the EU 7th Framework Programme under grant agreements 295261 (MEALS) and 318490 (SENSATION), by the DFG as part of SFB/TR 14 AVACS, by the ERC Advanced Investigators Grant 695614 (POWVER), by the CAS/SAFEA International Partnership Program for Creative Research Teams, by the National Natural Science Foundation of China (Grants No. 61472473, 61532019, 61550110249, 61550110506), by the Chinese Academy of Sciences Fellowship for International Young Scientists, and by the CDZ project CAP (GZ 1023).


  1. 1.
    Gurobi 4.0.2.
  2. 2.
  3. 3.
    PICOS: A Python interface for conic optimization solvers.
  4. 4.
    PRISM model checker.
  5. 5.
    Abate, A., El Ghaoui, L.: Robust model predictive control through adjustable variables: an application to path planning. In: CDC, pp. 2485–2490 (2004)Google Scholar
  6. 6.
    Ben-Tal, A., El Ghaoui, L., Nemirovski, A.: Robust Optimization. Princeton University Press, Princeton (2009)CrossRefzbMATHGoogle Scholar
  7. 7.
    Ben-Tal, A., Goryashko, A., Guslitzer, E., Nemirovski, A.: Adjustable robust solutions of uncertain linear programs. Math. Program. 99(2), 351–376 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Ben-Tal, A., Nemirovski, A.: Robust solutions of uncertain linear programs. Oper. Res. Lett. 25, 1–13 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Benedikt, M., Lenhardt, R., Worrell, J.: LTL model checking of interval Markov chains. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013 (ETAPS 2013). LNCS, vol. 7795, pp. 32–46. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Bertsimas, D., Brown, D.B., Caramanis, C.: Theory and applications of robust optimization. SIAM Rev. 53(3), 464–501 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Billingsley, P.: Probability and Measure. Wiley, New York (1979)zbMATHGoogle Scholar
  12. 12.
    Cattani, S., Segala, R.: Decision algorithms for probabilistic bisimulation. In: Brim, L., Jančar, P., Křetínský, M., Kučera, A. (eds.) CONCUR 2002. LNCS, vol. 2421, pp. 371–385. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Cattani, S., Segala, R., Kwiatkowska, M., Norman, G.: Stochastic transition systems for continuous state spaces and non-determinism. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 125–139. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Chatterjee, K., Sen, K., Henzinger, T.A.: Model-checking \(\omega \)-regular properties of interval Markov chains. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 302–317. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  15. 15.
    Chen, T., Han, T., Kwiatkowska, M.: On the complexity of model checking interval-valued discrete time Markov chains. Inf. Process. Lett. 113(7), 210–216 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Coste, N., Hermanns, H., Lantreibecq, E., Serwe, W.: Towards performance prediction of compositional models in industrial GALS designs. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 204–218. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  17. 17.
    Dantzig, G.B., Madansky, A.: On the solution of two-stage linear programs under uncertainty. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1. pp. 165–176 (1961)Google Scholar
  18. 18.
    Delahaye, B., Katoen, J.P., Larsen, K.G., Legay, A., Pedersen, M.L., Sher, F., Wasowski, A.: New results on abstract probabilistic automata. In: ACSD, pp. 118–127 (2011)Google Scholar
  19. 19.
    Delahaye, B., Larsen, K.G., Legay, A., Pedersen, M.L., Wąsowski, A.: Decision problems for interval Markov chains. In: Dediu, A.-H., Inenaga, S., Martín-Vide, C. (eds.) LATA 2011. LNCS, vol. 6638, pp. 274–285. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  20. 20.
    Fecher, H., Leucker, M., Wolf, V.: Don’t know in probabilistic systems. In: Valmari, A. (ed.) SPIN 2006. LNCS, vol. 3925, pp. 71–88. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  21. 21.
    Ferrer Fioriti, L.M., Hashemi, V., Hermanns, H., Turrini, A.: Deciding probabilistic automata weak bisimulation: theory and practice. Form. Asp. Comput. 28(1), 109–143 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Gebler, D., Hashemi, V., Turrini, A.: Computing behavioral relations for probabilistic concurrent systems. In: Remke, A., Stoelinga, M. (eds.) Stochastic Model Checking. LNCS, vol. 8453, pp. 117–155. Springer, Heidelberg (2014)Google Scholar
  23. 23.
    Givan, R., Leach, S.M., Dean, T.L.: Bounded-parameter Markov decision processes. Artif. Intell. 122(1–2), 71–109 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Goerigk, M.: ROPI-a robust optimization programming interface for C++. Optim. Methods Softw. 29(6), 1261–1280 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Grötschel, M., Lovász, L., Schrijver, A.: The ellipsoid method and its consequences in combinatorial optimization. Combinatorica 1(2), 169–197 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Hahn, E.M., Han, T., Zhang, L.: Synthesis for PCTL in parametric Markov decision processes. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 146–161. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  27. 27.
    Hashemi, V., Hatefi, H., Krčál, J.: Probabilistic bisimulations for PCTL model checking of interval MDPs (extended version). In: SynCoP. EPTCS, vol. 145, pp. 19–33 (2014)Google Scholar
  28. 28.
    Hashemi, V., Hermanns, H., Song, L., Subramani, K., Turrini, A., Wojciechowski, P.: Compositional bisimulation minimization for interval Markov decision processes. In: Dediu, A.-H., Janoušek, J., Martín-Vide, C., Truthe, B. (eds.) LATA 2016. LNCS, vol. 9618, pp. 114–126. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-30000-9_9 CrossRefGoogle Scholar
  29. 29.
    Hermanns, H., Katoen, J.P.: Automated compositional Markov chain generation for a plain-old telephone system. Sci. Comput. Program. 36(1), 97–127 (2000)CrossRefzbMATHGoogle Scholar
  30. 30.
    Iyengar, G.N.: Robust dynamic programming. Math. Oper. Res. 30(2), 257–280 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS, pp. 266–277 (1991)Google Scholar
  32. 32.
    Kanellakis, P.C., Smolka, S.A.: CCS expressions, finite state processes, and three problems of equivalence. Inf. Comput. 86(1), 43–68 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Khachyan, L.G.: A polynomial algorithm in linear programming. Sov. Math. Doklady 20(1), 191–194 (1979)Google Scholar
  35. 35.
    Kozine, I., Utkin, L.V.: Interval-valued finite Markov chains. Reliable Comput. 8(2), 97–113 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  37. 37.
    Löfberg, J.: Automatic robust convex programming. Optim. Methods Softw. 17(1), 115–129 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Nilim, A., El Ghaoui, L.: Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5), 780–798 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Paige, R., Tarjan, R.E.: Three partition refinement algorithms. SIAM J. Comput. 16(6), 973–989 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Puggelli, A.: Formal techniques for the verification and optimal control of probabilistic systems in the presence of modeling uncertainties. Ph.D. thesis, University of California, Berkeley (2014)Google Scholar
  41. 41.
    Puggelli, A., Li, W., Sangiovanni-Vincentelli, A.L., Seshia, S.A.: Polynomial-time verification of PCTL properties of MDPs with convex uncertainties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 527–542. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  42. 42.
    Segala, R.: Modeling and verification of randomized distributed real-time systems. Ph.D. thesis, MIT (1995)Google Scholar
  43. 43.
    Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 394–410. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  44. 44.
    Turrini, A., Hermanns, H.: Polynomial time decision algorithms for probabilistic automata. Inf. Comput. 244, 134–171 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Wolff, E.M., Topcu, U., Murray, R.M.: Robust control of uncertain Markov decision processes with temporal logic specifications. In: CDC, pp. 3372–3379 (2012)Google Scholar
  46. 46.
    Wu, D., Koutsoukos, X.D.: Reachability analysis of uncertain systems using bounded-parameter Markov decision processes. Artif. Intell. 172(8–9), 945–954 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Yi, W.: Algebraic reasoning for real-time probabilistic processes with uncertain information. Formal Techniques in Real-Time and Fault-Tolerant Systems. LNCS, vol. 863, pp. 680–693. Springer, Heidelberg (1994)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Ernst Moritz Hahn
    • 1
  • Vahid Hashemi
    • 2
    • 3
    Email author
  • Holger Hermanns
    • 3
  • Andrea Turrini
    • 1
  1. 1.State Key Laboratory of Computer Science, Institute of SoftwareChinese Academy of SciencesBeijingChina
  2. 2.Max Planck Institute for InformaticsSaarbrückenGermany
  3. 3.Department of Computer ScienceSaarland UniversitySaarbrückenGermany

Personalised recommendations