Energy-Utility Quantiles

  • Christel Baier
  • Marcus Daum
  • Clemens Dubslaff
  • Joachim Klein
  • Sascha Klüppelholz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8430)


The concept of quantiles is well-known in statistics, but its benefits for the formal quantitative analysis of probabilistic systems have been noticed only recently. To compute quantiles in Markov decision processes where the objective is a probability constraint for an until (i.e., constrained reachability) property with an upper reward bound, an iterative linear-programming (LP) approach has been proposed in a recent paper. We consider here a more general class of quantiles with probability or expectation objectives, allowing to reason about the trade-off between costs in terms of energy and some utility measure. We show how the iterative LP approach can be adapted for these types of quantiles and propose another iterative approach that decomposes the LP to be solved into smaller ones. This algorithm has been implemented and evaluated in case studies for quantiles where the objective is a probability constraint for until properties with upper reward bounds.


Markov Decision Process Reward Function Probability Constraint Path Property Nondeterministic Choice 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andova, S., Hermanns, H., Katoen, J.-P.: Discrete-time rewards model-checked. In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, pp. 88–104. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Baier, C., Daum, M., Dubslaff, C., Klein, J., Klüppelholz, S.: Energy-Utility Quantiles. Technical report, TU Dresden (2014),
  3. 3.
    Baier, C., et al.: Waiting for locks: How long does it usually take? In: Stoelinga, M., Pinger, R. (eds.) FMICS 2012. LNCS, vol. 7437, pp. 47–62. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Baier, C., Dubslaff, C., Klein, J., Klüppelholz, S., Wunderlich, S.: Probabilistic Model Checking for Energy-Utility Analysis. Festschrift (to appear 2014)Google Scholar
  5. 5.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT Press (2008)Google Scholar
  6. 6.
    Bertsekas, D., Tsitsiklis, J.: An analysis of stochastic shortest path problems. Mathematics of Operations Research 16(3), 580–595 (1991)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Bianco, A., de Alfaro, L.: Model checking of probabilistic and non-deterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  8. 8.
    Ciesinski, F., Baier, C., Größer, M., Klein, J.: Reduction techniques for model checking Markov decision processes. In: QEST 2008, pp. 45–54. IEEE (2008)Google Scholar
  9. 9.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, Department of Computer Science (1997)Google Scholar
  10. 10.
    de Alfaro, L.: Computing minimum and maximum reachability times in probabilistic systems. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 66–81. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  11. 11.
    Etessami, K., Kwiatkowska, M., Vardi, M., Yannakakis, M.: Multi-objective model checking of Markov decision processes. Logical Methods in Computer Science 4(4) (2008)Google Scholar
  12. 12.
    Forejt, V., Kwiatkowska, M., Norman, G., Parker, D.: Automated verification techniques for probabilistic systems. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 53–113. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Haverkort, B.: Performance of Computer Communication Systems: A Model-Based Approach. Wiley (1998)Google Scholar
  14. 14.
    Hinton, A., Kwiatkowska, M., Norman, G., Parker, D.: PRISM: A tool for automatic verification of probabilistic systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 441–444. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Katoen, J.-P., Zapreev, I., Hahn, E., Hermanns, H., Jansen, D.: The ins and outs of the probabilistic model checker MRMC. Perform. Eval. 68(2) (2011)Google Scholar
  16. 16.
    Kulkarni, V.: Modeling and Analysis of Stochastic Systems. Chapman & Hall (1995)Google Scholar
  17. 17.
    Kwiatkowska, M., Norman, G., Parker, D.: The PRISM benchmark suite. In: QEST 2012. IEEE Computer Society (2012)Google Scholar
  18. 18.
    Kwon, Y., Agha, G.: A Markov reward model for software reliability. In: IPDPS 2007, pp. 1–6. IEEE (2007)Google Scholar
  19. 19.
    Laroussinie, F., Sproston, J.: Model checking durational probabilistic systems. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 140–154. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  20. 20.
    Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (1994)Google Scholar
  21. 21.
    Serfling, R.J.: Approximation Theorems of Mathematical Statistics. Wiley (1980)Google Scholar
  22. 22.
    Ummels, M., Baier, C.: Computing quantiles in Markov reward models. In: Pfenning, F. (ed.) FOSSACS 2013. LNCS, vol. 7794, pp. 353–368. Springer, Heidelberg (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Christel Baier
    • 1
  • Marcus Daum
    • 1
  • Clemens Dubslaff
    • 1
  • Joachim Klein
    • 1
  • Sascha Klüppelholz
    • 1
  1. 1.Institute for Theoretical Computer ScienceTechnische Universität DresdenGermany

Personalised recommendations