Discrete-Time Rewards Model-Checked

  • Suzana Andova
  • Holger Hermanns
  • Joost-Pieter Katoen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2791)


This paper presents a model-checking approach for analyzing discrete-time Markov reward models. For this purpose, the temporal logic probabilistic CTL is extended with reward constraints. This allows to formulate complex measures – involving expected as well as accumulated rewards – in a precise and succinct way. Algorithms to efficiently analyze such formulae are introduced. The approach is illustrated by model-checking a probabilistic cost model of the IPv4 zeroconf protocol for distributed address assignment in ad-hoc networks.


Model Check Atomic Proposition Reward Structure Strongly Connect Component Path Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aziz, A., Sanwal, K., Singhal, V., Brayton, R.: Verifying continuous time Markov chains. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 269–276. Springer, Heidelberg (1996)Google Scholar
  2. 2.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: On the logical characterization of performability properties. In: Welzl, E., Montanari, U., Rolim, J.D.P. (eds.) ICALP 2000. LNCS, vol. 1853, pp. 780–792. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  3. 3.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE Transactions on Software Engineering 29(6), 524–545 (2003)CrossRefGoogle Scholar
  4. 4.
    Bernardo, M.: An algebra-based method to associate rewards with EMPA terms. In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (eds.) ICALP 1997. LNCS, vol. 1256, pp. 358–368. Springer, Heidelberg (1997)Google Scholar
  5. 5.
    Bohnenkamp, H., van der Stok, P., Hermanns, H., Vaandrager, F.W.: Cost optimisation of the IPv4 zeroconf protocol. In: Intl. Conf. on Dependable Systems and Networks, IEEE CS Press, Los Alamitos (2003) (to appear)Google Scholar
  6. 6.
    Cheshire, S., Adoba, B., Guttman, E.: Dynamic configuration of IPv4 link-local addresses (2002) (draft), available at
  7. 7.
    Ciardo, G., Muppala, J., Trivedi, K.S.: SPNP: Stochastic Petri Net Package. In: Proc. 3rd Int’l. Workshop on Petri Nets and Performance Models, pp. 142–151 (1989)Google Scholar
  8. 8.
    Clark, G., Gilmore, S., Hillston, J.: Specifying performance measures for PEPA. In: Katoen, J.-P. (ed.) AMAST-ARTS 1999, ARTS 1999, and AMAST-WS 1999. LNCS, vol. 1601, pp. 211–227. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  9. 9.
    Clarke, E., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (1999)Google Scholar
  10. 10.
    de Alfaro, L.: How to specify and verify the long-run average behavior of probabilistic systems. In: IEEE Symp. on Logic in Computer Science, pp. 454-465 (1998)Google Scholar
  11. 11.
    Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Form. Asp. of Comp. 6, 512–535 (1994)zbMATHCrossRefGoogle Scholar
  12. 12.
    Haverkort, B., Cloth, L., Hermanns, H., Katoen, J.-P., Baier, C.: Model checking performability properties. In: Intl. Conf. on Dependable Systems and Networks, pp. 103–113 (2002)Google Scholar
  13. 13.
    Kemeny, J.G., Snell, J.L.: Finite Markov Chains. Van Nostrand (1960)Google Scholar
  14. 14.
    Kulkarni, V.G.: Modeling and Analysis of Stochastic Systems. Chapman and Hall, Boca Raton (1995)zbMATHGoogle Scholar
  15. 15.
    Obal, W.D., Sanders, W.H.: State-space support for path-based reward variables. Performance Evaluation 35(3-4), 233–251 (1999)zbMATHCrossRefGoogle Scholar
  16. 16.
    Qureshi, M.A., Sanders, W.H.: Reward model solution method with impulse and rate rewards: An algorithm and numerical results. Performance Evaluation 20, 413–436 (1994)zbMATHCrossRefGoogle Scholar
  17. 17.
    Reibman, A., Trivedi, K.: Transient analysis of cumulative measures of Markov model behavior. Commun. Statist. Stochastic Models 5(4), 683–710 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Smith, R.M., Trivedi, K.S., Ramesh, A.V.: Performability analysis: measures, an algorithm and a case study. IEEE Tr. on Computers 37(4), 406–417 (1988)CrossRefGoogle Scholar
  19. 19.
    Sokolsky, O., Philippou, A., Lee, I., Christou, K.: Modeling and analysis of power-aware systems. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 409–425. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  20. 20.
    Voeten, J.P.M.: Performance evaluation with temporal rewards. Performance Evaluation 50, 189–218 (2002)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Suzana Andova
    • 1
  • Holger Hermanns
    • 1
    • 2
  • Joost-Pieter Katoen
    • 1
  1. 1.Formal Methods and Tools Group, Department of Computer ScienceUniversity of TwenteEnschedeThe Netherlands
  2. 2.Department of Computer Science Saarland UniversitySaarbrückenGermany

Personalised recommendations