Play to Test

  • Andreas Blass
  • Yuri Gurevich
  • Lev Nachmanson
  • Margus Veanes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3997)


Testing tasks can be viewed (and organized!) as games against nature. We study reachability games in the context of testing. Such games are ubiquitous. A single industrial test suite may involve many instances of a reachability game. Hence the importance of optimal or near optimal strategies for reachability games. One can use linear programming or the value iteration method of Markov decision process theory to find optimal strategies. Both methods have been implemented in an industrial model-based testing tool, Spec Explorer, developed at Microsoft Research.


Goal State Markov Decision Process Passive State Reasonable Strategy Cost Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Spec Explorer. (released January 2005), URL:
  2. 2.
    Alur, R., Courcoubetis, C., Yannakakis, M.: Distinguishing tests for nondeterministic and probabilistic machines. In: Proc. 27th Ann. ACM Symp. Theory of Computing, pp. 363–372 (1995)Google Scholar
  3. 3.
    Alur, R., Henzinger, T.A., Kupferman, O., Vardi, M.: Alternating refinement relations. In: Sangiorgi, D., de Simone, R. (eds.) CONCUR 1998. LNCS, vol. 1466, pp. 163–178. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  4. 4.
    Artho, C., Drusinsky, D., Goldberg, A., Havelund, K., Lowry, M., Pasareanu, C., Rosu, G., Visser, W.: Experiments with test case generation and runtime analysis. In: Börger, E., Gargantini, A., Riccobene, E. (eds.) ASM 2003. LNCS, vol. 2589, pp. 87–107. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Barnett, M., Grieskamp, W., Nachmanson, L., Schulte, W., Tillmann, N., Veanes, M.: Towards a tool environment for model-based testing with AsmL. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 264–280. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Barnett, M., Leino, R., Schulte, W.: The Spec# programming system: An overview. In: Barthe, G., Burdy, L., Huisman, M., Lanet, J.-L., Muntean, T. (eds.) CASSIS 2004. LNCS, vol. 3362, pp. 49–69. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Blass, A., Gurevich, Y., Nachmanson, L., Veanes, M.: Play to test. Technical Report MSR-TR-2005-04, Microsoft Research (January 2005) (Revised April 5, 2005)Google Scholar
  8. 8.
    Campbell, C., Grieskamp, W., Nachmanson, L., Schulte, W., Tillmann, N., Veanes, M.: Model-based testing of object-oriented reactive systems with Spec Explorer. Technical Report MSR-TR-2005-59, Microsoft Research (2005)Google Scholar
  9. 9.
    Chatterjee, K., Jurdziński, M., Henzinger, T.: Simple stochastic parity games. In: Baaz, M., Makowsky, J.A. (eds.) CSL 2003. LNCS, vol. 2803, pp. 100–113. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    de Alfaro, L.: Computing minimum and maximum reachability times in probabilistic systems. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 66–81. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  11. 11.
    de Alfaro, L.: Game models for open systems. In: Dershowitz, N. (ed.) Verification: Theory and Practice. LNCS, vol. 2772, pp. 269–289. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Filar, J., Vrieze, K.: Competitive Markov decision processes. Springer, New York (1996)CrossRefzbMATHGoogle Scholar
  13. 13.
    Grieskamp, W., Gurevich, Y., Schulte, W., Veanes, M.: Generating finite state machines from abstract state machines. In: ISSTA 2002. Software Engineering Notes, vol. 27, pp. 112–122. ACM, New York (2002)Google Scholar
  14. 14.
    Gujiwara, S., Bochman, G.V.: Testing non-deterministic state machines with fault-coverage. In: Kroon, J., Heijunk, R., Brinksma, E. (eds.) Protocol Test Systems, pp. 363–372 (1992)Google Scholar
  15. 15.
    Gurevich, Y.: Evolving Algebras 1993: Lipari Guide. In: Börger, E. (ed.) Specification and Validation Methods, pp. 9–36. Oxford University Press, Oxford (1995)Google Scholar
  16. 16.
    Gurevich, Y., Rossman, B., Schulte, W.: Semantic essence of AsmL. In: Theoretical Computer Science (2005), special issue dedicated to FMCO 2003, preliminary version available as Microsoft Research Technical Report MSR-TR-2004-27 (to appear)Google Scholar
  17. 17.
    Hartman, A., Nagin, K.: Model driven testing - AGEDIS architecture interfaces and tools. In: 1st European Conference on Model Driven Software Engineering, Nuremberg, Germany, pp. 1–11 (December 2003)Google Scholar
  18. 18.
    Jard, C., Jéron, T.: TGV: theory, principles and algorithms. In: The Sixth World Conference on Integrated Design and Process Technology, IDPT 2002, Pasadena, California (June 2002)Google Scholar
  19. 19.
    Kuliamin, V.V., Petrenko, A.K., Kossatchev, A.S., Bourdonov, I.B.: UniTesK: Model based testing in industrial practice. In: 1st European Conference on Model Driven Software Engineering, Nuremberg, Germany, pp. 55–63 (December 2003)Google Scholar
  20. 20.
    Nachmanson, L., Veanes, M., Schulte, W., Tillmann, N., Grieskamp, W.: Optimal strategies for testing nondeterministic systems. In: ISSTA 2004. Software Engineering Notes, vol. 29, pp. 55–64. ACM, New York (2004)Google Scholar
  21. 21.
    Petrenko, A., Yevtushenko, N., Bochmann, G.v.: Testing deterministic implementations from nondeterministic FSM specifications. In: Baumgarten, B., Burkhardt, H.-J., Giessler, A. (eds.) IFIP TC6 9th International Workshop on Testing of Communicating Systems, pp. 125–140. Chapman & Hall, Boca Raton (1996)CrossRefGoogle Scholar
  22. 22.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Series in Probability and Mathematical Statistics. A Wiley-Interscience, New York (1994)CrossRefzbMATHGoogle Scholar
  23. 23.
    Tasiran, S., Qadeer, S.: Runtime refinement checking of concurrent data structures. Electronic Notes in Theoretical Computer Science 113, 163–179 (January 2005), Proceedings of the Fourth Workshop on Runtime Verification (RV 2004)Google Scholar
  24. 24.
    Tretmans, J., Brinksma, E.: TorX: Automated model based testing. In: 1st European Conference on Model Driven Software Engineering, Nuremberg, Germany, pp. 31–43 (December 2003)Google Scholar
  25. 25.
    Veanes, M., Campbell, C., Schulte, W., Tillmann, N.: Online testing with model programs. In: ESEC/FSE 2005 (2005)Google Scholar
  26. 26.
    von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)zbMATHGoogle Scholar
  27. 27.
    Yannakakis, M.: Testing, optimization, and games. In: Proceedings of the Nineteenth Annual IEEE Symposium on Logic In Computer Science, LICS 2004, pp. 78–88. IEEE, Los Alamitos (2004)CrossRefGoogle Scholar
  28. 28.
    Yi, W., Larsen, K.G.: Testing probabilistic and nondeterministic processes. In: Testing and Verification XII, pp. 347–361. North Holland, Amsterdam (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Andreas Blass
    • 1
  • Yuri Gurevich
    • 2
  • Lev Nachmanson
    • 2
  • Margus Veanes
    • 2
  1. 1.University of MichiganAnn ArborUSA
  2. 2.Microsoft ResearchRedmondUSA

Personalised recommendations