Testing, Optimizaton, and Games

  • Mihalis Yannakakis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3142)


We discuss algorithmic problems arising in the testing of reactive systems, i.e. systems that interact with their environment. The goal is to design test sequences so that we can deduce desired information about the given system under test, such as whether it conforms to a given specification model, or whether it satisfies given requirement properties. Test generation can be approached from different points of view – as an optimization problem of minimizing cost and maximizing the effectiveness of the tests; as a game between tester and system under test; or as a learning problem. We touch on some of these aspects and related algorithmic questions.


State Machine Model Check Input Sequence Output Sequence Winning Strategy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aho, A.V., Dahbura, A.T., Lee, D., Uyar, M.U.: An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours. IEEE Trans. on Communication 39(11), 1604–1615 (1991)CrossRefGoogle Scholar
  2. 2.
    Alur, R., Courcoubetis, C., Yannakakis, M.: Distinguishing tests for nondeterministic and probabilistic machines. In: Proc. 27th Ann. ACM Symp. on Theory of Computing, pp. 363–372 (1995)Google Scholar
  3. 3.
    Alur, R., Yannakakis, M.: Model checking of hierarchical state machines. ACM Trans. on Progr. Languages and Systems 23(3), 273–303 (2001)CrossRefGoogle Scholar
  4. 4.
    Angluin, D.: Learning regular sets from queries and counterexamples. Inform. and Comp. 75, 87–106 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Apfelbaum, L.: Automated functional test generation. In: Proc. IEEE Autotestcon Conference (1995)Google Scholar
  6. 6.
    Booch, G., Jacobson, I., Rumbaugh, J.: Unified Modeling Language User Guide. Addison-Wesley, Reading (1997)Google Scholar
  7. 7.
    Brinksma, E., Tretmans, J.: Testing transition systems: An annotated bibliography. In: Cassez, F., Jard, C., Rozoy, B., Dermot, M. (eds.) MOVEP 2000. LNCS, vol. 2067, pp. 187–195. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  8. 8.
    Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. on Software Engineering SE-4(3), 178–187 (1978)CrossRefGoogle Scholar
  9. 9.
    Clarke, E.M., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (2000)Google Scholar
  10. 10.
    Edmonds, J., Johnson, E.L.: Matching, Euler tours and the Chinese postman. Mathematical Programming 5, 88–124 (1973)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Etessami, K., Yannakakis, M.: From Rule-Based to Automata-Based Testing. In: Proc. IFIP Joint Intl. Conf. FORTE XIII-PSTV XX, pp. 53–68. Kluwer, Dordrecht (2000)Google Scholar
  12. 12.
    Friedman, G., Hartman, A., Nagin, K., Shiran, T.: Projected state machine coverage for software testing. In: Proc. ISSTA (2002)Google Scholar
  13. 13.
    Frieze, A., Galbiati, G., Maffioli, F.: On the worst-case performance of some algorithms for the asymmetric traveling salesman problem. Networks 12, 23–39 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Groce, A., Peled, D., Yannakakis, M.: Adaptive model checking. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 357–370. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Harel, D.: Statecharts: A visual formalism for complex systems. Science of Computer Programming 8, 231–274 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Hennie, F.C.: Fault detection experiments for sequential circuits. In: Proc. 5th Annual Symp. Switching Circuit Theory and Logical Design, pp. 95–110 (1964)Google Scholar
  17. 17.
    Holzmann, G., Peled, D.A., Redberg, M.H.: Design tools for requirements engineering. Bell Labs Technical Journal 2, 86–95 (1997)CrossRefGoogle Scholar
  18. 18.
    Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 327–341. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Lee, D., Yannakakis, M.: On-line minimization of transition systems. In: Proc. 24th Ann. ACM Symp. on Theory of Computing, pp. 264–274 (1992)Google Scholar
  20. 20.
    Lee, D., Yannakakis, M.: Testing finite state machines: state identification and verification. IEEE Trans. on Computers 43(3), 306–320 (1994)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Lee, D., Yannakakis, M.: Optimization problems from feature testing of communication protocols. In: Proc. Intl. Conf. on Network Protocols, pp. 66–75 (1996)Google Scholar
  22. 22.
    Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - a survey. Proc. of IEEE 84(8), 1090–1123 (1996)CrossRefGoogle Scholar
  23. 23.
    Moore, E.F.: Gedanken-experiments on sequential machines. Automata Studies, Annals of Mathematics Studies, vol. 34, pp. 129–153. Princeton University Press, Princeton (1956)Google Scholar
  24. 24.
    Mosk-Aoyama, D., Yannakakis, M.: Testing Hierarchical Systems via Edge Covering (2004) (manuscript)Google Scholar
  25. 25.
    Naito, S., Tsunoyama, M.: Fault detection for sequential machines by transitions tours. In: Proc. IEEE Fault Tolerant Comput. Symp., pp. 238–243. IEEE Computer Society Press, Los Alamitos (1981)Google Scholar
  26. 26.
    Peled, D., Vardi, M., Yannakakis, M.: Black box checking. Journal of Automata, Languages and Combinatorics 7(2), 225–246 (2002)zbMATHMathSciNetGoogle Scholar
  27. 27.
    Petrenko, A.: Fault model-driven test derivation from finite state models: Annotated bibliography. In: Cassez, F., Jard, C., Rozoy, B., Dermot, M. (eds.) MOVEP 2000. LNCS, vol. 2067, pp. 196–205. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  28. 28.
    Reif, J.: The complexity of two-player games with incomplete information. J. Computer and System Sciences 29, 274–301 (1984)zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Rivest, R.L., Schapire, R.E.: Inference of finite automata using homing sequences. In: Proc. 21st Ann. Symp. on Theory of Computing, pp. 411–420 (1989)Google Scholar
  30. 30.
    Tretmans, J.: Test generation with inputs, outputs amd repetitive quiescence. Software - Concepts and Tools 17(3), 103–120 (1996)zbMATHGoogle Scholar
  31. 31.
  32. 32.
    Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (4), 98–108 (1973)MathSciNetGoogle Scholar
  33. 33.
    Yannakakis, M., Lee, D.: Testing finite state machines: fault detection. J. of Computer and System Sciences 50(2), 209–227 (1995)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Mihalis Yannakakis
    • 1
  1. 1.Department of Computer ScienceColumbia UniversityNew YorkUSA

Personalised recommendations