Black-Box Conformance Testing for Real-Time Systems

  • Moez Krichen
  • Stavros Tripakis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2989)


We propose a new framework for black-box conformance testing of real-time systems, where specifications are modeled as non-deterministic and partially-observable timed automata. We argue that such a model is essential for ease of modeling and expressiveness of specifications. The conformance relation is a timed extension of the input-output conformance relation of [29]. We argue that it is better suited for testing than previously considered relations such as bisimulation, must/may preorder or trace inclusion. We propose algorithms to generate two types of tests for this setting: analog-clock tests which measure dense time precisely and digital-clock tests which measure time with a periodic clock. The latter are essential for implementability, since only finite-precision clocks are available in practice. We report on a prototype tool and a small case study.


Test Suite Test Generation Label Transition System Periodic Clock Testing Framework 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alur, R., Dill, D.: A theory of timed automata. Theoretical Computer Science 126, 183–235 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Alur, R., Fix, L., Henzinger, T.: A determinizable class of timed automata. In: Dill, D.L. (ed.) CAV 1994. LNCS, vol. 818, Springer, Heidelberg (1994)Google Scholar
  3. 3.
    Belinfante, A., Feenstra, J., de Vries, R.G., Tretmans, J., Goga, N., Feijs, L., Mauw, S., Heerink, L.: Formal test automation: A simple experiment. In: 12th Int. Workshop on Testing of Communicating Systems, Kluwer, Dordrecht (1999)Google Scholar
  4. 4.
    Bensalem, S., Bozga, M., Krichen, M., Tripakis, S.: Testing conformance of realtime applications by automatic generation of observers. In: Runtime Verification (RV 2004) (2004) (to appear in ENTCS)Google Scholar
  5. 5.
    Berard, B., Petit, A., Diekert, V., Gastin, P.: Characterization of the expressive power of silent transitions in timed automata. Fundamenta Informaticae 36(2-3), 145–182 (1998)zbMATHMathSciNetGoogle Scholar
  6. 6.
    Bornot, S., Sifakis, J., Tripakis, S.: Modeling urgency in timed systems. In: de Roever, W.-P., Langmaack, H., Pnueli, A. (eds.) COMPOS 1997. LNCS, vol. 1536, p. 103. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  7. 7.
    Bozga, M., Fernandez, J.C., Ghirvu, L., Graf, S., Krimm, J.P., Mounier, L.: IF: a validation environment for timed asynchronous systems. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 543–547. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Braberman, V., Felder, M., Marre, M.: Testing timing behavior of real-time software. In: International Software Quality Week (1997)Google Scholar
  9. 9.
    Brat, G., Giannakopoulou, D., Goldberg, A., Havelund, K., Lowry, M., Pasareanu, C., Venet, A., Visser, W.: Experimental evaluation of V&V tools on martian rover software. In: SEI Software Model Checking Workshop (2003)Google Scholar
  10. 10.
    Brinksma, E., Tretmans, J.: Testing transition systems: An annotated bibliography. In: Cassez, F., Jard, C., Rozoy, B., Dermot, M. (eds.) MOVEP 2000. LNCS, vol. 2067, p. 187. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  11. 11.
    Cardell-Oliver, R.: Conformance test experiments for distributed real-time systems. In: ISSTA 2002, ACM Press, New York (2002)Google Scholar
  12. 12.
    Cardell-Oliver, R., Glover, T.: A practical and complete algorithm for testing real-time systems. In: Ravn, A.P., Rischel, H. (eds.) FTRTFT 1998. LNCS, vol. 1486, p. 251. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  13. 13.
    Clarke, D., Jéron, T., Rusu, V., Zinovieva, E.: STG: A symbolic test generation tool. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, p. 470. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  14. 14.
    Clarke, D., Lee, I.: Automatic generation of tests for timing constraints from requirements. In: 3rd Workshop on Object-Oriented Real-Time Dependable Systems, WORDS 1997 (1997)Google Scholar
  15. 15.
    Clarke, D., Lee, I.: Automatic test generation for the analysis of a real-time system: Case study. In: RTAS 1997 (1997)Google Scholar
  16. 16.
    En-Nouaary, A., Dssouli, R., Khendek, F., Elqortobi, A.: Timed test cases generation based on state characterization technique. In: RTSS 1998, IEEE, Los Alamitos (1998)Google Scholar
  17. 17.
    Fernandez, J.C., Jard, C., Jéron, T., Viho, G.: Using on-the-fly verification techniques for the generation of test suites. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, Springer, Heidelberg (1996)Google Scholar
  18. 18.
    Henzinger, T., Manna, Z., Pnueli, A.: What good are digital clocks? In: Kuich, W. (ed.) ICALP 1992. LNCS, vol. 623, Springer, Heidelberg (1992)Google Scholar
  19. 19.
    Hessel, A., Larsen, K., Nielsen, B., Pettersson, P., Skou, A.: Time-optimal real-time test case generation using UPPAAL. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 114–130. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  20. 20.
    Higashino, T., Nakata, A., Taniguchi, K., Cavalli, A.: Generating test cases for a timed I/O automaton model. In: IFIP Int’l Work. Test. Communicat. Syst., Kluwer, Dordrecht (1999)Google Scholar
  21. 21.
    ISO/IEC. Open Systems Interconnection Conformance Testing Methodology and Framework – Part 1: General Concept – Part 2 : Abstract Test Suite Specification – Part 3: The Tree and Tabular Combined Notation (TTCN). Technical Report 9646, International Organization for Standardization — Information Processing Systems — Open Systems Interconnection, Genève (1992) Google Scholar
  22. 22.
    Jard, C., Jéron, T., Morel, P.: Verification of test suites. In: TESTCOM 2000 (2000)Google Scholar
  23. 23.
    Khoumsi, A., Jéron, T., Marchand, H.: Test cases generation for nondeterministic real-time systems. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 131–146. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  24. 24.
    Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - A survey. Proceedings of the IEEE 84, 1090–1126 (1996)CrossRefGoogle Scholar
  25. 25.
    Nielsen, B., Skou, A.: Automated test generation from timed automata. In: Margaria, T., Yi, W. (eds.) TACAS 2001. LNCS, vol. 2031, p. 343. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  26. 26.
    Peleska, J.: Formal methods for test automation - hard real-time testing of controllers for the airbus aircraft family. In: IDPT 2002 (2002)Google Scholar
  27. 27.
    Sifakis, J., Yovine, S.: Compositional specification of timed systems. In: Puech, C., Reischuk, R. (eds.) STACS 1996. LNCS, vol. 1046, pp. 347–359. Springer, Heidelberg (1996)Google Scholar
  28. 28.
    Springintveld, J., Vaandrager, F., D’Argenio, P.: Testing timed automata. Theoretical Computer Science, 254 (2001)Google Scholar
  29. 29.
    Tretmans, J.: Testing concurrent systems: A formal approach. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 46–65. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  30. 30.
    Tretmans, J.: Testing techniques. Lecture notes, University of Twente, The Netherlands (2002)Google Scholar
  31. 31.
    Tripakis, S.: Fault diagnosis for timed automata. In: Damm, W., Olderog, E.-R. (eds.) FTRTFT 2002. LNCS, vol. 2469, p. 205. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  32. 32.
    Tripakis, S.: Folk theorems on the determinization and minimization of timed automata. In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, Springer, Heidelberg (2003)Google Scholar
  33. 33.
    Zhu, H., Hall, P., May, J.: Software unit test coverage and adequacy. ACM Computing Surveys 29(4) (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Moez Krichen
    • 1
  • Stavros Tripakis
    • 1
  1. 1.VERIMAG Centre EquationGièresFrance

Personalised recommendations