A Real-World Benchmark Model for Testing Concurrent Real-Time Systems in the Automotive Domain

  • Jan Peleska
  • Artur Honisch
  • Florian Lapschies
  • Helge Löding
  • Hermann Schmid
  • Peer Smuda
  • Elena Vorobev
  • Cornelia Zahlten
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7019)


In this paper we present a model for automotive system tests of functionality related to turn indicator lights. The model covers the complete functionality available in Mercedes Benz vehicles, comprising turn indication, varieties of emergency flashing, crash flashing, theft flashing and open/close flashing, as well as configuration-dependent variants. It is represented in UML2 and associated with a synchronous real-time systems semantics conforming to Harel’s original Statecharts interpretation. We describe the underlying methodological concepts of the tool used for automated model-based test generation, which was developed by Verified Systems International GmbH in cooperation with Daimler and the University of Bremen. A test suite is described as initial reference for future competing solutions. The model is made available in several file formats, so that it can be loaded into existing CASE tools or test generators. It has been originally developed and applied by Daimler for automatically deriving test cases, concrete test data and test procedures executing these test cases in Daimler’s hardware-in-the-loop system testing environment. In 2011 Daimler decided to allow publication of this model with the objective to serve as a ”real-world” benchmark supporting research of model based testing.


Test Suite System Under Test Test Execution Test Automation Tool Basic Control State 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Arcuri, A., Iqbal, M.Z., Briand, L.: Black-box system testing of real-time embedded systems using random and search-based testing. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 95–110. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Brummayer, R.: Efficient SMT Solving for Bit-Vectors and the Extensional Theory of Arrays. Ph.D. thesis, Johannes Kepler University Linz, Austria (November 2009)Google Scholar
  3. 3.
    Brummayer, R., Biere, A.: Local two-level and-inverter graph minimization without blowup. In: Proceedings of the 2nd Doctoral Workshop on Mathematical and Engineering Methods in Computer Science, MEMICS 2006 (2006)Google Scholar
  4. 4.
    Brummayer, R., Biere, A.: Lemmas on Demand for the Extensional Theory of Arrays. In: Proc. 6th Intl. Workshop on Satisfiability Modulo Theories (SMT 2008). ACM, New York (2008)Google Scholar
  5. 5.
    Conformiq Tool Suite (2010),
  6. 6.
    David, A., Larsen, K.G., Li, S., Nielsen, B.: Timed testing under partial observability. In: Proc. 2nd International Conference on Software Testing, Verification and Validation (ICST 2009), pp. 61–70. IEEE Computer Society, Los Alamitos (2009)CrossRefGoogle Scholar
  7. 7.
    Eén, N., Mishchenko, A., Sörensson, N.: Applying logic synthesis for speeding up SAT. In: Marques-Silva, J., Sakallah, K.A. (eds.) SAT 2007. LNCS, vol. 4501, pp. 272–286. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Harel, D., Naamad, A.: The statemate semantics of statecharts. ACM Transactions on Software Engineering and Methodology 5(4), 293–333 (1996)CrossRefGoogle Scholar
  9. 9.
    Harrold, M.J., Rothermel, G.: Siemens programs, hr variants,
  10. 10.
  11. 11.
    Jha, S., Limaye, R., Seshia, S.: Beaver: Engineering an Efficient SMT Solver for Bit-Vector Arithmetic. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 668–674. Springer, Heidelberg (2009), CrossRefGoogle Scholar
  12. 12.
    Jung, J., Sülflow, A., Wille, R., Drechsler, R.: SWORD v1.0. Tech. rep. (2009); SMTCOMP 2009: System DescriptionGoogle Scholar
  13. 13.
    Löding, H., Peleska, J.: Timed moore automata: test data generation and model checking. In: Proc. 3rd International Conference on Software Testing, Verification and Validation (ICST 2010). IEEE Computer Society, Los Alamitos (2010)Google Scholar
  14. 14.
    Lu, S., Li, Z., Quin, F., Tan, L., Zhou, P., Zhou, Y.: Bugbench: Benchmarks for evaluating bug detection tools. In: Workshop on the Evaluation of Software Defect Detection Tools (2005)Google Scholar
  15. 15.
  16. 16.
    Nielsen, B., Skou, A.: Automated test generation from timed automata. International Journal on Software Tools for Technology Transfer (STTT) 5, 59–77 (2003)CrossRefGoogle Scholar
  17. 17.
    Peleska, J., Honisch, A., Lapschies, F., Löding, H., Schmid, H., Smuda, P., Vorobev, E., Zahlten, C.: Embedded systems testing benchmark (2011),
  18. 18.
    Peleska, J., Vorobev, E., Lapschies, F.: Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 298–312. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Peleska, J., Vorobev, E., Lapschies, F., Zahlten, C.: Automated model-based testing with RT-Tester. Tech. rep. (2011),
  20. 20.
    RTCA, SC-167: Software Considerations in Airborne Systems and Equipment Certification, RTCA/DO-178B. RTCA (1992)Google Scholar
  21. 21.
    Springintveld, J.G., Vaandrager, F.W., D’Argenio, P.R.: Testing timed automata. Theoretical Computer Science 254(1-2), 225–257 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    European Committee for Electrotechnical Standardization: EN 50128 – Railway applications – Communications, signalling and processing systems – Software for railway control and protection systems. CENELEC, Brussels (2001)Google Scholar
  23. 23.
    Systems, S.: Enterprise architect 8.0 (2011),
  24. 24.
    Weißleder, S.: Test Models and Coverage Criteria for Automatic Model-Based Test Generation with UML State Machines. Doctoral thesis, Humboldt-University Berlin, Germany (2010)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Jan Peleska
    • 1
  • Artur Honisch
    • 3
  • Florian Lapschies
    • 1
  • Helge Löding
    • 2
  • Hermann Schmid
    • 3
  • Peer Smuda
    • 3
  • Elena Vorobev
    • 1
  • Cornelia Zahlten
    • 2
  1. 1.Department of Mathematics and Computer ScienceUniversity of BremenGermany
  2. 2.Verified Systems International GmbHBremenGermany
  3. 3.Daimler AGStuttgartGermany

Personalised recommendations