Scalable Incremental Test-case Generation from Large Behavior Models

  • Bernhard K. Aichernig
  • Dejan Ničković
  • Stefan Tiran
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9154)

Abstract

Model-based testing is a popular black-box testing technology that enables automation of test generation and execution, while achieving a given coverage. The application of this technology to large and complex systems is still a challenging problem, due to the state-space explosion resulting from the size of specification models.

In this paper, we evaluate a test-case generation approach that tackles this complexity along two axes. Firstly, our approach relies on a synchronous specification language for test models, thus avoiding the problem of interleaving actions. Secondly, our specification language enables incremental test-case generation by providing support for compositional modeling, in which each requirement or view of the system is expressed as a separate partial model. The individual requirement models are then naturally combined by conjunction, which is incrementally computed during the generation of tests.

We apply our test-case generation technique to two large industrial case studies: (1) an electronic control unit (ECU) of an agricultural device; and (2) a railway interlocking system. We demonstrate the scalability of our approach by creating a series of test models with increasing complexity and report on the experimental results.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aichernig, B.K., Brandl, H., Jöbstl, E., Krenn, W., Schlick, R., Tiran, S.: Killing strategies for model-based mutation testing. Software Testing, Verification and Reliability (Early view) (2014)Google Scholar
  2. 2.
    Aichernig, B.K., Jöbstl, E., Kegele, M.: Incremental refinement checking for test case generation. In: Veanes, M., Viganò, L. (eds.) TAP 2013. LNCS, vol. 7942, pp. 1–19. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  3. 3.
    Aichernig, B.K., Hörmaier, K., Lorber, F., Ničković, D., Tiran, S.: Require, test and trace IT. In: Núñez, M., Güdemann, M. (eds.) Formal Methods for Industrial Critical Systems. LNCS, vol. 9128, pp. 113–127. Springer, Heidelberg (2015) CrossRefGoogle Scholar
  4. 4.
    Aichernig, B.K., Lorber, F., Tiran, S.: Formal test-driven development with verified test cases. In: MODELSWARD 2014, pp. 626–635. SCITEPRESS, Lisbon (January 2014)Google Scholar
  5. 5.
    Arcaini, P., Gargantini, A.: Test generation for sequential nets of abstract state machines with information passing. Science of Computer Programming 94, Part 2(0), 93–108 (2014)Google Scholar
  6. 6.
    Arcaini, P., Gargantini, A., Riccobene, E.: An abstraction technique for testing decomposable systems by model checking. In: Seidl, M., Tillmann, N. (eds.) TAP 2014. LNCS, vol. 8570, pp. 36–52. Springer, Heidelberg (2014) Google Scholar
  7. 7.
    Berezin, S., Campos, S., Clarke, E.M.: Compositional reasoning in model checking. In: de Roever, W.-P., Langmaack, H., Pnueli, A. (eds.) COMPOS 1997. LNCS, vol. 1536, pp. 81–102. Springer, Heidelberg (1998) CrossRefGoogle Scholar
  8. 8.
    Bourhfir, C., Dssouli, R., Aboulhamid, E., Rico, N.: A guided incremental test case generation procedure for conformance testing for CEFSM specified protocols. In: Testing of Communicating Systems. IFIP, vol. 3, pp. 279–294. Springer, Heidelberg (1998)Google Scholar
  9. 9.
    Choi, Y., Heimdahl, M.P.E.: Model checking RSML-e requirements. In: HASE, pp. 109–118. IEEE Computer Society (2002)Google Scholar
  10. 10.
    Clarke, E., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guided abstraction refinement. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 154–169. Springer, Heidelberg (2000) CrossRefGoogle Scholar
  11. 11.
    Daca, P., Henzinger, T.A., Krenn, W., Ničković, D.: Compositional specifications for ioco testing. Technical Report IST-2014-148-v2+1, IST Austria (2014). http://repository.ist.ac.at/152/1/main_tr.pdf (visited on: March 27, 2014)
  12. 12.
    de Moura, L., Bjørner, N.S.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  13. 13.
    El-Fakih, K., Petrenko, A., Yevtushenko, N.: FSM test translation through context. In: Uyar, M.U., Duale, A.Y., Fecko, M.A. (eds.) TestCom 2006. LNCS, vol. 3964, pp. 245–258. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  14. 14.
    Fraser, G., Wotawa, F.: Creating test-cases incrementally with model-checkers. In: GI Jahrestagung (2). LNI, vol. 110, pp. 381–386. GI (2007)Google Scholar
  15. 15.
    Koo, H.-M., Mishra, P.: Functional test generation using design and property decomposition techniques. ACM Trans. Embed. Comput. Syst. 8(4), 32:1–32:33 (2009)Google Scholar
  16. 16.
    Krenn, W., Ničković, D., Tec, L.: Incremental language inclusion checking for networks of timed automata. In: Braberman, V., Fribourg, L. (eds.) FORMATS 2013. LNCS, vol. 8053, pp. 152–167. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  17. 17.
    Schwarzl, C., Peischl, B.: Test sequence generation from communicating UML state charts: An industrial application of symbolic transition systems. QSIC 2010, 122–131 (2010)Google Scholar
  18. 18.
    van der Bijl, M., Rensink, A., Tretmans, J.: Compositional testing with ioco. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 86–100. Springer, Heidelberg (2004) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Bernhard K. Aichernig
    • 1
  • Dejan Ničković
    • 2
  • Stefan Tiran
    • 1
    • 2
  1. 1.Institute for Software TechnologyGraz University of TechnologyGrazAustria
  2. 2.Austrian Institute of TechnologyViennaAustria

Personalised recommendations