Experimental Evaluation of a Novel Equivalence Class Partition Testing Strategy

  • Felix HübnerEmail author
  • Wen-ling Huang
  • Jan Peleska
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9154)


In this paper, a novel complete model-based equivalence class testing strategy is experimentally evaluated. This black-box strategy applies to deterministic systems with infinite input domains and finite internal state and output domains. It is complete with respect to a given fault model. This means that conforming behaviours will never be rejected, and all nonconforming behaviours inside a given fault domain will be uncovered. We investigate the question how this strategy performs for systems under test whose behaviours lie outside the fault domain. Furthermore, a strategy extension is presented, that is based on randomised data selection from input equivalence classes. While this extension is still complete with respect to the given fault domain, it also promises a higher test strength when applied against members outside this domain. This is confirmed by an experimental evaluation that compares mutation coverage achieved by the original and the extended strategy with the coverage obtained by random testing.


Model-based testing Equivalence class partition testing Adaptive random testing SysML State Transition Systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anand, S., Burke, E.K., Chen, T.Y., Clark, J.A., Cohen, M.B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86(8), 1978–2001 (2013)CrossRefGoogle Scholar
  2. 2.
    Arcuri, A., Iqbal, M.Z., Briand, L.: Black-Box system testing of real-time embedded systems using random and search-based testing. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 95–110. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  3. 3.
    Baier, C., Katoen, J.: Principles of model checking. MIT Press (2008)Google Scholar
  4. 4.
    Biere, A., Heljanko, K., Junttila, T., Latvala, T., Schuppan, V.: Linear encodings of bounded LTL model checking. Logical Methods in Computer Science 2(5) (November 2006), arXiv: 0611029, arXiv: cs/0611029
  5. 5.
    Braunstein, C., Haxthausen, A.E., Huang, W., Hübner, F., Peleska, J., Schulze, U., Vu Hong, L.: Complete model-based equivalence class testing for the ETCS ceiling speed monitor. In: Merz, S., Pang, J. (eds.) ICFEM 2014. LNCS, vol. 8829, pp. 380–395. Springer, Heidelberg (2014) Google Scholar
  6. 6.
    Braunstein, C., Huang, W.l., Peleska, J., Schulze, U., Hübner, F., Haxthausen, A.E., Hong, L.V.: A SysML test model and test suite for the ETCS ceiling speed monitor. Tech. rep., Embedded Systems Testing Benchmarks Site (April 30, 2014).
  7. 7.
    Chen, T.Y., Kuo, F.C., Merkel, R.G., Tse, T.H.: Adaptive random testing: the art of test case diversity. Journal of Systems and Software 83(1), 60–66 (2010)CrossRefGoogle Scholar
  8. 8.
    Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering SE 4(3), 178–186 (1978)Google Scholar
  9. 9.
    Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999) Google Scholar
  10. 10.
    Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Transactions on Software Engineering 17(6), 591–603 (1991)Google Scholar
  11. 11.
    Gaudel, M.-C.: Testing can be formal, too. In: Mosses, P.D., Nielsen, M. (eds.) CAAP 1995, FASE 1995, and TAPSOFT 1995. LNCS, vol. 915, pp. 82–96. Springer, Heidelberg (1995) CrossRefGoogle Scholar
  12. 12.
    Gill, A.: Introduction to the theory of finite-state machines. McGraw-Hill, New York (1962) Google Scholar
  13. 13.
    Huang, W.l., Peleska, J.: Complete model-based equivalence class testing. International Journal on Software Tools for Technology Transfer, 1–19 (2014).
  14. 14.
    Jaulin, L., Kieffer, M., Didrit, O., Walter, É.: Applied Interval Analysis. Springer, London (2001) CrossRefGoogle Scholar
  15. 15.
    Just, R.: The major mutation framework: efficient and scalable mutation analysis for java. In: Proceedings of the International Symposium on Software Testing and Analysis, ISSTA, July 23–25, pp. 433–436, San Jose, CA, USA (2014)Google Scholar
  16. 16.
    Ma, Y.S., Offutt, J., Kwon, Y.R.: MuJava: An Automated Class Mutation System: Research Articles. Softw. Test. Verif. Reliab. 15(2), 97–133 (2005).
  17. 17.
    Object Management Group: OMG Unified Modeling Language (OMG UML), superstructure, version 2.4.1. Tech. rep., OMG (2011)Google Scholar
  18. 18.
    Object Management Group: OMG Systems Modeling Language (OMG SysML\(^{TM}\)), Version 1.3. Tech. rep., Object Management Group (2012).
  19. 19.
    Peleska, J., Siegel, M.: Test automation of safety-critical reactive systems. South African Computer Journal 19, 53–77 (1997)Google Scholar
  20. 20.
    Peleska, J.: Industrial-strength model-based testing - state of the art and current challenges. In: Petrenko, A.K., Schlingloff, H. (eds.) Proceedings Eighth Workshop on Model-Based Testing. Electronic Proceedings in Theoretical Computer Science, vol. 111, pp. 3–28. Open Publishing Association, Rome (2013)Google Scholar
  21. 21.
    Petrenko, A., Simao, A., Maldonado, J.C.: Model-based testing of software and systems: Recent advances and challenges. Int. J. Softw. Tools Technol. Transf. 14(4), 383–386 (2012).
  22. 22.
    Springintveld, J., Vaandrager, F., D’Argenio, P.: Testing timed automata. Theoretical Computer Science 254(1–2), 225–257 (2001)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) FORTEST. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  24. 24.
    UNISIG: ERTMS/ETCS System Requirements Specification, Chapter 3, Principles, vol. Subset-026-3, chap. 3, issue 3.3.0 (February 2012)Google Scholar
  25. 25.
    Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceUniversity of BremenBremenGermany

Personalised recommendations