Complete model-based equivalence class testing

  • Wen-ling Huang
  • Jan PeleskaEmail author
ICTSS 2013


In this article, we present a model-based black-box equivalence partition testing strategy, together with a formal proof of its completeness properties. The results apply to reactive systems with large, possibly infinite input data types and finite internal and output data ranges that may be enumerated with acceptable effort. The investigation is performed on a semantic level and applies to all concrete test models whose behavioural semantics can be encoded as a variant of state transition systems. Test suite construction is performed in relation to a given fault model \(\mathcal{F}\) for which a finite black-box test suite can be constructed which is complete with respect to \(\mathcal{F}\). It is shown how the test suite generation can be effectively implemented by model-based testing tools, using propositional representations of behavioural model semantics and constraint solvers. A SysML model of the ceiling speed monitoring function of the European Train Control System is presented as a case study, to explain theory application to a concrete modelling formalism.


Model-based testing Equivalence class partition testing Complete test strategies State transition systems  SysML state machines 



The authors are grateful to Alexandre Petrenko and to the anonymous reviewers for their helpful comments and suggestions for improvements.


  1. 1.
    Anand, Saswat, Burke, Edmund K., Chen, Tsong Yueh, Clark, John A., Cohen, Myra B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 86(8), 1978–2001 (2013)CrossRefGoogle Scholar
  2. 2.
    Binder, R.V.: Testing object-oriented systems: models, patterns, and tools. Addison-Wesley, Reading (2000)Google Scholar
  3. 3.
    Brucker, A.D., Wolff, B.: On theorem prover-based testing. Formal Asp. Comput. 25(5), 683–721 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Chen, T.Y., Tse, T.H., Yu, Y.T.: Proportional sampling strategy: a compendium and some insights. J. Syst. Softw. 58(1), 65–81 (2001)Google Scholar
  5. 5.
    Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)Google Scholar
  6. 6.
    Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)Google Scholar
  7. 7.
    Dick, Jeremy, Faivre, Alain: Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock, J.C.P., Larsen, P.G. (eds.) FME ’93: Industrial-strength formal methods. Lecture notes in computer science, pp. 268–284. Springer, Berlin (1993)CrossRefGoogle Scholar
  8. 8.
    Doucet, F., Menarini, M., Krüger, I.H., Gupta, R.K., Talpin, J.-P.: A verification approach for gals integration of synchronous components. Electr. Notes Theor. Comput. Sci. 146(2), 105–131 (2006)CrossRefGoogle Scholar
  9. 9.
    European Committee for Electrotechnical Standardization: EN 50128—Railway applications—Communications, signalling and processing systems—Software for railway control and protection systems. CENELEC, Brussels (2001)Google Scholar
  10. 10.
    European Railway Agency: ERTMS—System Requirements Specification—UNISIG SUBSET-026 February 2012.
  11. 11.
    Frantzen, L., Tretmans, J., Willemse, Tim A.C.: Test generation based on symbolic specifications. In: Grabowski, J., Nielsen, B. (eds.) Formal approaches to software testing. Lecture notes in computer science, pp. 1–15. Springer, Berlin (2005)CrossRefGoogle Scholar
  12. 12.
    Fujiwara, S., Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)CrossRefGoogle Scholar
  13. 13.
    Gaudel, M.-C.: Testing can be formal, too. In: Mosses, P.D., Nielsen, M., Schwartzbach, M.I. (eds.) TAPSOFT. Lecture Notes in Computer Science, pp. 82–96. Springer, Berlin (1995)Google Scholar
  14. 14.
    Gnesi, S., Latella, D., Massink, M.: Formal test-case generation for uml statecharts. In: Ninth IEEE International Conference on Engineering Complex Computer Systems (ICECCS’04), iceccs, pp. 75–84 (2004)Google Scholar
  15. 15.
    Gill, A.: Introduction to the Theory of Finite-State Machines. McGraw-Hill, New York (1962)Google Scholar
  16. 16.
    Grieskamp, W., Gurevich, Y., Schulte, W., Veanes, M.: Generating finite state machines from abstract state machines. ACM SIGSOFT Softw. Eng. Notes 27(4), 112–122 (2002)CrossRefGoogle Scholar
  17. 17.
    Helke, S., Neustupny, T., Santen, T.: Automating test case generation from z specifications with isabelle. In: Bowen, J.P., Hinchey, M.G., Till, D. (eds.) ZUM ’97: The Z Formal Specification Notation, vol. 1212. Lecture Notes in Computer Science, pp. 52–71. Springer, Berlin (1997)Google Scholar
  18. 18.
    Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Katoen, J.-P., Stevens, P. (eds.) TACAS. Lecture Notes in Computer Science, pp. 327–341. Springer, Berlin (2002)Google Scholar
  19. 19.
    Huang, Wen-ling, Peleska, Jan: Exhaustive model-based equivalence class testing. In: Yenigün, Hüsnü, Yilmaz, Cemal, Ulrich, Andreas (eds.) Testing software and systems. Lecture notes in computer science, pp. 49–64. Springer, Berlin (2013)Google Scholar
  20. 20.
    Huang, W., Peleska, J., Schulze, U.: Comprehensive modelling for advanced systems of systems—test automation support. Public Document D34.1, COMPASS, October 2013.
  21. 21.
    ISO/DIS 26262-4 Road vehicles—Functional safety—Part 4: Product development: system level. Technical report, International Organization for Standardization (2009)Google Scholar
  22. 22.
    ISO/IEC/IEEE 29119-1:2013(e): Software and systems engineering—software testing—part 1: Concepts and definitions, Sept (2013)Google Scholar
  23. 23.
    ISO/IEC/IEEE 29119-2:2013(e): Software and systems engineering—software testing—part 2: Test processes. Sept (2013)Google Scholar
  24. 24.
    Kalaji, A.S., Hierons, R.M., Swift, S.: Generating feasible transition paths for testing from an extended finite state machine (efsm). In: ICST, IEEE Computer Society, Silver Spring, pp. 230–239 (2009)Google Scholar
  25. 25.
    Lapschies, F.: SONOLAR homepage. Accessed June 2014
  26. 26.
    Object Management Group: OMG Systems Modeling Language (\(\text{ OMG } \text{ SysML }^{{TM}}\)). Technical report, Object Management Group, 2010. OMG Document Number: formal/2010-06-02 (2010)Google Scholar
  27. 27.
    Object Management Group: OMG Unified Modeling Language (OMG UML), superstructure, version 2.4.1. Technical report, OMG (2011)Google Scholar
  28. 28.
    Peleska, J., Siegel, M.: Test automation of safety–critical reactive systems. S. Afr. Comput. J. 19, 53–77 (1997)Google Scholar
  29. 29.
    Peleska, J.: Industrial-strength model-based testing—state of the art and current challenges. In: Petrenko, A.K., Schlingloff, H. (eds.) Proceedings Eighth Workshop on Model-Based Testing, Rome, Italy, 17th March 2013. Electronic Proceedings in Theoretical Computer Science, vol. 111, pp. 3–28. Open Publishing Association (2013)Google Scholar
  30. 30.
    Peleska, J., Honisch, A., Lapschies, F., Löding, H., Schmid, H., Smuda, P., Vorobev, E., Zahlten, C.: A real-world benchmark model for testing concurrent real-time systems in the automotive domain. In: Wolff, B., Zaidi, F. (eds.) Testing Software and Systems. Proceedings of the 23rd IFIP WG 6.1 International Conference, ICTSS 2011. LNCS, vol. 7019, pp. 146–161, Nov 2011. IFIP WG 6.1, Springer, Berlin (2011)Google Scholar
  31. 31.
    Peleska, J., Vorobev, E., Lapschies, F.: Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) Nasa Formal Methods. Third International Symposium, NFM 2011. LNCS, vol. 6617, pp. 298–312. Springer, Berlin (2011)Google Scholar
  32. 32.
    Petrenko, A., Yevtushenko, N., Bochmann, G.v: Fault models for testing in context. In: Gotzhein, Reinhard, Bredereke, Jan (eds.) Formal description techniques IX—theory, application and tools, pp. 163–177. Chapman & Hall, London (1996)CrossRefGoogle Scholar
  33. 33.
    Petrenko, A., Simao, A., Maldonado, J.C.: Model-based testing of software and systems: recent advances and challenges. Int. J. Softw. Tools Technol. Transf. 14(4), 383–386 (2012)CrossRefGoogle Scholar
  34. 34.
    Ranise, S., Tinelli, C.: Satisfiability modulo theories. IEEE Mag. Intell. Syst. Trends Controv. 21(6), 71–81 (2006)Google Scholar
  35. 35.
    RTCA, SC-167 Software Considerations in Airborne Systems and Equipment Certification, RTCA/DO-178B RTCA (1992)Google Scholar
  36. 36.
    Spillner, A., Linz, T., Schaefer, H.: Software testing foundations. Dpunkt.verlag, Heidelberg (2006)Google Scholar
  37. 37.
    Springintveld, J.G., Vaandrager, F.W., D’Argenio, P.R.: Testing timed automata. Theor. Comput. Sci. 254(1–2), 225–257 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Tretmans, J.: Conformance testing with labelled transition systems: implementation relations and test generation. Comput. Netw. ISDN Syst. 29(1), 49–79 (1996)CrossRefGoogle Scholar
  39. 39.
    Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. Lecture Notes in Computer Science, vol. 4949, pp. 1–38. Springer, Berlin (2008)Google Scholar
  40. 40.
    UNISIG: ERTMS/ETCS System Requirements Specification, chap. 3. Principles, vol. Subset-026-3, chapt. 3. Issue 3.3.0 (2012)Google Scholar
  41. 41.
    Vasilevskii, M.P.: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceUniversity of BremenBremenGermany

Personalised recommendations