Formal Aspects of Computing

, Volume 29, Issue 2, pp 335–364 | Cite as

Complete model-based equivalence class testing for nondeterministic systems

  • Wen-ling Huang
  • Jan PeleskaEmail author
Original Article


The main objective of this article is to present a complete finite black-box testing theory for non-deterministic Kripke structures with possibly infinite input domains, but finite domains for internal state variables and outputs. To this end, an abstraction from Kripke structures of this sub-domain to finite state machines is developed. It is shown that every complete black-box testing theory for (deterministic or nondeterministic) finite state machines in the range of this abstraction induces a complete black-box input equivalence class partition testing (IECPT) theory for the Kripke structures under consideration. Additionally, it is shown that each of these IECPT theories can be combined with random testing, such that a random value is selected from an input equivalence class, whenever a representative from this class is required in a test step. Experiments have shown that this combination increases the test strength of equivalence class tests for systems under test (SUT) outside the fault domain, while we show here that this randomisation preserves the completeness property for SUT inside the domain. The investigations lead to several complete IECPT strategies which, to our best knowledge, were not known before for this sub-domain of Kripke structures. The elaboration and presentation of results is performed on a semantic level, so that the testing theories under consideration can be applied to models presented in any concrete formalism, whose behaviour is reflected by a member of our semantic category.


Model-based testing Equivalence class partition testing Random testing Kripke structures Nondeterminism Complete testing theories 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ABC+13.
    Anand S, Burke EK, Chen TY, Clark TY, Cohen MB, Grieskamp W, Harman M, Harrold MJ, McMinn P: An orchestrated survey of methodologies for automated software test case generation. J Syst Soft 86(8), 1978–2001 (2013)CrossRefGoogle Scholar
  2. BHH+14.
    Braunstein C, Haxthausen AE, Huang W-L, Hübner F, Peleska J, Schulze U, Hong LV (2014) Complete model-based equivalence class testing for the ETCS ceiling speed monitor. In: Merz S, Pang J (eds) Proceedings of the ICFEM 2014, Lecture Notes in Computer Science, vol 8829. Springer, Berlin, Heidelberg, pp 380–395Google Scholar
  3. CG11.
    Cavalcanti A, Gaudel M-C: Testing for refinement in Circus. Acta Inf 48(2), 97–147 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  4. CGP99.
    Clarke EM, Grumberg O, Peled DA: Model checking. The MIT Press, Cambridge (1999)Google Scholar
  5. Cho78.
    Chow TS: Testing software design modeled by finite-state machines. IEEE Trans Softw Eng SE- 4(3), 178–186 (1978)CrossRefzbMATHGoogle Scholar
  6. CHPW15.
    Cavalcanti A, Huang W-L, Peleska J, Woodcock J (2015) CSP and Kripke structures. In: Leucker M, Rueda C, Valencia FD (eds) Theoretical aspects of computing–ICTAC 2015—12th International Colloquium Cali, Colombia, October 29–31, 2015, Proceedings, Lecture Notes in Computer Science, vol 9399. Springer, Switzerland, pp 505–523Google Scholar
  7. DF93.
    Dick J, Faivre A (1993) Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock JCP, Larsen PG (eds) FME ’93: industrial-strength formal methods, Lecture Notes in Computer Science, vol 670. Springer, Berlin, Heidelberg, pp 268–284Google Scholar
  8. Dia08.
    Diaconescu R: Institution-independent model theory. Birkhäuser Verlag AG, Basel (2008)zbMATHGoogle Scholar
  9. FBK+91.
    Fujiwara S, Bochmann GV, Khendek F, Amalou M, Ghedamsi A: Test selection based on finite state models. IEEE Trans Softw Eng 17(6), 591–603 (1991)CrossRefGoogle Scholar
  10. FTW05.
    Frantzen L, Tretmans J, Willemse TAC (2005) Test generation based on symbolic specifications. In: Grabowski J, Nielsen B (eds) Formal approaches to software testing, Lecture Notes in Computer Science, vol 3395. Springer, Berlin, Heidelberg, pp 1–15Google Scholar
  11. Gau95.
    Gaudel M-C (1995) Testing can be formal, too. In: Mosses PD, Nielsen M, Schwartzbach MI (eds) TAPSOFT, Lecture Notes in Computer Science, vol 915. Springer, Heidelberg, pp 82–96Google Scholar
  12. GB92.
    Goguen JA, Burstall RM: Institutions: abstract model theory for specification and programming. J ACM 39(1), 95–146 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  13. GGSV02.
    Grieskamp W, Gurevich Y, Schulte W, Veanes M: Generating finite state machines from abstract state machines. ACM SIGSOFT Softw Eng Notes 27(4), 112–122 (2002)CrossRefGoogle Scholar
  14. GR14.
    Goguen J, Roşu G: Institution morphisms. Formal Aspects Comput 13(3), 274–307 (2014)zbMATHGoogle Scholar
  15. Hen88.
    Hennessy M: Algebraic theory of processes. MIT Press, Cambridge (1988)zbMATHGoogle Scholar
  16. HHP15.
    Hübner F, Huang W-L, Peleska J (2015) Experimental evaluation of a novel equivalence class partition testing strategy. In: Christian Blanchette J, Kosmatov N (eds) Tests and proofs–9th International Conference, TAP 2015, Held as Part of STAF 2015, L’Aquila, Italy, July 22–24, 2015. Proceedings, Lecture Notes in Computer Science, vol 9154. Springer, Switzerland, pp 155–172Google Scholar
  17. Hie04.
    Hierons RM: Testing from a nondeterministic finite state machine using adaptive state counting. IEEE Trans Comput 53(10), 1330–1342 (2004)CrossRefGoogle Scholar
  18. HJ98.
    Hoare CAR, Jifeng H: Unifying theories of programming. Prentice-Hall, Englewood Cliffs (1998)zbMATHGoogle Scholar
  19. HLSU02.
    Hong HS, Lee I, Sokolsky O, Ural H (2002) A temporal logic based theory of test coverage and generation. In: Katoen J-P, Stevens P (eds) TACAS, Lecture Notes in Computer Science, vol 2280. Springer, Heidelberg, pp 327–341Google Scholar
  20. HNS97.
    Helke S, Neustupny T, Santen T (1997) Automating test case generation from Z specifications with Isabelle. In: Bowen JP, Hinchey MG, Till D (eds) ZUM ’97: The Z formal specification notation, Lecture Notes in Computer Science, vol 1212. Springer, Berlin, Heidelberg, pp 52–71Google Scholar
  21. HP16.
    Huang W-L, Peleska J: Complete model-based equivalence class testing. STTT 18(3), 265–283 (2016)CrossRefGoogle Scholar
  22. KHS09.
    Kalaji AS, Hierons RM, Swift S (2009) Generating feasible transition paths for testing from an extended finite state machine (efsm). In: ICST. IEEE Computer Society, New York, pp 230–239Google Scholar
  23. Lap14.
    Lapschies F (2014) SONOLAR homepage.
  24. LvBP94.
    Luo G, Bochmann GV, Petrenko A: Test selection based on communicating nondeterministic finite-state machines using a generalized W-method. IEEE Trans Softw Eng 20(2), 149–162 (1994)CrossRefGoogle Scholar
  25. MR06.
    Mossakowski T, Roggenbach M (2006) Structured CSP—a process algebra as an institution. In: Fiadeiro JL, Schobbens P-Y (eds) Recent trends in algebraic development techniques, 18th International Workshop, WADT 2006, La Roche en Ardenne, Belgium, June 1–3, 2006, Revised Selected Papers, Lecture Notes in Computer Science, vol 4409. Springer, Heidelberg, pp 92–110Google Scholar
  26. NT81.
    Naito S, Tsunoyama M (1981) Fault detection for sequential machines by transition tours. In: Proc. IEEE Fault Tolerant Comput. Conf., pp 162–178Google Scholar
  27. Pel96.
    Peleska J (1996) Formal Methods and the Development of Dependable Systems. Number 9612. Christian-Albrechts-Universität Kiel, Institut fr Informatik und Praktische Mathematik, HabilitationsschriftGoogle Scholar
  28. Pel13.
    Peleska J (2013) Industrial-strength model-based testing - state of the art and current challenges. In: Petrenko AK, Schlingloff H (eds) Proceedings eighth workshop on model-based testing, Rome, Italy, 17th March 2013, Electronic Proceedings in Theoretical Computer Science, vol 111. Open Publishing Association, Sydney, pp 3–28Google Scholar
  29. PHH16.
    Peleska J, Huang W-L, Hübner F (2016) A novel approach to HW/SW integration testing of route-based interlocking system controllers. In: Lecomte T, Pinger R, Romanovsky A (eds) Reliability, safety, and security of railway systems. modelling, analysis, verification, and certification–First International Conference, RSSRail 2016, Paris, France, June 28–30, 2016, Proceedings, Lecture Notes in Computer Science, vol 9707. Springer, Switzerland, pp 32–49Google Scholar
  30. PS97.
    Peleska J, Siegel M: Test automation of safety-critical reactive systems. S Afr Comput J 19, 53–77 (1997)Google Scholar
  31. PS15.
    Petrenko A, Simão A: Generalizing the DS-methods for testing non-deterministic FSMs. Comput J 58(7), 1656–1672 (2015)CrossRefGoogle Scholar
  32. PVL11.
    Peleska J, Vorobev E, Lapschies F (2011) Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru M, Havelund K, Holzmann GJ, Joshi R (eds) Nasa formal methods, Third International Symposium, NFM 2011, LNCS, vol 6617, Pasadena. Springer, Heidelberg, pp 298–312Google Scholar
  33. PY11.
    Petrenko A, Yevtushenko N (2011) Adaptive testing of deterministic implementations specified by nondeterministic FSMs. In: Testing software and systems, Lecture Notes in Computer Science, vol 7019. Springer, Heidelberg, pp 162–178Google Scholar
  34. PY14.
    Petrenko A, Yevtushenko N (2014) Adaptive testing of nondeterministic systems with FSM. In: 15th International IEEE symposium on high-assurance systems engineering, HASE 2014, Miami Beach, FL, USA, January 9–11, 2014. IEEE Computer Society, New York, pp 224–228Google Scholar
  35. PYB96a.
    Petrenko A, Yevtushenko N, Bochmann GV (1996) Fault models for testing in context. In: Gotzhein R, Bredereke J (eds) Formal description techniques IX–Theory, application and tools. Chapman & Hall, London, pp 163–177Google Scholar
  36. PYB96b.
    Petrenko A, Yevtushenko N, Bochmann GV (1996) Testing deterministic implementations from nondeterministic FSM specifications. In: In testing of communicating systems, IFIP TC6 9th International Workshop on Testing of Communicating Systems. Chapman and Hall, London, pp 125–141Google Scholar
  37. Ros10.
    Roscoe AW: Understanding concurrent systems. Springer, London (2010)CrossRefzbMATHGoogle Scholar
  38. Sta72.
    Starke PH: Abstract automata. Elsevier, North-Holland (1972)zbMATHGoogle Scholar
  39. SVD01.
    Springintveld JG, Vaandrager FW, D’Argenio PR: Testing timed automata. Theor Comput Sci 254(1–2), 225–257 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  40. Tre96.
    Tretmans J: Conformance testing with labelled transition systems: implementation relations and test generation. Comput Netw ISDN Syst 29(1), 49–79 (1996)CrossRefGoogle Scholar
  41. Vas73.
    Vasilevskii MP: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)MathSciNetGoogle Scholar
  42. VHP15.
    Vu LH, Haxthausen AE, Peleska J (2015) Formal modeling and verification of interlocking systems featuring sequential release. In: Artho C, Ölveczky PC (eds) Formal techniques for safety-critical systems, Communications in Computer and Information Science, vol 476. Springer, Switzerland, pp 223–238Google Scholar

Copyright information

© British Computer Society 2016

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceUniversity of BremenBremenGermany

Personalised recommendations