Skip to main content
SpringerLink
Log in

Complete model-based equivalence class testing for nondeterministic systems

  • Original Article
  • Published:
Formal Aspects of Computing

Abstract

The main objective of this article is to present a complete finite black-box testing theory for non-deterministic Kripke structures with possibly infinite input domains, but finite domains for internal state variables and outputs. To this end, an abstraction from Kripke structures of this sub-domain to finite state machines is developed. It is shown that every complete black-box testing theory for (deterministic or nondeterministic) finite state machines in the range of this abstraction induces a complete black-box input equivalence class partition testing (IECPT) theory for the Kripke structures under consideration. Additionally, it is shown that each of these IECPT theories can be combined with random testing, such that a random value is selected from an input equivalence class, whenever a representative from this class is required in a test step. Experiments have shown that this combination increases the test strength of equivalence class tests for systems under test (SUT) outside the fault domain, while we show here that this randomisation preserves the completeness property for SUT inside the domain. The investigations lead to several complete IECPT strategies which, to our best knowledge, were not known before for this sub-domain of Kripke structures. The elaboration and presentation of results is performed on a semantic level, so that the testing theories under consideration can be applied to models presented in any concrete formalism, whose behaviour is reflected by a member of our semantic category.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anand S, Burke EK, Chen TY, Clark TY, Cohen MB, Grieskamp W, Harman M, Harrold MJ, McMinn P: An orchestrated survey of methodologies for automated software test case generation. J Syst Soft 86(8), 1978–2001 (2013)

    Article  Google Scholar 

  2. Braunstein C, Haxthausen AE, Huang W-L, Hübner F, Peleska J, Schulze U, Hong LV (2014) Complete model-based equivalence class testing for the ETCS ceiling speed monitor. In: Merz S, Pang J (eds) Proceedings of the ICFEM 2014, Lecture Notes in Computer Science, vol 8829. Springer, Berlin, Heidelberg, pp 380–395

  3. Cavalcanti A, Gaudel M-C: Testing for refinement in Circus. Acta Inf 48(2), 97–147 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Clarke EM, Grumberg O, Peled DA: Model checking. The MIT Press, Cambridge (1999)

    Google Scholar 

  5. Chow TS: Testing software design modeled by finite-state machines. IEEE Trans Softw Eng SE- 4(3), 178–186 (1978)

    Article  MATH  Google Scholar 

  6. Cavalcanti A, Huang W-L, Peleska J, Woodcock J (2015) CSP and Kripke structures. In: Leucker M, Rueda C, Valencia FD (eds) Theoretical aspects of computing–ICTAC 2015—12th International Colloquium Cali, Colombia, October 29–31, 2015, Proceedings, Lecture Notes in Computer Science, vol 9399. Springer, Switzerland, pp 505–523

  7. Dick J, Faivre A (1993) Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock JCP, Larsen PG (eds) FME ’93: industrial-strength formal methods, Lecture Notes in Computer Science, vol 670. Springer, Berlin, Heidelberg, pp 268–284

  8. Diaconescu R: Institution-independent model theory. Birkhäuser Verlag AG, Basel (2008)

    MATH  Google Scholar 

  9. Fujiwara S, Bochmann GV, Khendek F, Amalou M, Ghedamsi A: Test selection based on finite state models. IEEE Trans Softw Eng 17(6), 591–603 (1991)

    Article  Google Scholar 

  10. Frantzen L, Tretmans J, Willemse TAC (2005) Test generation based on symbolic specifications. In: Grabowski J, Nielsen B (eds) Formal approaches to software testing, Lecture Notes in Computer Science, vol 3395. Springer, Berlin, Heidelberg, pp 1–15

  11. Gaudel M-C (1995) Testing can be formal, too. In: Mosses PD, Nielsen M, Schwartzbach MI (eds) TAPSOFT, Lecture Notes in Computer Science, vol 915. Springer, Heidelberg, pp 82–96

  12. Goguen JA, Burstall RM: Institutions: abstract model theory for specification and programming. J ACM 39(1), 95–146 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  13. Grieskamp W, Gurevich Y, Schulte W, Veanes M: Generating finite state machines from abstract state machines. ACM SIGSOFT Softw Eng Notes 27(4), 112–122 (2002)

    Article  Google Scholar 

  14. Goguen J, Roşu G: Institution morphisms. Formal Aspects Comput 13(3), 274–307 (2014)

    MATH  Google Scholar 

  15. Hennessy M: Algebraic theory of processes. MIT Press, Cambridge (1988)

    MATH  Google Scholar 

  16. Hübner F, Huang W-L, Peleska J (2015) Experimental evaluation of a novel equivalence class partition testing strategy. In: Christian Blanchette J, Kosmatov N (eds) Tests and proofs–9th International Conference, TAP 2015, Held as Part of STAF 2015, L’Aquila, Italy, July 22–24, 2015. Proceedings, Lecture Notes in Computer Science, vol 9154. Springer, Switzerland, pp 155–172

  17. Hierons RM: Testing from a nondeterministic finite state machine using adaptive state counting. IEEE Trans Comput 53(10), 1330–1342 (2004)

    Article  Google Scholar 

  18. Hoare CAR, Jifeng H: Unifying theories of programming. Prentice-Hall, Englewood Cliffs (1998)

    MATH  Google Scholar 

  19. Hong HS, Lee I, Sokolsky O, Ural H (2002) A temporal logic based theory of test coverage and generation. In: Katoen J-P, Stevens P (eds) TACAS, Lecture Notes in Computer Science, vol 2280. Springer, Heidelberg, pp 327–341

  20. Helke S, Neustupny T, Santen T (1997) Automating test case generation from Z specifications with Isabelle. In: Bowen JP, Hinchey MG, Till D (eds) ZUM ’97: The Z formal specification notation, Lecture Notes in Computer Science, vol 1212. Springer, Berlin, Heidelberg, pp 52–71

  21. Huang W-L, Peleska J: Complete model-based equivalence class testing. STTT 18(3), 265–283 (2016)

    Article  Google Scholar 

  22. Kalaji AS, Hierons RM, Swift S (2009) Generating feasible transition paths for testing from an extended finite state machine (efsm). In: ICST. IEEE Computer Society, New York, pp 230–239

  23. Lapschies F (2014) SONOLAR homepage. http://www.informatik.uni-bremen.de/agbs/florian/sonolar/

  24. Luo G, Bochmann GV, Petrenko A: Test selection based on communicating nondeterministic finite-state machines using a generalized W-method. IEEE Trans Softw Eng 20(2), 149–162 (1994)

    Article  Google Scholar 

  25. Mossakowski T, Roggenbach M (2006) Structured CSP—a process algebra as an institution. In: Fiadeiro JL, Schobbens P-Y (eds) Recent trends in algebraic development techniques, 18th International Workshop, WADT 2006, La Roche en Ardenne, Belgium, June 1–3, 2006, Revised Selected Papers, Lecture Notes in Computer Science, vol 4409. Springer, Heidelberg, pp 92–110

  26. Naito S, Tsunoyama M (1981) Fault detection for sequential machines by transition tours. In: Proc. IEEE Fault Tolerant Comput. Conf., pp 162–178

  27. Peleska J (1996) Formal Methods and the Development of Dependable Systems. Number 9612. Christian-Albrechts-Universität Kiel, Institut fr Informatik und Praktische Mathematik, Habilitationsschrift

  28. Peleska J (2013) Industrial-strength model-based testing - state of the art and current challenges. In: Petrenko AK, Schlingloff H (eds) Proceedings eighth workshop on model-based testing, Rome, Italy, 17th March 2013, Electronic Proceedings in Theoretical Computer Science, vol 111. Open Publishing Association, Sydney, pp 3–28

  29. Peleska J, Huang W-L, Hübner F (2016) A novel approach to HW/SW integration testing of route-based interlocking system controllers. In: Lecomte T, Pinger R, Romanovsky A (eds) Reliability, safety, and security of railway systems. modelling, analysis, verification, and certification–First International Conference, RSSRail 2016, Paris, France, June 28–30, 2016, Proceedings, Lecture Notes in Computer Science, vol 9707. Springer, Switzerland, pp 32–49

  30. Peleska J, Siegel M: Test automation of safety-critical reactive systems. S Afr Comput J 19, 53–77 (1997)

    Google Scholar 

  31. Petrenko A, Simão A: Generalizing the DS-methods for testing non-deterministic FSMs. Comput J 58(7), 1656–1672 (2015)

    Article  Google Scholar 

  32. Peleska J, Vorobev E, Lapschies F (2011) Automated test case generation with SMT-solving and abstract interpretation. In: Bobaru M, Havelund K, Holzmann GJ, Joshi R (eds) Nasa formal methods, Third International Symposium, NFM 2011, LNCS, vol 6617, Pasadena. Springer, Heidelberg, pp 298–312

  33. Petrenko A, Yevtushenko N (2011) Adaptive testing of deterministic implementations specified by nondeterministic FSMs. In: Testing software and systems, Lecture Notes in Computer Science, vol 7019. Springer, Heidelberg, pp 162–178

  34. Petrenko A, Yevtushenko N (2014) Adaptive testing of nondeterministic systems with FSM. In: 15th International IEEE symposium on high-assurance systems engineering, HASE 2014, Miami Beach, FL, USA, January 9–11, 2014. IEEE Computer Society, New York, pp 224–228

  35. Petrenko A, Yevtushenko N, Bochmann GV (1996) Fault models for testing in context. In: Gotzhein R, Bredereke J (eds) Formal description techniques IX–Theory, application and tools. Chapman & Hall, London, pp 163–177

  36. Petrenko A, Yevtushenko N, Bochmann GV (1996) Testing deterministic implementations from nondeterministic FSM specifications. In: In testing of communicating systems, IFIP TC6 9th International Workshop on Testing of Communicating Systems. Chapman and Hall, London, pp 125–141

  37. Roscoe AW: Understanding concurrent systems. Springer, London (2010)

    Book  MATH  Google Scholar 

  38. Starke PH: Abstract automata. Elsevier, North-Holland (1972)

    MATH  Google Scholar 

  39. Springintveld JG, Vaandrager FW, D’Argenio PR: Testing timed automata. Theor Comput Sci 254(1–2), 225–257 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  40. Tretmans J: Conformance testing with labelled transition systems: implementation relations and test generation. Comput Netw ISDN Syst 29(1), 49–79 (1996)

    Article  Google Scholar 

  41. Vasilevskii MP: Failure diagnosis of automata. Kibernetika (Transl.) 4, 98–108 (1973)

    MathSciNet  Google Scholar 

  42. Vu LH, Haxthausen AE, Peleska J (2015) Formal modeling and verification of interlocking systems featuring sequential release. In: Artho C, Ölveczky PC (eds) Formal techniques for safety-critical systems, Communications in Computer and Information Science, vol 476. Springer, Switzerland, pp 223–238

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Bremen, Bremen, Germany

    Wen-ling Huang & Jan Peleska

Authors
  1. Wen-ling Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Peleska
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Peleska.

Additional information

Communicated by Rob Hierons

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Wl., Peleska, J. Complete model-based equivalence class testing for nondeterministic systems. Form Asp Comp 29, 335–364 (2017). https://doi.org/10.1007/s00165-016-0402-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00165-016-0402-2

Keywords

Search

Navigation