Test Generation by Constraint Solving and FSM Mutant Killing

  • Alexandre PetrenkoEmail author
  • Omer Nguena Timo
  • S. Ramesh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9976)


The problem of fault model-based test generation from formal models, in this case Finite State Machines, is addressed. We consider a general fault model which is a tuple of a specification, conformance relation and fault domain. The specification is a deterministic FSM which can be partially specified and not reduced. The conformance relation is quasi-equivalence, as all implementations in the fault domain are assumed to be completely specified FSMs. The fault domain is a set of all possible deterministic submachines of a given nondeterministic FSM, called a mutation machine. The mutation machine contains a specification machine and extends it with mutated transitions modelling potential faults. An approach for deriving a test suite which is complete (sound and exhaustive) for the given fault model is elaborated. It is based on our previously proposed method for analyzing the test completeness by logical encoding and SMT-solving. The preliminary experiments performed on an industrial controller indicate that the approach scales sufficiently well.


FSM Conformance testing Mutation testing Fault modelling Fault model-based test generation Test coverage Fault coverage analysis 



This work is supported in part by GM, NSERC and MEIE of Gouvernement du Québec.


  1. 1.
    Pomeranz, I., Sudhakar, M.R.: Test generation for multiple state-table faults in finite-state machines. IEEE Trans. Comput. 46(7), 783–794 (1997)CrossRefGoogle Scholar
  2. 2.
    Poage, J.F., McCluskey, Jr., E.J.: Derivation of optimal test sequences for sequential machines. In: Proceedings of the IEEE 5th Symposium on Switching Circuits Theory and Logical Design, pp. 121–132 (1964)Google Scholar
  3. 3.
    DeMilli, R.A., Offutt, J.A.: Constraint-based automatic test data generation. IEEE Trans. Softw. Eng. 17(9), 900–910 (1991)CrossRefGoogle Scholar
  4. 4.
    Grunsky, I.S., Petrenko, A.: Design of checking experiments with automata describing protocols. Automatic Control and Computer Sciences. Allerton Press Inc. USA. No. 4 (1988)Google Scholar
  5. 5.
    Hennie, F.C.: Fault detecting experiments for sequential circuits. In: Proceedings of the IEEE 5th Annual Symposium on Switching Circuits Theory and Logical Design. Princeton, pp. 95–110 (1964)Google Scholar
  6. 6.
    Koufareva, I., Petrenko, A., Yevtushenko, N.: Test generation driven by user–defined fault models. In: Proceedings of the 12th International Workshop on Testing of Communicating Systems, pp. 215–233 (1999)Google Scholar
  7. 7.
    Lee, D., Yannakakis, M.: Principles and methods of testing finite-state machines - a survey. Proc. IEEE 84(8), 1090–1123 (1996)CrossRefGoogle Scholar
  8. 8.
    Moore, E.F.: Gedanken experiments on sequential machines. In: Automata Studies, pp. 129–153. Princeton University Press (1956)Google Scholar
  9. 9.
    Petrenko, A., Yevtushenko, N.: Test suite generation for a FSM with a given type of implementation errors. In: Proceedings of IFIP 12th International Symposium on Protocol Specification, Testing, and Verification, pp. 229–243 (1992)Google Scholar
  10. 10.
    Petrenko, A., Yevtushenko, N., Bochmann, G.V.: Fault models for testing in context. In: Gotzhein, R., Bredereke, J. (eds.) Formal Description Techniques IX, pp. 163–178. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  11. 11.
    Vasilevskii, M.P.: Failure diagnosis of automata. Cybernetics, vol. 4, pp. 653–665. Plenum Publishing Corporation, New York (1973)Google Scholar
  12. 12.
    Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)CrossRefzbMATHGoogle Scholar
  13. 13.
    Vuong, S.T., Ko, K.C.: A novel approach to protocol test sequence generation. In: Global Telecommunications Conference, vol. 3, pp. 2–5. EEE (1990)Google Scholar
  14. 14.
    Petrenko, A., Boroday, S., Groz, R.: Confirming configurations in EFSM testing. IEEE Trans. Softw. Eng. 30(1), 29–42 (2004)CrossRefzbMATHGoogle Scholar
  15. 15.
    de Moura, L., Bjørner, N.S.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  16. 16.
    Bochmann, G.V., et al.: Fault models in testing. In: Proceedings of the IFIP TC6/WG6. 1 Fourth International Workshop on Protocol Test Systems. North-Holland Publishing Co., pp. 17–30 (1991)Google Scholar
  17. 17.
    Petrenko, A., Yevtushenko, N.: Testing from partial deterministic FSM specifications. IEEE Trans. Comput. 54(9), 1154–1165 (2005)CrossRefGoogle Scholar
  18. 18.
    Petrenko, A., Dury, A., Ramesh, S., Mohalik, S.: A method and tool for test optimization for automotive controllers. In: ICST Workshops, pp. 198–207. IEEE (2013)Google Scholar
  19. 19.
    Parr, T.: The Definitive ANTLR 4 Reference, vol. 2. Pragmatic Bookshelf, Raleigh (2013)Google Scholar
  20. 20.
    Petrenko, A, Nguena Timo, O., Ramesh, S.: Multiple mutation testing from FSM. In: Proceedings of the 35th IFIP WG 6.1 International Conference on Formal Techniques for Distributed Objects, Components, and Systems, pp. 222–238 (2016)Google Scholar
  21. 21.
    Belli, F., Budnik, C.J., Hollmann, A., Tuglular, T., Wong, W.E.: Model-based mutation testing - approach and case studies. Sci. Comput. Programm. 120, 25–48 (2016)CrossRefGoogle Scholar
  22. 22.
    El-Fakih, K., Dorofeeva, R., Yevtushenko, N., Bochmann, G.V.: FSM-based testing from user defined faults adapted to incremental and mutation testing. Programm. Comput. Softw. 38, 201–209 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Petrenko, A.: Checking experiments for symbolic input/output finite state machines. In: ICST Workshops, pp. 229–237. IEEE (2016)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2016

Authors and Affiliations

  • Alexandre Petrenko
    • 1
    Email author
  • Omer Nguena Timo
    • 1
  • S. Ramesh
    • 2
  1. 1.Computer Research Institute of Montreal, CRIMMontrealCanada
  2. 2.GM Global R&DWarrenUSA

Personalised recommendations