Advertisement

Toward testing from finite state machines with symbolic inputs and outputs

  • Alexandre Petrenko
Theme Section Paper

Abstract

After 60 or so years of development, the theory of checking experiments for FSM still continues to attract a lot of attention of research community. One of the reasons is that it offers test generation techniques which under well-defined assumptions guarantee complete fault coverage for a given fault model of a specification FSM. Checking experiments have already been extended to remove assumptions that the specification Mealy machine need to be reduced, deterministic, and completely specified, while keeping the input, output and state sets finite. In our recent work, we investigated possibilities of removing the assumption about the finiteness of the input set, introducing the model FSM with symbolic inputs. In this paper, we report the results that further lift the theory of checking experiments for Mealy machines with symbolic inputs and symbolic outputs. The former are predicates defined over input variables and the latter are output variable valuations computed by assignments on input variables. Both types of variables can have large or even infinite domains. Inclusion of assignments in the model complicates even output fault detection, as different assignments may produce the same output valuations for some input valuations. We address this issue by using a transition cover composed of symbolic inputs on which the assignments produce different outputs. The enhanced transition cover is then used in checking experiments, which detect assignment/output faults and more general transition faults under certain assumptions.

Keywords

Finite state machines Extended finite state machines Symbolic automata Conformance testing Checking experiments Fault model-based test generation 

Notes

Acknowledgements

This work was partially supported by MESI (Ministère de l’Économie, Science et Innovation of Gouvernement du Québec) and NSERC of Canada. The author acknowledges joint work with A. Simao published in [30], and several results from [30] are used in this paper. The author thanks the reviewers who carefully read the submitted version and made useful comments.

References

  1. 1.
    Anand, S., Burke, E.K., Chen, T.Y., Clark, J., Cohen, M.B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 86(8), 1978–2001 (2013)CrossRefGoogle Scholar
  2. 2.
    Bochmann, G.V., Das, A., Dssouli, R., Dubuc, M., Ghedamsi, A., Luo, G.: Fault models in testing. In: Proceedings of the IFIP TC6/WG6. 1 Fourth International Workshop on Protocol Test Systems, vol. IV, pp. 17–30. North-Holland Publishing Co. (1991)Google Scholar
  3. 3.
    Cheng, K.T., Krishnakumar, A.S.: Automatic functional test generation using the extended finite state machine model. In: Proceedings of 30th Design Automation Conference, pp. 86–91 (1993)Google Scholar
  4. 4.
    Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)CrossRefzbMATHGoogle Scholar
  5. 5.
    Chun, W., Amer, P.D.L.: Test case generation for protocols specified in Estelle. In: Proceedings of the IFIP TC6/WG6. 1 Third International Conference on Formal Description Techniques for Distributed Systems and Communication Protocols: Formal Description Techniques, vol. III, pp. 191–206. North-Holland Publishing Co (1990)Google Scholar
  6. 6.
    Dorofeeva, R., Yevtushenko, N., El-Fakih, K., Cavalli, A.: Experimental evaluation of FSM-based testing methods. In: Third IEEE International Conference on Software Engineering and Formal Methods, pp. 23–32. IEEE Computer Society (2005)Google Scholar
  7. 7.
    Fabbri, S., et al.: Mutation testing applied to validate specifications based on statecharts. In: Proceedings of 10th IEEE Symposium on Software Reliability Engineering, pp. 210–219 (1999)Google Scholar
  8. 8.
    Frantzen, L., Tretmans, J., Willemse, T.A.C.: Test generation based on symbolic specifications. In: Formal Approaches to Software Testing (FATES). Lecture Notes in Computer Science, vol. 3395, pp. 1–15 (2004)Google Scholar
  9. 9.
    Friedenthal, S., Moore, A., Steiner, R.: A practical guide to SysML: the systems modeling language. Morgan Kaufmann, San Francisco (2014)Google Scholar
  10. 10.
    Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)CrossRefGoogle Scholar
  11. 11.
    Glässer, U., Gotzhein, R., Prinz, A.: The formal semantics of SDL-2000: status and perspectives. Comput. Netw. 42(3), 343–358 (2003)CrossRefzbMATHGoogle Scholar
  12. 12.
    Harel, D., Naamad, A.: The statemate semantics of statecharts. ACM Trans. Softw. Eng. Methodol. 5(4), 293–333 (1996)CrossRefGoogle Scholar
  13. 13.
    Hennie, F.C.: Fault-detecting experiments for sequential circuits. In: Proceedings of Fifth Annual Symposium on Circuit Theory and Logical Design, pp. 95–110 (1965)Google Scholar
  14. 14.
    Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, pp. 327–341. Springer, Berlin (2002)Google Scholar
  15. 15.
    Huang, W., Peleska, J.: Exhaustive model-based equivalence class testing. In: International Conference on Testing Software and Systems. Lecture Notes in Computer Science, pp. 49–64. Springer, Berlin (2013)Google Scholar
  16. 16.
    Huang, W., Peleska, J.: Complete model-based equivalence class testing. Int. J. Softw. Tools Technol. Transf. (2014). doi: 10.1007/s10009-014-0356-8
  17. 17.
    Jéron, T., Veanes, M., Wolff, B. (ed.): Symbolic methods in testing. Report from Dagstuhl Seminar 13021 (2013)Google Scholar
  18. 18.
    Kalaji, A.S., Hierons, R.M., Swift, S.: Generating feasible transition paths for testing from an extended finite state machine (EFSM). In: International Conference on Software Testing, Verification and Validation, pp. 230–239. IEEE Computer Society, Silver Spring (2009)Google Scholar
  19. 19.
    Koufareva, I., Petrenko, A., Yevtushenko, N.: Test generation driven by user-defined fault models. In: Testing of Communicating Systems, pp. 215–233. Springer, Berlin (1999)Google Scholar
  20. 20.
    Li, X., Higashino, T., Higuchi, M., Taniguchi, K.: Automatic generation of extended UIO sequences for communication protocols in an EFSM model. In: Proceedings of Seventh International Workshop Protocol Test Systems, pp. 225–240 (1994)Google Scholar
  21. 21.
    Maler, O., Mens, I.: Learning regular languages over large alphabets. In: Tools and Algorithms for the Construction and Analysis of Systems 2014. Lecture Notes in Computer Science, vol. 8413, pp. 485–499 (2014)Google Scholar
  22. 22.
    Moore, E.F.: Gedanken-experiments on sequential machines. In: Automata Studies, vol. 34, pp. 129–153. Princeton University Press, Princeton (1956)Google Scholar
  23. 23.
    de Moura, L., Bjørner, N.: Z3: An Efficient SMT Solver. In: Ramakrishnan C.R., Rehof J. (eds) Tools and Algorithms for the Construction and Analysis of Systems, vol 4963, pp. 337–340. Springer, Berlin (2008)Google Scholar
  24. 24.
    Noord, Gv, Gerdemann, D.: Finite state transducers with predicates and identities. Grammars 4(3), 263–286 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Petrenko, A., Yevtushenko, N.: Test suite generation for a given type of implementation errors. In: Proceedings of IFIP XII International Conference Protocol Specification, Testing, and Verification, pp. 229–243 (1992)Google Scholar
  26. 26.
    Petrenko, A., Yevtushenko, N., Bochmann, G.: Fault models for testing in context. In: Gotzhein R., Bredereke J. (eds) Formal Description Techniques IX. IFIP Advances in Information and Communication Technology. Springer, Boston, MA (1996)Google Scholar
  27. 27.
    Petrenko, A., Boroday, S., Groz, R.: Confirming configurations in EFSM testing. IEEE Trans. Softw. Eng. 30(1), 29–42 (2004)CrossRefzbMATHGoogle Scholar
  28. 28.
    Petrenko, A., Yevtushenko, N.: Conformance tests as checking experiments for partial nondeterministic FSM. In: Proceedings of the 5th International Workshop on Formal Approaches to Testing of Software. Lecture Notes in Computer Science, vol. 3997, pp. 118–133 (2005)Google Scholar
  29. 29.
    Petrenko, A., Dury, A., Ramesh, S., Mohalik, S.: A method and tool for test optimization for automotive controllers. In: Proceedings of the 9th Workshop on Advances in Model Based Testing (A-MOST 2013) of the 6th IEEE International Conference on Software Testing, Verification and Validation (ICST 2013), Luxembourg (2013)Google Scholar
  30. 30.
    Petrenko, A., Simao, A.: Checking experiments for finite state machines with symbolic inputs. In: Proceedings of the 27th IFIP WG 6.1 International Conference on Testing Software and Systems, ICTSS 2015, UAE. Lecture Notes in Computer Science, vol. 9447, pp. 3–18 (2015)Google Scholar
  31. 31.
    Petrenko, A., Timo, O.N., Ramesh, S.: Test generation by constraint solving and FSM mutant killing. In Proceedings of the 28th IFIP International Conference on Testing Software and Systems, Graz (2016)Google Scholar
  32. 32.
    Petrenko, A., Timo, O.N., Ramesh, S.: Multiple mutation testing from FSM. In: Proceedings of the 35th IFIP WG 6.1 International Conference on Formal Techniques for Distributed Objects, Components, and Systems, pp. 222–238 (2016)Google Scholar
  33. 33.
    Petrenko, A.: Checking experiments for symbolic input/output finite state machines. In: ICST Workshops, pp. 229–237. IEEE (2016)Google Scholar
  34. 34.
    Rusu, V., Du Bousquet, L., Jeron, T.: An approach to symbolic test generation. In: Proceedings of Second International Conference on Integrating Formal Methods (IFM’00), pp. 338–357 (2000)Google Scholar
  35. 35.
    Tiwari, A.: Formal semantics and analysis methods for Simulink Stateflow models. Technical Report, SRI International (2002)Google Scholar
  36. 36.
    Vasilevskii, M.P.: Failure diagnosis of automata. Cybernetics 4, 653–665 (1973)Google Scholar
  37. 37.
    Veanes, M.: Applications of Symbolic Finite Automata. In: Konstantinidis S. (eds) Implementation and Application of Automata, pp. 16–23. Springer, Berlin (2013)Google Scholar
  38. 38.
    Veanes, M., Hooimeijer, P., Livshits, B., Molnar, D., Bjorner, N.: Symbolic finite state transducers: algorithms and applications. In: Proceedings of the 39th ACM Symposium on Principles of programming languages, pp. 137–150 (2012)Google Scholar
  39. 39.
    Wang, C.J., Liu, M.T.: Generating test cases for EFSM with given fault model. In: Proceedings of Twelfth Conference of the IEEE Computer and Communications Societies, pp. 774–781 (1993)Google Scholar
  40. 40.
    Watson, B.W.: Implementing and using finite automata toolkits. In: Extended Finite State Models of Language, pp. 19–36. Cambridge University Press, New York (1999)Google Scholar
  41. 41.
    Yannakakis, M., Lee, D.: Testing finite state machines: fault detection. J. Comput. Syst. Sci. 50(2), 209–227 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Yevtushenko, N., Petrenko, A.: Synthesis of test experiments in some classes of automata. Autom. Control Comput. Sci. 24(4), 50–55 (1990)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.CRIM, Centre de recherche informatique de MontréalMontrealCanada

Personalised recommendations