Safety-complete test suites

  • Wen-ling Huang
  • Sadik Özoguz
  • Jan PeleskaEmail author


In this paper, a novel safety-related variant of complete test suites for finite state machines is introduced. Under certain hypotheses which are similar to the ones used in the well-known W-Method and its improved versions, the new method guarantees to uncover every violation of safety properties from a certain well-defined class, while erroneous behaviour without safety relevance may remain undetected. While the method can be based on any of the known complete strategies for FSM testing, its most effective variant is based on the H-method, and this variant is presented in detail, denoted as the Safety-complete H-Method. It is guaranteed that application of the Safety-complete H-Method always results in less or equally many test cases than when applying the original H-Method. In well-defined situations that can be pre-determined from the reference model, the Safety-complete H-Method leads to a substantial reduction of test cases in comparison to the size of the analogous H test suites. We advocate this new test suite for situations, where exhaustive testing of the complete system is too expensive. In these cases, strong guarantees with respect to fault coverage should only be given for the errors representing safety violations, while it may be considered as acceptable if less critical errors remain undetected.


Model-based testing Complete testing theories Safety 



  1. Baier, C., & Katoen, J. (2008). Principles of model checking. Cambridge: MIT Press.zbMATHGoogle Scholar
  2. Chow, T.S. (1978). Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering SE, 4(3), 178–186.CrossRefGoogle Scholar
  3. Dorofeeva, R., El-Fakih, K., Yevtushenko, N. (2005). An improved conformance testing method. In: Wang, F. (ed) Formal Techniques for Networked and Distributed Systems - FORTE 2005, 25th IFIP WG 6.1 International Conference, Taipei, Taiwan, October 2-5, 2005, Proceedings, Springer, Lecture Notes in Computer Science, vol. 3731, pp. 204–218 . Scholar
  4. Fernandez, J., Mounier, L., Pachon, C. (2003). Property oriented test case generation. In: Petrenko, A., Ulrich, A. (eds) Formal Approaches to Software Testing, Third International Workshop on Formal Approaches to Testing of Software, FATES 2003, Montreal, Quebec, Canada, October 6th, 2003, Springer, Lecture Notes in Computer Science, vol. 2931, pp. 147–163 . Scholar
  5. Fujiwara, S., Gv, Bochmann, Khendek, F., Amalou, M., Ghedamsi, A. (1991). Test selection based on finite state models. IEEE Transactions on Software Engineering, 17(6), 591–603. Scholar
  6. Haxthausen, A.E., & Peleska, J. (2000). Formal development and verification of a distributed railway control system. IEEE Transaction on Software Engineering, 26(8), 687–701.CrossRefGoogle Scholar
  7. Huang, W., & Peleska, J. (2017a). Safety-complete test suites. In: Yevtushenko, N., Cavalli, AR., Yenigün, H. (eds) Testing Software and Systems - 29th IFIP WG 6.1 International Conference, ICTSS 2017, St. Petersburg, Russia, October 9-11, 2017, Proceedings, Springer, Lecture Notes in Computer Science, vol. 10533, pp. 145–161. Scholar
  8. Huang, Wl, & Peleska, J. (2017b). Complete model-based equivalence class testing for nondeterministic systems. Formal Aspects of Computing, 29(2), 335–364. Scholar
  9. Huang, W.L., Peleska, J., Schulze, U. (2014). Contract support for an evolving SoS. Tech. Rep. D34.3, COMPASS Comprehensive Modelling for Advanced Systems of Systems, available under
  10. Jorgensen, P.C. (2017). The craft of Model-Based testing. Boca Raton: CRC Press.CrossRefGoogle Scholar
  11. Li, S., & Qi, Z. (2004). Property-oriented testing: an approach to focusing testing efforts on behaviours of interest. In: Beydeda S., Gruhn V., Mayer J., Reussner R.H., Schweiggert F. (eds) Testing of Component-Based Systems and Software Quality, Proceedings of SOQUA 2004 (First International Workshop on Software Quality) and TECOS 2004 (Workshop Testing Component-Based Systems), GI, LNI, vol. 58, pp. 191–206.
  12. Luo, G., Bochmann, G., Petrenko, A. (1994). Test selection based on communicating nondeterministic finite-state machines using a generalized wp-method. IEEE Trans Software Eng, 20(2), 149–162. Scholar
  13. Luo, G., Petrenko, A.V., Bochmann, G. (1995). Selecting test sequences for partially-specified nondeterministic finite state machines, (pp. 95–110). Boston: Springer US. Scholar
  14. Machado, P.D.L., Silva, D.A., Mota, A.C. (2007). Towards property oriented testing. Electronic Notes in Theoretical Computer Science, 184(Supplement C), 3–19. Scholar
  15. Peleska, J., & Huang, Wl. (2017). Test automation - foundations and applications of model-based testing. University of Bremen, lecture notes, available under
  16. Petrenko, A., Yevtushenko, N., Lebedev, A., Das, A. (1993). Nondeterministic state machines in protocol conformance testing. In Rafiq, O. (Ed.) Protocol Test Systems, VI, Proceedings of the IFIP TC6/WG6.1 Sixth International Workshop on Protocol Test systems, Pau, France, 28-30 September, 1993, North-Holland, IFIP Transactions, vol. C-19 (pp. 363–378).Google Scholar
  17. Safra, S. (1988). On the complexity of omega-automata. In Proceedings of the 29th annual symposium on foundations of computer science. SFCS ’88 (pp. 319–327). Washington: IEEE Computer Society, DOI, (to appear in print).
  18. Simão, A, Petrenko, A., Yevtushenko, N. (2012). On reducing test length for FSMs with extra states. Software Testing, Verification and Reliability, 22(6), 435–454. Scholar
  19. Sistla, A.P. (1994). Safety, liveness and fairness in temporal logic. Formal Asp Comput, 6(5), 495–512. Scholar
  20. Vasilevskii, M.P. (1973). Failure diagnosis of automata. Kibernetika (Transl), 4, 98–108.MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceUniversity of BremenBremenGermany

Personalised recommendations