Abstract
In this paper, a novel safety-related variant of complete test suites for finite state machines is introduced. Under certain hypotheses which are similar to the ones used in the well-known W-Method and its improved versions, the new method guarantees to uncover every violation of safety properties from a certain well-defined class, while erroneous behaviour without safety relevance may remain undetected. While the method can be based on any of the known complete strategies for FSM testing, its most effective variant is based on the H-method, and this variant is presented in detail, denoted as the Safety-complete H-Method. It is guaranteed that application of the Safety-complete H-Method always results in less or equally many test cases than when applying the original H-Method. In well-defined situations that can be pre-determined from the reference model, the Safety-complete H-Method leads to a substantial reduction of test cases in comparison to the size of the analogous H test suites. We advocate this new test suite for situations, where exhaustive testing of the complete system is too expensive. In these cases, strong guarantees with respect to fault coverage should only be given for the errors representing safety violations, while it may be considered as acceptable if less critical errors remain undetected.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The Safety-complete H-Method is implemented in file fsm-generator.cpp.
References
Baier, C., & Katoen, J. (2008). Principles of model checking. Cambridge: MIT Press.
Chow, T.S. (1978). Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering SE, 4(3), 178–186.
Dorofeeva, R., El-Fakih, K., Yevtushenko, N. (2005). An improved conformance testing method. In: Wang, F. (ed) Formal Techniques for Networked and Distributed Systems - FORTE 2005, 25th IFIP WG 6.1 International Conference, Taipei, Taiwan, October 2-5, 2005, Proceedings, Springer, Lecture Notes in Computer Science, vol. 3731, pp. 204–218 . https://doi.org/10.1007/11562436_16.
Fernandez, J., Mounier, L., Pachon, C. (2003). Property oriented test case generation. In: Petrenko, A., Ulrich, A. (eds) Formal Approaches to Software Testing, Third International Workshop on Formal Approaches to Testing of Software, FATES 2003, Montreal, Quebec, Canada, October 6th, 2003, Springer, Lecture Notes in Computer Science, vol. 2931, pp. 147–163 . https://doi.org/10.1007/978-3-540-24617-6_11.
Fujiwara, S., Gv, Bochmann, Khendek, F., Amalou, M., Ghedamsi, A. (1991). Test selection based on finite state models. IEEE Transactions on Software Engineering, 17(6), 591–603. https://doi.org/10.1109/32.87284.
Haxthausen, A.E., & Peleska, J. (2000). Formal development and verification of a distributed railway control system. IEEE Transaction on Software Engineering, 26(8), 687–701.
Huang, W., & Peleska, J. (2017a). Safety-complete test suites. In: Yevtushenko, N., Cavalli, AR., Yenigün, H. (eds) Testing Software and Systems - 29th IFIP WG 6.1 International Conference, ICTSS 2017, St. Petersburg, Russia, October 9-11, 2017, Proceedings, Springer, Lecture Notes in Computer Science, vol. 10533, pp. 145–161. https://doi.org/10.1007/978-3-319-67549-7_9.
Huang, Wl, & Peleska, J. (2017b). Complete model-based equivalence class testing for nondeterministic systems. Formal Aspects of Computing, 29(2), 335–364. https://doi.org/10.1007/s00165-016-0402-2.
Huang, W.L., Peleska, J., Schulze, U. (2014). Contract support for an evolving SoS. Tech. Rep. D34.3, COMPASS Comprehensive Modelling for Advanced Systems of Systems, available under http://www.compass-research.eu/deliverables.html.
Jorgensen, P.C. (2017). The craft of Model-Based testing. Boca Raton: CRC Press.
Li, S., & Qi, Z. (2004). Property-oriented testing: an approach to focusing testing efforts on behaviours of interest. In: Beydeda S., Gruhn V., Mayer J., Reussner R.H., Schweiggert F. (eds) Testing of Component-Based Systems and Software Quality, Proceedings of SOQUA 2004 (First International Workshop on Software Quality) and TECOS 2004 (Workshop Testing Component-Based Systems), GI, LNI, vol. 58, pp. 191–206. http://subs.emis.de/LNI/Proceedings/Proceedings58/article3512.html.
Luo, G., Bochmann, G., Petrenko, A. (1994). Test selection based on communicating nondeterministic finite-state machines using a generalized wp-method. IEEE Trans Software Eng, 20(2), 149–162. https://doi.org/10.1109/32.265636. http://doi.ieeecomputersociety.org/10.1109/32.265636.
Luo, G., Petrenko, A.V., Bochmann, G. (1995). Selecting test sequences for partially-specified nondeterministic finite state machines, (pp. 95–110). Boston: Springer US. https://doi.org/10.1007/978-0-387-34883-4_6.
Machado, P.D.L., Silva, D.A., Mota, A.C. (2007). Towards property oriented testing. Electronic Notes in Theoretical Computer Science, 184(Supplement C), 3–19. https://doi.org/10.1016/j.entcs.2007.06.001. http://www.sciencedirect.com/science/article/pii/S157106610700432X.
Peleska, J., & Huang, Wl. (2017). Test automation - foundations and applications of model-based testing. University of Bremen, lecture notes, available under http://www.cs.uni-bremen.de/agbs/jp/papers/test-automation-huang-peleska.pdf.
Petrenko, A., Yevtushenko, N., Lebedev, A., Das, A. (1993). Nondeterministic state machines in protocol conformance testing. In Rafiq, O. (Ed.) Protocol Test Systems, VI, Proceedings of the IFIP TC6/WG6.1 Sixth International Workshop on Protocol Test systems, Pau, France, 28-30 September, 1993, North-Holland, IFIP Transactions, vol. C-19 (pp. 363–378).
Safra, S. (1988). On the complexity of omega-automata. In Proceedings of the 29th annual symposium on foundations of computer science. SFCS ’88 (pp. 319–327). Washington: IEEE Computer Society, DOI https://doi.org/10.1109/SFCS.1988.21948, (to appear in print).
Simão, A, Petrenko, A., Yevtushenko, N. (2012). On reducing test length for FSMs with extra states. Software Testing, Verification and Reliability, 22(6), 435–454. https://doi.org/10.1002/stvr.452. https://onlinelibrary.wiley.com/doi/abs/10.1002/stvr.452.
Sistla, A.P. (1994). Safety, liveness and fairness in temporal logic. Formal Asp Comput, 6(5), 495–512. https://doi.org/10.1007/BF01211865.
Vasilevskii, M.P. (1973). Failure diagnosis of automata. Kibernetika (Transl), 4, 98–108.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, Wl., Özoguz, S. & Peleska, J. Safety-complete test suites. Software Qual J 27, 589–613 (2019). https://doi.org/10.1007/s11219-018-9421-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-018-9421-y