Safety-complete test suites
- 41 Downloads
In this paper, a novel safety-related variant of complete test suites for finite state machines is introduced. Under certain hypotheses which are similar to the ones used in the well-known W-Method and its improved versions, the new method guarantees to uncover every violation of safety properties from a certain well-defined class, while erroneous behaviour without safety relevance may remain undetected. While the method can be based on any of the known complete strategies for FSM testing, its most effective variant is based on the H-method, and this variant is presented in detail, denoted as the Safety-complete H-Method. It is guaranteed that application of the Safety-complete H-Method always results in less or equally many test cases than when applying the original H-Method. In well-defined situations that can be pre-determined from the reference model, the Safety-complete H-Method leads to a substantial reduction of test cases in comparison to the size of the analogous H test suites. We advocate this new test suite for situations, where exhaustive testing of the complete system is too expensive. In these cases, strong guarantees with respect to fault coverage should only be given for the errors representing safety violations, while it may be considered as acceptable if less critical errors remain undetected.
KeywordsModel-based testing Complete testing theories Safety
- Dorofeeva, R., El-Fakih, K., Yevtushenko, N. (2005). An improved conformance testing method. In: Wang, F. (ed) Formal Techniques for Networked and Distributed Systems - FORTE 2005, 25th IFIP WG 6.1 International Conference, Taipei, Taiwan, October 2-5, 2005, Proceedings, Springer, Lecture Notes in Computer Science, vol. 3731, pp. 204–218 . https://doi.org/10.1007/11562436_16.CrossRefGoogle Scholar
- Fernandez, J., Mounier, L., Pachon, C. (2003). Property oriented test case generation. In: Petrenko, A., Ulrich, A. (eds) Formal Approaches to Software Testing, Third International Workshop on Formal Approaches to Testing of Software, FATES 2003, Montreal, Quebec, Canada, October 6th, 2003, Springer, Lecture Notes in Computer Science, vol. 2931, pp. 147–163 . https://doi.org/10.1007/978-3-540-24617-6_11.CrossRefGoogle Scholar
- Huang, W., & Peleska, J. (2017a). Safety-complete test suites. In: Yevtushenko, N., Cavalli, AR., Yenigün, H. (eds) Testing Software and Systems - 29th IFIP WG 6.1 International Conference, ICTSS 2017, St. Petersburg, Russia, October 9-11, 2017, Proceedings, Springer, Lecture Notes in Computer Science, vol. 10533, pp. 145–161. https://doi.org/10.1007/978-3-319-67549-7_9.CrossRefGoogle Scholar
- Huang, W.L., Peleska, J., Schulze, U. (2014). Contract support for an evolving SoS. Tech. Rep. D34.3, COMPASS Comprehensive Modelling for Advanced Systems of Systems, available under http://www.compass-research.eu/deliverables.html.
- Li, S., & Qi, Z. (2004). Property-oriented testing: an approach to focusing testing efforts on behaviours of interest. In: Beydeda S., Gruhn V., Mayer J., Reussner R.H., Schweiggert F. (eds) Testing of Component-Based Systems and Software Quality, Proceedings of SOQUA 2004 (First International Workshop on Software Quality) and TECOS 2004 (Workshop Testing Component-Based Systems), GI, LNI, vol. 58, pp. 191–206. http://subs.emis.de/LNI/Proceedings/Proceedings58/article3512.html.
- Luo, G., Bochmann, G., Petrenko, A. (1994). Test selection based on communicating nondeterministic finite-state machines using a generalized wp-method. IEEE Trans Software Eng, 20(2), 149–162. https://doi.org/10.1109/32.265636. http://doi.ieeecomputersociety.org/10.1109/32.265636.CrossRefGoogle Scholar
- Machado, P.D.L., Silva, D.A., Mota, A.C. (2007). Towards property oriented testing. Electronic Notes in Theoretical Computer Science, 184(Supplement C), 3–19. https://doi.org/10.1016/j.entcs.2007.06.001. http://www.sciencedirect.com/science/article/pii/S157106610700432X.CrossRefzbMATHGoogle Scholar
- Peleska, J., & Huang, Wl. (2017). Test automation - foundations and applications of model-based testing. University of Bremen, lecture notes, available under http://www.cs.uni-bremen.de/agbs/jp/papers/test-automation-huang-peleska.pdf.
- Petrenko, A., Yevtushenko, N., Lebedev, A., Das, A. (1993). Nondeterministic state machines in protocol conformance testing. In Rafiq, O. (Ed.) Protocol Test Systems, VI, Proceedings of the IFIP TC6/WG6.1 Sixth International Workshop on Protocol Test systems, Pau, France, 28-30 September, 1993, North-Holland, IFIP Transactions, vol. C-19 (pp. 363–378).Google Scholar
- Safra, S. (1988). On the complexity of omega-automata. In Proceedings of the 29th annual symposium on foundations of computer science. SFCS ’88 (pp. 319–327). Washington: IEEE Computer Society, DOI https://doi.org/10.1109/SFCS.1988.21948, (to appear in print).
- Simão, A, Petrenko, A., Yevtushenko, N. (2012). On reducing test length for FSMs with extra states. Software Testing, Verification and Reliability, 22(6), 435–454. https://doi.org/10.1002/stvr.452. https://onlinelibrary.wiley.com/doi/abs/10.1002/stvr.452.CrossRefGoogle Scholar