From Rule-Based to Automata-Based Testing
Rule based languages have been used extensively to provide a declarative description of causal relationships between events and conditions in a wide variety of systems. On the other hand, automata (finite state machine) based models provide a detailed operational specification of how the system state evolves over time. While rules are a convenient declarative mechanism, a finite automaton is more flexible and far more easily analyzed than a collection of rules with potentially complicated semantics.
In this paper, we address this dichotomy in the context of large scale testing. We describe a (probabilistic) rule based system, which has been developed and used over many years as the testing model of a large telephone switch. One important drawback of the rule-based system is that it is inflexible in terms of the test generation mechanisms it supports, essentially only allowing the generation of random tests.
We have designed and implemented a translation from the rule-based language to a probabilistic finite automaton model, and applied it to the switch model. Our translation makes use of several automata-theoretic algorithms to keep the size of the resulting automata manageable.
The automaton model allows much more flexible and targeted test generation. We have designed and implemented an assortment of targeted test generation algorithms which are applied to the probabilistic automaton.
KeywordsSystem testing rule based systems finite automata.
- Alur, R. and Yannakakis, M. (1998). Model checking of hierarchical state machines. In Proc. 6th ACM Symp. on Foundations of Software Engineering, pages 175–188.Google Scholar
- Baier, C., Engelen, B., and Majster-Cederbaum, M. (2000). Deciding bisimlarity and similarity for probabilistic processes Journal of Computer and System Sciences60(1):187–231 Google Scholar
- Beizer, B. (1990). Software Testing Techniques. Thompson.Google Scholar
- Brinksma, E. (1988). A theory for the derivation of tests. In Proc. IFIP WG6.1 8th PST V.Google Scholar
- Holzmann, G. J. (1990). Design and Validation of Protocols. Prentice-Hall.Google Scholar
- Holzmann, G. J. and Smith, M. H. (1999). A practical method for the verification of event driven systems. In Pro.c Intl. Conf. on Software Engineering, pages 597–608.Google Scholar
- Hull, R., Kumar, B., Llirbat, F., Simon, E., Su, J., and Zhou, G. (1999). Declarative workflows that support easy modification and dynamic browsing. In Proc. WA CC Google Scholar
- Lee, D. and Yannakakis, M. (1996a). Optimization problems from feature testing of communication protocols. In Proc. Intl. Conf. on Network Protocols, pages 66–75.Google Scholar
- Lee, D. and Yannakakis, M. (1996b). Principles and methods of testing finite state machines - a survey. Proc. of IEEE,84(8):10901123.Google Scholar
- Ullman, J. D. (1988). Principles of Database and Knowledge-base systems. Computer Science Press.Google Scholar
- Vardi, M. Y. and Wolper, P. (1986). An automata-theoretic approach to automatic program verification. In Kozen, D., editor, First IEEE Symp. on Logic in Computer Science (LICS), pages 322–331.Google Scholar
- Widom, J. and S. Ceri, E. (1996). Active Database Systems: Triggers and Rules for Advanced Database Processing. Morgan Kaufmann.Google Scholar