Automated Boundary Testing from Z and B
We present a method for black-box boundary testing from B and Z formal specifications. The basis of the method is to test every operation of the system at every boundary state using all input boundary values of that operation. The test generation process is highly automated. It starts by calculating boundary goals from Pre/Post predicates derived from the formal model. Then each boundary goal is instantiated to a reachable boundary state, by searching for a sequence of operations that reaches the boundary goal from the initial state. This process makes intensive use of a set-oriented constraint technology, both for boundary computation and to traverse the state space. The method was designed on the basis of industrial applications in the domain of critical software (Smart card and transportation). Application results show the effectiveness and the scalability of the method. In this paper, we give an overview of the method and focus on the calculation of the boundary goals and states.
Keywordsspecification-based testing boundary values set constraint solving B method Z notation
Unable to display preview. Download preview PDF.
- 1.E. Bernard, B. Legeard, X. Luck, and F. Peureux. Generation of test sequences from formal specifications: GSM 11.11 standard case-study. Submitted to the Journal of Software Practice and Experience, 2002.Google Scholar
- 2.M-C. Gaudel. Testing can be formal too. TAPSOFT’95: Theory and Practice of Software Development, LNCS 915:82–96, May 1995.Google Scholar
- 4.J.M. Spivey. The Z notation: A Reference Manual. Prentice-Hall, 2 nd edition, 1992. ISBN 0 13 978529 9.Google Scholar
- 5.C.B. Jones. Systematic Software Development Using VDM. Prentice-Hall, 2 nd edition, 1990.Google Scholar
- 6.J-R. Abrial. The B-BOOK: Assigning Programs to Meanings. Cambridge University Press, 1996. ISBN 0 521 49619 5.Google Scholar
- 9.L. Murray, D. Carrington, I. MacColl, J. McDonald, and P. Strooper. Formal derivation of finite state Machines for class testing. In Proceedings of the Z User Meeting (ZUM’98): The Z Formal Specification Notation, pages 42–59, Berlin, Germany, September 1998. Springer-Verlag.Google Scholar
- 10.L. van Aertryck, M. Benveniste, and D. le Metayer. CASTING: a formally based software test generation method. In 1st IEEE International Conference on Formal Engineering Methods (ICFEM’97), pages 99–112, 1997.Google Scholar
- 11.W. Grieskamp, Y. Gurevich, W. Schulte, and M. Veanes. Testing with abstract state machines. In Formal Methods and Tools for Computer Science (EURO-CAST’01)-Extended Abstracts, february 2001.Google Scholar
- 13.B. Legeard and F. Peureux. Generation of functional test sequences from B formal specifications-Presentation and industrial case-study. In 16th IEEE International Conference on Automated Software Engineering (ASE’01), San Diego, USA, November 2001.Google Scholar
- 14.F. Bouquet, J. Julliand, B. Legeard, and F. Peureux. Automatic reconstruction and generation of functional test patterns-application to the Java card transaction mechanism (confidential). Technical Report TR-01/02, LIFC-University of Franche-Comt and Schlumberger Montrouge Product Center, 2002.Google Scholar
- 15.N. Caritey, L. Gaspari, B. Legeard, and F. Peureux. Specification-based testing-Application on algorithms of Metro and RER tickets (confidential). Technical Report TR-03/01, LIFC-University of Franche-Comt and Schlumberger Besan on, 2001.Google Scholar
- 16.F. Ambert, B. Legeard, and E. Legros. Constraint Logic Programming on Sets and Multisets. In Proceedings of ICLP’94-Workshop on Constraint Languages and their use in Problem Modeling, pages 151–165, Ithaca, New York, November 1994.Google Scholar
- 17.F. Bouquet, B. Legeard, and F. Peureux. CLPS-B-A constraint solver for B. In Proceedings of the conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’02), Grenoble, France, April 2002.Google Scholar
- 18.ISO. Information Processing Systems, Open Systems Interconnection. OSI Conformance Testing Methodology and Framework-ISO 9646.Google Scholar
- 19.P. Behm, P. Desforges, and J.M. Meynadier. METEOR: An Industrial Success in Formal Development. In Second conference on the B method, Montpellier, France, 1998.Google Scholar
- 20.Clearsy, Europarc de Pichaury 13856 Aix-en-Provence Cedex 3-France. Atelier B Technical Support version 3, May 2001-http://www.atelierb.societe.com.
- 21.G.J. Myers. The Art of Software Testing. Wiley-InterScience, 1979.Google Scholar
- 22.A. Pretschner. Classical search strategies for test case generation with Constraint Logic Programming. In BRICS, editor, Proceedings of the Workshop on Formal Approaches to Testing of Software (FATES’01), pages 47–60, Aalborg, Denmark, August 2001.Google Scholar
- 23.D.J. Richardson and S.L. Aha T.O. O’Malley. Specification-based test oracles for reactive systems. In Proceedings of the 14 th International Conference on Software Engineering (ICSE’92), pages 105–118, Melbourne, Australia, May 1992. ACM Press.Google Scholar
- 24.Sun microsystems. Java Card 2.1.1 Virtual Machine Specification. http://java.sun.com/products/javacard/javacard21.html#specification, May 2000.