Automating formal specification-based testing

  • Michael R. Donat
III FASE FASE-8: Applications of Formal Methods to Software Engineering
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1214)

Abstract

This paper presents a technique for automatically generating logical schemata that specify groups of black-box test cases from formal specifications containing universal and existential quantification. These schemata are called test frames. Previous automated techniques have dealt with languages based on prepositional logic. Since this new technique deals with quantification it can be applied to more expressive specifications. This makes the technique applicable to specifications written at the system requirements level. The limitations imposed by quantification are discussed. Industrial needs are addressed by the capabilities of recognizing and augmenting existing test frames and by accommodating a range of specification-coverage schemes. The coverage scheme taxonomy introduced in this paper provides a standard for controlling the number of test frames produced. This technique is intended to automate portions of what is done manually by practitioners. Basing this technique on formal rules of logical derivation ensures that the test frames produced are logical consequences of the specification. It is expected that deriving test frames automatically will offset the cost of developing a formal specification. This tangible product makes formal specification more economically feasible for industry.

Keywords

Test Class Boolean Expression Disjunctive Normal Form Binary Decision Diagram Test Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

References

  1. 1.
    Jean-Raymond Abrial. Steam boiler control specification problem. In Jean-Raymond Abrial, Egon Börger, and Hans Langmaack, editors, Formal Methods for Industrial Applications: Specifying and Programming the Steam Boiler Control, volume 1165 of Lecture Notes in Computer Science, pages 500–509, October 1996. http://www.informatik.uni-kiel.de∼procos/dag9523/dag9523.html.Google Scholar
  2. 2.
    Pascal Bernard. A Z specification of the boiler. http://www.informatik.uni-kiel.-de∼procos/dag9523/bernard-fulltext.ps.Z, January 1996.Google Scholar
  3. 3.
    G. Bernot, M-C. Gaudel, and B. Marre. Software testing based on formal specifications. Software Engineering Journal, 6(6), November 1991.Google Scholar
  4. 4.
    Randal E. Bryant. Graph-based algorithms for boolean function manipulation. IEEE Transactions on Computers, C-35(8):677–691, August 1986.Google Scholar
  5. 5.
    Robert Büssow and Matthias Weber. A steam-boiler control specification with Statecharts and Z. In Jean-Raymond Abrial, Egon Borger, and Hans Langmaack, editors, Formal Methods for Industrial Applications: Specifying and Programming the Steam Boiler Control, volume 1165 of Lecture Notes in Computer Science, pages 109–128, October 1996.Google Scholar
  6. 6.
    Jeremy Dick and Alain Faivre. Automating the generation and sequencing of test cases from model-based specifications. In Formal Methods Europe '93, volume 670 of Lecture Notes in Computer Science, pages 268–284, 1993.Google Scholar
  7. 7.
    Michael R. Donat. Automating System-level Testing Based on Quantified Formal Specifications. PhD thesis, Department of Computer Science, University of British Columbia, Vancouver, B.C., Canada, 1997. In preparation.Google Scholar
  8. 8.
    Michael R. Garey and David S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H.Freeman and Company, San Francisco, 1979.Google Scholar
  9. 9.
    Marie-Claude Gaudel. Testing can be formal, too. In TAPSOFT: 6th International Joint Conference on Theory and Practice of Software Development, volume 915 of Lecture Notes in Computer Science, pages 82–96, 1995.Google Scholar
  10. 10.
    M.J.C. Gordon and T.F. Melham, editors. Introduction to HOL: A theorem proving environment for higher order logic. Cambridge University Press, 1993.Google Scholar
  11. 11.
    Jeffrey J. Joyce, Nancy Day, and Michael R. Donat. S: A machine readable specification notation based on higher order logic. In Thomas F. Melham and Juanito Camilleri, editors, Higher Order Logic Theorem Proving and Its Applications, 7th International Workshop, volume 859 of Lecture Notes in Computer Science, pages 285–299. Springer-Verlag, 1994.Google Scholar
  12. 12.
    Ian MacColl, David Carrington, and Philip Stocks. An experiment in specification-based testing. Technical Report 96-05, Software Verification Research Centre, Department of Computer Science, The University of Queensland, St. Lucia, QLD 4072, Australia, May 1996.Google Scholar
  13. 13.
    Christian P. Schinagl. VDM specification of the steam-boiler control using RSL notation. In Jean-Raymond Abrial, Egon Borger, and Hans Langmaack, editors, Formal Methods for Industrial Applications: Specifying and Programming the Steam Boiler Control, volume 1165 of Lecture Notes in Computer Science, pages 428–452, October 1996.Google Scholar
  14. 14.
    J. Michael Spivey. Understanding Z: A Specification language and its formal semantics. Cambridge University Press, 1988.Google Scholar
  15. 15.
    K. Toth and J. Joyce. Industrialization of formal methods through process definition. In 5th Annual Symposium of the National Council on Systems Enqineerinq NCOSE, July 1995.Google Scholar
  16. 16.
    Kalman Toth, Michael R. Donat, and Jeffrey J. Joyce. Generating test cases from formal specifications. In 6th Annual Symposium of the International Council on Systems Engineering. INCOSE, July 1996.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Michael R. Donat
    • 1
  1. 1.University of British ColumbiaVancouverCanada

Personalised recommendations