Using Model Checking to Generate Tests from Requirements Specifications

  • Angelo Gargantini
  • Constance Heitmeyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1687)

Abstract

Recently, many formal methods, such as the SCR (Software Cost Reduction) requirements method, have been proposed for improving the quality of software specifications. Although improved specifications are valuable, the ultimate objective of software development is to produce software that satisfies its requirements. To evaluate the correctness of a software implementation, one can apply black-box testing to determine whether the implementation, given a sequence of system inputs, produces the correct system outputs. This paper describes a specification-based method for constructing a suite of test sequences, where a test sequence is a sequence of inputs and outputs for testing a software implementation. The test sequences are derived from a tabular SCR requirements specification containing diverse data types, i.e., integer, boolean, and enumerated types. From the functions defined in the SCR specification, the method forms a collection of predicates called branches, which “cover” all possible software behaviors described by the specification. Based on these predicates, the method then derives a suite of test sequences by using a model checker’s ability to construct counterexamples. The paper presents the results of applying our method to four specifications, including a sizable component of a contractor specification of a real system.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    P. Ammann, P. Black, and W. Majurski. Using model checking to generate tests from specifications. In Proc. 2nd IEEE Intern. Conf. on Formal Engineering Methods (ICFEM’98), Brisbane, Australia, December 1998.Google Scholar
  2. 2.
    B. Beizer. Software Testing Techniques. Van Nostrand Reinhold, 1983.Google Scholar
  3. 3.
    R. Bharadwaj and C. Heitmeyer. Model checking complete requirements specifications using abstraction. Automated Software Eng. J., 6(1), January 1999.Google Scholar
  4. 4.
    M. R. Blackburn, R. D. Busser, and J. S. Fontaine. Automatic generation of test vectors for SCR-style specifications. In Proc. 12th Annual Conf. on Computer Assurance (COMPASS’ 97), Gaithersburg, MD, June 1997.Google Scholar
  5. 5.
    J. Callahan, F. Schneider, and S. Easterbrook. Specification-based testing using model checking. In Proc. SPIN Workshop, Rutgers University, August 1996. Tech. Report NASA-IVV-96-022.Google Scholar
  6. 6.
    P.-J. Courtois and David L. Parnas. Documentation for safety critical software. In Proc. 15th Int’l Conf. on Softw. Eng. (ICSE’ 93), Baltimore, MD, 1993.Google Scholar
  7. 7.
    S. Easterbrook and J. Callahan. Formal methods for verification and validation of partial specifications: A case study. Journal of Systems and Software, 1997.Google Scholar
  8. 8.
    A. Engels, L.M.G. Feijs, and S. Mauw. Test generation for intelligent networks using model checking. In Proc. TACAS’97, pages 384–398. Springer, 1997. in E. Brinksma, editor, LNCS 1217.Google Scholar
  9. 9.
    S. R. Faulk, L. Finneran, J. Kirby, Jr., S. Shah, and J. Sutton. Experience applying the CoRE method to the Lockheed C-130J. In Proc. 9th Annual Conf. on Computer Assurance (COMPASS’ 94), Gaithersburg, MD, June 1994.Google Scholar
  10. 10.
    S. Fujiwara, G. Bochmann, F. Khendek, M. Amalou, and A. Ghedamsi. Test selection based on finite state models. IEEE Trans. on Softw. Eng., 17(6), June 1991.Google Scholar
  11. 11.
    K. Havelund and N. Shankar. Experiments in theorem proving and model checking for protocol verification. In Proc. Formal Methods Europe (FME’96), pages 662–681. Springer-Verlag, March 1996. LNCS 1051.Google Scholar
  12. 12.
    C. Heitmeyer, J. Kirby, B. Labaw, M. Archer, and R. Bharadwaj. Using abstraction and model checking to detect safety violations in requirements specifications. IEEE Trans. on Softw. Eng., 24(11), November 1998.Google Scholar
  13. 13.
    C. Heitmeyer, J. Kirby, Jr., and B. Labaw. Tools for formal specification, verification, and validation of requirements. In Proc. 12th Annual Conf. on Computer Assurance (COMPASS’ 97), Gaithersburg, MD, June 1997.Google Scholar
  14. 14.
    C. L. Heitmeyer, R. D. Jeffords, and B. G. Labaw. Automated consistency checking of requirements specifications. ACM Trans. on Software Eng. and Methodology, 5(3):231–261, April–June 1996.CrossRefGoogle Scholar
  15. 15.
    K. Heninger, D. Parnas, J. Shore, and J. Kallander. Software requirements for the A-7E aircraft. Technical Report 3876, Naval Research Lab., Wash., DC, 1978.Google Scholar
  16. 16.
    G. J. Holzmann. The model checker SPIN. IEEE Trans. on Software Engineering, 23(5):279–295, May 1997.CrossRefMathSciNetGoogle Scholar
  17. 17.
    W. E. Howden. A functional approach to program testing and analysis. IEEE Trans. on Softw. Eng., 15:997–1005, October 1986.Google Scholar
  18. 18.
    R. Jeffords and C. Heitmeyer. Automatic generation of state invariants from requirements specifications. In Proc. Sixth ACM SIGSOFT Symp. on Foundations of Software Engineering, November 1998.Google Scholar
  19. 19.
    D. Mandrioli, S. Morasca, and A. Morzenti. Generating test cases for real-time systems from logic specifications. ACM Trans. on Computer Systems, 13(4):365–398, 1995.CrossRefGoogle Scholar
  20. 20.
    K. L. McMillan. Symbolic Model Checking. Kluwer Academic Publishers, Englewood Cliffs, NJ, 1993.MATHGoogle Scholar
  21. 21.
    S. P. Miller. Specifying the mode logic of a flight guidance system in CoRE and SCR. In Proc. 2nd ACM Workshop on Formal Methods in Software Practice (FMSP’98), 1998.Google Scholar
  22. 22.
    D. J. Richardson, S. L. Aha, and T. O’Malley. Specification-based test oracles for reactive systems. In Proc. 14th Intern. Conf. on Software Eng., pages 105–118. Springer, May 1992.Google Scholar
  23. 23.
    P. Stocks and D. Carrington. A framework for specification-based testing. IEEE Trans. on Softw. Eng., 22(11):777–793, November 1996.CrossRefGoogle Scholar
  24. 24.
    E. Weyuker, T. Goradia, and A. Singh. Automatically generating test data from a boolean specification. IEEE Trans. on Softw. Eng., 20:353–363, May 1994.MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Angelo Gargantini
    • 1
  • Constance Heitmeyer
    • 2
  1. 1.Politecnico di MilanoMilanoItaly
  2. 2.Naval Research LaboratoryWashingtonUSA

Personalised recommendations