Advertisement

Journal of Systems Integration

, Volume 1, Issue 3–4, pp 391–409 | Cite as

An integrated automatic test data generation system

  • A. Jefferson Offutt
Article

Abstract

The Godzilla automatic test data generator is an integrated collection of tools that implements a relatively new test data generation method—constraint-based testing—that is based on mutation analysis. Constraint-based testing integrates mutation analysis with several other testing techniques, including statement coverage, branch coverage, domain perturbation, and symbolic evaluation. Because Godzilla uses a rule-based approach to generate test data, it is easily extendible to allow new testing techniques to be integrated into the current system. This article describes the system that has been built to implement constraint-based testing. Godzilla's design emphasizes orthogonality and modularity, allowing relatively easy extensions. Godzilla's internal structure and algorithms are described with emphasis on internal structures of the system and the engineering problems that were solved during the implementation.

Key Words

constraints fault-based testing mutation testing software testing test data generation unit testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    A.T. Acree, “On mutation,” Ph.D. dissertation, Georgia Institute of Technology, Atlanta, GA, 1980.Google Scholar
  2. 2.
    A.T. Acree, T.A. Budd, R.A. DeMillo, R.J. Lipton, and F.G. Sayward, “Mutation analysis,” Technical Report GIT-ICS-79/08, School of Information and Computer Science, Georgia Institute of Technology, Atlanta, GA, September 1979.Google Scholar
  3. 3.
    L.A. Clarke, “A system to generate test data and symbolically execute programs,”IEEE Trans. Software Eng. vol. 2, no. 3, pp. 215–222, September 1976.Google Scholar
  4. 4.
    L.A. Clarke, A. Podgurski, D.J. Richardson, and S.J. Zeil, “A comparison of data flow path selection criteria,” inProc. Eighth Int. Conf. Software Eng., London UK, pp. 244–251, August 1985.Google Scholar
  5. 5.
    L.A. Clarke and D.J. Richardson, “Applications of symbolic evaluation,”J. Syst. Software, vol. 5, no. 1, pp. 15–35, 1985.Google Scholar
  6. 6.
    R.A. DeMillo, D.S. Guindi, K.N. King, W.M. McCracken, and A.J. Offutt, “An extended overview of the Mothra software testing environment,” inProc. Second Workshop Software Testing, Verification, and Analysis, Banff, Alberta, Canada, pp. 142–151, July 1988.Google Scholar
  7. 7.
    R.A. DeMillo, E.W. Krauser, R.J. Martin, A.J. Offutt, and E.H. Spafford, “The Mothra tool set,” inProc. 22nd Hawaii Int. Conf. Syst. Sci., Kailua-Kona, HI, pp. 275–284, January 1989.Google Scholar
  8. 8.
    R.A. DeMillo, R.J. Lipton, and F.G. Sayward, “Hints on test data selection: Help for the practicing programmer,”IEEE Computer vol. 11, no. 4, pp. 34–41, April 1978.Google Scholar
  9. 9.
    R.A. DeMillo, W.M. McCracken, R.J. Martin, and J.F. Passafiume,Software Testing and Evaluation. Benjamin/Cummings: Menlo Park, CA, 1987.Google Scholar
  10. 10.
    R.A. DeMillo and A.J. Offutt, “Constraint-based automatic test data generation,”IEEE Trans. Software Eng. vol. 17, no. 9, September 1991.Google Scholar
  11. 11.
    R.A. DeMillo and A.J. Offutt, “Experimental results of automatically generated adequate test sets,” inProc. Sixth Annual Pacific Northwest Software Quality Conf., Portland, OR, pp. 209–232, September 1988.Google Scholar
  12. 12.
    J.B. Goodenough, “A survey of program issues,” in P. Wegner, ed.,Research Directions in Software Technology. Prentice-Hall: Englewood Cliffs, NJ, 1979, pp. 316–340.Google Scholar
  13. 13.
    W.E. Howden, “Reliability of the path analysis testing strategy,”IEEE Trans. Software Eng. vol. 2, no. 3, pp. 208–215, September 1976.Google Scholar
  14. 14.
    W.E. Howden, “Weak mutation testing and completeness of test sets,”IEEE Trans. Software Eng. vol. 8, no. 4, pp. 371–379, July 1982.Google Scholar
  15. 15.
    K.N. King and A.J. Offutt, “A Fortran language system for mutation-based software testing,”Software Practice and Experience vol. 21, no. 7, pp. 686–718, July 1991.Google Scholar
  16. 16.
    R.J. Lipton and F.G. Sayward, “The status of research on program mutation,” inDigest for the Workshop on Software Testing and Test Documentation, pp. 355–373, December 1978.Google Scholar
  17. 17.
    Z. Manna,Mathematical Theory of Computation. McGraw-Hill: New York, 1974.Google Scholar
  18. 18.
    L.J. Morell, “Theoretical insights into fault-based testing,” inProc. Second Workshop on Software Testing, Verification, and Analysis, Banff, Alberta, Canada, pp. 45–62, July 1988.Google Scholar
  19. 19.
    G. Myers,The Art of Software Testing. Wiley: New York, 1979.Google Scholar
  20. 20.
    A.J. Offutt, “Automatic test data generation,” Ph.D. dissertation, Georgia Institute of Technology, Atlanta, GA, 1988. Technical Report GIT-ICS 88/28 (also released as Purdue University Software Engineering Research Center Technical Report SERC-TR-25-P).Google Scholar
  21. 21.
    A.J. Offutt, “The coupling effect: Fact or fiction?” inProc. Third Symp. on Software Testing, Analysis, and Verification, Key West, FL, pp. 131–140, December 1989.Google Scholar
  22. 22.
    A.J. Offutt, “Using mutation analysis to test software,” inProc. Seventh International Conference on Testing Computer Software, San Francisco, CA, pp. 65–77, June 1990.Google Scholar
  23. 23.
    S. Rapps and W.J. Weyuker, “Selecting software test data using data flow information,”IEEE Trans. Software Eng. vol. 11, no. 4, pp. 367–375, April 1985.Google Scholar
  24. 24.
    E.J. Weyuker, “Assessing test data adequacy through program inference,”ACM Trans. Programming Languages Syst. vol. 5, no. 4, pp. 641–655, October 1983.Google Scholar
  25. 25.
    M.R. Woodward and K. Halewood, “From weak to strong, dead or alive? An analysis of some mutation testing issues,” inProc. Second Workshop on Software Testing, Verification, and Analysis, Banff, Alberta, Canada, pp. 152–158, July 1988.Google Scholar

Copyright information

© Kluwer Academic Publishers 1991

Authors and Affiliations

  • A. Jefferson Offutt
    • 1
  1. 1.Department of Computer ScienceClemson UniversityClemson

Personalised recommendations