Abstract
The Godzilla automatic test data generator is an integrated collection of tools that implements a relatively new test data generation method—constraint-based testing—that is based on mutation analysis. Constraint-based testing integrates mutation analysis with several other testing techniques, including statement coverage, branch coverage, domain perturbation, and symbolic evaluation. Because Godzilla uses a rule-based approach to generate test data, it is easily extendible to allow new testing techniques to be integrated into the current system. This article describes the system that has been built to implement constraint-based testing. Godzilla's design emphasizes orthogonality and modularity, allowing relatively easy extensions. Godzilla's internal structure and algorithms are described with emphasis on internal structures of the system and the engineering problems that were solved during the implementation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A.T. Acree, “On mutation,” Ph.D. dissertation, Georgia Institute of Technology, Atlanta, GA, 1980.
A.T. Acree, T.A. Budd, R.A. DeMillo, R.J. Lipton, and F.G. Sayward, “Mutation analysis,” Technical Report GIT-ICS-79/08, School of Information and Computer Science, Georgia Institute of Technology, Atlanta, GA, September 1979.
L.A. Clarke, “A system to generate test data and symbolically execute programs,”IEEE Trans. Software Eng. vol. 2, no. 3, pp. 215–222, September 1976.
L.A. Clarke, A. Podgurski, D.J. Richardson, and S.J. Zeil, “A comparison of data flow path selection criteria,” inProc. Eighth Int. Conf. Software Eng., London UK, pp. 244–251, August 1985.
L.A. Clarke and D.J. Richardson, “Applications of symbolic evaluation,”J. Syst. Software, vol. 5, no. 1, pp. 15–35, 1985.
R.A. DeMillo, D.S. Guindi, K.N. King, W.M. McCracken, and A.J. Offutt, “An extended overview of the Mothra software testing environment,” inProc. Second Workshop Software Testing, Verification, and Analysis, Banff, Alberta, Canada, pp. 142–151, July 1988.
R.A. DeMillo, E.W. Krauser, R.J. Martin, A.J. Offutt, and E.H. Spafford, “The Mothra tool set,” inProc. 22nd Hawaii Int. Conf. Syst. Sci., Kailua-Kona, HI, pp. 275–284, January 1989.
R.A. DeMillo, R.J. Lipton, and F.G. Sayward, “Hints on test data selection: Help for the practicing programmer,”IEEE Computer vol. 11, no. 4, pp. 34–41, April 1978.
R.A. DeMillo, W.M. McCracken, R.J. Martin, and J.F. Passafiume,Software Testing and Evaluation. Benjamin/Cummings: Menlo Park, CA, 1987.
R.A. DeMillo and A.J. Offutt, “Constraint-based automatic test data generation,”IEEE Trans. Software Eng. vol. 17, no. 9, September 1991.
R.A. DeMillo and A.J. Offutt, “Experimental results of automatically generated adequate test sets,” inProc. Sixth Annual Pacific Northwest Software Quality Conf., Portland, OR, pp. 209–232, September 1988.
J.B. Goodenough, “A survey of program issues,” in P. Wegner, ed.,Research Directions in Software Technology. Prentice-Hall: Englewood Cliffs, NJ, 1979, pp. 316–340.
W.E. Howden, “Reliability of the path analysis testing strategy,”IEEE Trans. Software Eng. vol. 2, no. 3, pp. 208–215, September 1976.
W.E. Howden, “Weak mutation testing and completeness of test sets,”IEEE Trans. Software Eng. vol. 8, no. 4, pp. 371–379, July 1982.
K.N. King and A.J. Offutt, “A Fortran language system for mutation-based software testing,”Software Practice and Experience vol. 21, no. 7, pp. 686–718, July 1991.
R.J. Lipton and F.G. Sayward, “The status of research on program mutation,” inDigest for the Workshop on Software Testing and Test Documentation, pp. 355–373, December 1978.
Z. Manna,Mathematical Theory of Computation. McGraw-Hill: New York, 1974.
L.J. Morell, “Theoretical insights into fault-based testing,” inProc. Second Workshop on Software Testing, Verification, and Analysis, Banff, Alberta, Canada, pp. 45–62, July 1988.
G. Myers,The Art of Software Testing. Wiley: New York, 1979.
A.J. Offutt, “Automatic test data generation,” Ph.D. dissertation, Georgia Institute of Technology, Atlanta, GA, 1988. Technical Report GIT-ICS 88/28 (also released as Purdue University Software Engineering Research Center Technical Report SERC-TR-25-P).
A.J. Offutt, “The coupling effect: Fact or fiction?” inProc. Third Symp. on Software Testing, Analysis, and Verification, Key West, FL, pp. 131–140, December 1989.
A.J. Offutt, “Using mutation analysis to test software,” inProc. Seventh International Conference on Testing Computer Software, San Francisco, CA, pp. 65–77, June 1990.
S. Rapps and W.J. Weyuker, “Selecting software test data using data flow information,”IEEE Trans. Software Eng. vol. 11, no. 4, pp. 367–375, April 1985.
E.J. Weyuker, “Assessing test data adequacy through program inference,”ACM Trans. Programming Languages Syst. vol. 5, no. 4, pp. 641–655, October 1983.
M.R. Woodward and K. Halewood, “From weak to strong, dead or alive? An analysis of some mutation testing issues,” inProc. Second Workshop on Software Testing, Verification, and Analysis, Banff, Alberta, Canada, pp. 152–158, July 1988.
Author information
Authors and Affiliations
Additional information
Parts of this research were supported by Contract F30602-85-C-0255 through Rome Air Development Center while the author was a graduate student at the Georgia Institute of Technology.
Rights and permissions
About this article
Cite this article
Offutt, A.J. An integrated automatic test data generation system. Journal of Systems Integration 1, 391–409 (1991). https://doi.org/10.1007/BF02262722
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02262722