Skip to main content
SpringerLink
Log in

AI Planner Assisted Test Generation

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

This paper describes an AI planner assisted approach to generate test cases for system testing based on high level test objectives. We use four levels of test generation: the metaprocessor, the preprocessor, the AI planner, and the postprocessor levels. Test generation is based on an extended UML model of the system under test and a mapping of high-level test objectives into initial and goal conditions of the planner. Test objectives are derived from a series of interviews with professional testers. We suggest various options for test criteria related to test objectives. The AI planner was used to generate hundreds of test cases for a robot controlled tape silo. The planner generated tests within a reasonable time. It was successful for each test objective given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ammann, P.E., Black, P.E., and Majurski, W. 1998. Using model checking to generate tests from specications, Proceedings of the Second IEEE International Conference on Formal Engineering Methods, Brisbane, Australia, pp. 46-54.

  • Anderson, J.S. 1993. Automating Requirements Engineering Using Artificial Intelligence Techniques, Ph.D. Thesis, Dept. of Computer and Information Science, University of Oregon.

  • Barrett, A., Christianson, D., Kwok, C., Golden, K., Penberthy, S., Sun, Y., and Weld, D. 1995. UCPOP User's Manual, Technical Report 93-09-06d, University of Washington, Dept. of Computer Science and Engineering.

  • Bazzichi, F. and Spadafora, I. 1982. An automatic generator for compiler testing, IEEE Transactions on Software Engineering, 8(4): 343-353.

    Google Scholar 

  • Chow, T.S. 1977. Testing software design modeled by finite state machines, Proceedings of the First COMPSAC, pp. 58-64.

  • Coleman, D., Arnold, A., Bodoff, S., Dollin, C., Gilchrist, H., Hayes, F., and Jeremaes, P. 1994. Object oriented Development, The Fusion Method, Englewood Cliffs, Prentice Hall.

    Google Scholar 

  • Fickas, S. and Anderson, J. 1988. A proposed perspective shift: viewing specification design as a planning problem, Department of Computer and Information Science CIS-TR-88-15, University of Oregon, Eugene, OR.

    Google Scholar 

  • Fickas, S. and Helm, B.R. 1992. Knowledge representation and reasoning in the design of composite systems, IEEE Transactions on Software Engineering, SE-18(6): 470-482.

    Google Scholar 

  • Figliulo, T., von Mayrhauser, A., Shumway, M., and Karcich, R. 1996. Experiences with automated system testing and Sleuth, Proceedings of the IEEE Aerospace Applications Conference 1996, Snowmass, Colorado, Vol. 4, pp. 335-349.

    Google Scholar 

  • Fowler, M. 1997. UML Distilled, Applying the Standard Object Modeling Language, Reading. Addison-Wesley.

    Google Scholar 

  • Fujiwara, S., von Bochman, G., Khendek, F., Amalou, M., and Ghedamsi, A. 1991. Test selection based on finite state models, IEEE Transactions on Software Engineering, SE-17(10): 591-603.

    Google Scholar 

  • Hamlet, D. and Taylor, R. 1990. Partition testing does not inspire confidence, IEEE Transactions on Software Engineering, SE-16(12): 1402-1411.

    Google Scholar 

  • Howe, A.E., Dahlman, E., Hansen, C., Scheetz, M., and von Mayrhauser, A. 1999. Exploiting competitive planner performance, Proceedings of the European Conference on Planning (ECP-99), Durham, England, pp. 60-72.

  • Howe, A.E., von Mayrhauser, A., and Mraz, R.T. 1997. Test case generation as an AI planning problem, Automated Software Engineering, 4(1): 77-106.

    Google Scholar 

  • Huff, K. and Lesser, V. 1988. A plan-based intelligent assistant that supports the software development process, Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, pp. 97-106.

  • Huff, K. 1992. Software adaptation, Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse, pp. 63-66.

  • Ince, D.C. 1987. The automatic generation of test data, Computer Journal, 30(1): 63-69.

    Google Scholar 

  • Kasik, D.J. and George, H.G. 1996. Toward automatic generation of user test scripts, Proceedings of the Conference on Human Factors in Computing Systems: Common Ground, pp. 244-251.

  • Maurer, P. 1990. Generating test data withenh anced context-free grammars, IEEE Software, July: 50-55.

  • McDermott, D., Ghallab, M., Howe, A.E., Knobloch, C., Ram, A., Veloso, M., Weld, D., and Wilkins, D. 1999. The Planning Domain Definition Language, Technical Report, Yale University, Computer Science Department, New Haven, Connecticut, Release 1.7, March1999 (http://cswww.cs.yale.edu/homes/dvm).

    Google Scholar 

  • Memon, A.M., Pollack, M.E., and Soffa, M.L. 2000. Automated test oracles for GUIs, Proceedings of the Eighth International Symposium on the Foundations of Software Engineering, pp. 30-39. New York, NY, ACM Press.

    Google Scholar 

  • Memon, A.M., Pollack, M.E., and Soffa, M.L. 2001. Hierarchical GUI test case generation using automated planning, IEEE Transactions on Software Engineering, 27(2): 144-155.

    Google Scholar 

  • Ostrand, T.J. and Balcer, M.J. 1988. The category-partition method for specifying and generating functional tests, Communications of the ACM, 31(6): 676-686. New York, NY, ACM Press.

    Google Scholar 

  • Purdom, P.A. 1972. A sentence generator for testing parsers, BIT, 12(3): 366-375.

    Google Scholar 

  • Richardson, D.J., O'Malley, O., and Tittle, C. 1993. Approaches to speci.cation-based testing, Proceedings of the ACM Third Symposium on Software Testing, Analysis, and Verification (TAV3), pp. 86-96. New York, NY, ACM Press.

    Google Scholar 

  • Rist, R.S. 1992. Plan identification and re-use in programs, Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse, pp. 67-72.

  • Scheetz, M., von Mayrhauser, A., France, R., Dahlman, E., and Howe, A.E. 1999. Generating test cases from an OO model withan AI planning system, Proceedings of the International Symposium on Software Reliability Engineering, pp. 250-259. Los Alamitos, CA, IEEE Computer Society Press.

    Google Scholar 

  • Stocks, P. and Carrington, D. 1996. A framework for speci.cation-based testing, IEEE Transactions on Software Engineering, 22(11): 777-793.

    Google Scholar 

  • StorageTek. 1992. StorageTek 4400 Operator's Guide, Host Software Component (VM) Rel 1.2.0, StorageTek.

  • Tsoulakos, M., Duran, J.W., and Ntafos, S.C. 1993. On some reliability estimation problems in random and partition testing, IEEE Transactions on Software Engineering, SE-19(7): 687-697.

    Google Scholar 

  • von Mayrhauser, A., Walls, J., and Mraz, R.T. 1994. Testing applications using domain based testing and Sleuth, Proceedings of the Fifth International Software Reliability Engineering Symposium, pp. 206-215.

  • von Mayrhauser, A., Scheetz, M., Dahlman, M.E., and Howe, A.E. 1999. Trade-offs in planner representation for automated software testing, Proceedings of the IEEE Aerospace Conference, Section 7. 402: 1-10. Piscataway, NJ, IEEE Press.

    Google Scholar 

  • von Mayrhauser, A., Scheetz, M., Dahlman, E., and Howe, A.E. 2000. Planner based error recovery testing, Proceedings of the International Symposium on Software Reliability Engineering, pp. 186-195.

  • von Mayrhauser, A., Scheetz, M., France, R., and Dahlman, E. 2000. Generating test cases from an OO model withan AI planning system, IEEE Transactions on Reliability, 49(1): 26-36.

    Google Scholar 

  • Weld, D.S. 1999. Recent advances in AI planning, AI Magazine, 20(2): 93-122.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA, 99163

    Anneliese K. Amschler Andrews

  2. Computer Science Department, Colorado State University, Fort Collins, CO, 80523

    Chunhui Zhu, Michael Scheetz, Eric Dahlman & Adele E. Howe

Authors
  1. Anneliese K. Amschler Andrews
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunhui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Scheetz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric Dahlman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adele E. Howe
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Amschler Andrews, A.K., Zhu, C., Scheetz, M. et al. AI Planner Assisted Test Generation. Software Quality Journal 10, 225–259 (2002). https://doi.org/10.1023/A:1021686406575

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1021686406575

Search

Navigation