Advertisement

Software Quality Journal

, Volume 15, Issue 3, pp 241–263 | Cite as

Rapid goal-oriented automated software testing using MEA-graph planning

  • Manish GuptaEmail author
  • Jicheng Fu
  • Farokh B. Bastani
  • Latifur R. Khan
  • I.-Ling Yen
Article

Abstract

With the rapid growth in the development of sophisticated modern software applications, the complexity of the software development process has increased enormously, posing an urgent need for the automation of some of the more time-consuming aspects of the development process. One of the key stages in the software development process is system testing. In this paper, we evaluate the potential application of AI planning techniques in automated software testing. The key contributions of this paper include the following: (1) A formal model of software systems from the perspective of software testing that is applicable to important classes of systems and is amenable to automation using AI planning methods. (2) The design of a framework for an automated planning system (APS) for applying AI planning techniques for testing software systems. (3) Assessment of the test automation framework and a specific AI Planning algorithm, namely, MEA-Graphplan (Means-Ends Analysis Graphplan), algorithm to automatically generate test data. (4) A case study is presented to evaluate the proposed automated testing method and compare the performance of MEA-Graphplan with that of Graphplan. The empirical results show that for software testing, the MEA-Graphplan algorithm can perform computationally more efficiently and effectively than the basic Graph Planning algorithm.

Keywords

AI Planning Planning graph MEA-Graphplan Automated software testing 

References

  1. Avila, H. M., Aha, D. W., Nau, D. S., Weber, R., Breslow, L., & Yaman, F. (2001). SiN: Integrating case-based reasoning with task decomposition. IJCAI-2001, Seattle, August.Google Scholar
  2. Anderson, J. S. (1993). Automating requirements engineering using Artificial Intelligence Techniques. Ph.D. thesis, Dept. of Computer and Information Science, University of Oregon.Google Scholar
  3. Anderson, C., Mayrhauser, A., & Mraz, R. (1995). On the use of Neural Networks to guide Software Testing Activities. In Proc. of International Test Conference, Washington, DC.Google Scholar
  4. Anderson, C., Smith, D. E., & Weld, D. (1998). Conditional effects in graphplan. In Proc. of 4th Intl. Conf. on AI Planning Systems, June.Google Scholar
  5. Bacchus, F., & Ady, M. (2001). Planning with resources and concurrency: A forward chaining approach. International Joint Conference on Artificial Intelligence (IJCAI-2001), pp. 417–424.Google Scholar
  6. Blum, A., & Furst, M. (1997) Fast planning through planning graph analysis. Artificial Intelligence, 90, 281–300.CrossRefGoogle Scholar
  7. Bacchus, F., & Kabanza, F. (1996). Using temporal logic to control search in a forward chaining planner. In: M. Ghallab & A. Milani (Eds.), New directions in planning (pp. 141–153). IOS Press.Google Scholar
  8. Bacchus, F., & Kabanza, F. (2000). Using temporal logic to express search control knowledge for planning. Artificial Intelligence, 116, 123–191.MathSciNetCrossRefGoogle Scholar
  9. Borrajo, D., & Veloso, M. M. (1996). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 10, 1–34.Google Scholar
  10. Barrett, A., & Weld, D. (1994). Task-decomposition via plan parsing. In Proc. of AAAI–94, Seattle, WA, July.Google Scholar
  11. Chilenski, J. J., & Newcomb, P. H. (1994). Formal specification Tools for Test Coverage Analysis. In Proc. of Ninth Knowledge-Based Software Engineering Conference (pp. 59–68). Monterey, CA.Google Scholar
  12. Deason, W., Brown, D., Chang, K. H., & Cross, J. (1991). Rule-based software test data generator. IEEE Transactions on Knowledge and Data Engineering, 3(1), 108–117.CrossRefGoogle Scholar
  13. Erol, K., Hendler, J., & Nau, D. S. (1994). UMCP: A sound and complete procedure for hierarchical task-network planning. In Proc. of the International Conference on AI Planning Systems (AIPS), pp. 249–254, June.Google Scholar
  14. Estlin, T. A., & Mooney, R. J. (1996). Hybrid learning of search control for partial-order planning. New Directions in AI Planning (pp. 129–140). IOS Press.Google Scholar
  15. Fickas, S., & Anderson, J. (1988). A proposed perspective shift: Viewing specification design as a planning problem. Department of Computer and Information Science, CIS-TR-88-15. University of Oregon, Eugene, OR.Google Scholar
  16. Fickas, S., & Helm, B. R. (1992). Knowledge representation and reasoning in the design of composite systems. IEEE Transactions on Software Engineering, SE-18(6), 470–482.CrossRefGoogle Scholar
  17. Fikes, R., & Nilsson, N. (1971). STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2(3/4), 189–208.CrossRefGoogle Scholar
  18. Garagnani, M. (2000). Extending graphplan to domain axiom planning. In Proc. of the 19th Workshop of the UK Planning and Scheduling SIG (PLANSIG 2000) (pp. 275–276). Milton Keynes (UK), ISSN 1368–5708.Google Scholar
  19. Gupta, M., Bastani, F., Khan, L., & Yen, I. L. (2004). Automated test data generation using MEA-Graph Planning. In Proc. of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’04) (pp. 174–182). Boca Raton, Florida (USA).Google Scholar
  20. Huff, K. (1992). Software adaptation. In Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse (pp. 63–66). Stanford University.Google Scholar
  21. Huff, K., & Lesser, V. (1998). A plan-based intelligent assistant that supports the software development process. ACM SIGSOFT/SIGPLAN, Software Engineering Symposium on Practical Software Development Environments, November.Google Scholar
  22. Howe, A., Mayrhauser, A., & Mraz, R. (1997). Test case generation as an AI planning problem. Automated Software Engineering, 4(1), 77–106.CrossRefGoogle Scholar
  23. Koehler, J., Nebel, B., Hoffmann, J., & Dimopoulos, Y. (1997). Extending planning graphs to an ADL subset. In Proc. 4th European Conference on Planning (pp. 273–285). September.Google Scholar
  24. Kambhampati, R., Paeker, E., & Lambrecht, E. (1997). Understanding and extending graphplan. In Proc. 4th European Conference on Planning, September.Google Scholar
  25. McDermott, D. (1996). A heuristic estimator for means-ends analysis in planning. In Proc. 3rd Intl. Conf. AI Planning systems (pp. 142–149) May.Google Scholar
  26. Muñoz-Avila, H. (1998). Integrating twofold case retrieval and complete decision replay in CAPlan/CbC. PhD Thesis, University of Kaiserslautern.Google Scholar
  27. EL-Manzalawy, Y. (2006). Efficient planning with initial irrelevant facts. http://www.cs.iastate.edu/∼yasser/cs572pro.html (Feb 15, 2006)Google Scholar
  28. Minton, S., Carbonell, J. G., Knoblock, C. A., Kuokka, D. R., Etzioni, O., & Gil, Y. (1989). Explanation-based learning: A problem-solving perspective. Journal of Artificial Intelligence, 40(1–3), 63–118.CrossRefGoogle Scholar
  29. Mayrhauser, A., & Hines, S. C. (1993). Automated testing support for a robot tape library. In Proc. of the Fourth International Software Reliability Engineering Conference (pp. 6–14). November.Google Scholar
  30. Mraz, R. T., Howe, A. E., Mayrhauser, A., & Li, L. (1995). System testing with an AI planner. In Proc. Sixth International Symposium on Software Reliability Engineering (pp. 96 –105). Oct.Google Scholar
  31. Mayrhauser, A., Mraz, R. T., & Walls, J. (1994). Domain based regressing testing. In Proc. of the International conference on Software Maintenance (pp. 26–34). Sept.Google Scholar
  32. Memon, A. M., Pollack, M. E., & Soffa, M. L. (2001). Hierarchical GUI test case generation using automated planning. IEEE Transactions on Software Engineering, 27(2) February.CrossRefGoogle Scholar
  33. McAllester, D., & Rosenblitt, D. (1991). Systematic nonlinear planning. In Proc. 9th National Conference on Artificial Intelligence.Google Scholar
  34. Mayrhauser, A., Scheetz, M., Dahlman, E., & Howe, A. E. (2000). Planner based error recovery testing. In Proc. 11th International Symposium on Software Reliability Engineering (pp. 186–195). Oct.Google Scholar
  35. Muñoz-Avila, H., & Weberskirch, F. (1997). A case study on the mergeability of cases with a partial-order planner. In S. Steel & R. Alami (Eds.), Proc. of ECP-97 Recent Advances in AI Planning. Springer.Google Scholar
  36. Nau, D., Cao, Y., Lotem, A., & Muñoz-Avila, H. (1999). SHOP: Simple hierarchical ordered planner. IJCAI, 99, 968–973.Google Scholar
  37. Nebel, B., Dimopoulos, Y., & Koehler, J. (1997). Ignoring irrelevant facts and operators in plan generation. In Proc. 4th European Conference on Planning, SeptGoogle Scholar
  38. Parker, E. (1999). Making Graphplan Goal-Directed. In Proceedings of the 5th European Conference on Planning: Recent Advances in AI Planning, pp. 333–346.CrossRefGoogle Scholar
  39. Penberthy, J. S., & Weld, D. (1992). UCPOP: A sound, complete, partial order planner for ADL. In Proc. 3rd Intl. Conf. Principles of Knowledge Representation and Reasoning(pp. 103–114). Oct.Google Scholar
  40. Rist, R. S. (1992). Plan identification and re-use in programs. In Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse (pp. 67–72). Stanford University, March.Google Scholar
  41. Scheetz, M., Mayrhauser, A., France, R., Dahlman, E., & Howe, A. (1999). Generating test cases from OO model with an AI planning system. In Proc. of 10th International Symposium on Software Reliability Engineering (pp. 250–259). November 01–04.Google Scholar
  42. Veloso, M. M. (1994). Flexible strategy learning: Analogical replay of problem solving episodes. In Proc. of AAAI-94, the Twelfth National Conference on Artificial Intelligence (pp. 595–600). Seattle, WA: AAAI Press.Google Scholar
  43. Veloso, M. M., & Blythe, J. (1994). Linkability: Examining causal link commitments in partial-order planning. In Proc. of the Second International Conference on AI Planning Systems (pp. 170–175). JuneGoogle Scholar
  44. Veloso, M. M., Carbonell, J., Perez, M. A., Borrajo, D., Fink, E., & Blythe, J. (1995). Integrating planning and learning: The prodigy architecture. Journal of Experimental and Theoretical Artificial Intelligence, 7(1), 81–120.CrossRefGoogle Scholar
  45. Weld, D. S. (1994). An Introduction to least-commitment planning. AI Magazine, 15(4), 27–61.Google Scholar
  46. Weld, D. S. (1999). Recent advances in AI planning. AI Magazine, 20(2), 93–123.Google Scholar
  47. Yang, Q. (1990). Formalizing planning knowledge for hierarchical planning. Computational Intelligence Journal, 6(2), 12–24.CrossRefGoogle Scholar
  48. Yen, I-L., Bastani, F. B., Mohamed, F., Ma, H., & Linn, J. (2002). Application of AI planning techniques to automated code synthesis and testing. In Proc. 14th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’02). November.Google Scholar
  49. Zeil, S. J., & Wild, C. (1993). A knowledge base for software test refinement. Technical Report TR-93-14, Old Dominion University, Norfolk, VA.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Manish Gupta
    • 1
    Email author
  • Jicheng Fu
    • 1
  • Farokh B. Bastani
    • 1
  • Latifur R. Khan
    • 1
  • I.-Ling Yen
    • 1
  1. 1.Department of Computer ScienceUniversity of Texas at DallasDallasUSA

Personalised recommendations