Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Rapid goal-oriented automated software testing using MEA-graph planning

Abstract

With the rapid growth in the development of sophisticated modern software applications, the complexity of the software development process has increased enormously, posing an urgent need for the automation of some of the more time-consuming aspects of the development process. One of the key stages in the software development process is system testing. In this paper, we evaluate the potential application of AI planning techniques in automated software testing. The key contributions of this paper include the following: (1) A formal model of software systems from the perspective of software testing that is applicable to important classes of systems and is amenable to automation using AI planning methods. (2) The design of a framework for an automated planning system (APS) for applying AI planning techniques for testing software systems. (3) Assessment of the test automation framework and a specific AI Planning algorithm, namely, MEA-Graphplan (Means-Ends Analysis Graphplan), algorithm to automatically generate test data. (4) A case study is presented to evaluate the proposed automated testing method and compare the performance of MEA-Graphplan with that of Graphplan. The empirical results show that for software testing, the MEA-Graphplan algorithm can perform computationally more efficiently and effectively than the basic Graph Planning algorithm.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

References

  1. Avila, H. M., Aha, D. W., Nau, D. S., Weber, R., Breslow, L., & Yaman, F. (2001). SiN: Integrating case-based reasoning with task decomposition. IJCAI-2001, Seattle, August.

  2. Anderson, J. S. (1993). Automating requirements engineering using Artificial Intelligence Techniques. Ph.D. thesis, Dept. of Computer and Information Science, University of Oregon.

  3. Anderson, C., Mayrhauser, A., & Mraz, R. (1995). On the use of Neural Networks to guide Software Testing Activities. In Proc. of International Test Conference, Washington, DC.

  4. Anderson, C., Smith, D. E., & Weld, D. (1998). Conditional effects in graphplan. In Proc. of 4th Intl. Conf. on AI Planning Systems, June.

  5. Bacchus, F., & Ady, M. (2001). Planning with resources and concurrency: A forward chaining approach. International Joint Conference on Artificial Intelligence (IJCAI-2001), pp. 417–424.

  6. Blum, A., & Furst, M. (1997) Fast planning through planning graph analysis. Artificial Intelligence, 90, 281–300.

  7. Bacchus, F., & Kabanza, F. (1996). Using temporal logic to control search in a forward chaining planner. In: M. Ghallab & A. Milani (Eds.), New directions in planning (pp. 141–153). IOS Press.

  8. Bacchus, F., & Kabanza, F. (2000). Using temporal logic to express search control knowledge for planning. Artificial Intelligence, 116, 123–191.

  9. Borrajo, D., & Veloso, M. M. (1996). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 10, 1–34.

  10. Barrett, A., & Weld, D. (1994). Task-decomposition via plan parsing. In Proc. of AAAI–94, Seattle, WA, July.

  11. Chilenski, J. J., & Newcomb, P. H. (1994). Formal specification Tools for Test Coverage Analysis. In Proc. of Ninth Knowledge-Based Software Engineering Conference (pp. 59–68). Monterey, CA.

  12. Deason, W., Brown, D., Chang, K. H., & Cross, J. (1991). Rule-based software test data generator. IEEE Transactions on Knowledge and Data Engineering, 3(1), 108–117.

  13. Erol, K., Hendler, J., & Nau, D. S. (1994). UMCP: A sound and complete procedure for hierarchical task-network planning. In Proc. of the International Conference on AI Planning Systems (AIPS), pp. 249–254, June.

  14. Estlin, T. A., & Mooney, R. J. (1996). Hybrid learning of search control for partial-order planning. New Directions in AI Planning (pp. 129–140). IOS Press.

  15. Fickas, S., & Anderson, J. (1988). A proposed perspective shift: Viewing specification design as a planning problem. Department of Computer and Information Science, CIS-TR-88-15. University of Oregon, Eugene, OR.

  16. Fickas, S., & Helm, B. R. (1992). Knowledge representation and reasoning in the design of composite systems. IEEE Transactions on Software Engineering, SE-18(6), 470–482.

  17. Fikes, R., & Nilsson, N. (1971). STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2(3/4), 189–208.

  18. Garagnani, M. (2000). Extending graphplan to domain axiom planning. In Proc. of the 19th Workshop of the UK Planning and Scheduling SIG (PLANSIG 2000) (pp. 275–276). Milton Keynes (UK), ISSN 1368–5708.

  19. Gupta, M., Bastani, F., Khan, L., & Yen, I. L. (2004). Automated test data generation using MEA-Graph Planning. In Proc. of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’04) (pp. 174–182). Boca Raton, Florida (USA).

  20. Huff, K. (1992). Software adaptation. In Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse (pp. 63–66). Stanford University.

  21. Huff, K., & Lesser, V. (1998). A plan-based intelligent assistant that supports the software development process. ACM SIGSOFT/SIGPLAN, Software Engineering Symposium on Practical Software Development Environments, November.

  22. Howe, A., Mayrhauser, A., & Mraz, R. (1997). Test case generation as an AI planning problem. Automated Software Engineering, 4(1), 77–106.

  23. Koehler, J., Nebel, B., Hoffmann, J., & Dimopoulos, Y. (1997). Extending planning graphs to an ADL subset. In Proc. 4th European Conference on Planning (pp. 273–285). September.

  24. Kambhampati, R., Paeker, E., & Lambrecht, E. (1997). Understanding and extending graphplan. In Proc. 4th European Conference on Planning, September.

  25. McDermott, D. (1996). A heuristic estimator for means-ends analysis in planning. In Proc. 3rd Intl. Conf. AI Planning systems (pp. 142–149) May.

  26. Muñoz-Avila, H. (1998). Integrating twofold case retrieval and complete decision replay in CAPlan/CbC. PhD Thesis, University of Kaiserslautern.

  27. EL-Manzalawy, Y. (2006). Efficient planning with initial irrelevant facts. http://www.cs.iastate.edu/∼yasser/cs572pro.html (Feb 15, 2006)

  28. Minton, S., Carbonell, J. G., Knoblock, C. A., Kuokka, D. R., Etzioni, O., & Gil, Y. (1989). Explanation-based learning: A problem-solving perspective. Journal of Artificial Intelligence, 40(1–3), 63–118.

  29. Mayrhauser, A., & Hines, S. C. (1993). Automated testing support for a robot tape library. In Proc. of the Fourth International Software Reliability Engineering Conference (pp. 6–14). November.

  30. Mraz, R. T., Howe, A. E., Mayrhauser, A., & Li, L. (1995). System testing with an AI planner. In Proc. Sixth International Symposium on Software Reliability Engineering (pp. 96 –105). Oct.

  31. Mayrhauser, A., Mraz, R. T., & Walls, J. (1994). Domain based regressing testing. In Proc. of the International conference on Software Maintenance (pp. 26–34). Sept.

  32. Memon, A. M., Pollack, M. E., & Soffa, M. L. (2001). Hierarchical GUI test case generation using automated planning. IEEE Transactions on Software Engineering, 27(2) February.

  33. McAllester, D., & Rosenblitt, D. (1991). Systematic nonlinear planning. In Proc. 9th National Conference on Artificial Intelligence.

  34. Mayrhauser, A., Scheetz, M., Dahlman, E., & Howe, A. E. (2000). Planner based error recovery testing. In Proc. 11th International Symposium on Software Reliability Engineering (pp. 186–195). Oct.

  35. Muñoz-Avila, H., & Weberskirch, F. (1997). A case study on the mergeability of cases with a partial-order planner. In S. Steel & R. Alami (Eds.), Proc. of ECP-97 Recent Advances in AI Planning. Springer.

  36. Nau, D., Cao, Y., Lotem, A., & Muñoz-Avila, H. (1999). SHOP: Simple hierarchical ordered planner. IJCAI, 99, 968–973.

  37. Nebel, B., Dimopoulos, Y., & Koehler, J. (1997). Ignoring irrelevant facts and operators in plan generation. In Proc. 4th European Conference on Planning, Sept

  38. Parker, E. (1999). Making Graphplan Goal-Directed. In Proceedings of the 5th European Conference on Planning: Recent Advances in AI Planning, pp. 333–346.

  39. Penberthy, J. S., & Weld, D. (1992). UCPOP: A sound, complete, partial order planner for ADL. In Proc. 3rd Intl. Conf. Principles of Knowledge Representation and Reasoning(pp. 103–114). Oct.

  40. Rist, R. S. (1992). Plan identification and re-use in programs. In Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse (pp. 67–72). Stanford University, March.

  41. Scheetz, M., Mayrhauser, A., France, R., Dahlman, E., & Howe, A. (1999). Generating test cases from OO model with an AI planning system. In Proc. of 10th International Symposium on Software Reliability Engineering (pp. 250–259). November 01–04.

  42. Veloso, M. M. (1994). Flexible strategy learning: Analogical replay of problem solving episodes. In Proc. of AAAI-94, the Twelfth National Conference on Artificial Intelligence (pp. 595–600). Seattle, WA: AAAI Press.

  43. Veloso, M. M., & Blythe, J. (1994). Linkability: Examining causal link commitments in partial-order planning. In Proc. of the Second International Conference on AI Planning Systems (pp. 170–175). June

  44. Veloso, M. M., Carbonell, J., Perez, M. A., Borrajo, D., Fink, E., & Blythe, J. (1995). Integrating planning and learning: The prodigy architecture. Journal of Experimental and Theoretical Artificial Intelligence, 7(1), 81–120.

  45. Weld, D. S. (1994). An Introduction to least-commitment planning. AI Magazine, 15(4), 27–61.

  46. Weld, D. S. (1999). Recent advances in AI planning. AI Magazine, 20(2), 93–123.

  47. Yang, Q. (1990). Formalizing planning knowledge for hierarchical planning. Computational Intelligence Journal, 6(2), 12–24.

  48. Yen, I-L., Bastani, F. B., Mohamed, F., Ma, H., & Linn, J. (2002). Application of AI planning techniques to automated code synthesis and testing. In Proc. 14th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’02). November.

  49. Zeil, S. J., & Wild, C. (1993). A knowledge base for software test refinement. Technical Report TR-93-14, Old Dominion University, Norfolk, VA.

Download references

Author information

Correspondence to Manish Gupta.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gupta, M., Fu, J., Bastani, F.B. et al. Rapid goal-oriented automated software testing using MEA-graph planning. Software Qual J 15, 241–263 (2007). https://doi.org/10.1007/s11219-007-9018-3

Download citation

Keywords

  • AI Planning
  • Planning graph
  • MEA-Graphplan
  • Automated software testing