Mutation Operators for the Goal Agent Language

  • Sharmila Savarimuthu
  • Michael Winikoff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8245)


Testing multi-agent systems is a challenge, since by definition such systems are distributed, and are able to exhibit autonomous and flexible behaviour. One specific challenge in testing agent programs is developing a collection of tests (a “test suite”) that is adequate for testing a given agent program. In order to develop an adequate test suite, it is clearly important to be able to assess the adequacy of a given test suite. A well-established technique for assessing this is the use of mutation testing, where mutation operators are used to generate variants (“mutants”) of a given program, and a test suite is then assessed in terms of its ability to detect (“kill”) the mutants. However, work on mutation testing has focussed largely on the mutation of procedural and object-oriented languages. This paper is the first to propose a set of mutation operators for a cognitive agent-oriented programming language, specifically Goal. Our mutation operators are systematically derived, and are also guided by an exploration of the bugs found in a collection of undergraduate programming assignments written in Goal. In fact, in exploring these programs we also provide an additional contribution: evidence of the extent to which the two foundational hypotheses of mutation testing hold for Goal programs.


Mutation Testing Agent-Oriented Programming Languages Goal 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adra, S.F., McMinn, P.: Mutation operators for agent-based models. In: Proceedings of 5th International Workshop on Mutation Analysis. IEEE Computer Society (2010)Google Scholar
  2. 2.
    Ekinci, E.E., Tiryaki, A.M., Çetin, Ö., Dikenelli, O.: Goal-oriented agent testing revisited. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 173–186. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Gómez-Sanz, J.J., Botía, J., Serrano, E., Pavón, J.: Testing and debugging of MAS interactions with INGENIAS. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 199–212. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Hindriks, K.V.: Programming rational agents in Goal. In: Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) Multi-Agent Programming: Languages, Tools and Applications, ch. 4, pp. 119–157. Springer (2009)Google Scholar
  5. 5.
    Hindriks, K.V.: Programming rational agents in Goal (May 2011),
  6. 6.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)CrossRefGoogle Scholar
  7. 7.
    Johnson, M., Jonker, C., van Riemsdijk, B., Feltovich, P.J., Bradshaw, J.M.: Joint Activity Testbed: Blocks World for Teams (BW4T). In: Aldewereld, H., Dignum, V., Picard, G. (eds.) ESAW 2009. LNCS, vol. 5881, pp. 254–256. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Kim, S., Clark, J.A., McDermid, J.A.: The rigorous generation of Java mutation operators using HAZOP. Technical Report 2/8/99, Department of Computer Science, University of York (1999)Google Scholar
  9. 9.
    Low, C.K., Chen, T.Y., Rönnquist, R.: Automated test case generation for BDI agents. Journal of Autonomous Agents and Multi-Agent Systems 2(4), 311–332 (1999)CrossRefGoogle Scholar
  10. 10.
    Mathur, A.P.: Foundations of Software Testing. Pearson (2008) ISBN 978-81-317-1660-1Google Scholar
  11. 11.
    Miller, T., Padgham, L., Thangarajah, J.: Test coverage criteria for agent interaction testing. In: Weyns, D., Gleizes, M.P. (eds.) Proceedings of the 11th International Workshop on Agent Oriented Software Engineering, pp. 1–12 (2010)Google Scholar
  12. 12.
    Munroe, S., Miller, T., Belecheanu, R.A., Pěchouček, M., McBurney, P., Luck, M.: Crossing the agent technology chasm: Lessons, experiences and challenges in commercial applications of agents. Knowledge Engineering Review 21(4), 345–392 (2006)CrossRefGoogle Scholar
  13. 13.
    Nguyen, C.D., Perini, A., Tonella, P.: Experimental evaluation of ontology-based test generation for multi-agent systems. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 187–198. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Nguyen, C.D., Perini, A., Tonella, P.: Goal-Oriented Testing for MASs. International Journal of Agent-Oriented Software Engineering 4(1), 79–109 (2010)CrossRefGoogle Scholar
  15. 15.
    Offutt, A.: Investigations of the software testing coupling effect. ACM Transactions on Software Engineering and Methodology 1(1), 5–20 (1992)CrossRefGoogle Scholar
  16. 16.
    Pěchouček, M., Mařík, V.: Industrial deployment of multi-agent technologies: review and selected case studies. Journal of Autonomous Agents and Multi-Agent Systems 17, 397–431 (2008)CrossRefGoogle Scholar
  17. 17.
    van Riemsdijk, M.B., Hindriks, K.V., Jonker, C.M.: An empirical study of cognitive agent programs. Multiagent and Grid Systems 8(2), 187–222 (2012)Google Scholar
  18. 18.
    Saifan, A.A., Wahsheh, H.A.: Mutation operators for JADE mobile agent systems. In: Proceedings of the 3rd International Conference on Information and Communication Systems, ICICS (2012)Google Scholar
  19. 19.
    Thangarajah, J., Sardiña, S., Padgham, L.: Measuring plan coverage and overlap for agent reasoning. In: van der Hoek, W., Padgham, L., Conitzer, V., Winikoff, M. (eds.) International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2012, 3 Volumes, Valencia, Spain, June 4-8, pp. 1049–1056. IFAAMAS (2012)Google Scholar
  20. 20.
    Winikoff, M., Padgham, L.: Agent oriented software engineering. In: Weiss, G. (ed.) Multiagent Systems, ch. 5, 2nd edn. MIT Press (2013)Google Scholar
  21. 21.
    Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing for agent systems. In: 2nd International Working Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2007), pp. 10–18 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sharmila Savarimuthu
    • 1
  • Michael Winikoff
    • 1
  1. 1.University of OtagoNew Zealand

Personalised recommendations