Using Automatic Failure Detection for Cognitive Agents in Eclipse (AAMAS 2016 DEMONSTRATION)

  • Vincent J. Koeman
  • Koen V. Hindriks
  • Catholijn M. Jonker
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10093)


In order to reduce debugging effort and enable automated failure detection, we proposed an automated testing framework for detecting failures in cognitive agent programs in previous work. This approach is based on a minimal set of temporal operators that enable the specification of test conditions with sufficient expressiveness for detecting all failures in an existing failure taxonomy. We also introduced an according approach for specifying test templates that supports a programmer in writing tests. In this demonstration paper, the automated test framework for the Goal agent programming language that has been created for the Eclipse platform is introduced, with a focus on its practical aspects, i.e., how to use it to detect failures in cognitive agents. As fault localization is an important follow-up to failure detection, the integration of the test framework in the existing source-level debugger for Goal is discussed as well. In addition, an empirical evaluation of the automated testing framework implementation for Goal is presented based on the work of almost 200 novice agent programmers.


  1. 1.
    Bordini, R.H., Dastani, M., Winikoff, M.: Current issues in multi-agent systems development. In: O’Hare, G.M.P., Ricci, A., O’Grady, M.J., Dikenelli, O. (eds.) ESAW 2006. LNCS (LNAI), vol. 4457, pp. 38–61. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-75524-1_3 CrossRefGoogle Scholar
  2. 2.
    Bordini, R.H., Braubach, L., Gomez-sanz, J.J., Hare, G.O., Pokahr, A., Ricci, A.: A survey of programming languages and platforms for multi-agent systems. Informatica 30, 33–44 (2006)zbMATHGoogle Scholar
  3. 3.
    Caire, G., Cossentino, M., Negri, A.: Multi-agent systems implementation and testing. In: Proceedings of the 4th From Agent Theory to Agent Implementation Symposium, AT2AI-4 (2004)Google Scholar
  4. 4.
    Dastani, M.: Programming multi-agent systems. Knowl. Eng. Rev. 30, 394–418 (2015)CrossRefGoogle Scholar
  5. 5.
    Dix, J., Hindriks, K.V., Logan, B., Wobcke, W.: Engineering multi-agent systems (dagstuhl seminar 12342). Dagstuhl Rep. 2(8), 74–98 (2012)Google Scholar
  6. 6.
    Geer, D.: Eclipse becomes the dominant Java IDE. Computer 38(7), 16–18 (2005)CrossRefGoogle Scholar
  7. 7.
    Gomanyuk, S.: An approach to creating development environments for a wide class of programming languages. Program. Comput. Softw. 34(4), 225–236 (2008)CrossRefzbMATHGoogle Scholar
  8. 8.
    Hindriks, K.V.: Programming rational agents in GOAL. In: El Fallah Seghrouchni, A., Dix, J., Dastani, M., Bordini, R.H. (eds.) Multi-Agent Programming: Languages, Tools and Applications, pp. 119–157. Springer, US (2009)Google Scholar
  9. 9.
    Hindriks, K.V.: Debugging is explaining. In: Rahwan, I., Wobcke, W., Sen, S., Sugawara, T. (eds.) PRIMA 2012. LNCS (LNAI), vol. 7455, pp. 31–45. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-32729-2_3 CrossRefGoogle Scholar
  10. 10.
    Hindriks, K.V.: Programming cognitive agents in GOAL (2016).
  11. 11.
    Hindriks, K.V., Pasman, W., Koeman, V.J.: GOAL-Eclipse user manual (2016).
  12. 12.
    Houhamdi, Z.: Multi-agent system testing: a survey. Int. J. Adv. Comput. Sci. Appl. 2(6), 135–141 (2011)Google Scholar
  13. 13.
    ISO: ISO/IEC/IEEE 24765: 2010 systems and software engineering - vocabulary. Technical report, Institute of Electrical and Electronics Engineers, Inc. (2010)Google Scholar
  14. 14.
    Johnson, M., Jonker, C., Riemsdijk, B., Feltovich, P.J., Bradshaw, J.M.: Joint activity testbed: blocks world for teams (BW4T). In: Aldewereld, H., Dignum, V., Picard, G. (eds.) ESAW 2009. LNCS (LNAI), vol. 5881, pp. 254–256. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-10203-5_26 CrossRefGoogle Scholar
  15. 15.
    Koeman, V.J., Hindriks, K.V.: Designing a source-level debugger for cognitive agent programs. In: Chen, Q., Torroni, P., Villata, S., Hsu, J., Omicini, A. (eds.) PRIMA 2015. LNCS (LNAI), vol. 9387, pp. 335–350. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-25524-8_21 CrossRefGoogle Scholar
  16. 16.
    Koeman, V.J., Hindriks, K.V.: A fully integrated development environment for agent-oriented programming. In: Demazeau, Y., Decker, K.S., Bajo Pérez, J., de la Prieta, F. (eds.) PAAMS 2015. LNCS (LNAI), vol. 9086, pp. 288–291. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-18944-4_29 CrossRefGoogle Scholar
  17. 17.
    Koeman, V.J., Hindriks, K.V., Jonker, C.M.: Automating failure detection in cognitive agent programs. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS 2016, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, pp. 1237–1246 (2016)Google Scholar
  18. 18.
    Koeman, V.J., Hindriks, K.V., Jonker, C.M.: Using automatic failure detection for cognitive agents in Eclipse: (demonstration). In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS 2016, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, pp. 1507–1509 (2016)Google Scholar
  19. 19.
    Nguyen, C.D., Perini, A., Bernon, C., Pavón, J., Thangarajah, J.: Testing in multi-agent systems. In: Gleizes, M.-P., Gomez-Sanz, J.J. (eds.) AOSE 2009. LNCS, vol. 6038, pp. 180–190. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-19208-1_13 CrossRefGoogle Scholar
  20. 20.
    Runeson, P., Andersson, C., Thelin, T., Andrews, A., Berling, T.: What do we know about defect detection methods? IEEE Softw. 23(3), 82–90 (2006)CrossRefGoogle Scholar
  21. 21.
    Wielemaker, J., Schrijvers, T., Triska, M., Lager, T.: SWI-Prolog. Theory Pract. Logic Program. 12, 67–96 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Winikoff, M.: Novice programmers’ faults & failures in GOAL programs. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS 2014, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, pp. 301–308 (2014)Google Scholar
  23. 23.
    Zhang, Z., Thangarajah, J., Padgham, L.: Model based testing for agent systems. Softw. Data Technol. 22, 399–413 (2008)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Vincent J. Koeman
    • 1
  • Koen V. Hindriks
    • 1
  • Catholijn M. Jonker
    • 1
  1. 1.Delft University of TechnologyDelftThe Netherlands

Personalised recommendations