How Testable Are BDI Agents? An Analysis of Branch Coverage

  • Michael Winikoff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10002)


Before deploying a software system, it is important to assure that it will function correctly. Traditionally, this assurance is obtained by testing the system with a collection of test cases. However, since agent systems exhibit complex behaviour, it is not clear whether testing is even feasible. In this paper we extend our understanding of the feasibility of testing BDI agent programs by analysing their testability with respect to the all edges test adequacy criterion, and comparing with previous work that considered the all paths criterion. Our findings include that the number of tests required with respect to the all edges criterion is much lower than for the all paths criterion. We also compare BDI program testability with testability of (abstract) procedural programs.


Path Criterion Agent System Procedural Program Branch Coverage Edge Criterion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.): Multi-Agent Programming: Languages, Platforms and Applications. Springer, Heidelberg (2005)Google Scholar
  2. 2.
    Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.): Multi-Agent Programming: Languages, Tools and Applications. Springer, Heidelberg (2009)Google Scholar
  3. 3.
    Bordini, R.H., Fisher, M., Pardavila, C., Wooldridge, M.: Model checking AgentSpeak. In: Autonomous Agents and Multiagent Systems (AAMAS), pp. 409–416 (2003)Google Scholar
  4. 4.
    Bratman, M.E.: Intentions, Plans, and Practical Reason. Harvard University Press, Cambridge (1987)Google Scholar
  5. 5.
    Bratman, M.E., Israel, D.J., Pollack, M.E.: Plans and resource-bounded practical reasoning. Comput. Intell. 4, 349–355 (1988)CrossRefGoogle Scholar
  6. 6.
    Dastani, M., Hindriks, K.V., Meyer, J.-J.C. (eds.): Specification and Verification of Multi-agent systems. Springer, Berlin/Heidelberg (2010)zbMATHGoogle Scholar
  7. 7.
    Dennis, L.A., Fisher, M., Lincoln, N.K., Lisitsa, A., Veres, S.M.: Practical verification of decision-making in agent-based autonomous systems. In: Automated Software Engineering, 55 pages (2014)Google Scholar
  8. 8.
    Dennis, L.A., Fisher, M., Webster, M.P., Bordini, R.H.: Model checking agent programming languages. Autom. Softw. Eng. J. 19(1), 3–63 (2012)Google Scholar
  9. 9.
    Ekinci, E.E., Tiryaki, A.M., Çetin, Ö., Dikenelli, O.: Goal-oriented agent testing revisited. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 173–186. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-01338-6_13 CrossRefGoogle Scholar
  10. 10.
    Fisher, M., Dennis, L., Webster, M.: Verifying autonomous systems. Commun. ACM 56(9), 84–93 (2013)CrossRefGoogle Scholar
  11. 11.
    Gómez-Sanz, J.J., Botía, J., Serrano, E., Pavón, J.: Testing and debugging of MAS interactions with INGENIAS. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 199–212. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-01338-6_15 CrossRefGoogle Scholar
  12. 12.
    Jorgensen, P.: Software Testing: A Craftsman’s Approach, 2nd edn. CRC Press, Boca Raton (2002)CrossRefzbMATHGoogle Scholar
  13. 13.
    Mathur, A.P.: Foundations of Software Testing. Pearson (2008). ISBN 978-81-317-1660-1Google Scholar
  14. 14.
    Nguyen, C.D., Perini, A., Tonella, P.: Experimental evaluation of ontology-based test generation for multi-agent systems. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 187–198. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-01338-6_14 CrossRefGoogle Scholar
  15. 15.
    Padgham, L., Zhang, Z., Thangarajah, J., Miller, T.: Model-based test oracle generation for automated unit testing of agent systems. IEEE Trans. Softw. Eng. 39, 1230–1244 (2013)CrossRefGoogle Scholar
  16. 16.
    Raimondi, F., Lomuscio, A.: Automatic verification of multi-agent systems by model checking via ordered binary decision diagrams. J. Appl. Logic 5(2), 235–251 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Rao, A.S.: AgentSpeak(L): BDI agents speak out in a logical computable language. In: Velde, W., Perram, J.W. (eds.) MAAMAW 1996. LNCS, vol. 1038, pp. 42–55. Springer, Heidelberg (1996). doi: 10.1007/BFb0031845 CrossRefGoogle Scholar
  18. 18.
    Rao, A.S., Georgeff, M.P.: Modeling rational agents within a BDI-architecture. In: Allen, J., Fikes, R., Sandewall, E. (eds.) Principles of Knowledge Representation and Reasoning, Proceedings of the Second International Conference, pp. 473–484. Morgan Kaufmann (1991)Google Scholar
  19. 19.
    Vieira, R., Moreira, Á., Wooldridge, M., Bordini, R.H.: On the formal semantics of speech-act based communication in an agent-oriented programming language. J. Artif. Intell. Res. (JAIR) 29, 221–267 (2007)zbMATHGoogle Scholar
  20. 20.
    Winikoff, M., Cranefield, S.: On the testability of BDI agent systems. J. Artif. Intell. Res. (JAIR) 51, 71–131 (2014)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Winikoff, M., Cranefield, S.: On the testability of BDI agent systems (extended abstract). In: Journal Track of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 4217–4221 (2015)Google Scholar
  22. 22.
    Winikoff, M., Padgham, L., Harland, J., Thangarajah, J.: Declarative & procedural goals in intelligent agent systems. In: Proceedings of the Eighth International Conference on Principles of Knowledge Representation and Reasoning (KR), pp. 470–481. Morgan Kaufmann, Toulouse (2002)Google Scholar
  23. 23.
    Wooldridge, M., Fisher, M., Huget, M.-P., Parsons, S.: Model checking multi-agent systems with MABLE. In: Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 952–959 (2002)Google Scholar
  24. 24.
    Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing for agent systems. In: Second International Working Conference on Evaluation of Novel Approaches to Software Engineering (ENASE), pp. 10–18 (2007)Google Scholar
  25. 25.
    Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.University of OtagoDunedinNew Zealand

Personalised recommendations