Advertisement

Autonomous Agents and Multi-Agent Systems

, Volume 31, Issue 5, pp 1094–1132 | Cite as

BDI agent testability revisited

  • Michael Winikoff
Article

Abstract

Agent-based systems are deployed to solve a wide range of problems in a wide range of domains. Before software is deployed, it is important to obtain assurance that it will function correctly. Traditionally, this assurance is obtained by testing. However, there is an intuition that agents exhibit more complex behaviour than traditional software, which raises the question: how testable are agent systems? We focus on BDI agent programs, and analyse their testability with respect to the all edges test adequacy criterion (also known as “branch coverage”). Our results augment earlier results that considered the all paths criterion to provide a richer and more nuanced understanding of the testability of BDI agents. We show that the number of tests required with respect to the all edges criterion is much lower than that required with respect to the all paths criterion. We also show that, as for the previous analysis, BDI programs are harder to test than equivalently-sized procedural programs, even if exception handling is introduced. Overall, our conclusions lend strength to the earlier work, and motivate the need for work on formal methods for agent systems.

Keywords

Verification and validation of agent-based systems Belief–desire-intention (BDI) Testability 

References

  1. 1.
    Benfield, S. S., Hendrickson, J., & Galanti, D. (2006). Making a strong business case for multiagent technology. In P. Stone & G. Weiss (Eds.), Autonomous agents and multi-agent systems (AAMAS) (pp. 10–15). New york: ACM Press.Google Scholar
  2. 2.
    Bordini, R. H., Dastani, M., Dix, J., & El Fallah Seghrouchni, A. (Eds.). (2005). Multi-agent programming: Languages, platforms and applications. Berlin: Springer.zbMATHGoogle Scholar
  3. 3.
    Bordini, R. H., Dastani, M., Dix, J., & El Fallah Seghrouchni, A. (Eds.). (2009). Multi-agent programming: Languages, tools and applications. Berlin: Springer.zbMATHGoogle Scholar
  4. 4.
    Bordini, R. H., Fisher, M., Pardavila, C., & Wooldridge, M. (2003). Model checking AgentSpeak. In Autonomous agents and multiagent systems (AAMAS) (pp. 409–416).Google Scholar
  5. 5.
    Bordini, R. H., Hübner, J. F., & Wooldridge, M. (2007). Programming multi-agent systems in AgentSpeak using Jason. New York: Wiley. ISBN 0470029005.Google Scholar
  6. 6.
    Bratman, M. E. (1987). Intentions, plans, and practical reason. Cambridge, MA: Harvard University Press.Google Scholar
  7. 7.
    Bratman, M. E., Israel, D. J., & Pollack, M. E. (1988). Plans and resource-bounded practical reasoning. Computational Intelligence, 4, 349–355.CrossRefGoogle Scholar
  8. 8.
    Burmeister, B., Arnold, M., Copaciu, F., & Rimassa, G. (2008). BDI-agents for agile goal-oriented business processes. In Proceedings of the seventh international conference on autonomous agents and multiagent systems (AAMAS) [Industry Track] (pp. 37–44). IFAAMAS.Google Scholar
  9. 9.
    Busetta, P., Rönnquist, R., Hodgson, A., & Lucas, A. (1998). JACK intelligent agents—Components for intelligent agents in Java. Technical report, Agent Oriented Software Pty. Ltd, Melbourne, Australia. http://www.agent-software.com.
  10. 10.
    Cabral, B., & Marques, P. (2007). Exception handling: A field study in Java and .NET. In E. Ernst (Ed.), 21st European conference on object-oriented programming (ECOOP). LNCS (Vol. 4609, pp. 151–175). Berlin: Springer.Google Scholar
  11. 11.
    Dastani, M. (2008). 2APL: A practical agent programming language. Autonomous Agents and Multi-Agent Systems, 16(3), 214–248.CrossRefGoogle Scholar
  12. 12.
    Dastani, M., Hindriks, K. V., & Meyer, J.-J. C. (Eds.). (2010). Specification and verification of multi-agent systems. Berlin: Springer.zbMATHGoogle Scholar
  13. 13.
    Dennis, L. A., Fisher, M., Lincoln, N., Lisitsa, A., & Veres, S. M. (2016). Practical verification of decision-making in agent-based autonomous systems. Automated Software Engineering, 23(3), 305–359.CrossRefGoogle Scholar
  14. 14.
    Dennis, L. A., Fisher, M., Webster, M. P., & Bordini, R. H. (2012). Model checking agent programming languages. Automated Software Engineering Journal, 19(1), 3–63.CrossRefGoogle Scholar
  15. 15.
    d’Inverno, M., Kinny, D., Luck, M., & Wooldridge, M. (1998). A formal specification of dMARS. In M. Singh, A. Rao, & M. Wooldridge (Eds.), Fourth international workshop on agent theories, architectures, and languages. LNAI (Vol. 1365, pp. 155–176). Berlin: Springer.Google Scholar
  16. 16.
    Dorigo, M. & Stützle, T. (2004). Ant colony optimization. Cambridge, MA: MIT Press. ISBN 0-262-04219-3.Google Scholar
  17. 17.
    Ekinci, E. E., Tiryaki, A. M., Çetin, Ö., & Dikenelli, O. (2009). Goal-oriented agent testing revisited. In M. Luck & J. J. Gomez-Sanz (Eds.), Agent-oriented software engineering IX. Lecture notes in computer science (Vol. 5386, pp. 173–186). Berlin: Springer.Google Scholar
  18. 18.
    Fisher, M., Dennis, L., & Webster, M. (2013). Verifying autonomous systems. Communications of the ACM, 56(9), 84–93.CrossRefGoogle Scholar
  19. 19.
    Gomez-Sanz, J. J., Botía, J., Serrano, E., & Pavón, J. (2009). Testing and debugging of MAS interactions with INGENIAS. In M. Luck & J. J. Gomez-Sanz (Eds.), Agent-oriented software engineering IX. Lecture notes in computer science (Vol. 5386, pp. 199–212). Berlin: Springer.Google Scholar
  20. 20.
    Hindriks, K. V. (2009). Programming rational agents in Goal. In Bordini et al. [3], chapter 4 (pp. 119–157).Google Scholar
  21. 21.
    Hindriks, K. V., Boer, F. S. D., der Hoek, W. V., & Meyer, J.-J. C. (1999). Agent programming in 3APL. Autonomous Agents and Multi-Agent Systems, 2(4), 357–401.CrossRefGoogle Scholar
  22. 22.
    Huber, M. J. (1999). JAM: A BDI-theoretic mobile agent architecture. In Proceedings of the third international conference on autonomous agents (Agents’99) (pp. 236–243). New York: ACM Press.Google Scholar
  23. 23.
    Ingrand, F. F., Georgeff, M. P., & Rao, A. S. (1992). An architecture for real-time reasoning and system control. IEEE Expert, 7(6), 33–44.CrossRefGoogle Scholar
  24. 24.
    Jorgensen, P. (2002). Software testing: A Craftsman’s approach (2nd ed.). Boca Raton, FL: CRC Press.CrossRefzbMATHGoogle Scholar
  25. 25.
    Mathur, A. P. (2008). Foundations of software testing. Upper Saddle River: Pearson. ISBN 978-81-317-1660-1.Google Scholar
  26. 26.
    Moreira, A., & Bordini, R. (2002). An operational semantics for a BDI agent-oriented programming language. In J.-J. C. Meyer, & M. J. Wooldridge (Eds.), Proceedings of the workshop on logics for agent-based systems (LABS-02) (pp. 45–59).Google Scholar
  27. 27.
    Müller, J., & Fischer, K. (2014). Application impact of multi-agent systems and technologies: A survey. In O. Shehory & A. Sturm (Eds.), Agent-oriented software engineering (pp. 27–53). Berlin: Springer.Google Scholar
  28. 28.
    Munroe, S., Miller, T., Belecheanu, R., Pěchouček, M., McBurney, P., & Luck, M. (2006). Crossing the agent technology chasm: Lessons, experiences and challenges in commercial applications of agents. Knowledge Engineering Review, 21(4), 345–392.CrossRefGoogle Scholar
  29. 29.
    Nguyen, C. D., Perini, A., & Tonella, P. (2009). Experimental evaluation of ontology-based test generation for multi-agent systems. In M. Luck & J. J. Gomez-Sanz (Eds.), Agent-oriented software engineering IX. Lecture notes in computer science (Vol. 5386, pp. 187–198). Berlin: Springer.Google Scholar
  30. 30.
    Padgham, L., Zhang, Z., Thangarajah, J., & Miller, T. (2013). Model-based test oracle generation for automated unit testing of agent systems. IEEE Transactions on Software Engineering, 39(9), 1230–1244.CrossRefGoogle Scholar
  31. 31.
    Pokahr, A., Braubach, L., & Lamersdorf, W. (2005). Jadex: A BDI reasoning engine. In Bordini et al. [2] (pp. 149–174).Google Scholar
  32. 32.
    Raimondi, F., & Lomuscio, A. (2007). Automatic verification of multi-agent systems by model checking via ordered binary decision diagrams. Journal of Applied Logic, 5(2), 235–251.MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Rao, A. S. (1996). AgentSpeak(L): BDI agents speak out in a logical computable language. In W. V. de Velde, & J. Perrame, editors, Agents breaking away: Proceedings of the seventh european workshop on modelling autonomous agents in a multi-agent world (MAAMAW’96). LNAI (Vol. 1038, pp. 42–55). Berlin: Springer.Google Scholar
  34. 34.
    Rao, A. S. & Georgeff, M. P. (1991). Modeling rational agents within a BDI-architecture. In J. Allen, R. Fikes, & E. Sandewall (Eds.), Proceedings of the second international conference principles of knowledge representation and reasoning (pp. 473–484). San Mateo, CA: Morgan Kaufmann.Google Scholar
  35. 35.
    Rao, A. S., & Georgeff, M. P. (1992). An abstract architecture for rational agents. In C. Rich, W. Swartout, & B. Nebel (Eds.), Proceedings of the third international conference on principles of knowledge representation and reasoning (pp. 439–449). San Mateo, CA: Morgan Kaufmann.Google Scholar
  36. 36.
    Ryder, B. G., Smith, D., Kremer, U., Gordon, M., & Shah, N. (2000). A static study of Java exceptions using JESP. In D. A. Watt (Ed.), 9th international conference on compiler construction (CC). LNCS (Vol. 1781, pp. 67–81). Berlin: Springer.Google Scholar
  37. 37.
    Vieira, R., Moreira, Á., Wooldridge, M., & Bordini, R. H. (2007). On the formal semantics of speech-act based communication in an agent-oriented programming language. Journal of Artificial Intelligence Research (JAIR), 29, 221–267.zbMATHGoogle Scholar
  38. 38.
    Winikoff, M. (2005). JACK\(^{\text{TM}}\) intelligent agents: An industrial strength platform. In Bordini et al. [2] (pp. 175–193).Google Scholar
  39. 39.
    Winikoff, M. (2016). How testable are BDI agents? An analysis of branch coverage. In Engineering multi-agent systems (EMAS) post-proceedings. doi: 10.1007/978-3-319-50983-9_12.
  40. 40.
    Winikoff, M., & Cranefield, S. (2014). On the testability of BDI agent systems. Journal of Artificial Intelligence Research (JAIR), 51, 71–131.MathSciNetzbMATHGoogle Scholar
  41. 41.
    Winikoff, M., & Cranefield, S. (2015). On the testability of BDI agent systems (extended abstract). In Journal track of the international joint conference on artificial intelligence (IJCAI) (pp. 4217–4221).Google Scholar
  42. 42.
    Winikoff, M., Padgham, L., Harland, J., & Thangarajah, J. (2002). Declarative & procedural goals in intelligent agent systems. In Proceedings of the eighth international conference on principles of knowledge representation and reasoning (KR), Toulouse, France (pp. 470–481). San Mateo, CA: Morgan Kaufmann.Google Scholar
  43. 43.
    Wooldridge, M. (2002). An introduction to multiagent systems (2nd ed.). Chichester: Wiley. ISBN 0470519460.Google Scholar
  44. 44.
    Wooldridge, M., Fisher, M., Huget, M.-P., & Parsons, S. (2002). Model checking multi-agent systems with MABLE. In Autonomous agents and multi-agent systems (AAMAS) (pp. 952–959).Google Scholar
  45. 45.
    Zhang, Z., Thangarajah, J., & Padgham, L. (2007). Automated unit testing for agent systems. In Second international working conference on evaluation of novel approaches to software engineering (ENASE) (pp. 10–18).Google Scholar
  46. 46.
    Zhu, H., Hall, P. A. V., & May, J. H. R. (1997). Software unit test coverage and adequacy. ACM Computing Surveys, 29(4), 366–427.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  1. 1.Department of Information ScienceUniversity of OtagoDunedinNew Zealand

Personalised recommendations