Information Systems Frontiers

, Volume 16, Issue 2, pp 169–182 | Cite as

Beast methodology: An agile testing methodology for multi-agent systems based on behaviour driven development

  • Álvaro CarreraEmail author
  • Carlos A. Iglesias
  • Mercedes Garijo


This paper presents a testing methodology to apply Behaviour Driven Development (BDD) techniques while developing Multi-Agent Systems (MASs), termed BEhavioural Agent Simple Testing (BEAST) Methodology. This methodology is supported by the open source framework (BEAST Tool) which automatically generates test cases skeletons from BDD scenarios specifications. The developed framework allows the testing of MASs based on JADE or JADEX platforms. In addition, this framework offers a set of configurable Mock Agents with the aim of being able to execute tests while the MAS is under development. The BEAST Methodology presents transparent traceability from user requirements to test cases. Thus, the stakeholders can be aware of the project status. The methodology and the associated tool have been validated in the development of a MAS for fault diagnosis in FTTH (Fiber To The Home) networks. The results have been measured in quantifiable way obtaining a reduction of the tests implementation time.


Test Behaviour-driven development Multi-agent systems Mock-agents Agile Methodology 



This research work is supported by the Ministry of Economy and Competitiveness under the R&D project CALISTA (TEC2012-32457). The authors want to acknowledge the cooperation of Telefónica R&D, and Javier García-Algarra, Javier García-Ordás, Beatriz Fuentes, Raquel Toribio and Andrés Sedano-Frade.


  1. Adzic, G. (2009). Bridging the communication gap: specification by example and agile acceptance testing. UK: Neuri Limited.Google Scholar
  2. Adzic, G. (2011). Specification by example: how successful teams deliver the right software, 1st ed. Greenwich: Manning Publications Co.Google Scholar
  3. Agarwal, N., & Rathod, U. (2006). Defining success for software projects: an exploratory revelation. International Journal of Project Management, 24(4), 358–370.CrossRefGoogle Scholar
  4. Almeida, J., Iacob, M.E., Eck, P. (2007). Requirements traceability in model-driven development: applying model and transformation conformance. Information Systems Frontiers, 9(4), 327–342.CrossRefGoogle Scholar
  5. Bellifemine, F.L., Caire, G., Greenwood, D. (2007). Developing multi-agent systems with JADE, Wiley series in agent technology (Vol. 5). West Sussex:Wiley.CrossRefGoogle Scholar
  6. Bordini, R.H., Hübner, J.F., Wooldridge, M. (2007). Programming multi-agent systems in AgentSpeak using Jason (Vol. 8). West Sussex:Wiley.CrossRefGoogle Scholar
  7. Braubach, L., Pokahr, A., Lamersdorf, W. (2005). Jadex: A BDIagent system combining middleware and reasoning. In R.Unland, M. Calisti, M. Klusch, M. Walliser, S. Brantschen, T. Hempfling (Eds.), Software agent-based applications, platforms and development kits. Whitestein series in software agent technologies and autonomic computing (pp. 143–168). Basel: Birkhäuser.Google Scholar
  8. Clynch, N., & Collier, R. (2007). Sadaam: software agent development-an agile methodology. In Proceedings of the workshop of languages, methodologies, and development tools for multi-agent systems (LADS007). Durham.Google Scholar
  9. Coelho, R., Kulesza, U., von Staa, A., Lucena, C. (2006). Unit testing in multi-agent systems using mock agents and aspects. In Proceedings of the 2006 international workshop on software engineering for large-scale multi-agent systems, SELMAS 06 (pp. 83–90). New York: ACM.Google Scholar
  10. Dikenelli, O., Erdur, R.C., Gumus, O. (2005). Seagent: a platform for developing semantic web based multi agent systems. In Proceedings of the fourth international joint conference on autonomous agents and multiagent systems, AAMAS ’05 (pp. 1271–1272). New York: ACM.Google Scholar
  11. Evans, E. (2004). Domain-driven design: tackling complexity in the heart of software. Boston:Addison-Wesley Professional.Google Scholar
  12. Foundation TAS (2011). Apache maven project. Accessed 23 October 2012.
  13. García-Magariño, I., Gómez-Rodríguez, A., Gómez-Sanz, J., González-Moreno, J. (2009). Ingenias-scrum development process for multi-agent development. In International symposium on distributed computing and artificial intelligence 2008 (DCAI 2008) (pp. 108–117). Springer.Google Scholar
  14. Gómez-Sanz, J., Botía J., Serrano, E., Pavón, J. (2009). Testing and debugging of MAS interactions with INGENIAS. In M. Luck, & J. Gomez-Sanz (Eds.), Agent-oriented software engineering IX. Lecture notes in computer science (Vol. 5386, pp. 199–212). Heidelberg: Springer.Google Scholar
  15. Guerra-García, C., Caballero, I., Piattini, M. (2013). Capturing data quality requirements for web applications by means of dqwebre. Information Systems Frontiers, 15(3), 433–445.CrossRefGoogle Scholar
  16. Hamill, P. (2004). Unit test frameworks, 1st edn. California:O’Reilly.Google Scholar
  17. Haugset, B., Hanssen, G. (2008). Automated acceptance testing: A literature review and an industrial case study. In Agile, 2008. AGILE ’08. Conference (pp. 27–38).Google Scholar
  18. Houhamdi, Z. (2011). Multi-agent system testing: a survey. International Journal of Advanced Computer Science and Applications (IJACSA), 2(6), 135–141.Google Scholar
  19. Iglesias, C., Garijo, M., González, J., Velasco, J. (1998). Analysis and design of multiagent systems using mas-commonkads. In M. Singh, A. Rao, M. Wooldridge (Eds.), Intelligent agents IV agent theories, architectures, and languages. Lecture notes in computer science (Vol. 1365, pp. 313–327). Berlin: Springer.Google Scholar
  20. Luke, S., Cioffi-Revilla, C., Panait, L., Sullivan, K., Balan, G. (2005). Mason: a multiagent simulation environment. Simulation, 81(7), 517–527.CrossRefGoogle Scholar
  21. Marnewick, A., Pretorius, J.H., Pretorius, L. (2011). A perspective on human factors contributing to quality requirements: a cross-case analysis. In 2011 IEEE international conference on industrial engineering and engineering management (IEEM) (pp. 389–393).Google Scholar
  22. Martin, R., & Melnik, G. (2008). Tests and requirements, requirements and tests: a möbius strip. IEEE Software, 25(1), 54–59.CrossRefGoogle Scholar
  23. Mockito Project (2012). Mockito framework. Accessed 25 March 2012.
  24. Mugridge, R., & Cunningham, W. (2005). Fit for developing software: framework for integrated tests. Upper Saddle River, NJ:Prentice Hall.Google Scholar
  25. Nguyen, C.D. (2009). Testing techniques for software agents. PhD thesis. International Doctorate School in Information and Communication Technologies.Google Scholar
  26. Nguyen, D., Perini, A., Tonella, P. (2008). A goal-oriented software testing methodology. In M. Luck, & L. Padgham (Eds.), Agent oriented software engineering VIII. Lecture notes in computer science (Vol. 4951, pp. 58–72). Berlin: Springer.Google Scholar
  27. Nguyen, C.D., Perini, A., Tonella, P. (2010). Goal oriented test ing for mass. International Journal of Agent-Oriented Software Engineering, 4(1), 79–109.CrossRefGoogle Scholar
  28. Nguyen, C., Perini, A., Bernon, C., Pavón, J., Thangarajah, J. (2011). Testing in multi-agent systems. In M.P. Gleizes, & J. Gomez-Sanz (Eds.), Agent-oriented software engineering X. Lecture notes in computer science (Vol. 6038, pp. 180–190). Berlin: Springer.Google Scholar
  29. North, D. (2007). Introducing: behaviour-driven development. Accessed 28 March 2012.
  30. North, D. (2011). JBehave. A framework for behaviour driven development (BDD). Accessed 28 March 2012.
  31. Padgham, L., & Winikoff, M. (2003). Prometheus: a methodology for developing intelligent agents. In F. Giunchiglia, J. Odell, G. Weiß (Eds.), Agent-oriented software engineering III. Lecture notes in computer science (Vol. 2585, pp. 174–185). Berlin: Springer.Google Scholar
  32. Pavon, J., Gomez-Sanz, J.J., Fuentes, R. (2005). The ingenias methodology and tools. Agent-Oriented Methodologies, 9, 236–276.CrossRefGoogle Scholar
  33. Schwaber, K., & Sutherland, J. (2009). Scrum guide. Scrum Alliance, 19(6), 21.Google Scholar
  34. Solis, C., & Wang, X. (2011). A study of the characteristics of behaviour driven development. In 2011 37th EUROMICRO conference on software engineering and advanced applications (SEAA) (pp. 383–387).Google Scholar
  35. Sun, L., Ousmanou, K., Cross, M. (2010). An ontological modelling of user requirements for personalised information provision. Information Systems Frontiers, 12(3), 337–356.CrossRefGoogle Scholar
  36. Thangarajah, J., Jayatilleke, G., Padgham, L. (2011). Scenarios for system requirements traceability and testing. In The 10th international conference on autonomous agents and multiagent systems international foundation for autonomous agents and multiagent systems, AAMAS ’11 (Vol. 1, pp. 285–292). Richland.Google Scholar
  37. Tiryaki, A., ztuna, S., Dikenelli, O., Erdur, R. (2007). SUNIT: a unit testing framework for test driven development of multi-agent systems. In L. Padgham, & F. Zambonelli (Eds.), Agent-oriented software engineering VII. Lecture notes in computer science (Vol. 4405, pp. 156–173). Berlin: Springer.Google Scholar
  38. Uhrmacher, A.M., Uhrmacher, A., Weyns, D. (2009). Multi-agent systems: simulation and applications. Boca Raton, FL: CRC Press.Google Scholar
  39. Wooldridge, M., Jennings, N.R., Kinny, D. (2000). The Gaia methodology for agent-oriented analysis and design. Autonomous Agents and Multi-Agent Systems, 3(3), 285–312.Google Scholar
  40. Wynne, M., & Hellesøy, A. (2008). Cucumber. Behaviour driven development with elegance and joy. Accessed 28 March 2012.
  41. Zhang, Z., Thangarajah, J., Padgham, L. (2011). Automated testing for intelligent agent systems. In M.P. Gleizes, & J. Gomez-Sanz (Eds.), Agent-oriented software engineering X. Lecture notes in computer science (Vol. 6038, pp. 66–79). Berlin: 925 Springer.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Álvaro Carrera
    • 1
    Email author
  • Carlos A. Iglesias
    • 1
  • Mercedes Garijo
    • 1
  1. 1.Departamento de Ingeniería de Sistemas TelemáticosUniversidad Politécnica de MadridMadridSpain

Personalised recommendations