An Evaluation Method for Multi-Agent Systems

  • Pierpaolo Di Bitonto
  • Maria Laterza
  • Teresa Roselli
  • Veronica Rossano
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6070)


The growing employment of Multi-Agent Systems (MASs) in several domains of everyday life has provided the impetus for much research into new tools and methodologies for their design and implementation. But up to now, few works have focused on evaluation of these MASs, and none of these considered characteristics such as the rationality, the autonomy, the reactivity and the environment adaptability of the agents in the MAS. We believe these characteristics affect the whole performance of these systems and are connected to the complexity of the environment where the agents act. In this paper we propose an evaluation method for static multi-agent systems. The method, based on the Goal-Question-Metric approach, allows evaluation of these same MAS characteristics and combines two analysis perspectives of these systems: intra-agent and inter-agent. We also report the use of the defined approach to evaluate the GeCo_Automotive system’s MAS.


Multi-Agent Systems Goal-Question-Metric MAS evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jang, K.S., Nam, T.E., Wadhwa, B.: On measurement of Objects and Agents.,
  2. 2.
    Klügl, F.: Measuring Complexity of Multi-agent Simulations – An Attempt Using Metrics. In: Dastani, M., El Fallah Seghrouchni, A., Leite, J., Torroni, P. (eds.) LADS 2007. LNCS (LNAI), vol. 5118, pp. 123–138. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Król, D., Zelmozer, M.: Structural Performance Evaluation of Multi-Agent Systems. J. of Universal Computer Science 14, 1154–1178 (2008)Google Scholar
  4. 4.
    Hmida, F.B., Chaari, W.L., Tagina, M.: Performance Evaluation of Multiagent Systems: Communication Criterion. In: Carbonell, J.G., Siekmann, J. (eds.) KES-AMSTA 2008. LNCS (LNAI), vol. 4953, pp. 773–782. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Gutiérez, C., García-Magariño, I., Gómez-Sanz, J.J.: Evaluation of Multi-Agent System Communication in INGENIAS. In: Cabestany, J., Sandoval, F., Prieto, A., Corchado, J.M. (eds.) IWANN 2009. LNCS, vol. 5517, pp. 619–626. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Lee, S.K., Hwang, C.S.: Architecture modeling and evaluation for design of agent-based system. J. of Systems and Software 72, 195–208 (2004)CrossRefGoogle Scholar
  7. 7.
    Lass, R.N., Sultanik, E.A., Regli, W.C.: Metrics for Multiagent Systems. In: Madhavan, R., Tunstel, E., Messina, E. (eds.) Performance Evaluation and Benchmarking of Intelligent Systems. LNCS, pp. 1–19. Springer, US (2009)CrossRefGoogle Scholar
  8. 8.
    Basili, V., Caldiera, G., Rombach, H.: The Goal Question Metric Approach. In: Marciniak, J.J. (ed.) Encyclopedia of Soft. Eng., vol. 2, pp. 528–532. John Wiley & Sons, Inc., Chichester (1994)Google Scholar
  9. 9.
    Russell, S., Norvig, P.: Artificial intelligence: A modern approach. Prentice-Hall, Englewood Cliffs (1995)MATHGoogle Scholar
  10. 10.
    Wooldridge, M., Jennings, N.R.: Intelligent agents: Theory and practice. The Knowledge Engineering Review 10, 115–152 (1995)CrossRefGoogle Scholar
  11. 11.
    Nwana, H.S.: Software Agents: An Overview. The Knowledge Engineering Review 11, 205–244 (1996)CrossRefGoogle Scholar
  12. 12.
    Di Bitonto, P., Plantamura, V.L., Roselli, T., Rossano, V.: A taxonomy for cataloging LOs using IEEE educational metadata. In: 7th IEEE International Conference on Advanced Learning Technologies, pp. 139–141. IEEE Press, Los Alamitos (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Pierpaolo Di Bitonto
    • 1
  • Maria Laterza
    • 1
  • Teresa Roselli
    • 1
  • Veronica Rossano
    • 1
  1. 1.Department of Computer ScienceUniversity of BariBariItaly

Personalised recommendations