Towards the Automatic Identification of Faulty Multi-Agent Based Simulation Runs Using MASTER

  • Chris J. Wright
  • Phil McMinn
  • Julio Gallardo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7838)

Abstract

Testing a multi-agent based model is a tedious process that involves generating very many simulation runs, for example as a result of a parameter sweep. In practice, each simulation run must be inspected manually to gain complete confidence that the agent-based model has been implemented correctly and is operating according to expectations. We present MASTER, a tool which aims to semi-automatically detect when a simulation run has deviated from “normal” behaviour. A simulation run is flagged as “suspicious” when certain parameters traverse normal bounds determined by the modeller. These bounds are defined in reference to a small series of actual executions of the model deemed to be correct. The operation of MASTER is presented with two case studies, the first with the well-known “flockers” model supplied with the popular MASON agent-based modelling toolkit, and the second a skin tissue model written using another toolkit—FLAME.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    JUnit, http://www.junit.org (accessed: April 2012)
  2. 2.
  3. 3.
    Bentley, K., Gerhardt, H., Bates, P.: Agent-based simulation of notch-mediated tip cell selection in angiogenic sprout initialisation. Journal of Theoretical Biology 250, 25–36 (2008)CrossRefGoogle Scholar
  4. 4.
    Buchanan, M.: Meltdown modelling. Could agent-based computer models prevent another financial crisis? Nature 460(7256), 680–682 (2009)CrossRefGoogle Scholar
  5. 5.
    Chang, G., Roth, C.B., Reyes, C.L., Pornillos, O., Chen, Y.-J., Chen, A.P.: Science 314, 1875 (2006); Retraction of: Pornillos, et al.: Science 310(5756), 1950-1953; Reyes, Chang: Science 308(5724), 1028-1031; Chang, Roth: Science 293(5536), 1793–1800 Google Scholar
  6. 6.
    Coelho, R., Cirilo, E., Kulesza, U., von Staa, A., Rashid, A., Lucena, C.: JAT: A Test Automation Framework for Multi-Agent Systems. In: 2007 IEEE International Conference on Software Maintenance, pp. 425–434 (October 2007)Google Scholar
  7. 7.
    Cova, M., Kruegel, C., Vigna, G.: Detection and analysis of drive-by-download attacks and malicious javascript code. In: Proceedings of the International World Wide Web Conference (WWW 2010), pp. 281–290. ACM Press (2010)Google Scholar
  8. 8.
    Farmer, J., Foley, D.: The economy needs agent-based modelling. Nature 460(7256), 685–686 (2009)CrossRefGoogle Scholar
  9. 9.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)CrossRefGoogle Scholar
  10. 10.
    Kiran, M., Richmond, P., Holcombe, M., Chin, L.S., Worth, D., Greenough, C.: FLAME: simulating large populations of agents on parallel hardware architectures. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2010), pp. 1633–1636. ACM Press (2010)Google Scholar
  11. 11.
    Kruegel, C., Mutz, D., Valeur, F., Vigna, G.: On the detection of anomalous system call arguments. In: Snekkenes, E., Gollmann, D. (eds.) ESORICS 2003. LNCS, vol. 2808, pp. 326–343. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  12. 12.
    Luke, S., Cioffi-Revilla, C., Panait, L., Sullivan, K.: Mason: A new multi-agent simulation toolkit. In: Proceedings of the 2004 SwarmFest Workshop (2004)Google Scholar
  13. 13.
    Ma, Y.-S., Offutt, J., Kwon, Y.-R.: MuJava: An automated class mutation system. Journal of Software Testing, Verification and Reliability, 97–133 (2005)Google Scholar
  14. 14.
    Muaz Niazi, A.H., Kolberg, M.: Verification and validation of agent-based simulation using the vomas approach. In: Proceedings of the Third Workshop on Multi-Agent Systems and Simulation 2009, MASS 2009 (2009)Google Scholar
  15. 15.
    Nguyen, C., Perini, A.: Automated continuous testing of multi-agent systems. In: Workshop on Multi-Agent Systems (2007)Google Scholar
  16. 16.
    Simons, K.: Model error—evaluation of various finance models. New England Economic Review, 17–28 (1997)Google Scholar
  17. 17.
    Sun, T., McMinn, P., Coakley, S., Holcombe, M., Smallwood, R., MacNeil, S.: An integrated systems biology approach to understanding the rules of keratinocyte colony formation. Journal of the Royal Society Interface 4, 1077–1092 (2007)CrossRefGoogle Scholar
  18. 18.
    Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing for agent systems. In: 2nd International Working Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2007), pp. 10–18. Citeseer (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Chris J. Wright
    • 1
  • Phil McMinn
    • 1
  • Julio Gallardo
    • 1
  1. 1.Department of Computer ScienceUniversity of SheffieldPortobelloUK

Personalised recommendations