Validation of Agent-Based Simulation through Human Computation: An Example of Crowd Simulation

  • Pengfei Xing
  • Michael Lees
  • Hu Nan
  • T. Vaisagh Viswanthatn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7124)


Agent-based modeling as a methodology for understanding natural phenomena is becoming increasingly popular in many disciplines of scientific research. Validation is still a significant problem for agent-based modelers and while various validation methodologies have been proposed, none have been widely adopted. Data plays a key role in the validation of any simulation system, typically large amounts of observable real world data are necessary to compare with model outputs. However, the complex nature of the studied natural systems will often make data collection difficult. This is certainly true for crowd and egress simulation, where data is limited and difficult to collect. In this paper we propose a new technique for validation of agent-based models, particularly those which relate to human behavior. This methodology adopts ideas from the field of Human Computation as a means of collecting large amounts of contextual behavioral data. The key principle is to use games as a means of framing behavioral questions to try and capture people’s natural and instinctive decisions. We outline some key design challenges for such games and present one example game in the form of Escape. Escape is an egress based game where people are tasked to escape from rooms inhabited by other people. We show some preliminary studies which highlight some interesting applications of the game in addressing validation of behavioral based crowd and egress simulation.


Simulation Validation Games Human Computation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Banerjee, B., Kraemer, L.: Validation of agent based crowd egress simulation. In: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2010, vol. 1, pp. 1551–1552. International Foundation for Autonomous Agents and Multiagent Systems, Richland (2010)Google Scholar
  2. 2.
    Berrou, J., Beecham, J., Quaglia, P., Kagarlis, M., Gerodimos, A.: Calibration and validation of the legion simulation model using empirical data. In: Waldau, N., Gattermann, P., Knoflacher, H., Schreckenberg, M. (eds.) Pedestrian and Evacuation Dynamics 2005, pp. 167–181. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Epstein, J.M.: Why model? Journal of Artificial Societies and Social Simulation 11(4), 12 (2008)Google Scholar
  4. 4.
    Ferguson, N.M., Cummings, D.A.T., Fraser, C., Cajka, J.C., Cooley, P.C., Burke, D.S.: Strategies for mitigating an influenza pandemic. Nature 442(7101), 448–452 (2006)CrossRefGoogle Scholar
  5. 5.
    Ferscha, A., Zia, K.: On the efficiency of lifebelt based crowd evacuation. In: Proceedings of the 2009 13th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications, DS-RT 2009, pp. 13–20. IEEE Computer Society, Washington, DC (2009)CrossRefGoogle Scholar
  6. 6.
    Gonzalez, A.J.: Validation of human behavioral models. In: Proceedings of the Twelfth International Florida Artificial Intelligence Research Society Conference, pp. 489–493. AAAI Press (1999)Google Scholar
  7. 7.
    Guy, S.J., Chhugani, J., Curtis, S., Dubey, P., Lin, M., Manocha, D.: Pledestrians: a least-effort approach to crowd simulation. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA 2010, pp. 119–128. Eurographics Association, Aire-la-Ville (2010)Google Scholar
  8. 8.
    Helbing, D., Molnár, P.: Social force model for pedestrian dynamics. Physical Review E 51(5), 4282–4286 (1995)CrossRefGoogle Scholar
  9. 9.
    Jennings, N.R.: An agent-based approach for building complex software systems. Commun. ACM 44, 35–41 (2001)CrossRefGoogle Scholar
  10. 10.
    Klügl, F.: A validation methodology for agent-based simulations. In: Proceedings of the, ACM Symposium on Applied Computing, SAC 2008, pp. 39–43. ACM, New York (2008)Google Scholar
  11. 11.
    Luo, L., Zhou, S., Cai, W., Low, M.Y.-H., Lees, M.: Toward a generic framework for modeling human behaviors in crowd simulation. In: IAT, pp. 275–278. IEEE (2009)Google Scholar
  12. 12.
    Lux, T., Marchesi, M.: Scaling and criticality in a stochastic multi-agent model of a financial market. Nature 397(6719), 498–500 (1999)CrossRefGoogle Scholar
  13. 13.
    McGonical, J.: Gaming can make a better world. TED (February 2010),
  14. 14.
    Moss, S.: Alternative approaches to the empirical validation of agent-based models. Journal of Artificial Societies and Social Simulation 11(1), 5 (2008)Google Scholar
  15. 15.
    Pan, X., Han, C., Dauber, K., Law, K.: A multi-agent based framework for the simulation of human and social behaviors during emergency evacuations. AI & Society 22, 113–132 (2007), doi:10.1007/s00146-007-0126-1CrossRefGoogle Scholar
  16. 16.
    Pelechano, N., Stocker, C., Allbeck, J.M., Badler, N.I.: Being a part of the crowd: towards validating vr crowds using presence. In: AAMAS 2008 (1), pp. 136–142 (2008)Google Scholar
  17. 17.
    Sloot, P., Coveney, P., Dongarra, J.: Editorial board. Journal of Computational Science 1(4), CO2–CO2 (2010)Google Scholar
  18. 18.
    Sterman, J.: Testing behavioral simulation models by direct experiment. Working papers 1752-86. Massachusetts Institute of Technology (MIT), Sloan School of Management (1986)Google Scholar
  19. 19.
    Treuille, A., Cooper, S., Popović, Z.: Continuum crowds. In: SIGGRAPH 2006: ACM SIGGRAPH 2006 Papers, pp. 1160–1168. ACM, New York (2006)Google Scholar
  20. 20.
    von Ahn, L.: Games with a purpose. Computer 39, 92–94 (2006)CrossRefGoogle Scholar
  21. 21.
    von Ahn, L.: Human computation. In: Proceedings of the 4th International Conference on Knowledge Capture, K-CAP 2007, pp. 5–6. ACM, New York (2007)Google Scholar
  22. 22.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, pp. 319–326. ACM, New York (2004)CrossRefGoogle Scholar
  23. 23.
    von Ahn, L., Liu, R., Blum, M.: Peekaboom: a game for locating objects in images. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 55–64. ACM, New York (2006)Google Scholar
  24. 24.
    Weifeng, Y., Hai, T.K.: A novel algorithm of simulating multi-velocity evacuation based on cellular automata modeling and tenability condition. Physica A: Statistical Mechanics and its Applications 379(1), 250–262 (2007)CrossRefGoogle Scholar
  25. 25.
    Windrum, P., Fagiolo, G., Moneta, A.: Empirical validation of agent-based models: Alternatives and prospects. Journal of Artificial Societies and Social Simulation 10(2), 8 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Pengfei Xing
    • 1
  • Michael Lees
    • 1
  • Hu Nan
    • 1
  • T. Vaisagh Viswanthatn
    • 1
  1. 1.School of Computer EngineeringNanyang Technological UniversitySingapore

Personalised recommendations