This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Numerical simulation refers to simulation for finding solutions to mathematical models, normally for cases in which mathematics does not provide analytical solutions. Technical simulation stands for simulation with numerical models in computational sciences and engineering.
- 2.
Verification in the left quadrant of Fig. 8.1 is sometimes known as “internal validation”.
- 3.
From a technical point of view, in classical computer theory, verification amounts to ascertaining the validity of certain output as a function of given input, regardless of any interpretation given in terms of any theory or any phenomenon not strictly computational – it is pure inference in a closed world. But this would require us to assume that social processes are computational in a Church-Turing sense, which seems difficult to conceive. For an elaboration on this see (David et al. 2007).
- 4.
We consider “modules” and “sub-programs” as synonymous.
References
Amblard F, Bommel P, Rouchier J (2007) Assessment and validation of multi-agent models. In: Phan D, Amblard F (eds) Agent-based modelling and simulation in the social and human sciences. The Bardwell Press, Oxford, pp 93–114
Arai R, Watanabe S (2008) A quantitative comparison method for multi-agent based simulation in the feature space. In: David N, Sichman JS (eds) Multi-agent-based simulation IX, MABS 2008, revised selected papers (Lecture notes in artificial intelligence), vol 5269. Springer, Berlin, pp 154–166
Axelrod R (1997a) The dissemination of culture: a model with local convergence and global polarization. J Conf Resolut 41(2):203–226
Axelrod R (1997b) Advancing the art of simulation in the social sciences. In: Conte R, Hegselmann R, Terna P (eds) Simulating social phenomena. Springer, Berlin, pp 21–40
Axtell R, Axelrod R, Epstein J, Cohen M (1996) Aligning simulation models: a case study and results. Comp Math Organiz Theory 1(2):123–141
Barreteau O, Bousquet F, Attonaty JM (2001) Role-playing games for opening the black box of multi-agent systems: method and lessons of its application to Senegal River Valley irrigated systems. J Artif Soc Soc Simul 4(2). http://jasss.soc.surrey.ac.uk/4/2/5.html
Barreteau O et al (2013) Participatory approaches. Chapter 10 in this volume
Boero R, Squazzoni F (2005) Does empirical embeddedness matter? Methodological issues on agent-based models for analytical social science. J Artif Soc Soc Simul 8(4). http://jasss.soc.surrey.ac.uk/8/4/6.html
David N (2009) Validation and verification in social simulation: patterns and clarification of terminology. In: Squazzoni F (ed) Epistemological aspects of computer simulation in the social sciences, second international workshop, EPOS 2006, Brescia, 5–6 Oct 2006, revised selected and invited papers (Lecture notes in computer science), vol 5466. Springer, Berlin, pp 117–129
David N, Sichman J, Coelho H (2003) Towards an emergence-driven software process for agent-based-simulation. In: Sichman JS, Bousquet F, Davidsson P (eds) Multi-agent-based simulation II, third international workshop, MABS 2002, Bologna, 15–16 July 2002, revised papers (Lecture notes in artificial intelligence), vol 2581. Springer, Berlin, pp 89–104
David N, Marietto M, Sichman J, Coelho H (2004) The structure and logic of interdisciplinary research in agent-based social simulation. J Artif Soc Soc Simul 7(3). http://jasss.soc.surrey.ac.uk/7/3/4.html
David N, Sichman J, Coelho H (2007) Simulation as formal and generative social science: the very idea. In: Gershenson C, Aerts D, Edmonds B (eds) Worldviews, science and us: philosophy and complexity. World Scientific, Singapore, pp 266–284
David N, Caldas JC, Coelho H (2010) Epistemological perspectives on simulation III: selected papers of EPOS 2008. J Artif Soc Soc Simul 13(1), special section. http://jasss.soc.surrey.ac.uk/13/1/
Dean S et al (2000) Understanding Anasazi culture change through agent-based modeling. In: Kohler T, Gumerman G (eds) Dynamics in human and primate societies: agent-based modeling of social and spatial processes. Oxford University Press, New York/Oxford, pp 179–205
Edmonds B, Hales D (2003) Replication, replication and replication: some hard lessons from model alignment. J Artif Soc Soc Simul 6(4). http://jasss.soc.surrey.ac.uk/6/4/11.html
Edmonds B, Moss S (2005) From KISS to KIDS: an ‘anti-simplistic’ modelling approach. In: Davidsson P, Logan B, Takadama K (eds) Multi-agent and multi-agent-based simulation (Lecture notes in artificial intelligence), vol 3415. Springer, Berlin, pp 130–144
Frank U, Troitzsch KG (2005) Epistemological perspectives on simulation. J Artificial Soc Soc Simul 8(4). http://jasss.soc.surrey.ac.uk/8/4/7.html
Galán JM et al (2013) Checking simulations: detecting and avoiding errors and artefacts. Chapter 6 in this volume
Gilbert N (2008) Agent-based models, vol 153, Quantitative applications in the social sciences. Sage, London
Gross D, Strand R (2000) Can agent-based models assist decisions on large-scale practical problems? A philosophical analysis. Complexity 5(6):26–33
Kleijnen JPC (1995) Verification and validation of simulation models. Euro J Operat Res 82: 145–162
Law A, Kelton D (1991) Simulation modelling and analysis, 2nd edn. McGraw Hill, New York
Miller JH (1998) Active nonlinear tests (ANTs) of complex simulation models. Manage Sci 44(6):820–830
Moss S, Edmonds B (2005) Sociology and simulation: statistical and qualitative cross-validation. Am J Sociol 110(4):1095–1131
Munson L, Hulin C (2000) Examining the fit between empirical data and theoretical simulations. In: Ilgen DR, Hulin CL (eds) Computational modeling of behaviour in organisations: the third scientific discipline. American Psychological Association, Washington, DC, pp 69–84
Rouchier J, Cioffi-Revilla C, Polhill JG, Takadama K (2008) Progress in model-to-model analysis. J Artif Soc Soc Simul 11(2). http://jasss.soc.surrey.ac.uk/11/2/8.html
Sargent R (1999) Verification and validation of simulation models. In: Farrington PA, Nembhard HB, Sturrock DT, Evans GW (eds) Proceedings of the 1999 winter simulation conference, Squaw Peak, Phoenix, 5–8 Dec 1999, pp 39–48. http://www.informs-sim.org/wsc99papers/005.PDF
Sommerville I (1995) Software engineering, 5th edn. Addison Wesley, Boston
Squazzoni F (2009) Epistemological aspects of computer simulation in the social sciences, second international workshop, EPOS 2006, Brescia, Revised selected and invited papers (Lecture notes in computer science), vol 5466. Springer, Berlin, 5–6 Oct 2006
Takadama K, Suematsu Y, Sugimoto N, Nawa N, Shimohara K (2003) Cross-element validation in multi-agent-based simulation: switching learning mechanisms in agents. J Artif Soc Soc Simul 6(4). http://jasss.soc.surrey.ac.uk/6/4/6.html
Troitzsch K (2004) Validating simulation models. In: Horton G (ed) Proceedings of the 18th European simulation multiconference, ESM 2004, 13–16 June 2004, Magdeburg. SCS, Erlangen, pp 265–270
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Further Reading
Further Reading
Good introductions to validation and verification of simulation models in general are Sargent (1999) and Troitzsch (2004), the latter with a focus on social simulation. Validation of agent-based models in particular is addressed by Amblard and colleagues (Amblard et al. 2007).
For readers more interested in single aspects of V&V with regard to agent-based models in the context of social simulation, the following papers provide highly accessible starting points:
-
Edmonds and Hales (2003) demonstrate the importance of model replication (or model alignment) by means of a clear example.
-
Boero and Squazzoni (2005) examine the use of empirical data for model calibration and validation and argue that “the characteristics of the empirical target” influence the choice of validation strategies.
-
Moss and Edmonds (2005) discuss an approach for cross-validation that combines the involvement of stakeholders to validate the model qualitatively on the micro level with the application of statistical measures to numerical outputs to validate the model quantitatively on the macro level.
Finally, for a more in-depth epistemological perspective on verification and validation I would refer the inclined reader to a revised version of my EPOS 2006 paper (David 2009).
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
David, N. (2013). Validating Simulations. In: Edmonds, B., Meyer, R. (eds) Simulating Social Complexity. Understanding Complex Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-93813-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-93813-2_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-93812-5
Online ISBN: 978-3-540-93813-2
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)