The Quality of Social Simulation: An Example from Research Policy Modelling

Part of the Public Administration and Information Technology book series (PAIT, volume 10)

Abstract

This chapter deals with the assessment of the quality of a simulation. The first section points out the problems of the standard view and the constructivist view in evaluating social simulations. A simulation is good when we get from it what we originally would have liked to get from the target; in this, the evaluation of the simulation is guided by the expectations, anticipations, and experience of the community that uses it. This makes the user community view the most promising mechanism to assess the quality of a policy-modelling exercise. The second section looks at a concrete policy-modelling example to test this idea. It shows that the very first negotiation and discussion with the user community to identify their questions is highly user-driven, interactive, and iterative. It requires communicative skills, patience, willingness to compromise on both sides, and motivation to make the formal world of modellers and the narrative world of practical policy making meet. Often, the user community is involved in providing data for calibrating the model. It is not an easy issue to confirm the existence, quality, and availability of data and check for formats and database requirements. As the quality of the simulation in the eyes of the user will very much depend on the quality of the informing data and the quality of the model calibration, much time and effort need to be spent in coordinating this issue with the user community. Last but not least, the user community has to check the validity of simulation results and has to believe in their quality. Users have to be enabled to understand the model, to agree with its processes and ways to produce results, to judge similarity between empirical and simulated data, etc. Although the user community view might be the most promising, it is the most work-intensive mechanism to assess the quality of a simulation. Summarising, to trust the quality of a simulation means to trust the process that produced its results. This process includes not only the design and construction of the simulation model itself but also the whole interaction between stakeholders, study team, model, and findings.

This chapter deals with the assessment of the quality of a simulation. After discussing this issue on a general level, we apply and test the assessment mechanisms using an example from policy modelling.

References

  1. Ahrweiler P, Gilbert N (2005) Caffe Nero: the evaluation of social simulation. J Artif Soc Soc Simul 8(4):14Google Scholar
  2. Ahrweiler P, Pyka A, Gilbert N (2011) A New model for university-industry links in knowledge-based economies. J Prod Innov Manag 28:218–235CrossRefGoogle Scholar
  3. Ahrweiler P, Schilperoord M, Pyka A, Gilbert N (2014a, forthcoming): Testing policy options for horizon 2020 with SKIN. In: Gilbert N, Ahrweiler P, Pyka A (eds) Simulating knowledge dynamics in innovation networks. Springer, HeidelbergGoogle Scholar
  4. Ahrweiler P, Pyka A, Gilbert N (2014b, forthcoming): Simulating knowledge dynamics in innovation networks: an introduction. In: Gilbert N, Ahrweiler P, Pyka A (eds) Simulating knowledge dynamics in innovation networks. Springer, HeidelbergGoogle Scholar
  5. Axelrod R (1984) The evolution of cooperation. Basic Books, New YorkGoogle Scholar
  6. Balzer W, Moulines CU, Sneed JD (1987) An architectonic for science. The structuralist program. Reidel, DordrechtCrossRefGoogle Scholar
  7. Baudrillard J (1988) Jean Baudrillard selected writings. Polity Press, CambridgeGoogle Scholar
  8. Ben-Naim E, Krapivsky P, Redner S (2003) Bifurcations and patterns in compromise processes. Phys D 183:190–204CrossRefGoogle Scholar
  9. Benenson I (2005) The city as a human-driven system. Paper presented at the workshop on modelling urban social dynamics. University of Surrey, Guildford, April 2005.Google Scholar
  10. Bruch E (2005) Dynamic models of neighbourhood change. Paper presented at the workshop on modelling urban social dynamics. University of Surrey, Guildford, April 2005.Google Scholar
  11. Carrier M (1994) The completeness of scientific theories. On the derivation of empirical indicators within a theoretical framework: the case of physical geometry. Kluwer, DordrechtGoogle Scholar
  12. Castelacci F, Fevolden A, Blom M (2014) R & D policy support and industry concentration: a SKIN model analysis of the European defence industry. In: Gilbert N, Ahrweiler P, PykaA (eds) Simulating knowledge dynamics in innovation networks. Heidelberg, SpringerGoogle Scholar
  13. Chalmers D, French R, Hofstadter D (1995) High-level perception, representation, and analogy. In: Hofstadter D (ed) Fluid concepts and creative analogies. Basic Books, New York, pp 165–191Google Scholar
  14. Cole O (2000) White-box testing. Dr. Dobb’s Journal, March 2000, pp 23–28Google Scholar
  15. Deffuant G, Neau D, Amblard F, Weisbuch G (2000) Mixing beliefs among interacting agents. Advances in complex systems. Adv Complex Syst 3:87–98CrossRefGoogle Scholar
  16. Doran J, Gilbert N (1994) Simulating Societies: an Introduction. In: Doran J, Gilbert N (eds) Simulating societies: the computer simulation of social phenomena. UCL Press, London, pp 1–18Google Scholar
  17. Droste W (1994) Sieger sehen anders aus (Winners look different). Schulenburg, HamburgGoogle Scholar
  18. Gellner E (1990) Pflug, Schwert und Buch. Grundlinie der Menschheitsgeschichte (Plough, Sword and Book. Foundations of human history). Klett-Cotta, StuttgartGoogle Scholar
  19. Gilbert N (1997) A simulation of the structure of academic science, Sociological Research Online 2(1997). http://www.socresonline.org.uk/socresonline/2/2/3.html
  20. Gilbert N, Troitzsch K (1997) Simulation for the social scientist. Open University Press, BuckinghamGoogle Scholar
  21. Gilbert N, Ahrweiler P, Pyka A (2007) Learning in innovation networks: some simulation experiments. Phys A: Stat Mech Appl 378(1):667–693Google Scholar
  22. Gilbert N, Ahrweiler P, Pyka A (eds) (2014, forthcoming) Simulating knowledge dynamics in innovation networks. Springer, HeidelbergCrossRefGoogle Scholar
  23. Glasersfeld E von (1987) Siegener Gespräche über Radikalen Konstruktivismus (Siegen Diskussions on Radical Constructivism). In: Schmidt SJ (ed) Der Diskurs des Radikalen Konstruktivismus. Suhrkamp, Frankfurt a. M., pp 401–440Google Scholar
  24. Harbodt S (1974) Computer simulationen in den Sozialwissenschaften (Computer simulations in the social sciences). Rowohlt, ReinbekGoogle Scholar
  25. Kértesz A (1993) Artificial intelligence and the sociology of scientific knowledge. Lang, Frankfurt, a. M.Google Scholar
  26. Korber M, Paier M (2014) Simulating the effects of public funding on research in life sciences: direct research funds versus tax incentives. In: Gilbert N, Ahrweiler P, Pyka A (eds) Simulating knowledge dynamics in innovation networks. Springer, HeidelbergGoogle Scholar
  27. Nickles T (1989) ∈ntegrating the science studies disciplines. In: Fuller S, de Mey M, Shinn T, Woolgar S (eds) The cognitive turn. Sociological and psychological perspectives on science. Kluwer, Dordrecht, pp 225–256Google Scholar
  28. Norris C (1992) Uncritical theory. Lawrence and Wishart, LondonGoogle Scholar
  29. Pyka A, Gilbert N, Ahrweiler P (2003) Simulating Innovation networks. In: Pyka A, Küppers G (eds) Innovation networks—theory and practice. Edward Elgar, Cheltenham, pp 169–198Google Scholar
  30. Pyka A, Gilbert N, Ahrweiler P (2007) Simulating knowledge generation and distribution processes in innovation collaborations and networks. Cybern Syst 38(7):667–693CrossRefGoogle Scholar
  31. Quine W (1977) Ontological relativity. Columbia University Press, ColumbiaGoogle Scholar
  32. Schilperoord M, Ahrweiler P (2014, forthcoming) Towards a prototype policy laboratory for simulating innovation networks. In: Gilbert N, Ahrweiler P, Pyka A (eds) Simulating knowledge dynamics in innovation networks. Springer, HeidelbergGoogle Scholar
  33. Scholz R, Nokkala T, Ahrweiler P, Pyka A, Gilbert N (2010) The agent-based NEMO model (SKEIN): simulating European Framework programmes. In: Ahrweiler P (ed) Innovation in complex social systems, Routledge studies in global competition. Routledge, London, pp 300–314Google Scholar
  34. Searle J (1997) The construction of social reality. Free Press, New YorkGoogle Scholar
  35. Weisbuch G (2004) Bounded confidence and social networks. Special Issue: application of complex networks in biological information and physical systems. Eur Phys JB 38:339–343Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.EA European Academy of Technology and Innovation Assessment GmbHBad Neuenahr-AhrweilerGermany
  2. 2.University of SurreyGuildfordUK

Personalised recommendations