The Users’ Judgements—The Stakeholder Approach to Simulation Validation

  • Nicole J. SaamEmail author
Part of the Simulation Foundations, Methods and Applications book series (SFMA)


This article presents a sociological perspective on the stakeholder approach to simulation validation using the validation of socio-ecological simulation models as an example. I develop an argument-based approach to simulation validation which can be applied in the natural and social sciences and argue that it is the constructionist camp of action researchers which has to consider the stakeholders’ judgements as an indispensable point of reference for simulation validation. Only the stakeholders can validate that the model makes explicit their tacit knowledge. Only the stakeholders’ willingness to accept and act upon the scenarios can decide issues of credibility. Obtaining the stakeholders’ judgements in such a framework is an iterative communicative procedure that requires a strong background in qualitative methods of empirical social research as well as gaming simulation.


Simulation validation Argument-based approach Action research Participatory modelling Stakeholder 



The author thanks Claus Beisbart for helpful discussions concerning this manuscript.


  1. Ahrweiler, P., & Gilbert, N. (2009). The epistemologies of social simulation research. In F. Squazzoni (Ed.), Epistemological aspects of computer simulation in the social sciences (pp. 12–28). Heidelberg: Springer.Google Scholar
  2. Ahrweiler, P., & Gilbert, N. (2015). The quality of social simulation—an example from research policy modelling. In M. Janssen, M. Wimmer & A. Deljoo (Eds.), Policy practice and digital science—integrating complex systems, social simulation and public administration in policy research (pp. 35–55). Springer, Heidelberg.Google Scholar
  3. Barbour, R. (2007). Doing focus groups. London: Sage.CrossRefGoogle Scholar
  4. Barlas, Y., & Carpenter, S. (1990). Philosophical roots of model validation two paradigms. System Dynamics Review, 6, 148–166.CrossRefGoogle Scholar
  5. Barnaud, C., Bousquet, F., & Trebuil, G. (2008). Multi-agent simulations to explore rules for rural credit in a highland farming community of Northern Thailand. Ecological Economics, 66, 615–627.CrossRefGoogle Scholar
  6. Barreteau, O., Bousquet, F., & Attonaty, J. M. (2001). Role-playing games for opening the black box of multi-agent systems. Method and lessons of its application to Senegal river valley irrigated systems. Journal of Artificial Societies and Social Simulation, 4.
  7. Barreteau, O., et al. (2013). Participatory approaches. In B. Edmonds & R. Meyers (Eds.), Simulating social complexity. understanding complex systems (pp. 197–234). Heidelberg: Springer.Google Scholar
  8. Baumberger, C., Knutti,R., & Hirsch Hadorn, G. (2017). Building confodence in climate model projections. An analysis of inferences from fit. WIREs Climate Change, 8, e454.Google Scholar
  9. Bhaskar, R. (1978). A realist theory of science. Brighton, England: Harvester Press.Google Scholar
  10. Berger, P., & Luckmann, T. (1966). The social construction of reality. New York: Doubleday.Google Scholar
  11. Bryman, A. (2012). Social research methods (4th ed.). Oxford: Oxford University Press.Google Scholar
  12. Castella, J.-C., Trung, T. N., & Boissau,S. (2005). Participatory simulation of land-use changes in the Northern Mountains of Vietnam. The combined use of an agent-based model, a role-playing game, and a geographic information system. Ecology and Society, 10, 27.Google Scholar
  13. Cho, J., & Trent, A. (2006). Validity in qualitative research revisited. Qualitative Research, 6, 319–340.CrossRefGoogle Scholar
  14. Coghlan, D., & Brydon-Miller, M. (2014). The SAGE encyclopedia of action research (Vol. 1 and 2). London: Sage.Google Scholar
  15. Dewey, J. (1900). The school and society. Chicago: Chicago University Press.Google Scholar
  16. Dewey, J. (1902). The child and the curriculum. Chicago: Chicago University Press.Google Scholar
  17. Dewey, J. (1991a). Logic: The theory of inquiry. Carbondale: Southern Illinios University Press.Google Scholar
  18. Dewey, J. (1991[1927]). The public and its problems. Athens: Ohio University Press.Google Scholar
  19. Dray, A., Perez, P., Jones, N., Le Page, C., D’Aquino, P., & Auatabu, T. (2006). The atollgame experience: from knowledge engineering to a computer-assisted role playing game. Journal of Artificial Societies and Social Simulation, 9.
  20. Dumrongrojwatthana, P., & Trébuil, G. (2011). Northern Thailand case: Gaming and simulation for co-learning and collective action. Companion modelling for collaborative landscape management between herders and foresters. In A. van Paassen, J. van den Berg, E. Steingröver, R. Werkman & B. Pedroli (Eds.), Knowledge in action. The search for collaborative research for sustainable landscape development (pp. 191–219). Wageningen Academic Publishers.Google Scholar
  21. Étienne, M. (Ed.). (2014). Companion modelling. A participatory approach to support sustainable development. Dordrecht: Springer.Google Scholar
  22. Feinstein, A. H., & Cannon, H. M. (2003). A hermeneutical approach to external validation of simulation models. Simulation & Gaming, 34, 186–197.CrossRefGoogle Scholar
  23. Flick, U. (2014). An introduction to qualitative research (5th ed.). London/Thousand Oaks, CA/ Dehli: Sage.Google Scholar
  24. Gadamer, H.-G. (2002 [1960]). Truth and method. Joel Weinsheimer. (2 revised ed.). New York: Continuum.Google Scholar
  25. Glasersfeld, E. V. (1982). An interpretation of piaget’s constructivism. Revue Internationale de Philosophie, 36, 612–635.Google Scholar
  26. Greenwood, D. J., & Levin, M. (1998). Introduction to action research: Social research for social change. Thousand Oaks: Sage.Google Scholar
  27. Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park: Sage.Google Scholar
  28. Guba, E. G., & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and emerging confluences. In N. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (pp. 191–215). Thousand Oaks: Sage.Google Scholar
  29. Gubrium, J. F., & Holstein, J. A. (Eds.). (2001). Handbook of interview research. London: Sage.Google Scholar
  30. Habermas, J. (1984). Theory of communicative action. In T. McCarthy (Ed.), Reason and the rationalization of society (Vol. 1). Boston: Beacon Press.Google Scholar
  31. Heron, J., & Reason, P. (1997). A participatory inquiry paradigma. Qualitative Inquiry, 3, 274–294.CrossRefGoogle Scholar
  32. Hershberg, R. M. (2014). Constructivism. In D. Coghlan & M. Brydon-Miller (Eds.), The SAGE encyclopedia of action research (Vol. 1 and 2, pp. 182–186). London: Sage.Google Scholar
  33. Houston, S. (2014). Critical realism. In D. Coghlan & M. Brydon-Miller (Eds.), The SAGE encyclopedia of action research (Vol. 1 and 2, pp. 219–222). London: Sage.Google Scholar
  34. Irvine, S. R., Levary, R. R., & McCoy, M. S. (1998). The impact of judgemental biases on the validation of simulation models. Simulation & Gaming, 29, 152–164.CrossRefGoogle Scholar
  35. James, W. (1907). Pragmatism’s conception of truth. The Journal of Philosophy, Psychology and Scientific Methods, 4, 141–155.CrossRefGoogle Scholar
  36. Johansson, A., & Lindhult, E. (2008). Emancipation or workability? Critical versus pragmatic scientific orientation in action research. Action Research, 6, 95–115.CrossRefGoogle Scholar
  37. Kemmis, S. (2008). Critical theory and participatory action research. In P. Reason & H. Bradbury (Eds.), Handbook of action research. Participative inquiry and practice (2nd ed., pp. 121–138). London: Sage.Google Scholar
  38. Klabbers, J. H. G. (2009). The magic circle. Principles of gaming & simulation (3rd ed.). Rotterdam: Sense Publishers.Google Scholar
  39. Kleindorfer, G. B., O’Neill, L., & Ganeshan, R. (1998). Validation in simulation: Various positions in the philosophy of science. Management Science, 44, 1087–1099.CrossRefGoogle Scholar
  40. Koro-Ljungberg, M. (2008). Validity and validation in the making in the context of qualitative research. Qualitative Health Research, 7, 983–989.CrossRefGoogle Scholar
  41. Kvale, S. (1995). The social construction of validity. Qualitative Inquiry, 1, 19–40.CrossRefGoogle Scholar
  42. Lather, P. (1986). Issues of validity in openly ideological research. Interchange, 17, 63–84.CrossRefGoogle Scholar
  43. Lather, P. (2007). Validity, qualitative. In G. Ritzer (Ed.), The blackwell encyclopedia of sociology (pp. 5161–5165). Oxford: Blackwell Publishing.Google Scholar
  44. Lincoln, Y. S., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills: Sage.CrossRefGoogle Scholar
  45. Lincoln, Y. S., Lynham, S. A., & Guba, E. G. (2011). Paradigmatic controversies, contradictions, and emerging confluences, revisited. In N. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research revisited (pp. 97–128). Thousand Oaks: Sage.Google Scholar
  46. Moss, S. (2008). Alternative approaches to the empirical validation of agent-based models. Journal of Artificial Societies and Social Simulation, 11,
  47. Murray-Smith, D. J. (2015). Testing and validation of computer simulation models: Principles, methods and applications. Cham: Springer.CrossRefGoogle Scholar
  48. Naivinit, W., Le Page, C., Trébuil, G., & Gajaseni, N. (2010). Participatory agent-based modeling and simulation of rice production and labor migrations in Northeast Thailand. Environmental Modelling and Software, 25, 1345–1358.CrossRefGoogle Scholar
  49. Nicholas, M. C., & Hathcoat, J. D. (2014). Ontology. In: D. Coghlan & M. Brydon-Miller (Eds.), The SAGE encyclopedia of action research (Vol. 1 and 2, pp. 570–572). London: Sage.Google Scholar
  50. Parker, W. S. (2009). Confirmation and adequacy-for-purpose in climate modeling. Proceedings of the Aristotelian Society, Supplementary, 83, 233–249.CrossRefGoogle Scholar
  51. Reason, P., & Bradbury, H. (Eds.). (2008). Handbook of action research. Participative inquiry and practice (2nd ed.). London: Sage.Google Scholar
  52. Sargent, R. G. (2013). Verification and validation of simulation models. Journal of Simulation, 7, 12–24.CrossRefGoogle Scholar
  53. Scheele, B. (Ed.). (1992). Struktur-Lege-Verfahren als Dialog-Konsens-Methodik. Münster: Aschendorff.Google Scholar
  54. Scheele, B., & Groeben, N. (1986). Methodological aspects of illustrating the cognitive-reflexive function of aesthetic communication. Poetics, 15, 527–554.CrossRefGoogle Scholar
  55. Scheele, B., & Groeben, N. (2010). Dialog-Konsens-Methoden. In G. Mey & K. Mruck (Eds.), Handbuch qualitative Forschung in der Psychologie (pp. 506–523). Wiesbaden: VS.Google Scholar
  56. Scheurich, J. J. (1996). The masks of validity: A deconstructive investigation. International Journal of Qualitative Studies in Education, 9, 49–60.CrossRefGoogle Scholar
  57. Schlesinger, S., Crosbie, R. E., et al. (1979). Terminology for model credibility. Simulation, 32, 103–104.CrossRefGoogle Scholar
  58. Schubert, C. (2015). Situating technological and societal futures. Pragmatist engagements with computer simulations and social dynamics. Technology in Society, 40, 4–13.CrossRefGoogle Scholar
  59. Shotter, J. (2014). Social constructionism. In D. Coghlan & M. Brydon-Miller (Eds.), The SAGE encyclopedia of action research (Vol. 1 and 2, pp. 704–707). London: Sage.Google Scholar
  60. Simon, C., & Étienne, M. (2010). A companion modelling approach applied to forest management planning. Environmental Modelling and Software, 25, 1371–1384.CrossRefGoogle Scholar
  61. Steinberg, S. R. (2014). Critical constructivism. In D. Coghlan & M. Brydon-Miller (Eds.), The SAGE encyclopedia of action research (Vol. 1 and 2, pp. 203–206). London: Sage.Google Scholar
  62. Toulmin, S. (1950). An examination of reason in the place of ethics. Cambridge: Cambridge University Press.Google Scholar
  63. Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press.Google Scholar
  64. Voinov, A., & Bousquet, F. (2010). Modelling with stakeholders. Environmental Modelling and Software, 25, 1268–1281.CrossRefGoogle Scholar
  65. Wagner, Rudolph F. (2003). Clinical case formulation in the research program “subjective theories”. Patients with chronic diseases. European Journal of Psychological Assessment, 19, 185–194.CrossRefGoogle Scholar
  66. Weidemann, D. (2009). A cultural psychological approach to analyze intercultural learning. Potential and limits of the structure formation technique. Forum Qualitative Social Research, 10, 43.
  67. Worrapimphonga, K., Gajaseni, N., Le Page, C., & Bousquet, F. (2010). A companion modeling approach applied to fishery management. Environmental Modelling and Software, 25, 1334–1344.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute for SociologyFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany

Personalised recommendations