Challenges to Simulation Validation in the Social Sciences. A Critical Rationalist Perspective

  • Michael MäsEmail author
Part of the Simulation Foundations, Methods and Applications book series (SFMA)


I reflect on challenges to the validation of theoretical models from the perspective of a critical rationalist seeking to develop true explanations of empirical phenomena. I illustrate my arguments with examples from the rich literature on social-influence models, a field that has profited from contributions from various disciplines such as physics, and mathematics. While this field is characterized by a large number of competing formal models, it has been criticized for having failed to generate reliable explanations and predictions, because of a lack of empirical research validating models. I list five challenges to model validation in the social sciences: First, social-scientific theories are based on many obscure concepts. Second, many social-scientific concepts are latent. Third, the representation of time is unclear in many models. Forth, in most social settings, various processes influence dynamics in parallel. Fifth, context dependencies limit the development of general models.


Model validation Social-influence Social networks 


  1. Abelson, R. P. (1964). Mathematical models of the distribution of attitudes under controversy. In N. Frederiksen & H. Gulliksen (Eds.), Contributions to mathematical psychology (pp. 142–160). New York: Rinehart Winston.Google Scholar
  2. Ahrweiler, P., & Gilbert, N. (2015). The quality of social simulation: An example from research policy modelling. In M. Janssen, M. Wimmer, & A. Deljoo (Eds.), Policy practice and digital science (pp. 35–55). Heidelberg/New York: Springer.Google Scholar
  3. Axelrod, R. (1997). The dissemination of culture-A model with local convergence and global polarization. Journal of Conflict Resolution, 41(2), 203–226.Google Scholar
  4. Baldassarri, D., & Bearman, P. (2007). Dynamics of political polarization. American Sociological Review, 72(5), 784–811.Google Scholar
  5. Bikhchandani, S., Hirshleifer, D., & Welch, I. (1992). A theory of fads, fashion, custom, and cultural-change as informational cascades. Journal of Political Economy, 100(5), 992–1026.Google Scholar
  6. Bonacich, P., & Philip, L. (2012). Introduction to mathematical sociology. Princeton and Oxford: Princeton University Press.zbMATHGoogle Scholar
  7. Boudon, R. (1981). The logic of social action. An introduction to sociological analysis. London: Routledge and Kegan Paul.Google Scholar
  8. Bourdieu, P. (1984). Distinction: A social critique of the judgment of taste. Cambridge, MA: Harvard University Press.Google Scholar
  9. Boxell, L., Gentzkow, M., & Shapiro, J. M. (2017). Greater internet use is not associated with faster growth in political polarization among US demographic groups. Proceedings of the National Academy of Sciences of the United States of America, 114(40), 10612–10617.Google Scholar
  10. Bramson, A., Grim, P., Singer, D. J., Fisher, S., Berger, W., Sack, G., & Flocken, C. (2016). Disambiguation of social polarization concepts and measures. The Journal of Mathematical Sociology.Google Scholar
  11. Brewer, M. B. (1991). The social self-On being the same and different at the same time. Personality and Social Psychology Bulletin, 17(5), 475–482.Google Scholar
  12. Brousmiche, K.-L., Kant, J.-D., Sabouret, N., & Prenot-Guinard, F. (2016). From beliefs to attitudes: Polias, a model of attitude dynamics based on cognitive modeling and field data. Journal of Artificial Societies and Social Simulation, 19(4).Google Scholar
  13. Byrne, D. (1971). The attraction paradigm. New York, London: Academic Press.Google Scholar
  14. Castellano, C., Fortunato, S., & Loreto, V. (2009). Statistical physics of social dynamics. Reviews of Modern Physics, 81(2), 591–646.Google Scholar
  15. Chattoe-Brown, E. (2014). Using agent based modelling to integrate data on attitude change. Sociological Research Online, 19(1).Google Scholar
  16. Clemm von Hohenberg, B., Mäs, M., & Pradelski, B. S. R. (2017). Micro influence and macro dynamics of opinion formation. SSRN.
  17. Conte, R., Gilbert, N., Bonelli, G., Cioffi-Revilla, C., Deffuant, G., Kertesz, J., et al. (2012). Manifesto of computational social science. The European Physical Journal Special Topics, 214, 325–346.Google Scholar
  18. David, N. (2009). Validation and verification in social simulation: Patterns and clarification of terminology. In F. Squazzoni (Ed.), EPOS 2006: Epistemological Aspects of Computer Simulation in the Social Sciences (pp. 117–129). Berlin: Springer.Google Scholar
  19. Deffuant, G., Huet, S., & Amblard, F. (2005). An individual-based model of innovation diffusion mixing social value and individual benefit. American Journal of Sociology, 110(4), 1041–1069.Google Scholar
  20. DiMaggio, P., Evans, J., & Bryson, B. (1996). Have Americans’ social attitudes become more polarized? American Journal of Sociology, 102(3), 690–755.Google Scholar
  21. Elias, N. (1969). The civilizing process. In The history of manners (Vol. I). Oxford: Blackwell.Google Scholar
  22. Esteban, J. M., & Ray, D. (1994). On the measurement of polarization. Econometrica, 62(4), 819–851.zbMATHGoogle Scholar
  23. Everett, J. A. C. (2013). The 12 item social and economic conservatism Scale (SECS). PLoS One, 8(12), e82131. (Edited by Pete Roma. Public Library of Science).Google Scholar
  24. Flache, A., & Mäs, M. (2008a). Why do faultlines matter? A computational model of how strong demographic faultlines undermine team cohesion. Simulation Modelling Practice and Theory, 16(2), 175–191.Google Scholar
  25. Flache, A., & Mäs, M. (2008b). How to get the timing right. A computational model of the effects of the timing of contacts on team cohesion in demographically diverse teams. Computational and Mathematical Organization Theory, 14(1), 23–51.zbMATHGoogle Scholar
  26. Flache, A., & Macy, M. W. (2011a). Small worlds and cultural polarization. The Journal of Mathematical Sociology, 35(1–3), 146–176.MathSciNetGoogle Scholar
  27. Flache, A., & Macy, M. W. (2011b). Local convergence and global diversity: From interpersonal to social influence. Journal of Conflict Resolution, 55(6), 970–995.Google Scholar
  28. Flache, A., Mäs, M., Feliciani, T., Chattoe-Brown, E., Deffuant, G., Huet, S., & Lorenz, J. (2017). Models of social influence: Towards the next frontiers. Journal of Artificial Societies and Social Simulation, 20(4).Google Scholar
  29. Friedkin, N. E., & Johnsen, E. C. (2011). Social influence network theory. New York: Cambridge University Press.zbMATHGoogle Scholar
  30. Garrison, L. A., & Babcock, D. S. (2009). Alcohol consumption among college students: An agent-based computational simulation. Complexity, 14(6), 35–44 (Wiley Subscription Services, Inc., A Wiley Company).Google Scholar
  31. Gilbert, N., & Troitzsch, K. G. (1999). Simulation for the social scientist. Buckingham Philadelphia: Open University Press.Google Scholar
  32. Gilbert, N., & Ahrweiler, P. (2005). Caffè Nero: The evaluation of social simulation. Journal of Artificial Societies and Social Simulation, 8(4).Google Scholar
  33. Golder, S. A., & Macy, M. W. (2014). Digital footprints: Opportunities and challenges for online social research. Annual Review of Sociology, 40(1), 129–152.Google Scholar
  34. Grow, A., & Flache, A. (2011). How attitude certainty tempers the effects of faultlines in demographically diverse teams. Computational and Mathematical Organization Theory, 17(2), 196–224.zbMATHGoogle Scholar
  35. Hedström, P., & Udehn, L. (2009). Analytical sociology and theories of the middle range. In P. Hedström, & P. Bearman (Eds.), The oxford handbook (pp. 25–47). Oxford University Press.Google Scholar
  36. Hedström, P., & Ylikoski, P. (2010). Causal mechanisms in the social sciences. Annual Review of Sociology, 36(1), 49–67.Google Scholar
  37. Hegselmann, R., & Krause, U. (2002). Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of Artificial Societies and Social Simulation, 5(3).Google Scholar
  38. Hegselmann, R., Schelling, T. C., & Sakoda, J. M. (2017). The intellectual, technical, and social history of a model. Journal of Artificial Societies and Social Simulation (JASSS), 20(3), 15.Google Scholar
  39. Hempel, C. G., & Oppenheim, P. (1948). Studies in the logic of explanation. Philosophy of Science, 15, 135–175.Google Scholar
  40. Hogg, M. A., Turner, J. C., & Davidson, B. (1990). Polarized norms and social frames of reference: A test of the self-categorization theory of group polarization. Basic and Applied Social Psychology, 11(1), 77–100.Google Scholar
  41. Huckfeldt, R., Johnson, P. E., & Sprague, J. (2004). Political disagreement. The survival of diverse opinions within communication networks. Cambridge University Press.Google Scholar
  42. Iyengar, S., & Hahn, K. S. (2009). Red media, blue media: Evidence of ideological selectivity in media use. Journal of Communication, 59(1), 19–39.Google Scholar
  43. Keijzer, M. A., Mäs, M., & Flache, A. (2018). Online social networks foster cultural isolation. Groningen.Google Scholar
  44. Lau, D. C., & Keith Murnighan, J. (2005). Interactions within groups and subgroups: The effects of demographic faultlines. Academy of Management Journal, 48(4), 645–659.Google Scholar
  45. Lazarsfeld, P. F., & Merton, R. K. (1954). Friendship and social process: A substantive and methodological analysis. In M. Berger, T. Abel, & C. H. Page (Eds.), Freedom and Control in Modern Society (pp. 18–66). New York, Toronto, London: Van Nostrand.Google Scholar
  46. Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A. L., Brewer, D., et al. (2009). Computational social science. Science, 323(5915), 721–723.Google Scholar
  47. Lewis, K., Gonzalez, M., & Kaufman, J. (2012). Social selection and peer influence in an online social network. Proceedings of the National Academy of Sciences of the United States of America, 109(1), 68–72.Google Scholar
  48. Liu, C. C., & Srivastava, S. B. (2015). Pulling closer and moving apart: Interaction, identity, and influence in the U.S. Senate, 1973 to 2009. American Sociological Review, 80(1), 192–217.Google Scholar
  49. Loehlin, J. C., & Alexander Beaujean, A. (2016). Latent variable models: An introduction to factor, path, and structural equation analysis (5th ed.). New York and London: Taylor & Francis.Google Scholar
  50. Lorenz, J. (2005). A stabilization theorem for dynamics of continuous opinions. Physica a-Statistical Mechanics and Its Applications, 355(1), 217–223.MathSciNetGoogle Scholar
  51. Lorenz, J. (2007). Continuous opinion dynamics under bounded confidence: A survey. International Journal of Modern Physics C, 18(12), 1819–1838.zbMATHGoogle Scholar
  52. Lorenz, J. (2017). Modeling the evolution of ideological landscapes through opinion dynamics. In W. Jager, R. Verbrugge, A. Flache, G. de Roo, L. Hoogduin, & C. Hemelrijk (Eds.), Advances in social simulation 2015 (pp. 255–266). Springer International Publishing.Google Scholar
  53. Macy, M. W., Kitts, J., Flache, A., & Benard, S. (2003). Polarization and dynamic networks. A hopfield model of emergent structure. In R. Breiger, K. Carley, & P. Pattison (Eds.), Dynamic Social Network Modeling and Analysis: Workshop Summary and Papers (pp. 162–173). Washington, DC: The National Academies Press.Google Scholar
  54. Mark, N. P. (2003). Culture and competition: Homophily and distancing explanations for cultural niches. American Sociological Review, 68(3), 319–345.Google Scholar
  55. Marsden, P. V., & Friedkin, N. E. (1993). Network studies of social-influence. Sociological Methods & Research, 22(1), 127–151.Google Scholar
  56. Mäs, M., & Flache, A. (2013). Differentiation without distancing. Explaining opinion bi-polarization without assuming negative influence. Plos One 8(11).Google Scholar
  57. Mäs, M., & Bischofberger, L. (2015). Will the personalization of online social networks foster opinion polarization. SSRN Electronic Journal.
  58. Mäs, M., Flache, A., & Helbing, D. (2010). Individualization as driving force of clustering phenomena in humans. PLoS Computational Biology, 6(10), e1000959.Google Scholar
  59. Mäs, M., Flache, A., Takács, K., & Jehn, K. A. (2013). In the short term we divide, in the long term we unite: Demographic crisscrossing and the effects of faultlines on subgroup polarization. Organization Science, 24(3), 716–736.Google Scholar
  60. Mäs, M., Flache, A., & Kitts, J. A. (2014). cultural integration and differentiation in groups and organizations. In V. Dignum & F. Dignum (Eds.), Perspectives on culture and agent-based simulations. Cham: Springer.Google Scholar
  61. Mason, W. A., Conrey, F. R., & Smith, E. R. (2007). Situating social influence processes: Dynamic, multidirectional flows of influence within social networks. Personality and Social Psychology Review, 11(3), 279–300.Google Scholar
  62. McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27, 415–444.Google Scholar
  63. Merton, R. K. (1957). Social theory and social structure. Ill: Glencoe.Google Scholar
  64. Nagel, E. (1979). The structure of science: Problems in the logic of scientific explanation. Indianapolis, Cambridge: Hackett.Google Scholar
  65. Opp, K. D. (2009). Theories of political protest and social movements: A multidisciplinary introduction, critique, and synthesis. New York: Routledge.Google Scholar
  66. Opp, K. D., & Gern, C. (1993). Dissident groups, personal networks, and spontaneous cooperation-The East-German revolution of 1989. American Sociological Review, 58(5), 659–680.Google Scholar
  67. Popper, K. R. (1959). The logic of scientific discovery. London: Hutchinson.zbMATHGoogle Scholar
  68. Postmes, T., Spears, R., Sakhel, K., & De Groot, D. (2001). Social influence in computer-mediated communication: The effects of anonymity on group behavior. Personality and Social Psychology Bulletin, 27(10), 1243–1254.Google Scholar
  69. Rosenbaum, M. E. (1986). The repulsion hypothesis: On the nondevelopment of relationships. Journal of Personality and Social Psychology, 51(6), 1156–1166.Google Scholar
  70. Sakoda, J. M. (1971). The checkerboard model of social interaction. The Journal of Mathematical Sociology, 1(1), 119–132 (Taylor & Francis Group).Google Scholar
  71. Salganik, M. J. (2017). Bit by bit: Social research in the digital age.Google Scholar
  72. Salzarulo, L. (2006). A continuous opinion dynamics model based on the principle of meta-contrast. Journal of Artificial Societies and Social Simulation, 9(1).Google Scholar
  73. Schelling, T. C. (1971). Dynamic models of segregation. Journal of Mathematical Sociology, 1, 143–186.zbMATHGoogle Scholar
  74. Slater, M. D. (2007). Reinforcing spirals: The mutual influence of media selectivity and media effects and their impact on individual behavior and social identity. Communication Theory, 17(3), 281–303.Google Scholar
  75. Steglich, C., Snijders, T. A. B., & Pearson, M. (2010). Dynamic networks and behavior: Separating selection from influence. Sociological Methodology, 40(1), 329–393.Google Scholar
  76. Tajfel, H. (1978). Social categorization, social identity and social comparison. In Differentiation between social groups: Studies in the social psychology of intergroup relations.Google Scholar
  77. Tajfel, H., & Turner, J. C. (1986). The social identity theory of intergroup behavior. In S. Worchel, & W. G. Austin (Eds.), Psychology of intergroup relations (pp. 7–24). Chicago: Nelson-Hall Publishers.Google Scholar
  78. Takács, K., Flache, A., & Mäs, M. (2016). Discrepancy and disliking do not induce negative opinion shifts. PLoS One, 11(6), e0157948.Google Scholar
  79. Turner, J. H. (1974). The structure of sociological theory. Chicago, Illinois: The Dorsey Press.Google Scholar
  80. Turner, J. H. (1995). Macrodynamics. Toward a theory on the organization of human populations. New Brunswick, N.J: Rutgers University Press.Google Scholar
  81. Wimmer, A., & Lewis, K. (2010). Beyond and below racial homophily: ERG models of a friendship network documented on Facebook. American Journal of Sociology, 116(2), 583–642.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Sociology/ICSUniversity of GroningenGroningenThe Netherlands

Personalised recommendations