Regression Metamodels for Sensitivity Analysis in Agent-Based Computational Demography

  • André GrowEmail author
Part of the The Springer Series on Demographic Methods and Population Analysis book series (PSDE, volume 41)


Agent-based computational simulation models can be complex and this can make it difficult to understand which processes are driving model behaviour. Sensitivity analysis by means of metamodels can greatly facilitate the understanding of the behaviour of complex simulation models. However, this method has so far largely been neglected in agent-based computational demography, with few exceptions. In this chapter, I illustrate how sensitivity analysis can be conducted by means of regression metamodels. I argue that this type of metamodel is particularly attractive for use in agent-based computational demography due to the fact that most demographers have at least a basic understanding of multiple regression. This makes this type of metamodel highly accessible and easy to communicate. After describing the basics of regression metamodels, I illustrate their use by conducting a sensitivity analysis of an agent-based model of educational assortative mating that is based on data on the structure of Belgian marriage markets between 1921 and 2012. I close the chapter with a discussion of the benefits and limitations of regression metamodels and point the reader to further readings on this topic.


Parameter Combination Male Agent Experimental Region Marriage Market Female Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP/2007–2013)/ERC Grant Agreement no. 312290 for the GENDERBALL project. I thank Jan Van Bavel, Hideko Matsuo, and two anonymous reviewers for helpful comments on earlier versions of this chapter.

Supplementary material (73 mb)
Appendices (ZIP 75031 kb).


  1. Antony, J. (2003). Design of experiments for engineers and scientists. Oxford: Butterworth-Heinmann.Google Scholar
  2. Bijak, J., Hilton, J., Silverman, E., & Cao, V. D. (2013). Reforging the wedding ring: Exploring a semi-artificial model of population for the United Kingdom with Gaussian process emulators. Demographic Research, 29, 729–766.CrossRefGoogle Scholar
  3. Billari, F. C., Ongaro, F., & Prskawetz, A. (2003). Introduction: Agent-based computational demography. In F. C. Billari & A. Prskawetz (Eds.), Agent-based computational demography: Using simulation to improve our understanding of demographic behaviour (pp. 1–17). Heidelberg: Physica Verlag.Google Scholar
  4. Billari, F. C., Prskawetz, A., Aparicio Diaz, B., & Fent, T. (2007). The “Wedding-Ring”: An agent-based marriage model based on social interaction. Demographic Research, 17, 59–82.Google Scholar
  5. Bonabeau, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences of the United States of America, 99(Suppl. 3), 7280–7287.CrossRefGoogle Scholar
  6. Coutts, S. R., & Yokomizo, H. (2014). Meta-models as a straightforward approach to the sensitivity analysis of complex models. Population Ecology, 56(1), 7–19.CrossRefGoogle Scholar
  7. De Hauw, Y., Grow, A., Van Bavel, J. (2015). The shifting gender balance in higher education and assortative mating in Europe. Paper presented at the 2015 Annual Meeting of the Population Association of America, San Diego, USA.Google Scholar
  8. De Mulder, W., Grow, A., Molenberghs, G., & Verbeke, G. (2015). Application of statistical emulation to an agent-based model: Assortative mating and the reversal of gender inequality in education in Belgium. In H. Friedl & H. Wagner (Eds.), Proceedings of the 30th international workshop on statistical modelling, vol. 1 (pp. 139–144). Linz.Google Scholar
  9. England, P., & McClintock, E. A. (2009). The gendered double standard of aging in US marriage markets. Population and Development Review, 35(4), 797–816.CrossRefGoogle Scholar
  10. Epstein, J. M. (1999). Agent-based computational models and generative social science. Complexity, 4(5), 41–60.CrossRefGoogle Scholar
  11. Esteve, A., García-Román, J., & Permanyer, I. (2012). The gender-gap reversal in education and its effect on union formation: The end of hypergamy? Population and Development Review, 38(3), 535–546.CrossRefGoogle Scholar
  12. Fox, J. (2008). Applied regression analysis and generalized linear models (2nd ed.). Thousand Oaks: Sage.Google Scholar
  13. Grow, A., & Van Bavel, J. (2015). Assortative mating and the reversal of gender inequality in education in Europe: An agent-based model. PLoS ONE, 10(6), e0127806.CrossRefGoogle Scholar
  14. Kalmijn, M. (1994). Assortative mating by cultural and economic occupational status. American Journal of Sociology, 100(2), 422–452.CrossRefGoogle Scholar
  15. KC, S., Barakat, B., Goujon, A., Skirbekk, V., Sanderson, W. C., & Lutz, W. (2010). Projection of populations by level of educational attainment, age, and sex for 120 countries for 2005–2050. Demographic Research, 22, 383–472.CrossRefGoogle Scholar
  16. Kleijnen, J. P. C. (1992). Sensitivity analysis of simulation experiments: Regression analysis and statistical design. Mathematics and Computers in Simulation, 34(3–4), 297–315.CrossRefGoogle Scholar
  17. Kleijnen, J. P. C. (2005). An overview of the design and analysis of simulation experiments for sensitivity analysis. European Journal of Operational Research, 164(2), 287–300.CrossRefGoogle Scholar
  18. Kleijnen, J. P. C. (2008). Design and analysis of simulation experiments. Tilburg: Springer.Google Scholar
  19. Kleijnen, J. P. C., & Sargent, R. G. (2000). A methodology for fitting and validating metamodels in simulation. European Journal of Operational Research, 120(1), 14–29.CrossRefGoogle Scholar
  20. Kleijnen, J. P. C., van Ham, G., & Rotmans, J. (1992). Techniques for sensitivity analysis of simulation models: A case study of the CO2 greenhouse effect. Simulation, 58(6), 410–417.CrossRefGoogle Scholar
  21. Kleijnen, J. P. C., Sanchez, S. M., Lucas, T. W., & Cioppa, T. M. (2005). A user’s guide to the brave new world of designing simulation experiments. INFORMS Journal on Computing, 17(3), 263–289.CrossRefGoogle Scholar
  22. Law, A. M., & Kelton, D. W. (2000). Simulation modeling and analysis (3rd ed.). Boston: McGraw-Hill.Google Scholar
  23. Lenth, R. V. (2009). Response-surface methods in R, using rsm. Journal of Statistical Software, 32(7), 1–17.CrossRefGoogle Scholar
  24. Ligmann-Zielinska, A., Kramer, D. B., Cheruvelil, K. S., & Soranno, P. A. (2014). Using uncertainty and sensitivity analysis in sociological agent-based models to improve their analytical performance and policy relevance. PLoS ONE, 9(10), e109779.CrossRefGoogle Scholar
  25. Lorscheid, I., Heine, B.-O., & Meyer, M. (2012). Opening the “Black Box” of simulations: Increased transparency and effective communication through the systematic design of experiments. Computational and Mathematical Organization Theory, 18(1), 22–62.CrossRefGoogle Scholar
  26. Lutz, W., Goujon, A., KC, S., & Sanderson, W. C. (2007). Reconstruction of populations by age, sex and level of educational attainment for 120 countries for 1970–2000. Vienna Yearbook of Population Research, 5, 193–235.CrossRefGoogle Scholar
  27. Macy, M. W., & Flache, A. (2009). Social dynamics from the bottom up: Agent-based models of social interaction. In P. Hedström & P. Bearman (Eds.), The oxford handbook of analytical sociology (pp. 245–268). Oxford: Oxford University Press.Google Scholar
  28. Macy, M. W., & Willer, R. (2002). From factors to actors: Computational sociology and agent-based modeling. Annual Review of Sociology, 28, 143–166.CrossRefGoogle Scholar
  29. McCarthy, M. A., Burgman, M. A., & Ferson, S. (1995). Sensitivity analysis for models of population viability. Biological Conservation, 73(2), 93–100.CrossRefGoogle Scholar
  30. Myers, R. H., & Montgomery, D. C. (1995). Response surface methodology: Process and product optimization using designed experiments. New York: Wiley.Google Scholar
  31. Oakley, J., & O’Hagan, A. (2002). Bayesian inference for the uncertainty distribution of computer model outputs. Biometrika, 89(4), 769–784.CrossRefGoogle Scholar
  32. Oppenheimer, V. K. (1988). A theory of marriage timing. American Journal of Sociology, 94(3), 563–591.CrossRefGoogle Scholar
  33. R Core Team. (2014). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
  34. Rao, C. R. (1959). Some problems involving linear hypothesis in multivariate analysis. Biometrika, 46(1/2), 49–58.CrossRefGoogle Scholar
  35. Schelling, T. C. (1971). Dynamic models of segregation. Journal of Mathematical Sociology, 1(2), 143–186.CrossRefGoogle Scholar
  36. Schwartz, C. R. (2013). Trends and variation in assortative mating: Causes and consequences. Annual Review of Sociology, 39, 451–470.CrossRefGoogle Scholar
  37. Silverman, E., Bijak, J., Hilton, J., Cao, V. D., & Noble, J. (2013). When demography met social simulation: A tale of two modelling approaches. Journal of Artificial Societies and Social Simulation, 16(4), 9.CrossRefGoogle Scholar
  38. Simão, J., & Todd, P. M. (2002). Modeling mate choice in monogamous mating systems with courtship. Adaptive Behavior, 10(2), 113–136.CrossRefGoogle Scholar
  39. Simpson, T. W., Peplinski, J. D., Koch, P. N., & Allen, J. K. (2001). Metamodels for computer-based engineering design: Survey and recommendations. Engineering with Computers, 17(2), 129–150.CrossRefGoogle Scholar
  40. Smaldino, P. E., & Schank, J. C. (2011). Human mate choice is a complex system. Complexity, 17(5), 11–22.CrossRefGoogle Scholar
  41. Thiele, J. C., Kurth, W., & Grimm, V. (2014). Facilitating parameter estimation and sensitivity analysis of agent-based models: A cookbook using NetLogo and R. Journal of Artificial Societies and Social Simulation, 17(3), 11.CrossRefGoogle Scholar
  42. Todd, P. M., Hills, T., & Hendrickson, A. (2013). Modeling reproductive decisions with simple heuristics. Demographic Research, 29, 641–662.CrossRefGoogle Scholar
  43. Tunali, S., & Batzman, I. (2000). Dealing with the least squares regression assumptions in simulation metamodeling. Computers and Industrial Engineering, 38(2), 307–320.CrossRefGoogle Scholar
  44. Van Bavel, J. (2012). The reversal of gender inequality in education, union formation and fertility in Europe. Vienna Yearbook of Population Research, 10, 127–154. CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Centre for Sociological ResearchUniversity of Leuven (KU Leuven)LeuvenBelgium

Personalised recommendations