Computational Economics

, Volume 44, Issue 4, pp 507–536 | Cite as

Efficient Sampling and Meta-Modeling for Computational Economic Models

  • Isabelle Salle
  • Murat Yıldızoğlu


Extensive exploration of simulation models comes at a high computational cost, all the more when the model involves a lot of parameters. Economists usually rely on random explorations, such as Monte Carlo simulations, and basic econometric modeling to approximate the properties of computational models. This paper aims to provide guidelines for the use of a much more efficient method that combines a parsimonious sampling of the parameter space using a specific design of experiments (DoE), with a well-suited metamodeling method first developed in geostatistics: kriging. We illustrate these guidelines by following them in the analysis of two simple and well known economic models: Nelson and Winter’s industrial dynamics model, and Cournot oligopoly with learning firms. In each case, we show that our DoE experiments can catch the main effects of the parameters on the models’ dynamics with a much lower number of simulations than the Monte-Carlo sampling (e.g. 85 simulations instead of 2,000 in the first case). In the analysis of the second model, we also introduce supplementary numerical tools that may be combined with this method, for characterizing configurations complying with a specific criterion (social optimal, replication of stylized facts, etc.). Our appendix gives an example of the R-project code that can be used to apply this method on other models, in order to encourage other researchers to quickly test this approach on their models.


Computational economics Exploration of agent-based models  Design of experiments Meta-modeling 



We are grateful to two anonymous referees for their comments and suggestions, and to the participants of the Lipari summer school on “Data mining and modelling of complex techno-socio-economic systems” (July 2012, Italy) for useful comments and discussions. We are responsible for all remaining errors.

Supplementary material


  1. Besag, J., & Clifford, P. (1991). Sequential monte carlo p-values. Biometrika, 78(2), 301–304.CrossRefGoogle Scholar
  2. Booth, J., & Butler, R. (1999). An importance sampling algorithm for exact conditional tests in log-linear models. Biometrika, 86(2), 321–332.CrossRefGoogle Scholar
  3. Box, G., & Draper, N. (1987). Empirical model building and responses surfaces. New York: Wiley.Google Scholar
  4. Cary, N. (2010). JMP \({\textregistered }\) 9 modeling and multivariate methods. Cary: SAS Institute Inc.Google Scholar
  5. Cioppa, T., (2002). Efficient nearly orthogonal and space-filling experimental designs for high-dimensional complex models. Naval postgraduate school: Doctoral Dissertation in philosophy in operations research.Google Scholar
  6. Durrande, N., Ginsbourger, O., & Roustant, O. (2012). Additive covariance kernels for high-dimensional Gaussian process modeling. Annales de la Faculté des Sciences de Toulouse Tome, 21(3), 481–499.CrossRefGoogle Scholar
  7. Fang, K., Lin, D., Winker, P., & Zhang, Y. (2000). Uniform design: theory and application. Technometrics, 42(3), 237–248.CrossRefGoogle Scholar
  8. Fisher, R. A. (1935). The design of experiments (9th ed.). New York: Macmillan.Google Scholar
  9. Frey, C., & Patil, S. (2002). Identification and review of sensitivity analysis methods. Risk Analysis, 22(3), 553.CrossRefGoogle Scholar
  10. Goupy, J., & Creighton, L. (2007). Introduction to design of experiments with JMP examples (3rd ed.). Cary: SAS Institute Inc.Google Scholar
  11. Herbst, E., & Schorfheide, F. (2013). Sequential monte carlo sampling for DSGE models, working paper 19152. National Bureau of Economic Research.Google Scholar
  12. Iman, R., & Helton, J. (1988). An investigation of uncertainty and sensitivity analysis techniques for computer models. Risk Analysis, 8, 71–90.CrossRefGoogle Scholar
  13. Jeong, S., Murayama, M., & Yamamoto, K. (2005). Efficient optimization design method using kriging model. Journal of Aircraft, 42, 413–420.CrossRefGoogle Scholar
  14. Jourdan, A. (2005). Planification d’experiences numeriques. Revue MODULAD, 33, 63–73.Google Scholar
  15. Krige, D. G. (1951). A statistical approach to some basic mine valuation problems on the Witwatersrand. Journal of the Chemical, Metallurgical and Mining Society of South Africa, 52(6), 119–139.Google Scholar
  16. Masters, T. (1993). Practical neural network recipes in C++. New York: Academic Press.Google Scholar
  17. Matheron, G. (1963). Principles of geostatistics. Economic Geology, 58, 1246.CrossRefGoogle Scholar
  18. Mebane, W. J., & Sekhon, J. (2011). Genetic optimization using derivatives: the rgenoud package for R. Journal of Statistical Software, 42(11), 1–26.Google Scholar
  19. Miller, J., & Page, S. (2007). Complex adaptive systems. Princeton: Princeton University Press.Google Scholar
  20. Nelson, R. R., & Winter, S. G. (1978). Forces generating and limiting concentration under schumpeterian competition. Bell Journal of Economics, 9(2), 524–548.CrossRefGoogle Scholar
  21. Nelson, R. R., & Winter, S. G. (1982). The schumpeterian tradeoff revisited. American Economic Review, 72(1), 114–132.Google Scholar
  22. Oeffner, M. (2008). Agent-based Keynesian macroeconomics—an evolutionary model embedded in an agent-based computer simulation. Doctoral dissertation: Bayerische Julius - Maximilians Universitat, Wurzburg.Google Scholar
  23. R Development Core Team. (2013). R: A language and environment for statistical computing, R Foundation for statistical computing, Vienna. ISBN 3-900051-07-0.
  24. Roustant, O., Ginsbourger, D., & Deville, Y. (2010). DiceKriging, diceOptim: two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. Journal of Statistical Software, 55(2), 100.Google Scholar
  25. Sacks, J., Welch, W., Mitchell, T., & Wynn, H. (1989). Design and analysis of computer experiments. Statistical Science, 4(4), 409.CrossRefGoogle Scholar
  26. Salle, I., Sénégas, M., & Yıldızoğlu, M. (2012). How transparent should a Central Bank be? an ABM assessment. avril: mimeo, Bordeaux University.Google Scholar
  27. Saltelli, A., Tarantola, S., & Chan, K. (1999). A quantitative model-independent method for global sensitivity analysis of model output. Technometrics, 41(1), 39–56.CrossRefGoogle Scholar
  28. Sanchez, S. M., (2005). Work smarter, not harder: Guidelines for designing simulation experiments, In M. E. Kuhl, N. M. Steiger, F. B. Armstrong, & J. A. Joines (eds.) Proceedings of the 2005 Winter Simulation Conference. Software available at
  29. Silva, I., Assuncao, R., & Costa, M. (2009). Power of the sequential monte carlo test. Sequential Analysis, 28(2), 163–174.CrossRefGoogle Scholar
  30. Tesfatsion, L., & Judd, K. L. (Eds.). (2006). Handbook of computational economics. Agent-based computational economics (Vol. 2). Amsterdam: North-Holland.Google Scholar
  31. Vallée, T., & Yıldızoğlu, M. (2009). Convergence in the finite Cournot oligopoly with social and individual learning. Journal of Economic Behavior & Organization, 72(2), 670–690.CrossRefGoogle Scholar
  32. van Beers, W., & Kleijnen, J. (2004). Kriging interpolation in simulation: a survey. In R. G. Ingalls, M. D. Rossetti, J. S. Smith, & B. A. Peters (Eds.), Handbook of statistics. New York: Elsevier.Google Scholar
  33. Wang, G., & Shan, S. (2007). Review of metamodeling techniques in support of engineering design optimization. Journal of Mechanical Design, 129, 370.CrossRefGoogle Scholar
  34. Welch, W. J., Buck, R. J., Sacks, J., Wynn, H. P., Mitchell, T. J., & Morris, M. D. (1992). Screening, predicting, and computer experiments. Technometrics, 34(1), 15–25.CrossRefGoogle Scholar
  35. Ye, K. (1998). Orthogonal column latin hypercubes and their application in computer experiments. Journal of the American Statistical Association, 93(444), 1430–1439.CrossRefGoogle Scholar
  36. Yıldızoğlu, M. (2001). Connecting adaptive behaviour and expectations in models of innovation: The potential role of artificial neural networks. European Journal of Economic and Social Systems, 15(3), 51–65.Google Scholar
  37. Yıldızoğlu, M., Sénégas, M.-A., & Zumpe, M. (2012). Learning the optimal buffer-stock consumption rule of Carroll. Macroeconomic Dynamics, 5, 255.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.CeNDEF, Amsterdam School of EconomicsUniversity of AmsterdamAmsterdamThe Netherlands
  2. 2.Tinbergen InstituteAmsterdamThe Netherlands
  3. 3.GREThA (UMR CNRS 5113)Université de BordeauxPessacFrance

Personalised recommendations