Empirical validation of simulated models through the GSL-div: an illustrative application

  • Francesco LampertiEmail author
Regular Article


A major concern about the use of simulation models regards their relationship with the empirical data. The identification of a suitable indicator quantifying the distance between the model and the data would help and guide model selection and output validation. This paper proposes the use of a new criterion, called GSL-div and developed in Lamperti (Econ Stat, 2017., to assess the degree of similarity between the dynamics observed in the data and those generated by the numerical simulation of models. As an illustrative application, this approach is used to distinguish between different versions of the well known asset pricing model with heterogeneous beliefs proposed in Brock and Hommes (J Econ Dyn Control 22(8–9):1235–1274, 1998. Once the discrimination ability of the GSL-div is proved, model’s dynamics are directly compared with actual data coming from two major stock market indexes (EuroSTOXX 50 for Europe and CSI 300 for China). Results show that the model, once calibrated, is fairly able to track the evolution of both the two indexes, even though a better fit is reported for the Chinese stock market. However, I also find that many different combinations of traders’ behavioural rules are compatible with the same observed dynamics. Within this heterogeneity, an emerging common trait is found: to be empirically valid, the model has to account for a strong trend following component, which might either come from a unique trend type that heavily extrapolates information from past observations or the combinations of different types with milder, or even opposite, attitudes towards the trend.


Simulated models Empirical validation Model selection GSL-div 



The author would like to thank Mattia Guerini, Napoletano, Andrea Roventini, all the participants to the 2016 CEF conference in Bordeaux, two anonymous referees and the editorial board for valuable comments and suggestions. All the shortcomings are the author’s.


  1. Alfarano S, Lux T, Wagner F (2005) Estimation of agent-based models: the case of an asymmetric herding model. Comput Econ 26(1):19–49CrossRefGoogle Scholar
  2. Alfarano S, Lux T, Wagner F (2006) Estimation of a simple agent-based model of financial markets: an application to australian stock and foreign exchange data. Phys A Stat Mech Appl 370(1):38–42CrossRefGoogle Scholar
  3. Amilon H (2008) Estimation of an adaptive stock market model with heterogeneous agents. J Empir Finance 15(2):342–362. CrossRefGoogle Scholar
  4. Banerjee AV (1992) A simple model of herd behavior. Q J Econ 107(3):797–817CrossRefGoogle Scholar
  5. Barde S (2016) Direct comparison of agent-based models of herding in financial markets. J Econ Dyn Control 73:329–353. CrossRefGoogle Scholar
  6. Barde S (2016b) A practical, accurate, information criterion for Nth order markov processes. Comput Econ 50:281–324CrossRefGoogle Scholar
  7. Bianchi C, Cirillo P, Gallegati M, Vagliasindi PA (2007) Validating and calibrating agent-based models: a case study. Comput Econ 30(3):245–264CrossRefGoogle Scholar
  8. Bianchi C, Cirillo P, Gallegati M, Vagliasindi PA (2008) Validation in agent-based models: An investigation on the cats model. J Econ Behav Organ 67(3):947–964CrossRefGoogle Scholar
  9. Boswijk HP, Hommes CH, Manzan S (2007) Behavioral heterogeneity in stock prices. J Econ Dyn Control 31(6):1938–1970. CrossRefGoogle Scholar
  10. Brock WA, Hommes CH (1997) A rational route to randomness. Econometrica 65(5):1059–1095CrossRefGoogle Scholar
  11. Brock WA, Hommes CH (1998) Heterogeneous beliefs and routes to chaos in a simple asset pricing model. J Econ Dyn Control 22(8–9):1235–1274. CrossRefGoogle Scholar
  12. Canova F, Sala L (2009) Back to square one: identification issues in dsge models. J Monet Econ 56(4):431–449CrossRefGoogle Scholar
  13. Chiarella C, Iori G, Perelló J (2009) The impact of heterogeneous trading rules on the limit order book and order flows. J Econ Dyn Control 33(3):525–537CrossRefGoogle Scholar
  14. Dawid H, Fagiolo G (2008) Agent-based models for economic policy design: introduction to the special issue. J Econ Behav Organ 67(2):351–354CrossRefGoogle Scholar
  15. Dosi G, Fagiolo G, Roventini A (2010) Schumpeter meeting keynes: A policy-friendly model of endogenous growth and business cycles. J Econ Dyn Control 34(9):1748–1767CrossRefGoogle Scholar
  16. Dosi G, Fagiolo G, Napoletano M, Roventini A (2013) Income distribution, credit and fiscal policies in an agent-based keynesian model. J Econ Dyn Control 37(8):1598–1625CrossRefGoogle Scholar
  17. Dosi G, Fagiolo G, Napoletano M, Roventini A, Treibich T (2015) Fiscal and monetary policies in complex evolving economies. J Econ Dyn Control 52:166–189. CrossRefGoogle Scholar
  18. Fabretti A (2012) On the problem of calibrating an agent based model for financial markets. J Econ Interact Coord 8(2):277–293. CrossRefGoogle Scholar
  19. Fagiolo G, Guerini M, Lamperti F, Moneta A, Roventini A (2017) Validation of agent-based models in economics and finance. LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant’Anna School of Advanced Studies, Pisa, ItalyGoogle Scholar
  20. Franke R (2016) Competitive moment matching of a new-keynesian and an old-keynesian model. J Econ Interact Coord. Google Scholar
  21. Franke R, Westerhoff F (2011) Estimation of a structural stochastic volatility model of asset pricing. Comput Econ 38(1):53–83. CrossRefGoogle Scholar
  22. Franke R, Westerhoff F (2012) Structural stochastic volatility in asset pricing dynamics: estimation and model contest. J Econ Dyn Control 36(8):1193–1211. CrossRefGoogle Scholar
  23. Franke R, Westerhoff F (2016) Why a simple herding model may generate the stylized facts of daily returns: explanation and estimation. J Econ Interact Coord 11(1):1–34. CrossRefGoogle Scholar
  24. Gallegati M, Richiardi M (2009) Agent based modelling in economics and complexity. In: Meyer RA (ed) Encyclopedia of complexity and sistem science. Springer, New YorkGoogle Scholar
  25. Giannone D, Reichlin L, Sala L (2006) Vars, common factors and the empirical validation of equilibrium business cycle models. J Econ 132(1):257–279CrossRefGoogle Scholar
  26. Gilli M, Winker P (2003) A global optimization heuristic for estimating agent based models. Comput Stat Data Anal 42(3):299–312CrossRefGoogle Scholar
  27. Gourieoux C, Monfort A (1997) Simulation based econometric methods. Oxford University Press, OxfordCrossRefGoogle Scholar
  28. Grazzini J (2012) Analysis of the emergent properties: Stationarity and ergodicity. J Artif Soc Soc Simul 15(2):7CrossRefGoogle Scholar
  29. Grazzini J, Richiardi M (2015) Estimation of ergodic agent-based models by simulated minimum distance. J Econ Dyn Control 51:148–165. CrossRefGoogle Scholar
  30. Grazzini J, Richiardi MG, Tsionas M (2017) Bayesian estimation of agent-based models. J Econ Dyn Control 77:26–47. CrossRefGoogle Scholar
  31. Gualdi S, Tarzia M, Zamponi F, Bouchaud JP (2015) Tipping points in macroeconomic agent-based models. J Econ Dyn Control 50:29–61. CrossRefGoogle Scholar
  32. Guerini M, Moneta A (2016) A method for agent-based model validation. LEM Papers Series 2016/16, Laboratory of Economics and Management (LEM), Sant’Anna School of Advanced Studies, Pisa, ItalyGoogle Scholar
  33. Jacob Leal S, Napoletano M, Roventini A, Fagiolo G (2015) Rock around the clock: an agent-based model of low- and high-frequency trading. J Evol Econ 26(1):49–76. CrossRefGoogle Scholar
  34. Kukacka J, Barunik J (2016) Simulated ML estimation of financial agent-based models. Working papers IES 2016/07, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies.
  35. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22:49–86CrossRefGoogle Scholar
  36. Lamperti F (2017) An information theoretic criterion for empirical validation of simulation models. Econ Stat. Google Scholar
  37. Lamperti F, Roventini A, Sani A (2017) Agent based model calibration using machine learning surrogates. LEM Papers Series forthcoming, Laboratory of Economics and Management (LEM), Sant’Anna School of Advanced Studies, Pisa, ItalyGoogle Scholar
  38. LeBaron B, Winker P (2008) Introduction to the special issue on agent-based models for economic policy advice. J Econ Stat 228(2+3):141–148Google Scholar
  39. Leombruni R, Richiardi M, Saam NJ, Sonnessa M (2006) A common protocol for agent-based social simulation. J Artif Soc Soc Simul 9(1):15Google Scholar
  40. Lin J (1991) Divergence measures based on the shannon entropy. IEEE Trans Inf Theory 37:145–151CrossRefGoogle Scholar
  41. Lux T, Marchesi M (2000) Volatility clustering in financial markets: a microsimulation of interacting agents. Int J Theor Appl Finance 3(04):675–702CrossRefGoogle Scholar
  42. Lux T, Zwinkels RC (2017) Empirical validation of agent-based models.
  43. Manson S (2002) Validation and verification of multi-agent systems. In: Janssen MA (ed) Complexity and ecosystem management. Edward Elgar, CheltenhamGoogle Scholar
  44. Marks R (2007) Validating simulation models: A general framework and four applied examples. Comput Econ 30(3):265–290. CrossRefGoogle Scholar
  45. Marks R (2013) Validation and model selection: three similarity measures compared. Complex Econ 2(1):41–61CrossRefGoogle Scholar
  46. Paccagnini A (2009) Model validation in the DSGE approach: a survey, working paper, MimeoGoogle Scholar
  47. Pellizzari P, Forno AD (2006) A comparison of different trading protocols in an agent-based market. J Econ Interact Coord 2(1):27–43. CrossRefGoogle Scholar
  48. Pyka A, Fagiolo G (2007) Agent-based modelling: a methodology for neo-schumpeterian economics. In: Pyka H (ed) Elgar companion to neo-schumpeterian economics. Edward Elgar, CheltenhamGoogle Scholar
  49. Recchioni MC, Tedeschi G, Gallegati M (2015) A calibration procedure for analyzing stock price dynamics in an agent-based framework. J Econ Dyn Control 60:1–25. CrossRefGoogle Scholar
  50. Rosen R (1985) Anticipatory systems: philosophical, mathematical, and methodological foundations. Pergamon, OxfordGoogle Scholar
  51. Shannon CE (1948) A mathematical theory of communication. Bell Syst Techn J 27:379–423CrossRefGoogle Scholar
  52. Teräsvirta T (1994) Specification, estimation, and evaluation of smooth transition autoregressive models. J Am Stat Assoc 89(425):208–218. Google Scholar
  53. Windrum P, Fagiolo G, Moneta A (2007) Empirical validation of agent-based models: Alternatives and prospects. J Artif Soc Soc Simul 10(2):8Google Scholar
  54. Winker P, Gilli M, Jeleskovic V (2007) An objective function for simulation based inference on exchange rate data. J Econ Interact Coord 2(2):125–145. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Institute of Economics and LEMScuola Superiore Sant’AnnaPisaItaly

Personalised recommendations