Computational Economics

, Volume 30, Issue 3, pp 195–226 | Cite as

A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems

  • Giorgio FagioloEmail author
  • Alessio Moneta
  • Paul Windrum


This paper addresses the methodological problems of empirical validation in agent-based (AB) models in economics and how these are currently being tackled. We first identify a set of issues that are common to all modelers engaged in empirical validation. We then propose a novel taxonomy, which captures the relevant dimensions along which AB economics models differ. We argue that these dimensions affect the way in which empirical validation is carried out by AB modelers and we critically discuss the main alternative approaches to empirical validation being developed in AB economics. We conclude by focusing on a set of (as yet) unresolved issues for empirical validation that require future research.


Methodology Agent-based computational economics Simulation models Empirical validation Calibration History-friendly modeling 


B41 B52 C63 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Arthur W.B. (1988). Competing technologies: An overview. In: Dosi G., Freeman C., Nelson R., Silverberg G., Soete L. (eds) Technical change and economic theory. London, Pinter, pp. 590–607Google Scholar
  2. Arthur W.B. (1994). Increasing returns and path-dependency in economics. Ann Arbor, University of Michigan PressGoogle Scholar
  3. Barreteau, O. (2003). Our companion modeling approach. Journal of Artificial Societies and Social Simulation, 6, 1.Google Scholar
  4. Brenner, T. (2004). Agent learning representation—advice in modelling economic learning. Papers on Economics and Evolution #0416, Jena: Max Planck Institute.Google Scholar
  5. Brenner, T., & Murmann, J. P. (2003). The use of simulations in developing robust knowledge about causal processes: Methodological considerations and an application to industrial evolution. Papers on Economics and Evolution #0303, Jena: Max Planck Institute.Google Scholar
  6. Brenner, T., & Werker, C. (2008). A taxonomy of inference in simulation models. Computational Economics,
  7. Brock W. (1999). Scaling in economics: A reader’s guide. Industrial and Corporate Change 8: 409–446CrossRefGoogle Scholar
  8. Carr E.H. (1961). What is history?. London, MacmillanGoogle Scholar
  9. Chattoe, E. (2002). Building empirically plausible multi-agent systems: A case study of innovation diffusion. In K. Dautenhahn (Ed.), Socially intelligent agents: Creating relationships with computers and robots. Dordrecht: Kluwer.Google Scholar
  10. Cowan R., Foray D. (2002). Evolutionary economics and the counterfactual threat: On the nature and role of counterfactual history as an empirical tool in economics. Journal of Evolutionary Economics 12(5): 539–562CrossRefGoogle Scholar
  11. Dawid, H. (2006). Agent-based models of innovation and technological change. In L. Tesfatsion & K. Judd (Eds.), Handbook of computational economics II: Agent-based computational economics. North-Holland: Elsevier.Google Scholar
  12. Doran J. (1997). From computer simulation to artificial societies. SCS Transactions on Computer Simulation 14(2): 69–78Google Scholar
  13. Dosi G., Freeman C., Fabiani S. (1994). The process of economic development: Introducing some stylized facts and theories on technologies, firms and institutions. Industrial and Corporate Change 3: 1–46CrossRefGoogle Scholar
  14. Dosi G., Marengo L., Fagiolo G. (2005). Learning in evolutionary environment, In: Dopfer K. (eds) Evolutionary principles of economics. Cambridge, Cambridge University PressGoogle Scholar
  15. Dosi G., Nelson R.R. (1994). An introduction to evolutionary theories in economics. Journal of Evolutionary Economics 4: 153–172CrossRefGoogle Scholar
  16. Edmonds, B., & Moss, S. (2005). From KISS to KIDS—an ‘anti-simplistic’ modelling approach. In P. Davidsson, B. Logan, & K. Takadama (Eds.), Multi agent based simulation 2004 (Vol. 3415, pp. 130–144). Lecture notes in artificial intelligence, Springer.Google Scholar
  17. Fagiolo, G. (1998). Spatial interactions in dynamic decentralized economies: A review. In P. Cohendet, P. Llerena, H. Stahn, & G. Umbhauer (Eds.), The economics of networks. Interaction and Behaviours. Berlin - Heidelberg: Springer Verlag.Google Scholar
  18. Fagiolo G., Dosi G. (2003). Exploitation, exploration and innovation in a model of endogenous growth with locally interacting agents. Structural Change and Economic Dynamics 14: 237–273CrossRefGoogle Scholar
  19. Fagiolo G., Dosi G., Gabriele R. (2004a). Matching, bargaining, and wage Setting in an evolutionary model of labor market and output dynamics. Advances in Complex Systems 14: 237–273Google Scholar
  20. Fagiolo G., Marengo L., ValenteM. (2004b). Endogenous networks in random population games. Mathematical Population Studies 11: 121–147CrossRefGoogle Scholar
  21. Frenken, K. (2005). History, state and prospects of evolutionary models of technical change: A review with special emphasis on complexity theory. The Netherlands: Utrecht University, mimeo.Google Scholar
  22. Friedman M. (1953). The methodology of positive economics, in essays in positive economics. Chicago, University of Chicago PressGoogle Scholar
  23. Gallegati M., Delli Gatti D., Di Guilmi C., Gaffeo E., Giulioni G., Palestrini A. (2005). A new approach to business fluctuations: Heterogeneous interacting agents, scaling laws and financial fragility. Journal of Economic Behavior Organization 56: 489–512CrossRefGoogle Scholar
  24. Gallegati M., Giulioni G., Palestrini A., Delli Gatti D. (2003). Financial fragility, patterns of firms’ entry and exit and aggregate dynamics. Journal of Economic Behavior and Organization 51: 79–97CrossRefGoogle Scholar
  25. Gilbert N., Troitzsch K. (1999). Simulation for the social scientist. Milton Keynes, Open University PressGoogle Scholar
  26. Gilbert, N. (2004). Open problems in using agent-based models in industrial and labor dynamics. In R. Leombruni & M. Richiardi (Eds.), Industry and labor dynamics: The agent-based computational approach (pp. 401–405). Singapore: World Scientific.Google Scholar
  27. Haavelmo T. (1944). The probability approach in econometrics. Econometrica 12: 1–115CrossRefGoogle Scholar
  28. Kagel J.H., Roth A.E. (Eds.), (1995). The handbook of experimental economics, (Vol. 1). Princeton, N.J: Princeton University PressGoogle Scholar
  29. Kaldor N. (1961). Capital accumulation and economic growth. In: Lutz F.A., Hague D.C. (eds) The theory of capital. London, Macmillan, pp. 177–222Google Scholar
  30. Klejinen, J. (2000). Validation of models: Statistical techniques and data availability. Proceedings of 1999 Winter Simulation Conference. San Diego, CAGoogle Scholar
  31. Koesrindartoto, D., Sun, J., & Tesfatsion, L. (2005). An agent-based computational laboratory for testing the economic reliability of wholesale power market designs. IEEE Power Engineering Society Conference Proceedings. (Vol. 3, pp. 2818–2823), San Francisco, California USA.Google Scholar
  32. Kuhn T. (1962). The structure of scientific revolutions. Chicago, Chicago University PressGoogle Scholar
  33. Kwásnicki W. (1998). Simulation methodology in evolutionary economics. In: Schweitzer F., Silverberg G. (eds) Evolution und selbstorganisation in der konomie. Berlin, Duncker and HumblotGoogle Scholar
  34. Lakatos, I. (1970). Falsification and the methodology of scientific research programmes. In I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 91–196). Cambridge: Cambridge University Press.Google Scholar
  35. Lane D. (1993a). Artificial worlds and economics, part I. Journal of Evolutionary Economics 3: 89–107CrossRefGoogle Scholar
  36. Lane D. (1993b). Artificial worlds and economics, part II. Journal of Evolutionary Economics 3: 177–197CrossRefGoogle Scholar
  37. Leamer E.E. (1978). Specification searches, ad hoc inference with nonexperimental data. New York, John WileyGoogle Scholar
  38. Leombruni, R. (2002). The methodological status of agent-based simulations. Working Paper No. 19, LABORatorio R. Revelli, Centre for Employment Studies, Turin, Italy.Google Scholar
  39. Leombruni, R., Richiardi, M., Saam, N., & Sonnessa, M. (2006). A common protocol for agent-based social simulation, Journal of Artificial Societies and Social Simulation, 9(1) <//>.Google Scholar
  40. Liebowitz S.J., Margolis S.E. (1990). The fable of the keys. Journal of Law and Economics 22: 1–26CrossRefGoogle Scholar
  41. Lucas, R. (1976). Econometric policy evaluation: A critique. In K. Brunner & A. H. Meltzer (Eds.), The phillips curve and labor markets. Carnegie-Rochester Conference Series on Public Policy. (Vol. 1. pp. 161–168). North Holland: Spring, Amsterdam.Google Scholar
  42. Lucas, R., & Sargent, T. (1979). After Keynesian macroeconomics. Reprinted in R. Lucas & T. Sargent (Eds.), Rational expectations and econometric practice. 1981, (pp. 295–320). London: Allen & Unwin.Google Scholar
  43. Mäki U. (1992). On the method of isolation in economics. Poznan Studies in the Philosophy of the Sciences and the Humanities 26: 19–54Google Scholar
  44. Mäki, U. (1994). Reorienting the assumptions issue. In R. Backhouse (Ed.), New Directions in economic methodology. London and New York: Routledge.Google Scholar
  45. Mäki, U. (1998). Realism. In J. B. Davis, D. Wade Hands, & U. Mäki (Eds.), The handbook of economic methodology. (pp. 404–409), Cheltenham, UK: Edward Elgar.Google Scholar
  46. Malerba F., Nelson R.R., Orsenigo L., Winter S.G. (1999). History friendly models of industry evolution: The computer industry. Industrial and Corporate Change 8: 3–41CrossRefGoogle Scholar
  47. Malerba, F., & Orsenigo, L. (2001). Innovation and market structure in the dynamics of the pharmaceutical industry and biotechnology: Towards a history friendly model. Conference in Honour of Richard Nelson and Sydney Winter, Aalborg. 12th–15th June 2001.Google Scholar
  48. Marks, B. (2005). Agent-based market design. Australian Graduate School of Management, mimeo.Google Scholar
  49. Marks, B. (2007). Validating simulation models: A general framework and four applied examples. Computational Economics,
  50. Nelson R.R. (1995). Recent evolutionary theorizing about economic change. Journal of Economic Literature 33: 48–90Google Scholar
  51. Nelson R.R., Winter S.G. (1982). An evolutionary theory of economic change. Cambridge, Harvard University PressGoogle Scholar
  52. Plott C.R., Smith V.L. (eds) (1998). Handbook of experimental economics results. North Holland, Elsevier pressGoogle Scholar
  53. Pyka, A., & Fagiolo, G. (2005). Agent-based modelling: A methodology for neo-schumpeterian economics. In H. Hanusch & A. Pyka (Eds.), The elgar companion to neo-schumpeterian economics. Cheltenham: Edward Elgar.Google Scholar
  54. Richiardi, M. (2003). The promises and perils of agent-based computational economics. Working Paper No. 29, LABORatorio R. Revelli, Centre for Employment Studies, Turin, Italy.Google Scholar
  55. Sargent, R. P. (1998). Verification and validation of simulation models. Proceedings of 1998 Winter Simulation Conference. San Diego, CA.Google Scholar
  56. Sawyer K.R., Beed C., Sankey H. (1997). Underdetermination in economics. The Duhem-Quine Thesis, Economics and Philosophy 13: 1–23Google Scholar
  57. Schorfheide F. (2000). Loss function-based evaluation of DSGE models. Journal of Applied Econometrics 15: 645–670CrossRefGoogle Scholar
  58. Silverberg G., Dosi G., Orsenigo L. (1988). Innovation, diversity and diffusion: A self-organisation model, Economic Journal 98: 1032–1054Google Scholar
  59. Silverberg, G., & Verspagen, B. (1995). Evolutionary theorizing on economic growth. Working Paper, WP-95–78. Laxenburg, Austria: IIASA.Google Scholar
  60. Tesfatsion, L. (1997). How economists can get a life. In W. Arthur, S, Durlauf, & D. Lane (Eds.), The economy as an evolving complex system II. MA, Addison-Wesley: Santa Fe Institute, Santa Fe and Reading.Google Scholar
  61. Tesfatsion, L. (2002). Agent-based computational economics: Growing economies from the bottom up. Working Paper 1, Iowa State University, Department of Economics.Google Scholar
  62. Valente, M. (2005). Qualitative simulation modelling. Faculty of Economics, University of L’Aquila, L’Aquila, Italy, mimeo.Google Scholar
  63. Vega-Redondo F. (1996). Evolution, games, and economic behavior. Oxford, Oxford University PressGoogle Scholar
  64. Werker, C., & Brenner, T. (2004). Empirical calibration of simulation models. Papers on Economics and Evolution # 0410, Jena: Max Planck Institute for Research into Economic Systems.Google Scholar
  65. Windrum P. (1999). Simulation models of technological innovation: A review. American Behavioral Scientist 42(10): 1531–1550CrossRefGoogle Scholar
  66. Windrum P., Birchenhall C. (1998). Is life cycle theory a special case? Dominant designs and the emergence of market niches through co-evolutionary learning. Structural Change and Economic Dynamics 9: 109–134CrossRefGoogle Scholar
  67. Windrum, P. (2004). Neo-Schumpeterian simulation models, merit research memoranda 2004-004, MERIT, University of Maastricht. In H. Hanusch & A. Pyka (Eds.), The elgar companion to neo- schumpeterian economics. Cheltenham: Edward Elgar, forthcoming.Google Scholar
  68. Windrum, P. (2007). Neo-Schumpeterian simulation models. In H. Hanusch & A. Pyka (Eds.), The elgar companion to neo-schumpeterian economics. Cheltenham: Edward Elgar.Google Scholar
  69. Wooldridge M., Jennings N.R. (1995). Intelligent agents: Theory and practice, Knowledge Engineering Review 10: 115–152Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Laboratory of Economics and ManagementSant’Anna School of Advanced StudiesPisaItaly
  2. 2.Evolutionary Economics GroupMax Planck Institute of EconomicsJenaGermany
  3. 3.Manchester Metropolitan University Business SchoolManchesterUK

Personalised recommendations