Computational Economics

, Volume 30, Issue 3, pp 245–264 | Cite as

Validating and Calibrating Agent-Based Models: A Case Study

  • Carlo Bianchi
  • Pasquale CirilloEmail author
  • Mauro Gallegati
  • Pietro A. Vagliasindi


In this paper we deal with some validation and calibration experiments on a modified version of the Complex Adaptive Trivial System (CATS) model proposed in Gallegati et al. (2005 Journal of Economic Behavior and Organization, 56, 489–512). The CATS model has been extensively used to replicate a large number of scaling types stylized facts with a remarkable degree of precision. For such purposes, the simulation of the model has been performed entering ad hoc parameter values and using the same initial set up for all the agents involved in the experiments. Nowadays alternative robust and reliable validation techniques for determining whether the simulation model is an acceptable representation of the real system are available. Moreover many distributional and goodness-of-fit tests have been developed while several graphical tools have been proposed to give the researcher a quick comprehension of actual and simulated data. This paper discusses some validation experiments performed with the modified CATS model. In particular starting from a sample of Italian firms included in the CEBI database, we perform several ex-post validation experiments over the simulation period 1982–2000. In the experiments, the model parameters have been estimated using actual data and the initial set up consists of a sample of agents in 1982. The CATS model is then simulated over the period 1982–2000. Using alternative validation techniques, the simulations’ results are ex-post validated with respect to the actual data. The results are promising in that they show the good capabilities of the CATS model in reproducing the observed reality. Finally we have performed a first calibration experiment via indirect inference, in order to ameliorate our estimates. Even in this case, the results are interesting.


Validation Calibration Agent-based models Indirect inference Size distribution Tail analysis EVT 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Axelrod R. (1997). Advancing the art of simulation in the social sciences. In: Conte R., Hegselmann R., Terna P. (eds) Simulating social phenomena. Berlin, Springer-Verlag, pp. 21–40Google Scholar
  2. Axtell R. (2000). Why agents? On the varied motivations for agent computing in the social sciences. Center on Social and Economic Dynamics, Working Paper 17.Google Scholar
  3. Axtell R. (2001). Zipf’s distribution of US firms sizes. Sciences 293: 1818–1820Google Scholar
  4. Axtell R., Axelrdod R., Epstein J.M., Cohen M.D. (1996). Aligning simulation models: A case study and results. Computational and Mathematical Organization Theory 1, 123–141CrossRefGoogle Scholar
  5. Bianchi C., Cirillo P., Gallegati M., Vagliasindi P. (2007). Validation in agent-based models: An investigation on the CATS model. Journal of Economic Behaviour and Organization, forthcoming.Google Scholar
  6. Bottazzi G., Secchi A. (2005). Explaining the distribution of firm growth rates. Rand Journal of Economics 37, 234–263Google Scholar
  7. Carley, K. (1996). Validating computational models. Working Paper: Scholar
  8. Cirillo, P. (2007). Some considerations about Gibrat’s law in Italy. Economics Letters, forthcoming.Google Scholar
  9. Embrechts P., Mikosch T., Kluppelberg C. (1997). Modelling extremal events. Berlin and New York, Springer-VerlagGoogle Scholar
  10. Epstein J. (1999). Agent-based computational models and generative social sciences. Complexity 4, 41–60CrossRefGoogle Scholar
  11. Fagiolo G., Moneta A., Windrum P. (2007). Empirical validation of agent-based models: Alternatives and Prospects. Journal of Artificial Societies and Social Simulation 10(2): 8Google Scholar
  12. Fujiwara Y. (2004). Zipf law in firms bankruptcy. Physica A 337, 219–230CrossRefGoogle Scholar
  13. Gabaix X., Gopikrishnan P., Plerou V., Stanley H.E. (2003). A theory of power law distributions in financial markets fluctuations. Nature 423, 267–270CrossRefGoogle Scholar
  14. Gaffeo E., Di Guilmi C., Gallegati M. (2003). Power law scaling in the world income distribution. Economics Bullettin 15, 1–7Google Scholar
  15. Gallegati M., Giulioni G., Palestrini A., Delli Gatti D. (2003a). Financial fragility, patterns of firms’ entry and exit and aggregate dynamics. Journal of Economic Behavior and Organization 51, 79–97CrossRefGoogle Scholar
  16. Gallegati M., Giulioni G., Kichiji N. (2003b). Complex dynamics and financial fragility in an agent-based model. Advances in Complex Systems 6, 770–779CrossRefGoogle Scholar
  17. Gallegati M., Delli Gatti D., Di Guilmi C., Gaffeo E., Giulioni G., Palestrini A. (2004). Business cycles fluctuations and firms’ size distribution dynamics. Advances in Complex Systems 7, 1–18CrossRefGoogle Scholar
  18. Gallegati M., Delli Gatti D., Di Guilmi C., Gaffeo E., Giulioni G., Palestrini A. (2005). A new approach to business fluctuations: Heterogeneous interacting agents, scaling laws and financial fragility. Journal of Economic Behavior and Organization 56, 489–512CrossRefGoogle Scholar
  19. Gallegati M., Delli Gatti D., Gaffeo E., Giulioni G., Kirman A., Palestrini A., Russo A. (2007). Complex dynamics and empirical evidence. Information Science 177: 1202–1221Google Scholar
  20. Gilli M., Winker P. (2003). A global optimization heuristic for estimating agent-based models. Computational Statistics and Data Analysis 42, 299–312CrossRefGoogle Scholar
  21. Gourieroux C., Monfort A. (1996). Simulation-based econometric methods. Oxford, Oxford University PressGoogle Scholar
  22. Greenwald B.C., Stiglitz J.E. (1990). Macroeconomic models with equity and credit rationing. In: Hubbard R. (eds) Information, capital markets and investment. Chicago, Chicago University PressGoogle Scholar
  23. Greenwald B.C., Stiglitz J.E. (1993). Financial market imperfections and business cycles. The Quarterly Journal of Economics 108, 77–114CrossRefGoogle Scholar
  24. Hahn F. (1982). Money and inflation. Oxford, Blackwell PublishingGoogle Scholar
  25. Hall B.E. (1987). The relationship between firm size and growth. Journal of Industrial Economics 35, 583–606CrossRefGoogle Scholar
  26. Ijiri Y., Simon H.A. (1977). Skew distributions and the size of business firms. Amsterdam, North HollandGoogle Scholar
  27. Kaldor N. (1965). Capital accumulation and economic growth. In: Lutz F.A., Hague D.C. (eds) The theory of capital Proceedings of a Conference held by the International Economic Association. London, MacMillanGoogle Scholar
  28. Kleiber C., Kotz S. (2003). Statistical size distributions in economics and actuarial sciences. New York, WileyGoogle Scholar
  29. Kleijnen J.P.C. (1998). Experimental design for sensitivity analysis, optimization and validation of simulation models. In: Banks J. (eds) Handbook of simulation (Chap 6). New York, WileyGoogle Scholar
  30. Klevmarken N.A. (1998). Statistical inference in microsimulation models: Incorporating external information. Working Paper of Uppsala University, Department of Economics.Google Scholar
  31. Mandelbrot B. (1960). The Pareto-Lévy law and the distribution of income. International Economic Review 1, 79–106CrossRefGoogle Scholar
  32. Okuyama K., Takayasu H., Takayasu M. (1999). Zipf’s law in income distribution of companies. Physica A 269, 125–131CrossRefGoogle Scholar
  33. Prabhakar M.D.N., Xie M., Jiang R. (2003). Weibull models. New York, WileyGoogle Scholar
  34. Quandt R.E. (1966a). On the size distribution of firms. American Economic Review 56, 416–432Google Scholar
  35. Quandt R.E. (1966b). Old and new methods of estimation and the pareto distribution. Metrika 10, 55–82CrossRefGoogle Scholar
  36. Ramsden J., Kiss-Haypal G. (2000). Company size distribution in different countries. Physica A 277, 220–227CrossRefGoogle Scholar
  37. Sargent T.J. (1998). Verification and validation in simulation models. Proceedings of 1998 Winter Simulation Conference, pp. 52–64.Google Scholar
  38. Shao J. (2003). Mathematical statistics. New York, Springer-VerlagGoogle Scholar
  39. Simon H.A. (1955). On a class of skew distribution functions. Biometrika 42, 425–440Google Scholar
  40. Stanley M., Amaral L., Buldyrev S., Havling S., Leshorn H., Maas P., Salinger M., Stanley E. (1996). Scaling behavior in the growth of companies. Nature 379, 804–806CrossRefGoogle Scholar
  41. Subbotin M.T. (1923). The law of frequency of error. Mathematicheskii Sbornik 31, 296–301Google Scholar
  42. Tesfatsion, L. (2007). Website on Validation of ACE: Scholar
  43. Tesfatsion L., Judd K. (2006). Handbook of computational economics 2. Amsterdam: North Holland.Google Scholar
  44. Troitzsch, K. (2004). Validating simulation models. Proceedings of the 18th European Simulation Multiconference, pp. 98–106.Google Scholar
  45. Vagliasindi P., Cirillo P., Verga G. (2006). Imprese e mercato del credito in un modello agent-based. Rivista Internazionale di Scienze Sociali 114, 459–486Google Scholar
  46. Winker, P., & Gilli, M. (2001). Indirect estimation of parameters of agent based models of financial markets. Working Paper presented at the 2001 International Conference on Computing in Economics and Finance of the Society for Computational Economics.Google Scholar
  47. Zipf G.K. (1932). Selective studies and the principle of relative frequency in language. Cambridge, Cambridge PressGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Carlo Bianchi
    • 1
  • Pasquale Cirillo
    • 2
    Email author
  • Mauro Gallegati
    • 3
  • Pietro A. Vagliasindi
    • 4
  1. 1.Dipartimento di Scienze EconomicheUniversità di PisaPisaItaly
  2. 2.IMQUniversità BocconiMilanItaly
  3. 3.DEA/SIECUniversità Politecnica delle MarcheAnconaItaly
  4. 4.Dipartimento di Diritto, Economia e Finanza InternazionaleUniversità di ParmaParmaItaly

Personalised recommendations