Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Validating and Calibrating Agent-Based Models: A Case Study

Abstract

In this paper we deal with some validation and calibration experiments on a modified version of the Complex Adaptive Trivial System (CATS) model proposed in Gallegati et al. (2005 Journal of Economic Behavior and Organization, 56, 489–512). The CATS model has been extensively used to replicate a large number of scaling types stylized facts with a remarkable degree of precision. For such purposes, the simulation of the model has been performed entering ad hoc parameter values and using the same initial set up for all the agents involved in the experiments. Nowadays alternative robust and reliable validation techniques for determining whether the simulation model is an acceptable representation of the real system are available. Moreover many distributional and goodness-of-fit tests have been developed while several graphical tools have been proposed to give the researcher a quick comprehension of actual and simulated data. This paper discusses some validation experiments performed with the modified CATS model. In particular starting from a sample of Italian firms included in the CEBI database, we perform several ex-post validation experiments over the simulation period 1982–2000. In the experiments, the model parameters have been estimated using actual data and the initial set up consists of a sample of agents in 1982. The CATS model is then simulated over the period 1982–2000. Using alternative validation techniques, the simulations’ results are ex-post validated with respect to the actual data. The results are promising in that they show the good capabilities of the CATS model in reproducing the observed reality. Finally we have performed a first calibration experiment via indirect inference, in order to ameliorate our estimates. Even in this case, the results are interesting.

This is a preview of subscription content, log in to check access.

References

  1. Axelrod R. (1997). Advancing the art of simulation in the social sciences. In: Conte R., Hegselmann R., Terna P. (eds) Simulating social phenomena. Berlin, Springer-Verlag, pp. 21–40

  2. Axtell R. (2000). Why agents? On the varied motivations for agent computing in the social sciences. Center on Social and Economic Dynamics, Working Paper 17.

  3. Axtell R. (2001). Zipf’s distribution of US firms sizes. Sciences 293: 1818–1820

  4. Axtell R., Axelrdod R., Epstein J.M., Cohen M.D. (1996). Aligning simulation models: A case study and results. Computational and Mathematical Organization Theory 1, 123–141

  5. Bianchi C., Cirillo P., Gallegati M., Vagliasindi P. (2007). Validation in agent-based models: An investigation on the CATS model. Journal of Economic Behaviour and Organization, forthcoming.

  6. Bottazzi G., Secchi A. (2005). Explaining the distribution of firm growth rates. Rand Journal of Economics 37, 234–263

  7. Carley, K. (1996). Validating computational models. Working Paper: www.econ.iastate.edu/tesfatsi/EmpVal/EmpVal.Carley.pdf

  8. Cirillo, P. (2007). Some considerations about Gibrat’s law in Italy. Economics Letters, forthcoming.

  9. Embrechts P., Mikosch T., Kluppelberg C. (1997). Modelling extremal events. Berlin and New York, Springer-Verlag

  10. Epstein J. (1999). Agent-based computational models and generative social sciences. Complexity 4, 41–60

  11. Fagiolo G., Moneta A., Windrum P. (2007). Empirical validation of agent-based models: Alternatives and Prospects. Journal of Artificial Societies and Social Simulation 10(2): 8

  12. Fujiwara Y. (2004). Zipf law in firms bankruptcy. Physica A 337, 219–230

  13. Gabaix X., Gopikrishnan P., Plerou V., Stanley H.E. (2003). A theory of power law distributions in financial markets fluctuations. Nature 423, 267–270

  14. Gaffeo E., Di Guilmi C., Gallegati M. (2003). Power law scaling in the world income distribution. Economics Bullettin 15, 1–7

  15. Gallegati M., Giulioni G., Palestrini A., Delli Gatti D. (2003a). Financial fragility, patterns of firms’ entry and exit and aggregate dynamics. Journal of Economic Behavior and Organization 51, 79–97

  16. Gallegati M., Giulioni G., Kichiji N. (2003b). Complex dynamics and financial fragility in an agent-based model. Advances in Complex Systems 6, 770–779

  17. Gallegati M., Delli Gatti D., Di Guilmi C., Gaffeo E., Giulioni G., Palestrini A. (2004). Business cycles fluctuations and firms’ size distribution dynamics. Advances in Complex Systems 7, 1–18

  18. Gallegati M., Delli Gatti D., Di Guilmi C., Gaffeo E., Giulioni G., Palestrini A. (2005). A new approach to business fluctuations: Heterogeneous interacting agents, scaling laws and financial fragility. Journal of Economic Behavior and Organization 56, 489–512

  19. Gallegati M., Delli Gatti D., Gaffeo E., Giulioni G., Kirman A., Palestrini A., Russo A. (2007). Complex dynamics and empirical evidence. Information Science 177: 1202–1221

  20. Gilli M., Winker P. (2003). A global optimization heuristic for estimating agent-based models. Computational Statistics and Data Analysis 42, 299–312

  21. Gourieroux C., Monfort A. (1996). Simulation-based econometric methods. Oxford, Oxford University Press

  22. Greenwald B.C., Stiglitz J.E. (1990). Macroeconomic models with equity and credit rationing. In: Hubbard R. (eds) Information, capital markets and investment. Chicago, Chicago University Press

  23. Greenwald B.C., Stiglitz J.E. (1993). Financial market imperfections and business cycles. The Quarterly Journal of Economics 108, 77–114

  24. Hahn F. (1982). Money and inflation. Oxford, Blackwell Publishing

  25. Hall B.E. (1987). The relationship between firm size and growth. Journal of Industrial Economics 35, 583–606

  26. Ijiri Y., Simon H.A. (1977). Skew distributions and the size of business firms. Amsterdam, North Holland

  27. Kaldor N. (1965). Capital accumulation and economic growth. In: Lutz F.A., Hague D.C. (eds) The theory of capital Proceedings of a Conference held by the International Economic Association. London, MacMillan

  28. Kleiber C., Kotz S. (2003). Statistical size distributions in economics and actuarial sciences. New York, Wiley

  29. Kleijnen J.P.C. (1998). Experimental design for sensitivity analysis, optimization and validation of simulation models. In: Banks J. (eds) Handbook of simulation (Chap 6). New York, Wiley

  30. Klevmarken N.A. (1998). Statistical inference in microsimulation models: Incorporating external information. Working Paper of Uppsala University, Department of Economics.

  31. Mandelbrot B. (1960). The Pareto-Lévy law and the distribution of income. International Economic Review 1, 79–106

  32. Okuyama K., Takayasu H., Takayasu M. (1999). Zipf’s law in income distribution of companies. Physica A 269, 125–131

  33. Prabhakar M.D.N., Xie M., Jiang R. (2003). Weibull models. New York, Wiley

  34. Quandt R.E. (1966a). On the size distribution of firms. American Economic Review 56, 416–432

  35. Quandt R.E. (1966b). Old and new methods of estimation and the pareto distribution. Metrika 10, 55–82

  36. Ramsden J., Kiss-Haypal G. (2000). Company size distribution in different countries. Physica A 277, 220–227

  37. Sargent T.J. (1998). Verification and validation in simulation models. Proceedings of 1998 Winter Simulation Conference, pp. 52–64.

  38. Shao J. (2003). Mathematical statistics. New York, Springer-Verlag

  39. Simon H.A. (1955). On a class of skew distribution functions. Biometrika 42, 425–440

  40. Stanley M., Amaral L., Buldyrev S., Havling S., Leshorn H., Maas P., Salinger M., Stanley E. (1996). Scaling behavior in the growth of companies. Nature 379, 804–806

  41. Subbotin M.T. (1923). The law of frequency of error. Mathematicheskii Sbornik 31, 296–301

  42. Tesfatsion, L. (2007). Website on Validation of ACE: http://www.econ.iastate.edu/tesfatsi/empvalid.htm

  43. Tesfatsion L., Judd K. (2006). Handbook of computational economics 2. Amsterdam: North Holland.

  44. Troitzsch, K. (2004). Validating simulation models. Proceedings of the 18th European Simulation Multiconference, pp. 98–106.

  45. Vagliasindi P., Cirillo P., Verga G. (2006). Imprese e mercato del credito in un modello agent-based. Rivista Internazionale di Scienze Sociali 114, 459–486

  46. Winker, P., & Gilli, M. (2001). Indirect estimation of parameters of agent based models of financial markets. Working Paper presented at the 2001 International Conference on Computing in Economics and Finance of the Society for Computational Economics.

  47. Zipf G.K. (1932). Selective studies and the principle of relative frequency in language. Cambridge, Cambridge Press

Download references

Author information

Correspondence to Pasquale Cirillo.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bianchi, C., Cirillo, P., Gallegati, M. et al. Validating and Calibrating Agent-Based Models: A Case Study. Comput Econ 30, 245–264 (2007). https://doi.org/10.1007/s10614-007-9097-z

Download citation

Keywords

  • Validation
  • Calibration
  • Agent-based models
  • Indirect inference
  • Size distribution
  • Tail analysis
  • EVT