Skip to main content
SpringerLink
Log in

Calibration of Agent-Based Models by Means of Meta-Modeling and Nonparametric Regression

  • Published:
Computational Economics Aims and scope Submit manuscript

Abstract

Taking agent-based models to the data is still very challenging for researchers. In this paper we propose a new method to calibrate the model parameters based on indirect inference, which consists in minimizing the distance between real and artificial data. Basically, we first introduce a nonparametric regression meta-model to approximate the relationship between model parameters and distance. Then the meta-model is estimated by local polynomial regression estimation on a small sample of parameter vectors drawn from the parameter space of the ABM. Finally, once the distance has been estimated we can pick the parameter vector minimizing it. One innovative feature of the method is the sampling scheme, based on sampling at the same time both the parameter vectors and the seed of the random numbers generator in a random fashion, which permits to average out the effect of randomness without resorting to Monte Carlo simulations. A battery of simple calibration exercises performed on an agent-based macro model shows that the method allows to minimize the distance with good precision using relatively few simulations of the model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and material

Data are available from the authors upon request.

Code availability

Codes are available from the authors upon request.

Notes

  1. Nonetheless, more advanced validation methods have recently been put forth like for example the one in Guerini and Moneta (2017), which is based on the comparison of the causal relationships implied by a structural VAR model estimated on both real and simulated data.

  2. There are a few exceptions, though, e.g. Alfarano et al. (2005), Caiani et al. (2016), who obtain some closed-form solutions depending on the parameters.

  3. See for example Fagiolo et al. (2017) and Lux and Zwinkels (2018) for two extensive surveys and Platt (2020) for a test of different methods.

  4. The data can be used directly or through summary statistics.

  5. When \(p=1\) this method is called local linear regression.

  6. For exposition convenience we suppose the parameter space \(\Theta \) to be univariate.

  7. We will ignore the role of initial conditions. If the ABM is ergodic, this is totally legitimate.

  8. The whole procedure, including the calculation of the local polynomial estimators, was implemented in MATLAB. The code is available upon request.

  9. Although we can say nothing about heteroskedasticity in the error terms. Even when present, however, this issue can just decrease the efficiency of the estimators.

  10. The i-th in-sample fitted value \({\hat{f}}(\theta _i)\) is just Eq. (15) with \(\theta _i\) replacing \(\theta _0\).

  11. Note that here we use different symbols from the original papers in order to avoid confusion with other variable and parameter names.

  12. Estimating the distance function over the whole parameter space takes a fraction of second in all the univariate cases and less than one second in the bivariate case.

  13. As real data we used average U.S. quarterly macro data retrieved from FRED dataset, obtaining \(g_r=0.0078\) and \(\pi _r=0.0077\).

  14. The definition of the boundaries of the parameter space and its discretization are open issues in calibration that we cannot resolve here. For all the three parameters we simply chose a reasonable range of values such that the ABM did not show a degenerated behavior.

  15. In all the examples the boundaries of H are based on the boundaries of the parameter space. If in fact h is too small, then perfect collinearity issues do not allow to invert the matrix \(B' \Sigma B\) in Eq. 7, whereas if h is too large the estimated regression function trivially becomes a straight line.

References

  • Alfarano, S., Lux, T., & Wagner, F. (2005). Estimation of agent-based models: The case of an asymmetric herding model. Computational Economics, 26(1), 19–49.

    Article  Google Scholar 

  • Barde, S., & van der Hoog, S. (2017). An empirical validation protocol for large-scale agent-based models. University of Kent, School of Economics Discussion Papers

  • Bargigli, L., Riccetti, L., Russo, A., & Gallegati, M. (2020). Network calibration and metamodeling of a financial accelerator agent-based model. Journal of Economic Interaction and Coordination, 15(2), 413–440.

    Article  Google Scholar 

  • Bianchi, C., Cirillo, P., Gallegati, M., & Vagliasindi, P. A. (2007). Validating and calibrating agent-based models: A case study. Computational Economics, 30, 245–264.

    Article  Google Scholar 

  • Bianchi, C., Cirillo, P., Gallegati, M., & Vagliasindi, P. A. (2008). Validation in agent-based models: an investigation on the CATS model. Journal of Economic Behavior and Organization, 67, 947–964.

    Article  Google Scholar 

  • Caiani, A., Godin, A., Caverzasi, E., Gallegati, M., Kinsella, S., & Stiglitz, J. (2016). Agent based-stock flow consistent macroeconomics: Towards a benchmark model. Journal of Economic Dynamics and Control, 69, 375–408.

    Article  Google Scholar 

  • Chen, S., & Desiderio, S. (2018). Computational evidence on the distributive properties of monetary policy. Economics- The Open-Access, Open-Assessment E-Journal, 12(2018–62), 1–32.

    Google Scholar 

  • Chen, S., & Desiderio, S. (2020). Job duration and inequality. Economics- The Open-Access, Open-Assessment E-Journal, 14(2020–9), 1–26.

    Google Scholar 

  • Chen, S., & Desiderio, S. (2021). A regression-based calibration method for agent-based models. Computational Economics (published on-line), https://doi.org/10.1007/s10614-021-10106-9.

  • Fabretti, A. (2013). On the problem of calibrating an agent-based model for financial markets. Journal of Economic Interaction and Coordination, 8(2), 277–293.

    Article  Google Scholar 

  • Fagiolo, G., Guerini, M., Lamperti, F., Moneta, A., & Roventini, A. (2017). Validation of agent-based models in economics an finance. LEM Working Paper Series, Scuola Superiore Sant’Anna, Pisa.

  • Fagiolo, G., Moneta, A., & Windrum, P. (2007). A critical guide to empirical validation of agent-based models in economics: Methodologies, procedures, and open problems. Computational Economics, 30, 195–226.

    Article  Google Scholar 

  • Gilli, M., & Winker, P. (2003). A global optimization heuristic for estimating agent-based models. Computational Statistics and Data Analysis, 42, 299–312.

    Article  Google Scholar 

  • Gourieroux, C., Monfort, A., & Renault, E. (1993). Indirect Inference. Journal of Applied Econometrics, 8, 85–118.

    Article  Google Scholar 

  • Grazzini, J., & Richiardi, M. (2015). Estimation of ergodic agent-based models by simulated minimum distance. Journal of Economic Dynamics and Control, 51, 148–165.

    Article  Google Scholar 

  • Grazzini, J., Richiardi, M., & Tsionas, M. (2017). Bayesian estimation of agent-based models. Journal of Economic Dynamics and Control, 77, 26–47.

    Article  Google Scholar 

  • Guerini, M., & Moneta, A. (2017). A method for agent-based models validation. Journal of Economic Dynamics and Control, 82, 125–141.

    Article  Google Scholar 

  • Judd, K., & Tesfatsion, L. (Eds.). (2006). Handbook of computational economics II: Agent-based models. Amsterdam: North-Holland.

    Google Scholar 

  • Kukacka, J., & Barunik, J. (2017). Estimation of financial agent-based models with simulated maximum likelihood. Journal of Economic Dynamics and Control, 85, 21–45.

    Article  Google Scholar 

  • Lamperti, F. (2018). Empirical validation of simulated models through the GSL-div: an illustrative application. Journal of Economic Interaction and Coordination, 13, 143–171.

    Article  Google Scholar 

  • Lamperti, F., Roventini, A., & Sani, A. (2018). Agent-based model calibration using machine learning surrogates. Journal of Economic Dynamics and Control, 90, 366–389.

    Article  Google Scholar 

  • Lux, T., & Zwinkels, R. (2018). Empirical validation of agent-based models. In C. Hommes & B. LeBaron (Eds.), Handbook of Computational Economics (Vol. 4, pp. 437-488). Available at SSRN: https://ssrn.com/abstract=2926442 or https://doi.org/10.2139/ssrn.2926442.

  • McFadden, D. (1989). A method of simulated moments for estimation of discrete response models without numerical integration. Econometrica, 57(5), 995–1026.

    Article  Google Scholar 

  • Platt, D. (2020). A comparison of economic agent-based model calibration methods. Journal of Economic Dynamics and Control, 113, 1003859.

    Article  Google Scholar 

  • Recchioni, M. C., Tedeschi, G., & Gallegati, M. (2015). A calibration procedure for analyzing stock price dynamics in an agent-based framework. Journal of Economic Dynamics and Control, 60, 1–25.

    Article  Google Scholar 

  • Salle, I., & Yildizoglu, M. (2014). Efficient sampling and meta-modeling for computational economic models. Computational Economics, 44(4), 507–536.

    Article  Google Scholar 

  • Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., et al. (2008). Global sensitivity analysis. The primer. New Jersey: John Wiley & Sons.

    Google Scholar 

  • Tsybakov, A. (2009). Introduction to nonparametric estimation. New York: Springer.

    Book  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the financial support by the Natural Science Foundation of Guangdong Province, under Grant No. 2018A030310148. We also wish to thank Giovanni Cerulli and all the participants to IWcee19-VII International Workshop on Computational Economics and Econometrics at CNR, Rome, 3-5 July 2019 for useful suggestions.

Funding

This work was funded by Natural Science Foundation of Guangdong Province, under Grant No. 2018A030310148.

Author information

Authors and Affiliations

  1. Business School, Shantou University, 243 Daxue Road, Shantou, 515063, China

    Siyan Chen & Saul Desiderio

Authors
  1. Siyan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saul Desiderio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saul Desiderio.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

In this section we propose the general implementation of our method in pseudo-code, divided into four algorithms (Tables 6, 7, 8 and 9). The pseudo-code is intended to be completed using all the relevant formulas and equations presented in the paper.

Table 6 Algorithm A to generate samples \(\{\theta _i\}^n_{i=1}\) and \(\{d_i\}^n_{i=1}\)
Full size table
Table 7 Algorithm B to choose the optimal bandwidth \(h^*\)
Full size table
Table 8 Algorithm C to estimate the minimum-distance \(\theta ^*\)
Full size table
Table 9 Algorithm D to compute local polynomial estimates
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Desiderio, S. Calibration of Agent-Based Models by Means of Meta-Modeling and Nonparametric Regression. Comput Econ 60, 1457–1478 (2022). https://doi.org/10.1007/s10614-021-10188-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10614-021-10188-5

Keywords

JEL Classifications

Search

Navigation