Network calibration and metamodeling of a financial accelerator agent based model


We introduce a simple financially constrained production framework in which heterogeneous firms and banks maintain multiple credit connections. The parameters of credit market interaction are estimated from real data in order to reproduce a set of empirical regularities of the Japanese credit market. We then pursue the metamodeling approach, i.e. we derive a reduced form for a set of simulated moments \(h(\theta ,s)\) through the following steps: (1) we run agent-based simulations using an efficient sampling design of the parameter space \(\Theta \); (2) we employ the simulated data to estimate and then compare a number of alternative statistical metamodels. Then, using the best fitting metamodels, we study through sensitivity analysis the effects on h of variations in the components of \(\theta \in \Theta \). Finally, we employ the same approach to calibrate our agent-based model (ABM) with Japanese data. Notwithstanding the fact that our simple model is rejected by the evidence, we show th at metamodels can provide a methodologically robust answer to the question “does the ABM replicate empirical data?”.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. 1.

    For a review see Chen et al. (2012).

  2. 2.

    A network involving n firms and m banks connected by l links is said to be sparse when \(l \ll n \times m \), otherwise it is said to be dense. The Japanese credit market studied in Bargigli and Gallegati (2011), whose most recent data are employed in this paper, had \(l = 21,811\) connections over a maximum of \(n \times m = 2,674 \times 182 = 486,668\) in 2005.

  3. 3.

    By topological property we mean any observable which is defined on a binary network or on the binary representation of a weighted network. The latter is obtained from the binary representation of its weighted links, which is defined, for each couple of nodes (ij), as \(a_{ij} = 1(w_{ij}>0)\), where is the Indicator function and \(w_{ij}\) is the strength of the relationship between i and j.

  4. 4.

    Admittedly, with this choice we introduce potentially a small survivor bias in the model, since surviving firms are typically larger. However, the number of firm defaults is very limited over the parameter space we use for simulations and we choose the median (instead of the mean) in order to minimize the bias.

  5. 5.

    For more details see

  6. 6.

    In detail, we employ random intercepts in a generalized linear mixed model estimated with the R Core Team (2015) package lme4.

  7. 7.

    The first 200 periods are discarded to get rid of transient dynamics that could introduce a bias in model statistics. Moreover, the long period of simulation does not represent a long-run analysis but a repeated business cycle analysis. In other words, we do not consider the presence of a trend in time-series by construction.

  8. 8.


  9. 9.

  10. 10.



  1. Alfarano S, Lux T, Wagner F (2005) Estimation of agent-based models: the case of an asymmetric herding model. Comput Econ 26(1):19–49

  2. Barde S, van der Hoog S (2017) An empirical validation protocol for large-scale agent-based models, studies in economics 1712, school of economics. University of Kent, Canterbury

  3. Bargigli L, Gallegati M (2011) Random digraphs with given expected degree sequences: a model for economic networks. J Econ Behav Organ 78(3):396–411

  4. Bargigli L, Gallegati M, Riccetti L, Russo A (2014) Network analysis and calibration of the “leveraged network-based financial accelerator”. J Econ Behav Organ 99:109–125

  5. Brenner T, Werker C (2007) A taxonomy of inference in simulation models. Comput Econ 30(3):227–244

  6. Campolongo F, Saltelli A, Tarantola S (2000) Sensitivity analysis as an ingredient of modeling. Stat Sci 15(4):377–395

  7. Canova F, Sala L (2009) Back to square one: identification issues in DSGE models. J Monet Econ 56:431–449

  8. Chamberlain G (1980) Analysis of covariance with qualitative data. Rev Econ Stud 47(1):225–238

  9. Chen S-H, Chang C-L, Du Y-R (2012) Agent-based economic models and econometrics. Knowl Eng Rev 27(6):187–219

  10. Cioppa TM, Lucas TW (2007) Efficient nearly orthogonal and space-filling latin hypercubes. Technometrics 49(1):45–55

  11. Dancik GM, Jones DE, Dorman KS (2010) Parameter estimation and sensitivity analysis in an agent-based model of leishmania major infection. J Theor Biol 262(3):398–412

  12. Fagiolo G, Moneta A, Windrum P (2007) A critical guide to empirical validation of agent-based models in economics: methodologies, procedures, and open problems. Comput Econ 30(3):195–226

  13. Gallegati M, Palestrini A, Delli Gatti D, Scalas E (2006) Aggregation of heterogeneous interacting agents: the variant representative agent framework. J Econ Interact Coord 1(1):5–19

  14. Gouriéroux C, Monfort A (1996) Simulation-based econometric methods. CORE lectures. Oxford University Press, Oxford

  15. Grazzini J, Richiardi M (2015) Estimation of ergodic agent-based models by simulated minimum distance. J Econ Dyn Control 51:148–165

  16. Greenwald B, Stiglitz JE (1993) Financial market imperfections and business cycles. Q J Econ 108(77–114):1993

  17. Kirman AP (1992) Whom or what does the representative individual represent? J Econ Perspect 6(2):117–136

  18. Manzan S, Westerhoff F-H (2007) Heterogeneous expectations, exchange rate dynamics and predictability. J Econ Behav Organ 64(1):111–128

  19. Nakagawa S, Schielzeth H (2013) A general and simple method for obtaining \(R^2\) from generalized linear mixed-effects models. Methods Ecol Evol 4(2):133–142

  20. Park J, Newman MEJ (2004) Statistical mechanics of networks. Phys Rev E 2004(70):066117

  21. R Core Team (2015) (2015) A language and environment for statistical computing. R (2015) Foundation for Statistical Computing, Vienna, Austria

  22. Riccetti L, Russo A, Gallegati M (2013) Leveraged network-based financial accelerator. J Econ Dyn Control 37(8):1626–1640

  23. Roustant O, Ginsbourger D, Deville Y (2012) DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. J Stat Softw 51(1):1–55

  24. Salle I, Yildizoglu M (2014) Efficient sampling and meta-modeling for computational economic models. Comput Econ 44(4):507–536

  25. Shin HS (2008) Risk and liquidity in a system context. J Financ Intermed 17(3):315–329

  26. Stoker TM (1993) Empirical approaches to the problem of aggregation over individuals. J Econ Lit 31(4):1827–1874

  27. Valente M (2005) Qualitative simulation modelling. Faculty of Economics, University of L’Aquila (Italy), Mimeo

Download references


We thank all the participants of the DISEI Department seminar of University of Florence held on November 17th 2015, the DISES Department seminar of Polytechnic University of Marche held on March 3rd 2016, the CEF2016 conference held on June 26–28 2016 in Bordeaux for their useful comments. A special thanks to Yoshi Fujiwara for providing the Japanese credit market data. All the usual disclaimers apply.

Author information

Correspondence to Alberto Russo.



Efficient sampling of \(\Theta \)

In order to simulate the ABM, we have to specify the value of its parameter vector \(\theta \). In general, some parameters can be set at a value which comes from the literature, experimental studies and empirical data. For other parameters, we can define an appropriate range of variation and study the behavior of the model within that range using Montecarlo simulations.

Random sampling from a uniform distribution, which is a common choice in Montecarlo exercises, is inefficient because it generates a high number of redundant sampling points (points very close to each other), while leaving some parts of the parameter space unexplored. A common alternative is importance sampling, which however requires prior information. A proper “design of experiment” (DOE) delivers instead a parsimonious sample which is nevertheless representative of the parameter space. In particular, representative samples are said to be “space filling”, since they cover as uniformly as possible the domain of variation.

The sampling scheme we adopt for the subspace of free parameters \(\theta = (r_{cb},\delta ,\mu )\), specified in Table 5, is the one suggested by Cioppa and Lucas (2007) and employed by Salle and Yildizoglu (2014). This scheme is based on Nearly Orthogonal Latin Hypercubes (NOLH). In the context of sampling theory, a square grid representing the location of sample points for a couple of parameters is a Latin square if there is only one sample point in each row and each column. A Latin hypercube is the generalization of this concept to an arbitrary number of dimensions, whereby each sample point is the only one in each axis-aligned hyperplane containing it. This property ensures that sample points are non collapsing, i.e. that the 1-dimensional projections of sample points along each axis are space filling. In fact, with this scheme, the sampled values of each parameter appear once and only once.

Basic Latin Hypercube schemes may display correlations between the columns of the \(k \times n\) design matrix X, where k is the number of parameters and n is the sample size for each parameter, especially when k is lower but close to n. Instead, an orthogonal design is convenient because it gives uncorrelated estimates of the coefficients in linear regression models and improves the performance of statistical estimation in general. In practice, in orthogonal sampling, the sample space is divided into equally probable subspaces. All sample points in the orthogonal LH scheme are then chosen simultaneously making sure that the total ensemble of sample points is a Latin Hypercube and that each subspace is sampled with the same density.

Table 7 Design of Experiment (DOE) for simulations of Sect. 4
Table 8 Design of Experiment (DOE) for simulations of Sect. 5

The NOLH scheme of Cioppa and Lucas (2007) improves the space filling properties of the resulting sample when \(k \lessapprox n\) at the cost of introducing a small maximal correlation of 0.03 between the columns of X. Furthermore, no assumptions regarding the homoskedasticity of errors or the shape of the response surface (like linearity) are required to obtain this scheme. The values of \(\theta = (r_{cb},\delta ,\mu )\) obtained from this scheme and used for simulations of Sect. 4 are reported in Table 7, while those employed for simulations of Sect. 5 are reported in Table 8.

Kriging regression

In the metamodeling selection exercise of Sect. 4, we estimate various Kriging models. These are generalized regression models, potentially allowing for heteroskedastic and correlated errors. The approach is widely used for ABM metamodeling in various fields (see e.g. Salle and Yildizoglu 2014; Dancik et al. 2010 and references therein). Using generalized regression is convenient since some of the parameters of our model are related to random distributions which naturally affect the variability of model output. The Kriging approach (Roustant et al. 2012) resorts to feasible generalized least squares by assuming a stationary correlation kernel \(K(h) = K(\theta _i-\theta _j)\), where \(\theta _i,\theta _j\) are distinct points in the parameter space \(\Theta \). K(h) takes the following general form:

$$\begin{aligned} K(h) = \prod _{j=1}^{d}g(h_j,\lambda _j) \end{aligned}$$

where d is the dimension of \(\Theta \), and \(\lambda = (\lambda _1,\dots ,\lambda _d)\) is a vector of parameters to be determined. In particular, we employ for g the specifications of Table 9.

Table 9 Correlation kernels (Roustant et al. 2012)

Since we work with noisy, potentially heteroskedastic observations, in our estimation the covariance matrix of residuals is determined as follows:

$$\begin{aligned} C = \sigma ^2 \, R + \text {diag}(\tau ) \end{aligned}$$

where R is the correlation matrix with elements \(R_{ij} = K(\theta _i-\theta _j)\) and \(\tau = (\tau _1^2,\dots ,\tau _n^2)\) is the vector containing the observed variance of model output at fixed points of the parameter space and n is the size of the NOLH design. ML estimation is performed on the “concentrated” multivariate Gaussian log-likelihood, obtained by substituting the vector of regression coefficients with their generalized least square estimator. The “concentrated” log-likelihood is a function of \(\sigma \) and \(\lambda \), which are the optimization variables of the estimation. The solution is obtained numerically through the quasi-Newton algorithm provided by the DiceKriging R Core Team (2015) package (Roustant et al. 2012).

Sensitivity analysis

Campolongo et al. (2000) define sensitivity analysis (SA) as the study of how uncertainty in the output of a model can be apportioned to different sources of uncertainty in the model input. In this respect, SA techniques should satisfy the two main requirements of being global and model free. By global, one means that SA must take into consideration the entire joint distribution of parameters. Global methods are opposed to local methods, which take into consideration the variation of one parameter at a time, e.g. by computing marginal effects of each parameter. By model independent, one means that no assumptions on the model functional relationship with its inputs, such as linearity, are required.

Campolongo et al. (2000) propose a global approach based on the decomposition of variance:

$$\begin{aligned} V(h)&= \sum _i^k V_i + \sum _{i<j} V_{ij} + \sum _{i<j<m} V_{ijm} + \dots + V_{12\dots k} \\ V_i&= \mathbb {V}_{\theta _{i}}\left[ \mathbb {E}_{\theta _{-i}}\left( h|\theta _i = x \right) \right] \\ V_{ij}&= \mathbb {V}_{\theta _{(i,j)}}\left[ \mathbb {E}_{\theta _{-(i,j)}}\left( h|\theta _i = x, \theta _j = y \right) \right] - V_{i} - V_{j}\\ \dots \end{aligned}$$

where h is a generic vector of moments. We see that \(V_i\) represents the variance of the main effect of parameter i, while all the other terms are related to interaction effects. From this general formula we can obtain the contribution of interaction effects \(S_{Ii}\) involving the parameter \(\theta _i\) as follows:

$$\begin{aligned} S_{Ii}&= S_{Ti} - S_i \end{aligned}$$
$$\begin{aligned} S_i&= \frac{V_i}{V}\end{aligned}$$
$$\begin{aligned} S_{Ti}&= \frac{\mathbb {E}_{\theta _{-i}}\left[ \mathbb {V}_{\theta _{i}}(h|\theta _{-i})\right] }{V} = 1 - \frac{\mathbb {V}_{\theta _{-i}}\left[ \mathbb {E}_{\theta _i}(h|\theta _{-i})\right] }{V} = 1 - \frac{V_{-i}}{V} \end{aligned}$$

The multidimensional integral of the last line can be evaluated numerically using the extended FAST method described in Campolongo et al. (2000). The results of Fig. 5 show, for each parameter in \(\theta \), the main effect (C.2) and the interaction effect (C.1) on the components of \(h = (m,v,fb)\).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bargigli, L., Riccetti, L., Russo, A. et al. Network calibration and metamodeling of a financial accelerator agent based model. J Econ Interact Coord (2018) doi:10.1007/s11403-018-0217-8

Download citation