Network calibration and metamodeling of a financial accelerator agent based model

Abstract

We introduce a simple financially constrained production framework in which heterogeneous firms and banks maintain multiple credit connections. The parameters of credit market interaction are estimated from real data in order to reproduce a set of empirical regularities of the Japanese credit market. We then pursue the metamodeling approach, i.e. we derive a reduced form for a set of simulated moments $h(\theta ,s)$ through the following steps: (1) we run agent-based simulations using an efficient sampling design of the parameter space $\Theta$; (2) we employ the simulated data to estimate and then compare a number of alternative statistical metamodels. Then, using the best fitting metamodels, we study through sensitivity analysis the effects on h of variations in the components of $\theta \in \Theta$. Finally, we employ the same approach to calibrate our agent-based model (ABM) with Japanese data. Notwithstanding the fact that our simple model is rejected by the evidence, we show th at metamodels can provide a methodologically robust answer to the question “does the ABM replicate empirical data?”.

Appendices

Appendices

Efficient sampling of $\Theta$

In order to simulate the ABM, we have to specify the value of its parameter vector $\theta$. In general, some parameters can be set at a value which comes from the literature, experimental studies and empirical data. For other parameters, we can define an appropriate range of variation and study the behavior of the model within that range using Montecarlo simulations.

Random sampling from a uniform distribution, which is a common choice in Montecarlo exercises, is inefficient because it generates a high number of redundant sampling points (points very close to each other), while leaving some parts of the parameter space unexplored. A common alternative is importance sampling, which however requires prior information. A proper “design of experiment” (DOE) delivers instead a parsimonious sample which is nevertheless representative of the parameter space. In particular, representative samples are said to be “space filling”, since they cover as uniformly as possible the domain of variation.

The sampling scheme we adopt for the subspace of free parameters $\theta = (r_{cb},\delta ,\mu )$, specified in Table 5, is the one suggested by Cioppa and Lucas (2007) and employed by Salle and Yildizoglu (2014). This scheme is based on Nearly Orthogonal Latin Hypercubes (NOLH). In the context of sampling theory, a square grid representing the location of sample points for a couple of parameters is a Latin square if there is only one sample point in each row and each column. A Latin hypercube is the generalization of this concept to an arbitrary number of dimensions, whereby each sample point is the only one in each axis-aligned hyperplane containing it. This property ensures that sample points are non collapsing, i.e. that the 1-dimensional projections of sample points along each axis are space filling. In fact, with this scheme, the sampled values of each parameter appear once and only once.

Basic Latin Hypercube schemes may display correlations between the columns of the $k \times n$ design matrix X, where k is the number of parameters and n is the sample size for each parameter, especially when k is lower but close to n. Instead, an orthogonal design is convenient because it gives uncorrelated estimates of the coefficients in linear regression models and improves the performance of statistical estimation in general. In practice, in orthogonal sampling, the sample space is divided into equally probable subspaces. All sample points in the orthogonal LH scheme are then chosen simultaneously making sure that the total ensemble of sample points is a Latin Hypercube and that each subspace is sampled with the same density.

Table 7 Design of Experiment (DOE) for simulations of Sect. 4

Table 8 Design of Experiment (DOE) for simulations of Sect. 5

The NOLH scheme of Cioppa and Lucas (2007) improves the space filling properties of the resulting sample when $k \lessapprox n$ at the cost of introducing a small maximal correlation of 0.03 between the columns of X. Furthermore, no assumptions regarding the homoskedasticity of errors or the shape of the response surface (like linearity) are required to obtain this scheme. The values of $\theta = (r_{cb},\delta ,\mu )$ obtained from this scheme and used for simulations of Sect. 4 are reported in Table 7, while those employed for simulations of Sect. 5 are reported in Table 8.

Kriging regression

In the metamodeling selection exercise of Sect. 4, we estimate various Kriging models. These are generalized regression models, potentially allowing for heteroskedastic and correlated errors. The approach is widely used for ABM metamodeling in various fields (see e.g. Salle and Yildizoglu 2014; Dancik et al. 2010 and references therein). Using generalized regression is convenient since some of the parameters of our model are related to random distributions which naturally affect the variability of model output. The Kriging approach (Roustant et al. 2012) resorts to feasible generalized least squares by assuming a stationary correlation kernel $K(h) = K(\theta _i-\theta _j)$, where $\theta _i,\theta _j$ are distinct points in the parameter space $\Theta$. K(h) takes the following general form:

$$\begin{aligned} K(h) = \prod _{j=1}^{d}g(h_j,\lambda _j) \end{aligned}$$

(B.1)

where d is the dimension of $\Theta$, and $\lambda = (\lambda _1,\dots ,\lambda _d)$ is a vector of parameters to be determined. In particular, we employ for g the specifications of Table 9.

Table 9 Correlation kernels (Roustant et al. 2012)

Since we work with noisy, potentially heteroskedastic observations, in our estimation the covariance matrix of residuals is determined as follows:

$$\begin{aligned} C = \sigma ^2 \, R + \text {diag}(\tau ) \end{aligned}$$

(B.2)

where R is the correlation matrix with elements $R_{ij} = K(\theta _i-\theta _j)$ and $\tau = (\tau _1^2,\dots ,\tau _n^2)$ is the vector containing the observed variance of model output at fixed points of the parameter space and n is the size of the NOLH design. ML estimation is performed on the “concentrated” multivariate Gaussian log-likelihood, obtained by substituting the vector of regression coefficients with their generalized least square estimator. The “concentrated” log-likelihood is a function of $\sigma$ and $\lambda$, which are the optimization variables of the estimation. The solution is obtained numerically through the quasi-Newton algorithm provided by the DiceKriging R Core Team (2015) package (Roustant et al. 2012).

Sensitivity analysis

Campolongo et al. (2000) define sensitivity analysis (SA) as the study of how uncertainty in the output of a model can be apportioned to different sources of uncertainty in the model input. In this respect, SA techniques should satisfy the two main requirements of being global and model free. By global, one means that SA must take into consideration the entire joint distribution of parameters. Global methods are opposed to local methods, which take into consideration the variation of one parameter at a time, e.g. by computing marginal effects of each parameter. By model independent, one means that no assumptions on the model functional relationship with its inputs, such as linearity, are required.

Campolongo et al. (2000) propose a global approach based on the decomposition of variance:

$$\begin{aligned} V(h)&= \sum _i^k V_i + \sum _{i<j} V_{ij} + \sum _{i<j<m} V_{ijm} + \dots + V_{12\dots k} \\ V_i&= \mathbb {V}_{\theta _{i}}\left[ \mathbb {E}_{\theta _{-i}}\left( h|\theta _i = x \right) \right] \\ V_{ij}&= \mathbb {V}_{\theta _{(i,j)}}\left[ \mathbb {E}_{\theta _{-(i,j)}}\left( h|\theta _i = x, \theta _j = y \right) \right] - V_{i} - V_{j}\\ \dots \end{aligned}$$

where h is a generic vector of moments. We see that $V_i$ represents the variance of the main effect of parameter i, while all the other terms are related to interaction effects. From this general formula we can obtain the contribution of interaction effects $S_{Ii}$ involving the parameter $\theta _i$ as follows:

$$\begin{aligned} S_{Ii}&= S_{Ti} - S_i \end{aligned}$$

(C.1)

$$\begin{aligned} S_i&= \frac{V_i}{V}\end{aligned}$$

(C.2)

$$\begin{aligned} S_{Ti}&= \frac{\mathbb {E}_{\theta _{-i}}\left[ \mathbb {V}_{\theta _{i}}(h|\theta _{-i})\right] }{V} = 1 - \frac{\mathbb {V}_{\theta _{-i}}\left[ \mathbb {E}_{\theta _i}(h|\theta _{-i})\right] }{V} = 1 - \frac{V_{-i}}{V} \end{aligned}$$

(C.3)

The multidimensional integral of the last line can be evaluated numerically using the extended FAST method described in Campolongo et al. (2000). The results of Fig. 5 show, for each parameter in $\theta$, the main effect (C.2) and the interaction effect (C.1) on the components of $h = (m,v,fb)$.