AMonte Carlo Study of Time Varying Coefficient ( TVC ) Estimation

A number of recent papers have proposed a time-varying-coefficient (TVC) procedure that, in theory, yields consistent parameter estimates in the presence of measurement errors, omitted variables, incorrect functional forms, and simultaneity. The key element of the procedure is the selection of a set of driver variables. With an ideal driver set the procedure is both consistent and efficient. However, in practice it is not possible to know if a perfect driver set exists. We construct a number of Monte Carlo experiments to examine the performance of the methodology under (i) clearly-defined conditions and (ii) a range of model misspecifications. We also propose a new Bayesian search technique for the set of driver variables underlying theTVCmethodology. Experiments are performed to allow for incorrectly specified functional form, omitted variables, measurement errors, unknown nonlinearity and endogeneity. In all cases except the last, the technique works well in reasonably small samples.


Introduction
A series of papers have proposed the use of time-varying coefficient (TVC) models to uncover the bias-free estimates of a set of model coefficients in the presence of omitted variables, measurement error and an unknown true functional form. 1 There have also been a reasonably-large number of successful applications of the technique. 2 However, it is difficult to establish the usefulness of a technique strictly through applications since we can never be certain of the accuracy of the results. This paper attempts to bridge the gap between the asymptotic theoretical results of the theoretical papers and the apparently good The technique is motivated by an important theorem that was first proved by Swamy and Mehta (1975) and has recently been confirmed by Granger (2008) who quoted a proof that he attributed to Hal White. This theorem states that any nonlinear function may be exactly represented by a linear relationship with timevarying parameters. The importance of this theorem is that it allows us to capture an unknown true functional form in this framework. The parameters of this timevarying-coefficient model are, of course, not consistent estimates of the true functional form since they will be contaminated by the usual biases due to omitted variables, measurement error and simultaneity. The technique being investigated here allows us, in principal, to decompose the TVCs into two components; we associate the first component with the true nonlinear structure, which we interpret as the derivative of the dependent variable with respect to each of the independent variables in the unknown, nonlinear, true function; we associate the second component with the biases emanating from misspecification, and which we then 1 Swamy, Chang, Mehta and Tavlas (2003), Swamy, Tavlas, Hall and Hondroyiannis (2010), Swamy, Hall and Tavlas (2012), Swamy, Hall and Tavlas, (2014), Swamy, Mehta, Tavlas and Hall (2014). 2 Empirical applications include Hall, Hondroyiannis, Swamy, and Tavlas (2009), Hall, Hondroyiannis, Swamy and Tavlas (2010), Tavlas, Swamy, Hall and Kenjegaliev (2013), Hondroyiannis, Kenjegaliev, Hall, Swamy and Tavlas (2013), Kenjegaliev, Hall, Tavlas and Swamy (2013), Hall, Swamy, and Tavlas (2017). 1 remove from the TVC to give us our consistent estimates. Potentially, this technique offers an interesting way forward in dealing with model misspecification. It has generally been applied in a time series setting but it can equally well be interpreted as a cross section 3 or panel estimation technique.
The remainder of this paper is structured as follows. Section 2 outlines the basic (TVC) theoretical framework. Section 3 discusses some computational issues associated with estimating the model. Section 4 reports on a series of Monte Carlo experiments. Section 5 concludes. An Appendix provides details on the computational methods used in the Monte Carlo simulations.

The Theoretical Framework
We follow Swamy, Tavlas, Hall and Hondroyiannis (2010) who set the groundwork for uncovering causal economic laws. We assume where * * t t , x e are the true determinants of t y . Alternatively we can represent this relationship by We have the auxiliary equations: Substituting (3) into (2) gives: To deal with errors in variables we assume: Substituting into (4), we obtain: where It is straightforward to obtain the following In matrix notation we have where 0t β is redefined as α +  (8) By assumption, ( )

Computational aspects
Under a normality assumption in both t u and t r ε the likelihood function is where the parameter vector is Coupled with a prior, ( ) p θ , by Bayes' theorem we get the kernel posterior distribution: We assume a standard non-informative prior: where d is the dimensionality of r Σ . The prior of π will be detailed below.
Markov Chain Monte Carlo (MCMC) techniques can be used to obtain a sample (s) { ,s 1,...,S} θ = that converges in distribution to the posterior p( | Y) θ . One efficient MCMC strategy is the following.
Selecting the drivers Suppose we have (12) and instead of (13) we have where (m) t z is a potential set of drivers (subset) from a universe Equations (12) and (20) where δ is a vector of unknown parameters and its elements can be j {0,1} δ ∈ .
The prior implies a mixture of two normals: If 0j κ is "small" and 1j κ is "large", then, when j 0 δ = chances are that variable j will be excluded from the model while if j 1 δ = chances are that variable j will be included in the model. The prior for the indicator parameter δ is: and we set 1 j 2 q = . For 0j κ and 1j κ , George, Sun and Ni (2008)  For the elements of c we follow a similar approach. If j c corresponds to a diagonal element it is always included in the model. If not, we use a mixture-ofnormals SSVS approach as above.

Model I: Incorrect Functional Form
The true model is = γ + γ + γ , =, , x x t 1 … T and we have omitted the nonlinear term. The driver is We have t~i idN(0,1) ε 6 and t x~iidN(1,1) . In this case the correlation between t z and 2 t x is If the correlation were equal to 1, then this would be a perfect driver as it exactly recreates the missing quadratic term. The estimation procedure would then be unbiased and efficient. If the correlation were zero, then zt would contain no information about the missing nonlinearity. We are, therefore, interested in varying this correlation and seeing how low the correlation can fall before the estimator ceases to be useful.
In this case the true effect is γ + γ 1 2 t x , that is, the derivative of y with respect to x . There are no omitted variables or other misspecifications other than the nonlinearity so the set S2 is empty and the estimate of the derivative is given x . Table 1 gives the results of this set of Monte Carlo experiments for sample sizes of 50, 100, 200 and 1000. When the correlation between z and x is very high, then even for small samples the bias is very small and the standard deviation of the results is also small at around 1%. As the sample size grows, both the bias and the standard deviation fall, and the estimator is clearly consistent and efficient. As we look across the table, where the correlation between the driver and the true x variable falls the estimation procedure still does very well until the correlation falls to about 0.5; at that point the bias and the standard error begins to rise quite substantially. This happens even more clearly with the very large sample size of T=1000 where both the bias and standard error are very small until the correlation falls below 0.5.
We estimate the TVC model = β + β t 0t 1t t1 y x and again use a driver = α + ε t t2 t z x and we see how well the estimator performs as the correlation between t z and t2 x falls. The correlation between t z and t2 x is α ρ = α + 2 1 .
In this case, the true effect is γ 1 and the bias free estimate is β − π − 1t 1 t t z e .
The results of this experiment are given in Table 2. The results show a similar picture to case 1 above. Both the bias and the standard deviation clearly decrease as the sample size increases. Even for the smallest sample size both the bias and the standard deviation are quite small while the correlation between the driver and the misspecification is above 0.5. Again, as the correlation falls below 0.5 the bias and standard deviation rise quite quickly.

Model III: Measurement Error
The third model deals with measurement error, so we generate data from = γ + γ The results of this experiment are reported in Table 3. The results are entirely consistent with the results in the earlier two cases. The technique is clearly consistent, as the sample rises the bias falls considerably. Even for a small sample the bias is quite low for correlations between the driver and the measurement error which is 0.5 or above.

Detecting Irrelevant Drivers
8 Next, we examine whether the SSVS 5 procedure, which we have not applied so far, can correctly identify the drivers t1 t2 z ,z . To this end, we construct ten other drivers, say t2 t ,12 z ,...,z from a multivariate normal distribution with zero means and equal correlations of 0.70. In Table 4 we report the equivalent of Table 3 plus the proportion of cases, say Π * , in which SSVS has correctly excluded t2 t ,12 z ,...,z from the set of possible drivers 6 .
There is again a remarkable cut off at the correlation level of 0.5. Above this level the true driver set is correctly identified in around 60% of cases and for the largest sample in over 90% of cases, even for small samples. Once the correlation falls below 0.5, however, the proportion of correct identifications falls dramatically. An obvious conclusion here is that when we have drivers that are effective enough so that we will get reasonably good parameter estimates, the SSVS algorithm is very effective at detecting them.

Model IV: A More Complex Nonlinearity
The true model is ( ) , −π π . We have t~i idN(0,1) ε and t x~iidN(0,1) ordered from smallest to largest. The drivers, that is powers of t x are selected through the SSVS procedure. We set the maximum value of J to 10.
The results for this exercise are given in Table 5. In this case for δ in the range 0.1 to 5 the bias remains very small, as does the standard deviation. There is also a noticeable reduction in both bias and standard deviation as the sample size increases.

Model V: An Endogeneity Experiment
In this experiment we have: t 1 2 t1 3 t2 t y x x u = γ + γ + γ + . The correlation between t u and tj x is 0.80 ( 1, 2 j = ) so that endogeneity is quite strong in this model. The results for this experiment are given in Table 6. Here the table has some rather different results than the earlier tables. The bias remains quite high, even for quite high correlations for the sample size of 50 or 100. It is only for much larger sample sizes that the bias becomes negligible. For larger sample sizes the bias remains small again for correlations above 0.5 and the SSVS selection procedure works reasonably well.

Conclusions
This paper has investigated the performance of the TVC estimation procedure in a Monte Carlo setting. The key element of TVC estimation is the identification and selection of a set of driver variables. With an ideal driver set, it is straightforward to show that the procedure is both consistent and efficient.
However, in practice it is not possible to know if we have a perfect driver set.
Therefore, we need to know how the procedure performs when the driver set is less than perfect. In this paper, we dealt with this issue in a Monte Carlo setting.
We construct a number of Monte Carlo experiments to examine the performance of the methodology under (i) clearly-defined conditions and (ii) a range of model misspecifications. We also propose a new Bayesian search technique for the set of driver variables underlying the TVC methodology. Experiments are performed to allow for incorrectly specified functional form, omitted variables, measurement errors, unknown nonlinearity and endogeneity. Our broad conclusion is that, even for relatively small samples, the technique works well so long as the correlation between the driver set and the misspecification in the model is greater than about 0.5. Both the bias and the efficiency of the estimators also improve as the sample size grows, but again a correlation of over 0.5 seems to be required. The only caveat to this result is that if we are considering strong simultaneity bias; in that case the sample size needs to be quite large (over 500) before the technique works reasonably well. Finally, we find that the SSVS technique also seems to perform well in finding an appropriate driver set from a much larger set of possible drivers.

APPENDIX
This Appendix provides details on the computational methods used in the Monte Carlo simulations. Following Girolami and Calderhead (2011) where B denotes the D-dimensional Brownian motion. The first-order Euler discretization provides the following candidate generation mechanism: ( ) Here Y denotes the available data. The Brownian motion of the Riemann manifold is given by: The discrete form of the above stochastic differential equations is: , .
and the acceptance probability has the standard Metropolis form: The gradient and the Hessian are computed using analytic derivatives. All computations are performed in Fortran 77 making extensive use of IMSL subroutines.
The Metropolis-Hastings procedure we use is a simple random walk whose candidate generating density is a multivariate Student-t distribution with 5 degrees of freedom 7 and covariance equal to a scaled version of the covariance obtained from the Langevin Diffusion MCMC. The scale parameter is adjusted so that approximately 25% of the draws are accepted.
7 This guarantees the existence of moments up to order four. Notes: Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, Bias is the percent absolute bias; s.d. is the standard deviation of the bias. Notes: Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, Bias is the percent absolute bias; s.d. is the standard deviation of the bias. Notes: Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, Bias is the percent absolute bias; s.d. is the standard deviation of the bias. Notes: Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, Bias is the percent absolute bias; s.d. is the standard deviation of the bias. * Π is the proportion of times the correct driver set is selected Notes: δ is the degree of missing nonlinearity given as in Section 4.5 above. T is the sample size. Bias is the percent absolute bias; s.d. is the standard deviation of the bias.