A Monte Carlo Study of Time Varying Coefficient (TVC) Estimation

A number of recent papers have proposed a time-varying-coefficient (TVC) procedure that, in theory, yields consistent parameter estimates in the presence of measurement errors, omitted variables, incorrect functional forms, and simultaneity. The key element of the procedure is the selection of a set of driver variables. With an ideal driver set the procedure is both consistent and efficient. However, in practice it is not possible to know if a perfect driver set exists. We construct a number of Monte Carlo experiments to examine the performance of the methodology under (i) clearly-defined conditions and (ii) a range of model misspecifications. We also propose a new Bayesian search technique for the set of driver variables underlying the TVC methodology. Experiments are performed to allow for incorrectly specified functional form, omitted variables, measurement errors, unknown nonlinearity and endogeneity. In all cases except the last, the technique works well in reasonably small samples.

variables, measurement error and an unknown true functional form. 1 There have also been a reasonably-large number of successful applications of the technique. 2 However, it is difficult to establish the usefulness of a technique strictly through applications since we can never be certain of the accuracy of the results. This paper attempts to bridge the gap between the asymptotic theoretical results of the theoretical papers and the apparently good performance of the applied papers by constructing a set of Monte Carlo experiments to examine (1) how well the technique performs under clearlydefined conditions and (2) the limits on the technique's ability to perform successfully under a broad range of model misspecifications.
The technique is motivated by an important theorem that was first proved by Swamy and Mehta (1975) and has recently been confirmed by Granger (2008) who quoted a proof that he attributed to Hal White. This theorem states that any nonlinear function may be exactly represented by a linear relationship with time-varying parameters. The importance of this theorem is that it allows us to capture an unknown true functional form in this framework. The parameters of this time-varying-coefficient model are, of course, not consistent estimates of the true functional form since they will be contaminated by the usual biases due to omitted variables, measurement error and simultaneity. The technique being investigated here allows us, in principal, to decompose the TVCs into two components; we associate the first component with the true nonlinear structure, which we interpret as the derivative of the dependent variable with respect to each of the independent variables in the unknown, nonlinear, true function; we associate the second component with the biases emanating from misspecification, and which we then remove from the TVC to give us our consistent estimates. Potentially, this technique offers an interesting way forward in dealing with model misspecification. It has generally been applied in a time series setting but it can equally well be interpreted as a cross section 3 or panel estimation technique.
The remainder of this paper is structured as follows. Section 2 outlines the basic (TVC) theoretical framework. Section 3 discusses some computational issues associated with estimating the model. Section 4 reports on a series of Monte Carlo experiments. Section 5 concludes. An "Appendix" provides details on the computational methods used in the Monte Carlo simulations.

The Theoretical Framework
We follow Swamy et al. (2010) who set the groundwork for uncovering causal economic laws. We assume 1 Swamy et al. (2003Swamy et al. ( , 2010Swamy et al. ( , 2012Swamy et al. ( , 2015. 2 Empirical applications include Hall et al. (2009Hall et al. ( , 2010Hall et al. ( , 2017, Tavlas et al. (2013), Hondroyiannis et al. (2013), Kenjegaliev et al. (2013). 3 Time varying coefficients are meaningless in a cross section setting; in such a setting the coefficients vary across the individual units in the cross section. We simply re-interpret the t-subscript as i-subscripts.
where x * t , e * t are the true determinants of y t . Alternatively we can represent this relationship by We have the auxiliary equations: Substituting (3) into (2) gives: To deal with errors in variables we assume: x t x * t + w t .
Substituting into (4), we obtain: where β 0t β t β 0t , β xt and D wt , D xt are diagonal containing w t and x t , respectively, along the diagonal. Finally, we assume there exists a vector z t of drivers such that where z et 1, z t . Under the assumption: It is straightforward to obtain the following In matrix notation we have a D x , and D x diag x e1 , . . . , x eT . A restrictive version of (7) is where β 0t is redefined as α 0t + v t and β t is independent of α 2t v * t u t and (8) as where the first row of r post-multiplied by z et does not contain the mean of α 2t v * t , and the first element of ε rt is independent of α 2t v * t . The error vectors ε t and ε rt are introduced in (8) and (13). Substituting we have Defining z et ⊗ x et X t , and π r vec( r ), we have: By assumption, E u t + x et ε rt 0 and var u t + x et ε rt σ 2 u + x et σ 2 r r x et .

Computational Aspects
Under a normality assumption in both u t and ε rt the likelihood function is where the parameter vector is θ π r , σ r , σ u , vech( r ) . Coupled with a prior, p(θ), by Bayes' theorem we get the kernel posterior distribution: We assume a standard non-informative prior: where d is the dimensionality of Σ r . The prior of π will be detailed below. Markov Chain Monte Carlo (MCMC) techniques can be used to obtain a sample {θ (s) , s 1, . . . , S} that converges in distribution to the posterior p(θ|Y). One efficient MCMC strategy is the following.
Assuming that different non-zero elements of C r are c 1 , . . . , c p the new parameter vector is π r and c [c 0 , c 00 , c 1 , . . . , c p ] ∈ R p+2 . Drawings from the conditional posterior distribution of c|π r , Y can be realized using the Girolami and Calderhead (2011) Metropolis Adjusted Langevin Diffusion method described in the "Appendix".
Selecting the drivers Suppose we have (12) and instead of (13) we have where z (m) t is a potential set of drivers (subset) from a universe Z {z t1 , . . . , z tG }. Equations (12) and (20) define different models indexed by m. As searching through all possible combinations of variables in Z is infeasible, we follow the Stochastic Search Variable Selection (SSVS) approach of George et al. (2008). 4 The SSVS involves a specific prior of the form: where δ is a vector of unknown parameters and its elements can be δ j ∈ {0, 1} . Also D diag d 2 1 , . . . , d G 2 : The prior implies a mixture of two normals: If κ 0j is "small" and κ 1j is "large", then, when δ j 0 chances are that variable j will be excluded from the model while if δ j 1 chances are that variable j will be included in the model. The prior for the indicator parameter δ is: and we set q j 1 2 . For κ 0j and κ 1j , George et al. (2008) propose a semi-automatic procedure based on κ 2 0j c 0v π j and κ 2 1j c 1v π j for c 0 1 10 , c 1 10 andv π j is any preliminary estimate of the variance of π j .
For the elements of c we follow a similar approach. If c j corresponds to a diagonal element it is always included in the model. If not, we use a mixture-of-normals SSVS approach as above.

Monte Carlo results
In all cases below γ 0 γ 1 γ 2 .1. The number of Monte Carlo simulations is set to 10,000. All ε t j ∼ iid N (0, 1). In case IV, we set σ ε .3.

Model I: Incorrect Functional Form
The true model is y t γ 0 + γ 1 x t + 1 2 γ 2 x 2 t , t 1, . . . , T and we have omitted the nonlinear term. The driver is z t αx t +ε t . We have ε t ∼ iidN(0,1) and x t ∼ iidN(1,1). In this case the correlation between z t and x 2 t is ρ .
If the correlation were equal to 1, then this would be a perfect driver as it exactly recreates the missing quadratic term. The estimation procedure would then be unbiased and efficient. If the correlation were zero, then z t would contain no information about the missing nonlinearity. We are, therefore, interested in varying this correlation and seeing how low the correlation can fall before the estimator ceases to be useful.
In this case the true effect is γ 1 + γ 2 x t , that is, the derivative of y with respect to x. There are no omitted variables or other misspecifications other than the nonlinearity so the set S 2 is empty and the estimate of the derivative is given by γ 1 + γ 2 x t β 1 − ε t . Table 1 gives the results of this set of Monte Carlo experiments for sample sizes of 50, 100, 200 and 1000. When the correlation between z and x is very high, then even for small samples the bias is very small and the standard deviation of the results is also small at around 1%. As the sample size grows, both the bias and the standard deviation fall, and the estimator is clearly consistent and efficient. As we look across the table, where the correlation between the driver and the true x variable falls the estimation procedure still does very well until the correlation falls to about .5; at that point the bias and the standard error begins to rise quite substantially. This happens even more clearly with the very large sample size of T 1000 where both the bias and standard error are very small until the correlation falls below .5.

Model II: Omitted Variables
The second model focuses on omitted variables. The true model is y t γ 0 + γ 1 x t1 + γ 2 x t2 . The x t1 , x t2 are correlated: x t2 γx t1 + ξ t , ξ t , x t1 ∼ iidN(0,1). The squared correlation between the two variables is ρ 2 12 γ 2 γ 2 +1 . We set γ 2 so that this is .80. .520 Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, bias is the percent absolute bias; SD is the standard deviation of the bias We estimate the TVC model y t β 0t + β 1t x t1 and again use a driver z t αx t2 + ε t and we see how well the estimator performs as the correlation between z t and x t2 falls. The correlation between z t and x t2 is ρ α √ α 2 +1 . In this case, the true effect is γ 1 and the bias free estimate is β 1t − π 1 z t − e t . The results of this experiment are given in Table 2. The results show a similar picture to case 1 above. Both the bias and the standard deviation clearly decrease as the sample size increases. Even for the smallest sample size both the bias and the standard deviation are quite small while the correlation between the driver and the misspecification is above .5. Again, as the correlation falls below .5 the bias and standard deviation rise quite quickly.

Model III: Measurement Error
The third model deals with measurement error, so we generate data from y t γ 0 +γ 1 x t then create y * t y t + ε t1 and x * t x t + ε t2 then we estimate the TVC model y * t β 0 t + β 1t x * t and use two z's as drivers z t1 α 1 ε t1 + ε t3 and z t2 α 2 ε t2 + ε t4 and again see how things change as α gets bigger.
The results of this experiment are reported in Table 3. The results are entirely consistent with the results in the earlier two cases. The technique is clearly consistent, as the sample rises the bias falls considerably. Even for a small sample the bias is quite low for correlations between the driver and the measurement error which is .5 or above.

Detecting Irrelevant Drivers
Next, we examine whether the SSVS 5 procedure, which we have not applied so far, can correctly identify the drivers z t1 , z t2 . To this end, we construct ten other drivers, say z t2 , . . . , z t, 12 from a multivariate normal distribution with zero means and equal correlations of .70. In Table 4 we report the equivalent of Table 3 plus the proportion of cases, say * , in which SSVS has correctly excluded z t2 , . . . , z t,12 from the set of possible drivers. 6 There is again a remarkable cut off at the correlation level of .5. Above this level the true driver set is correctly identified in around 60% of cases and for the largest sample in over 90% of cases, even for small samples. Once the correlation falls below .5, however, the proportion of correct identifications falls dramatically. An obvious conclusion here is that when we have drivers that are effective enough so that we 5 An alternative to using the SSVS procedure would be the LASSO prior. The procedures are similar in terms of timing and purpose. There is some evidence that both perform well (Pavlou et al. 2016) and in a similar manner but further work is needed in this area. 6 There is an issue here as to whether we need to start from a superset of drivers which includes the true ones. Clearly, if we do this then this is an ideal situation and the Monte Carlo tells us how well the procedure performs. However, from a theoretical point of view what we need is that the superset includes variables be highly correlated with the true drivers. In a data rich environment this would not be a strong restriction. Bai and Ng (2010) prove that common factors which drive all the variables in a system are valid instrumental variables. By the same reasoning, we could construct a set of factors from a large set of variables which would work well as drivers. .381 Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, bias is the percent absolute bias; SD is the standard deviation of the bias .255 Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, bias is the percent absolute bias; SD is the standard deviation of the bias .0% .0% .0% .0% Corr ρ is the degree of correlation between the true driver and the driver used. T is the sample size, bias is the percent absolute bias; SD is the standard deviation of the bias. * is the proportion of times the correct driver set is selected δ is the degree of missing nonlinearity given as in Sect. 4.5 above. T is the sample size. Bias is the percent absolute bias; SD is the standard deviation of the bias will get reasonably good parameter estimates, the SSVS algorithm is very effective at detecting them.

Model IV: A More Complex Nonlinearity
The true model is y t γ 0 + γ 1 x t + exp −δx 2 t + ε t , t 1,…,T and we have omitted the nonlinear term. The drivers form a Fourier basis cos(jx t ), sin(jx t ), j = 1, . . . J after transforming all series to lie in (− π, π). We have ε t ∼ iidN(0,1) and x t ∼ iidN(0,1) ordered from smallest to largest. The drivers, that is powers of x t are selected through the SSVS procedure. We set the maximum value of J to 10.
We again estimate the TVC model y t β 0 t + β 1t x t1 and this time the derivative of y with respect to x is γ 1 − 2δx t exp(−δx 2 t ). Our estimate of this is again given by β 1t − e t .
The results for this exercise are given in Table 5. In this case for δ in the range .1-5 the bias remains very small, as does the standard deviation. There is also a noticeable reduction in both bias and standard deviation as the sample size increases.

Model V: An Endogeneity Experiment
In this experiment we have: y t γ 1 + γ 2 x t1 + γ 3 x t2 + u t . The correlation between u t and x tj is .80 (j = 1, 2) so that endogeneity is quite strong in this model. Our drivers are four variables z t5 , . . . , z t8 orthogonal to the error u t and four drivers z t1 , . . . , z t4 which are correlated with the error u t but they are orthogonal to each other as well as orthogonal to the other four drivers. The degree of correlation between the drivers and u t is ρ. We are interested in * , the proportion of cases where all the drivers z t1 , . . . , Corr ρ is the degree of correlation between the true driver and the driver used. * is the proportion of cases where all the drivers z t1 , . . . , z t4 are included in the model and the drivers z t5 , . . . , z t8 are all excluded. T is the sample size. Bias is the percent absolute bias; SD is the standard deviation of the bias z t4 are included in the model and the drivers z t5 , . . . , z t8 are all excluded. Of course, we do not force the correct drivers in final estimation. The results for this experiment are given in Table 6. Here the table has some rather different results than the earlier tables. The bias remains quite high, even for quite high correlations for the sample size of 50 or 100. It is only for much larger sample sizes that the bias becomes negligible. For larger sample sizes the bias remains small again for correlations above .5 and the SSVS selection procedure works reasonably well.

Conclusions
This paper has investigated the performance of the TVC estimation procedure in a Monte Carlo setting. The key element of TVC estimation is the identification and selection of a set of driver variables. With an ideal driver set, it is straightforward to show that the procedure is both consistent and efficient. However, in practice it is not possible to know if we have a perfect driver set. Therefore, we need to know how the dB i (t) |G(θ(t))| −1/2 d