Testing for spatial error dependence in probit models
Abstract
In this note, we compare three test statistics that have been suggested to assess the presence of spatial error autocorrelation in probit models. We highlight the differences between the tests proposed by Pinkse and Slade (J Econom 85(1):125–254, 1998), Pinkse (Asymptotics of the Moran test and a test for spatial correlation in Probit models, 1999; Advances in Spatial Econometrics, 2004) and Kelejian and Prucha (J Econom 104(2):219–257, 2001), and compare their properties in a extensive set of Monte Carlo simulation experiments both under the null and under the alternative. We also assess the conjecture by Pinkse (Asymptotics of the Moran test and a test for spatial correlation in Probit models, 1999) that the usefulness of these test statistics is limited when the explanatory variables are spatially correlated. The Kelejian and Prucha (J Econom 104(2):219–257, 2001) generalized Moran’s I statistic turns out to perform best, even in medium sized samples of several hundreds of observations. The other two tests are acceptable in very large samples.
Keywords
Spatial econometrics Spatial probit Moran’s IJEL Classification
C21 C251 Introduction
In contrast to the situation in the standard linear model, relatively little is known about tests against spatial error autocorrelation in specifications with limited dependent variables, such as a probit model. Three different test statistics have been proposed in the literature, respectively by Pinkse and Slade (1998), Pinkse (1999); Pinkse (2004) and Kelejian and Prucha (2001). Even though they have the same asymptotic distribution, they tend to yield different results in empirical practice.^{1}
Very few systematic findings are available regarding the relative performance of these tests in finite samples. The only small sample results to date are contained in a limited simulation experiment in Novo (2001) and in Amaral and Anselin (2011). The results in Novo (2001) are based on a small number of replications (2,000) and, therefore, suffer from a lack of precision in the reported rejection frequencies. Also, his experiment was limited to sample sizes up to \(N = 225\). Amaral and Anselin (2011) only considered one test, the Kelejian and Prucha (2001) generalized Moran’s I test, but for both probit and tobit models.
As is well known, ignoring spatial error autocorrelation in a probit model has more serious consequences than in the standard linear regression. In the latter, the main problem is one of lack of efficiency (e.g., Anselin 1988). However, in a probit model, ignoring spatially autocorrelated errors results not only in inefficiency but also inconsistency for the standard maximum likelihood estimator (e.g., Fleming 2004). In part, this is due to the fact that most spatial processes for spatial error autocorrelation are also heteroskedastic Anselin (2006). A better understanding of the properties of test statistics developed to detect this form of misspecification is therefore not only of theoretical interest, but also has ramifications for empirical practice.
In this note, we extend the study in Amaral and Anselin (2011) to a comparative assessment of the three test statistics in the probit model. We highlight the similarities and differences between the three test statistics and carry out a series of Monte Carlo simulation experiments to compare their size and power. We also assess a conjecture formulated in Pinkse (1999), where it was suggested that the tests may have limited usefulness in the presence of spatially correlated explanatory variables. As stated in the beginning of this introduction, we consider only spatial autocorrelation in standard probit models, not models with a spatial lag.
2 Model and test statistics
2.1 Probit and spatial probit
2.2 Residuals
Unlike the standard linear model, there is no unambiguous estimate for the residual in the probit model, since the true residual vector \(\mathbf y^* -\mathbf X \hat{\beta }\) is unobserved. Instead, the estimated residual needs to be based on the difference between the observed \(y_i\) and a predicted value \(\hat{\Phi }_i\), the cumulative standard normal distribution evaluated at \(\mathbf x^{\prime } _i\hat{\beta }\).
2.3 Test statistics
3 Design of the experiments
We consider the performance of the three test statistics in a series of Monte Carlo simulation experiments in which we manipulate sample size and the value of the spatial autoregressive parameter. We use seven different sample sizes, each consisting of a regular grid, ranging from \(7\times 7\) grid cells (N\(=\) 49) to \(625\times 625\) cells (N\(=\) 390,625), with \(N={49, 100, 225, 625, 2{,}500, 15{,}625, 390{,}625}\). We use thirteen different values for the spatial autoregressive parameter, with \(\lambda =\{-0.8,-0.5,-0.3,-0.1,-0.05,-0.01,0.0,0.01,0.05,0.1,0.3,0.5,0.8\}\). \(\mathbf W \) is defined considering all neighbouring regions according to rook contiguity.
Each experiment consists of 10,000 replications.^{3} A nominal Type I error of 0.05 is used throughout, which leads to an associated sample standard deviation in each simulation run of \(\sqrt{0.05 \times 0.95/10{,}000}=0.0022\). In other words, rejection frequencies within the range [0.0478 – 0.0522] are within one standard deviation of the it true value of 0.05, and frequencies within the range [0.0456–0.0544] are within two standard deviations of the true value.
4 Relative performance of the test statistics
4.1 Size and distribution under the null hypothesis
Size of tests
\(N\) | \({MI}\) | \({LM_{PS}}\) | \({LM_P}\) |
---|---|---|---|
49 | 0.0473 | 0.0153 | 0.0355 |
100 | 0.0476 | 0.0295 | 0.0437 |
225 | 0.0434 | 0.0389 | 0.0427 |
625 | 0.0484 | 0.0470 | 0.0514 |
2,500 | 0.0478 | 0.0487 | 0.0470 |
15,625 | 0.0483 | 0.0498 | 0.0495 |
390,625 | 0.0490 | 0.0498 | 0.0502 |
We assess the extent to which the test statistics obtain their asymptotic distribution by means of a Kolmogorov–Smirnov test. We take the null hypothesis to be a \(\chi ^2(1)\) distribution. To make all test statistics comparable, we use the square of \(MI\) or \(I^2\) from Eq. 15. Also, it should be noted that (Pinkse and Slade (1998), p. 131) did not derive an asymptotic distribution for the \(LM_{PS}\) statistic, but instead proposed a bootstrap procedure. For the purposes of this exercise, we compare its distribution to a \(\chi ^2(1)\).
Kolmogorov–Smirnov test against \(\chi ^2(1)\) distribution
\(N\) | \(I^2\) | \(LM_{PS}\) | \(LM_P\) |
---|---|---|---|
49 | 0.0139 | 0.1825 | 0.0383 |
(0.289) | (0.000) | (0.000) | |
100 | 0.0136 | 0.1047 | 0.0178 |
(0.313) | (0.000) | (0.084) | |
225 | 0.0158 | 0.0596 | 0.0188 |
(0.165) | (0.000) | (0.058) | |
625 | 0.0098 | 0.0313 | 0.0102 |
(0.723) | (0.000) | (0.676) | |
2,500 | 0.0134 | 0.0244 | 0.0139 |
(0.331) | (0.005) | (0.289) | |
15,625 | 0.0087 | 0.0096 | 0.0100 |
(0.844) | (0.746) | (0.699) | |
390,625 | 0.0119 | 0.0157 | 0.0117 |
(0.478) | (0.170) | (0.500) |
4.2 Power of the test statistics
Rejection frequency: spatial autoregressive error
\(N\) | \(\lambda \) | Sp. Error | Sp. Error (adjusted) | ||||
---|---|---|---|---|---|---|---|
\(MI\) | \(LM_{PS}\) | \(LM_P\) | \(MI\) | \(LM_{PS}\) | \(LM_P\) | ||
49 | 0.01 | 0.047 | 0.014 | 0.034 | 0.050 | 0.049 | 0.051 |
0.05 | 0.045 | 0.014 | 0.031 | 0.046 | 0.046 | 0.046 | |
0.1 | 0.043 | 0.015 | 0.03 | 0.046 | 0.043 | 0.045 | |
0.3 | 0.064 | 0.022 | 0.047 | 0.067 | 0.068 | 0.067 | |
0.5 | 0.137 | 0.070 | 0.115 | 0.142 | 0.156 | 0.147 | |
0.8 | 0.526 | 0.430 | 0.505 | 0.534 | 0.569 | 0.551 | |
100 | 0.01 | 0.049 | 0.028 | 0.045 | 0.051 | 0.050 | 0.053 |
0.05 | 0.051 | 0.028 | 0.047 | 0.053 | 0.048 | 0.055 | |
0.1 | 0.048 | 0.029 | 0.045 | 0.050 | 0.051 | 0.053 | |
0.3 | 0.109 | 0.065 | 0.101 | 0.112 | 0.100 | 0.113 | |
0.5 | 0.296 | 0.220 | 0.293 | 0.301 | 0.294 | 0.315 | |
0.8 | 0.882 | 0.837 | 0.883 | 0.884 | 0.883 | 0.891 | |
225 | 0.01 | 0.049 | 0.042 | 0.049 | 0.054 | 0.053 | 0.054 |
0.05 | 0.051 | 0.040 | 0.048 | 0.057 | 0.051 | 0.054 | |
0.1 | 0.060 | 0.047 | 0.057 | 0.068 | 0.059 | 0.064 | |
0.3 | 0.207 | 0.166 | 0.208 | 0.220 | 0.195 | 0.224 | |
0.5 | 0.604 | 0.514 | 0.605 | 0.619 | 0.558 | 0.624 | |
0.8 | 0.997 | 0.995 | 0.997 | 0.997 | 0.997 | 0.998 | |
625 | 0.01 | 0.051 | 0.044 | 0.050 | 0.052 | 0.047 | 0.048 |
0.05 | 0.056 | 0.054 | 0.058 | 0.057 | 0.056 | 0.057 | |
0.1 | 0.082 | 0.074 | 0.084 | 0.084 | 0.078 | 0.083 | |
0.3 | 0.467 | 0.384 | 0.480 | 0.470 | 0.397 | 0.475 | |
0.5 | 0.943 | 0.904 | 0.949 | 0.945 | 0.909 | 0.948 | |
0.8 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
2,500 | 0.01 | 0.050 | 0.051 | 0.050 | 0.052 | 0.052 | 0.053 |
0.05 | 0.090 | 0.082 | 0.093 | 0.093 | 0.084 | 0.096 | |
0.1 | 0.237 | 0.206 | 0.241 | 0.242 | 0.210 | 0.247 | |
0.3 | 0.976 | 0.957 | 0.980 | 0.977 | 0.958 | 0.981 | |
0.5 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.8 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
15,625 | 0.01 | 0.062 | 0.059 | 0.063 | 0.065 | 0.059 | 0.064 |
0.05 | 0.340 | 0.293 | 0.345 | 0.346 | 0.295 | 0.347 | |
0.1 | 0.871 | 0.816 | 0.881 | 0.873 | 0.816 | 0.881 | |
0.3 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.5 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.8 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
390,625 | 0.01 | 0.337 | 0.283 | 0.337 | 0.340 | 0.283 | 0.337 |
0.05 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.1 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.3 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.5 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
0.8 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
4.3 Spatial autocorrelation in the explanatory variables
Rejection frequency: spatially correlated regressors
\(N\) | \(\gamma \) | AR(X) | MA(X) | ||||
---|---|---|---|---|---|---|---|
\(MI\) | \(LM_{PS}\) | \(LM_P\) | \(MI\) | \(LM_{PS}\) | \(LM_P\) | ||
625 | \(-\)0.8 | 0.0513 | 0.0435 | 0.0575 | 0.0493 | 0.0451 | 0.0509 |
\(-\)0.3 | 0.0548 | 0.0513 | 0.0508 | 0.0532 | 0.0442 | 0.0490 | |
0 | 0.0484 | 0.0470 | 0.0514 | 0.0484 | 0.0470 | 0.0514 | |
0.3 | 0.0482 | 0.0456 | 0.0463 | 0.0484 | 0.0474 | 0.0471 | |
0.8 | 0.0493 | 0.0388 | 0.0575 | 0.0505 | 0.0470 | 0.0527 | |
2,500 | \(-\)0.8 | 0.0488 | 0.0436 | 0.0569 | 0.0551 | 0.0491 | 0.0582 |
\(-\)0.3 | 0.0468 | 0.0505 | 0.0467 | 0.0503 | 0.0516 | 0.0513 | |
0 | 0.0478 | 0.0487 | 0.0470 | 0.0478 | 0.0487 | 0.0470 | |
0.3 | 0.0489 | 0.0489 | 0.0490 | 0.0532 | 0.0529 | 0.0545 | |
0.8 | 0.0481 | 0.0440 | 0.0582 | 0.0518 | 0.0473 | 0.0560 | |
15,625 | \(-\)0.8 | 0.0521 | 0.0519 | 0.0612 | 0.0471 | 0.0497 | 0.0514 |
\(-\)0.3 | 0.0496 | 0.0492 | 0.0493 | 0.0482 | 0.0492 | 0.0485 | |
0 | 0.0483 | 0.0498 | 0.0495 | 0.0483 | 0.0498 | 0.0495 | |
0.3 | 0.0521 | 0.0468 | 0.0509 | 0.0477 | 0.0498 | 0.0477 | |
0.8 | 0.0521 | 0.0499 | 0.0597 | 0.0500 | 0.0503 | 0.0538 | |
390,625 | \(-\)0.8 | 0.0495 | 0.0527 | 0.0568 | 0.0530 | 0.0503 | 0.0550 |
\(-\)0.3 | 0.0487 | 0.0477 | 0.0475 | 0.0490 | 0.0492 | 0.0493 | |
0 | 0.0490 | 0.0498 | 0.0502 | 0.0490 | 0.0498 | 0.0502 | |
0.3 | 0.0499 | 0.0473 | 0.0491 | 0.0539 | 0.0520 | 0.0541 | |
0.8 | 0.0491 | 0.0503 | 0.0571 | 0.0530 | 0.0537 | 0.0570 |
The rejection frequencies for \(LM_P\) indeed tend to be somewhat elevated, but only for very large values of the spatial parameter (\(| \gamma |=0.8\)). Interestingly, the effect on \(LM_{PS}\) works in the other direction, yielding a few cases of under-rejection. Overall, the \(MI\) statistic does not seem to be affected by spatial autocorrelation in the regressors, especially for \(N > 625\). The difference between the test statistics may be due to the way the residuals are calculated, since \(MI\) uses the “naive” residuals, whereas \(LM_P\) is based on the generalized Cox–Snell residuals, which involve \(\mathbf x \) in the weighting factor as well.
5 Conclusion
Our simulation experiments are the first systematic evaluation of the properties of the three tests proposed in the literature to assess spatial error autocorrelation in a probit model. They demonstrated that of the three tests, \(MI\) is overall the most reliable. It is unbiased across the widest range of sample sizes and achieves its asymptotic distribution under the null, even for \(N = 49\). The two other statistics also perform well under the null, but require larger sample sizes (\(N>~2{,}500\)) to obtain the \(\chi ^2(1)\) asymptotic distribution. All three test have good power against the alternative, especially in the larger sample sizes. Finally, the \(MI\) test is not affected by spatial correlation in the regressors, whereas there is a slight effect on the two other tests.
Footnotes
- 1.
For example, using the Columbus data from Anselin (1988) (N\(=\) 49), truncated in the same fashion as in McMillen (1992) and others, with y\(=\) 1 for crime\(>40\), yields values of respectively 2.48, 1.22 and 2.89 for the statistics in a probit estimation of crime on income and housing values. Whereas the first two values do not reject the null, the last statistic weakly rejects the null.
- 2.
Recall that the LM test statistic for spatial error autocorrelation in the standard linear regression takes the form \(LM=[(\mathbf e^{\prime } \mathbf W \mathbf e ) / \hat{\sigma }^2]^2 / tr(\mathbf{WW }+\mathbf{W^{\prime }W })\), where \(\mathbf e \) is a vector of OLS residuals, \(tr\) is a trace operator and \(\hat{\sigma }^2\) is the usual estimate of error variance ((Anselin 1988, p. 104)). Using the notation from Eq. 5, \(v=\hat{\sigma }^4 tr(\mathbf{WW }+\mathbf{W^{\prime }W })\).
- 3.
For all computations, we used PySAL, a Python library for spatial analysis Rey and Anselin (2007).
- 4.
All the experiments were also performed for irregular layouts and unbalanced samples. The results were similar to those for a balanced sample with regular lattice and were omitted from this paper.
- 5.
The spatial transformation induces a change in the mean and variance of the \(x\) variables. In order to ensure that the sample remains balanced and that the approximate \(R^2\) in the samples is comparable to the other samples, we carry out a transformation of the \(\mathbf x \) vector and adjust the variance of the error term \(\varepsilon \) such that the \(R^2 \approx 0.67\) in all settings. The transformation of the \(\mathbf x \) vector required to maintain a balanced sample is \(\mathbf x \sim -2(1-\gamma )^{-\theta }+U(-5,5)\).
- 6.
We take the 95th percentile of the distribution under the null obtained from our simulations as the “correct” critical value, rather than the value of 3.84 for a \(\chi ^2(1)\). This correction will become negligible for the larger sample sizes, since all three tests achieve their asymptotic distribution. The correction is most pronounced for \(LM_{PS}\) in the smaller samples.
Notes
Acknowledgments
This project was supported by Award No. 2009-SQ-B9-K101 by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect those of the Department of Justice.
References
- Amaral, P., Anselin, L.: Finite sample properties of Moran’s I test for spatial autocorrelation in Probit and Tobit models—empirical evidence. Working Paper 2011–7, GeoDa Center for Geospatial Analysis and Computation, Arizona State University, Tempe, AZ (2011)Google Scholar
- Anselin, L.: Spatial Econometrics: Methods and Models. Kluwer, Dordrecht (1988)CrossRefGoogle Scholar
- Anselin, L.: Spatial econometrics. In: Mills, T.C., Patterson, K. (eds.) Palgrave Handbook of Econometrics. Econometric Theory, vol. 1, pp. 901–969. Palgrave Macmillan, Basingstoke (2006)Google Scholar
- Beron, K.J., Vijverberg, W.P.: Probit in a spatial context: a Monte Carlo analysis. In: Anselin, L., Florax, R.J., Rey, S.J. (eds.) Advances in Spatial Econometrics, pp. 169–195. Springer, Heidelberg (2004)CrossRefGoogle Scholar
- Cox, D.R., Snell, E.J.: A general definition of residuals. J. R. Stat. Soc. Ser. B 39, 248–275 (1968)Google Scholar
- Fleming, M.: Techniques for estimating spatially dependent discrete choice models. In: Anselin, L., Florax, R.J., Rey, S.J. (eds.) Advances in Spatial Econometrics, pp. 145–168. Springer, Heidelberg (2004)CrossRefGoogle Scholar
- Hendry, D.F.: A comment on specification searches in spatial econometrics: the relevance of Hendry’s methodology. Reg. Sci. Urban Econ. 36, 309–312 (2006)CrossRefGoogle Scholar
- Kelejian, H., Prucha, I.: On the asymptotic distribution of the Moran I test statistic with applications. J. Econom. 104(2), 219–257 (2001)CrossRefGoogle Scholar
- McMillen, D.: Probit with spatial autocorrelation. J. Reg. Sci. 32(3), 335–348 (1992)CrossRefGoogle Scholar
- Novo, A.: Spatial probit models: statistical inference and estimation. Monte Carlo simulations and an application to contagious currency crises. Phd thesis, University of Illinois, Urbana-Champaign, IL (2001)Google Scholar
- Pinkse, J.: Asymptotics of the Moran test and a test for spatial correlation in Probit models. Working paper, Department of Economics, University of British Columbia, Vancouver, BC (1999)Google Scholar
- Pinkse, J.: Moran-flavored tests with nuisance parameter. In: Anselin, L., Florax, R.J., Rey, S.J. (eds.) Advances in Spatial Econometrics, pp. 67–77. Springer, Heidelberg (2004)CrossRefGoogle Scholar
- Pinkse, J., Slade, M.E.: Contracting in space: an application of spatial statistics to discrete-choice models. J. Econom. 85(1), 125–254 (1998)CrossRefGoogle Scholar
- Rey, S., Anselin, L.: PySAL, a Python library of spatial analytical methods. Rev. Reg. Stud. 37(5–27), 1 (2007)Google Scholar