A Review on Dimension-Reduction Based Tests For Regressions

Chapter
First Online:

Abstract

Curse of dimensionality is a big obstacle for constructing efficient goodness-of-fit tests for regression models with large or moderate number of covariates. To alleviate this difficulty, numerous efforts have been devoted in the last two decades. This review intends to collect and comment on the developments in this aspect. To make the paper self-contained, basic ideas on goodness-of-fit testing for regression models are also briefly reviewed, and the main classes of methods and their advantages and disadvantages are presented. Further, the difficulty caused by the dimensionality (number of covariates) is then discussed. The relevant dimension reduction methodologies are presented. Further, as a dedication to Stute’s 70th birthday, we also include a section to summarize his great contributions other than the results in dimension reduction-based tests.

Keywords

Curse of dimensionality Dimension reduction Goodness-of-fit Model checking Parametric regression models Projection pursuit 
This is a preview of subscription content, log in to check access.

References

  1. Bierens, H. J. (1982). Consistent model especification tests. Journal of Econometrics, 20, 105–134.Google Scholar
  2. Bierens, H. J. (1990). A consistent conditional moment test of functional form. Ecomometrica, 58, 1443–1458.Google Scholar
  3. Conde-Amboage, M. and González-Manteiga, W. (2015). A lack-of-fit test for quantile regression models with high-dimensional covariates. Computational Statistics & Data Analysis, 88, 128–138.Google Scholar
  4. Delgado, M. A. and Gonzáles-Manteiga, W. (2001). Significance testing in nonparametric regression based on the bootstrap. Annals of Statistics, 29, 1469–1507.Google Scholar
  5. Dette, H. (1999). A consistent test for the functional form of a regression based on a difference of variance estimates. Annals of Statistics, 27, 1012–1050.Google Scholar
  6. Dette, H., Neumeyer, N. and Van Keilegom, I. (2007). A new test for the parametric form of the variance function in nonparametric regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69, 903–917.Google Scholar
  7. Escanciano, J. C. (2006). A consistent diagnostic test for regression models using projections. Econometric Theory, 22, 1030–1051.Google Scholar
  8. Fan, J. and Jiang, J. (2007). Nonparametric inference with generalized likelihood ratio tests. Test, 16, 409–444.Google Scholar
  9. Fan, J., Zhang, C. and Zhang, J. (2001) Generalized likelihood ratio statistics and Wilks phenomenon. Annals of Statistics, 29, 153–193.Google Scholar
  10. Fan, Y. and Li, Q. (1996). Consistent model specifiation tests: omitted variables and semiparametric functional forms. Econometrica, 64, 865–890.Google Scholar
  11. Fan, Y. and Li, Q. (2000). Consistent Model Specification Tests: Kernel-Based Tests Versus Bierens’ ICM Tests. Econometric Theory, 16, 1016–1041.Google Scholar
  12. Feng, Z., Wen, X., Yu Z. and Zhu, L. X. (2013). On partial sufficient dimension reduction with applications to partially linear multi-index models. Journal of the American Statistical Association, 501, 237–246.Google Scholar
  13. Ferreira, E. and Stute, W. (2004). Testing for differences between conditional means in a time series context. Journal of the American Statistical Association, 99, 169–174.Google Scholar
  14. González-Manteiga, W. and Crujeiras, R. M. (2013a). An updated review of Goodness-of-Fit tests for regression models. Test, 22, 361–411.Google Scholar
  15. González-Manteiga, W. and Crujeiras, R. M. (2013b). Rejoinder on: An updated review of Goodness-of-Fit tests for regression models. Test, 22, 442–447.Google Scholar
  16. Guo, X., Wang, T. and Zhu, L. X. (2016). Model checking for parametric single-index models: a dimension reduction model-adaptive approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78, 1013–1035.Google Scholar
  17. Härdle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Annals of Statistics, 21, 1926–1947.Google Scholar
  18. Hart, J. (1997). Nonparametric smoothing and lack-of-fit tests. Springer, Berlin.Google Scholar
  19. Huber, J. P. (1985). Projection Pursuit, Annals of Statistics, 13, 435–475.Google Scholar
  20. Huskova, M. and Meintanis, S. (2009). Goodness-of-fit tests for parametric regression models based on empirical characteristic functions. Kybernetika, 45, 960–971.Google Scholar
  21. Huskova, M. and Meintanis, S. (2010). Test for the error distribution in nonparametric possibly heterocedastic regression models. Test, 19, 92–112.Google Scholar
  22. Khmaladze, E. V. (1981). Martingale Approach in the Theory of Goodness-of-fit Tests. Theor. Prob. Appl., 26, 240–257.Google Scholar
  23. Khmaladze, E. V. and Koul, H. L. (2009). Goodness-of-fit problem for errors in nonparametric regression: distribution free approach. Annals of Statistics, 37, 3165–3185.Google Scholar
  24. Koul, H. L. and Ni, P. P. (2004). Minimum distance regression model checking. Journal of Statistical Planning and Inference, 119,109–141.Google Scholar
  25. Koul, H. L. and Song, W. X. (2009). Minimum distance regression model checking with Berkson measurement errors. Annals of Statistics, 37, 132–156.Google Scholar
  26. Koul, H. L. and Stute, W. (1999). Nonparametric model checks for time series. Annals of Statistics, 27, 204–236.Google Scholar
  27. Koul, H. L., Stute, W. and Li, F. (2005). Model diagnosis for setar time series. Statistica Sinica, 15, 795–817.Google Scholar
  28. Koul, H. L., Xie, C. L. and Zhu, L. X. (2016). An adaptive-to-model test for parametric single-index errors-in-variables models. Submitted.Google Scholar
  29. Kumbhakar, S. C., Park, B. U., Simar, L. and Tsionas, E. G. (2007), Nonparametric stochastic frontiers: a local likelihood approach. Journal of Econometrics, 137, 1–27.Google Scholar
  30. Lavergne, P., Maistre, S. and Patilea, V. (2015). A significance test for covariates in nonparametric regression. Electronic Journal of Statistics, 9, 643–678.Google Scholar
  31. Lavergne, P. and Patilea, V. (2008). Breaking the curse of dimensionality in nonparametric testing. Journal of Econometrics, 143, 103–122.Google Scholar
  32. Lavergne, P. and Patilea, V. (2012). One for all and all for One: regression checks with many regressors. Journal of Business & Economic Statistics, 30, 41–52.Google Scholar
  33. Lavergne, P. and Vuong, Q. (2000). Nonparametric significance testing. Econometric Theory, 16, 576–601.Google Scholar
  34. Lin, Z. J., Li, Q. and Sun. Y. G. (2014), A consistent nonparametric test of parametric regression functional form in fixed effects panel data models. Journal of Econometrics, 178(1), 167–179.Google Scholar
  35. Ma, S. J., Zhang. J., Sun, Z. H. and Liang, H. (2014). Integrated conditional moment test for partially linear single index models incorporating dimension-reduction. Electronic Journal of Statistics, 8, 523–542.Google Scholar
  36. Maistre, S., Lavergne, P. and Patilea, V. (2017). Powerful nonparametric checks for quantile regression. Journal of Statistical Planning and Inference, 180, 13–29.Google Scholar
  37. Maistre, S. and Patilea, V. (2017). Nonparametric model checks of single-index assumptions. Statistica Sinica, Online.Google Scholar
  38. Niu, C. Z., Guo, X. and Zhu, L. X. (2016). Enhancements of nonparametric generalized likelihood ratio test: bias-correction and dimension reduction. Working paper.Google Scholar
  39. Niu, C. Z. and Zhu, L. X. (2016). A robust adaptive-to-model enhancement test for parametric single-index models. Working paper.Google Scholar
  40. Srihera, R. and Stute, W. (2010). Nonparametric comparison of regression functions. Journal of Multivariate Analysis, 101, 2039–2059.Google Scholar
  41. Stute, W. (1984). Asymptotic normality of nearest neighbor regression function estimates. Annals of Statistics, 12, 917–926.Google Scholar
  42. Stute, W. (1997). Nonparametric model checks for regression. Annals of Statistics, 25, 613–641.Google Scholar
  43. Stute, W. and Gonzáles-Manteiga, W. (1996). NN goodness-of-fit tests for linear models. Journal of Statistical Planning and Inference, 53, 75–92.Google Scholar
  44. Stute, W., Gonzáles-Manteiga, W. and Presedo-Quindimil, M. (1993). Boostrap based goodness-of-fit tests. Metrika, 40, 243–256.Google Scholar
  45. Stute, W., Gonzáles-Manteiga, W. and Presedo-Quindimil, M. (1998a). Bootstrap approximation in model checks for regression. Journal of the American Statistical Association, 93, 141–149.Google Scholar
  46. Stute, W., Gonzáles-Manteiga, Sánchez-Sellero, C. (2000). Nonparametric model checks in censored regression. Communication in Statistics-Theory and Methods, 29, 1611–1629.Google Scholar
  47. Stute, W., Presendo-Quindimil, M., Gonzáles-Manteiga, W. and Koul, H. L. (2006). Model checks for higher order time series. Statistics & Probability Letters, 76, 1385–1396.Google Scholar
  48. Stute, W., Thies, S. and Zhu, L. X. (1998b). Model checks for regression: An innovation process approach. Annals of Statistics, 26, 1916–1934.Google Scholar
  49. Stute, W., Xu, W.L. and Zhu, L. X. (2008). Model diagnosis for parametric regression in high dimensional spaces. Biometrika, 95, 451–467.Google Scholar
  50. Stute, W. and Zhu, L. X. (2002). Model checks for generalized linear models. Scandinavian Journal of Statistics, 29, 535–546.Google Scholar
  51. Stute, W. and Zhu, L. X. (2005). Nonparametric checks for single-index models. Annals of Statistics, 33, 1048–1083.Google Scholar
  52. Su, J. Q. and Wei, L. J. (1991). A lack of fit test for the mean function in a generalized linear model. Journal of the American Statistical Association, 86, 420–426.Google Scholar
  53. Tan, F. L., Zhu, X. H. and Zhu, L. X. (2017). A projection-based adaptive-to-model test for regressions. Statistica Sinica, Online.Google Scholar
  54. Van Keilegom, I., Gonzáles-Manteiga, W. and Sánchez Sellero, C. (2008). Goodness-of-fit tests in parametric regression based on the estimation of the error distribution. Test, 17, 401–415.Google Scholar
  55. Xia, Q., Xu, W. L. and Zhu. L. X. (2015). Consistently determining the number of factors in multivariate volatility modelling. Statistica Sinica, 25, 1025–1044.Google Scholar
  56. Xia, Y. C. (2009). Model check for multiple regressions via dimension reduction. Biometrika, 96, 133–148.Google Scholar
  57. Xia, Y. C., Tong, H., Li, W. K. and Zhu, L. X. (2002). An adaptive estimation of dimension reduction space. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64, 363–410.Google Scholar
  58. Zhang, C. and Dette, H. (2004). A power comparison between nonparametric regression tests. Statistics and Probability Letters, 66, 289–301.Google Scholar
  59. Zhang, T. and Wu, W. B. (2011). Testing parametric assumptions of trends of a nonstationary time series. Biometrika, 98, 599–614.Google Scholar
  60. Zheng, J. X. (1996). A consistent test of functional form via nonparametric estimation techniques. Journal of Econometrics, 75, 263–289.Google Scholar
  61. Zhu, L. P., Wang, T., Zhu, L.X. and Ferré, L. (2010). Sufficient dimension reduction through discretization-expectation estimation. Biometrika, 97, 295–304.Google Scholar
  62. Zhu, L. X. (2003), Model checking of dimension-reduction type for regression. Statistica Sinica, 13, 283–296.Google Scholar
  63. Zhu, L. X. and An, H. Z. (1992). A test method for nonlinearity of regression model. Journal of Math., 12, 391–397. In ChineseGoogle Scholar
  64. Zhu, L. X. and Li, R. (1998). Dimension-reduction type test for linearity of a stochastic model. Acta Mathematicae Applicatae Sinica, 14, 165–175.Google Scholar
  65. Zhu, X. H., Guo, X. and Zhu, L. X. (2017). An adaptive-to-model test for partially parametric single-index models. Statistics and Computing, 27, 1193–1204.Google Scholar
  66. Zhu, X. H., Chen, F., Guo, X. and Zhu, L. X. (2016). Heteroscedasticity checks for regression models: A dimension-reduction based model adaptive approach. Computational Statistics & Data Analysis, 103, 263–283.Google Scholar
  67. Zhu, X. H. and Zhu, L. X. (2016). Dimension-reduction based significance testing in nonparametric regression. Working paper.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of StatisticsBeijing Normal UniversityBeijingChina
  2. 2.Department of MathematicsHong Kong Baptist UniversityKowloon TongHong Kong

Personalised recommendations