Advertisement

Model Selection

  • Hannes LeebEmail author
  • Benedikt M. Pötscher
Chapter
First Online:

Abstract

We provide an overview of the vast and rapidly growing area of model selection in statistics and econometrics.

Keywords

Model Selection Bayesian Information Criterion Candidate Model American Statistical Association Model Selection Criterion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ahmed, S. E. and Basu, A. K. (2000): Least squares, preliminary test and Stein-type estimation in general vector AR(p) models. Statistica Neerlandica 54, 47-66.zbMATHMathSciNetCrossRefGoogle Scholar
  2. Akaike, H. (1969): Fitting autoregressive models for prediction. Annals of the Institute of Statistical Mathematics 21, 243-247.zbMATHMathSciNetCrossRefGoogle Scholar
  3. Akaike, H. (1970): Statistical predictor identification. Annals of the Institute of Statistical Mathematics 22, 203-217.zbMATHMathSciNetCrossRefGoogle Scholar
  4. Akaike, H. (1973): Information theory and an extension of the maximum likelihood principle B.N. Petrov and F. Csaki Second International Symposium on Information Theory. Akadémiai Kiadó, Budapest.Google Scholar
  5. Allen, D. M. (1971): Mean square error of prediction as a criterion for selecting variables. Technometrics 13, 469-475.zbMATHCrossRefGoogle Scholar
  6. Allen, D. M. (1974): The relationship between variable selection and data augmentation and a method for prediction. Technometrics 16, 125-127.zbMATHMathSciNetCrossRefGoogle Scholar
  7. Amemiya, T. (1980): Selection of regressors. International Economic Review 21, 331-354.zbMATHMathSciNetCrossRefGoogle Scholar
  8. An, H. Z. and Chen, Z. G. (1986): The identification of ARMA processes. Journal of Applied Probability 23A, 75-87.MathSciNetGoogle Scholar
  9. An, H. Z. and Gu, L. (1985): On the selection of regression variables. Acta Mathematicae Applicatae Sinica 2, 27-36.zbMATHCrossRefGoogle Scholar
  10. Anderson, T. W. (1962): The choice of the degree of a polynomial regression as a multiple decision problem. Annals of Mathematical Statistics 33, 255-265.zbMATHMathSciNetCrossRefGoogle Scholar
  11. Anderson, T. W. (1963): Determination of the order of dependence in normally distributed time series Rosenblatt, M. (Ed.): Time Series Analysis, 425-446. Wiley, New York.Google Scholar
  12. Bancroft, T. A. and Han, C. P. (1977): Inference based on conditional specification: A note and a bibliography. International Statistical Review 45, 117-127.zbMATHMathSciNetGoogle Scholar
  13. Baraud, Y. (2002): Model selection for regression on a random design. ESAIM Probability and Statistics 6, 127-146.zbMATHMathSciNetCrossRefGoogle Scholar
  14. Barron, A. R. (1991): Complexity regularization with application to artificial neural networks Nonparametric functional estimation and related topics (Spetses, 1990), NATO Advanced Study Institute Series C Mathematical and Physical Sciences 335, 561-576. Kluwer, Dordrecht.Google Scholar
  15. Barron, A. R. (1999): Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems Bayesian Statistics, (Alcoceber, 1998) 6, 27-52. Oxford University Press, Oxford.Google Scholar
  16. Barron, A. R. and Cover, T. M. (1991): Minimum complexity density estimation. IEEE Transactions on Information Theory 37, 1034-1054. (Corrections: IEEE Transactions on Information Theory 37, 1738.)zbMATHMathSciNetCrossRefGoogle Scholar
  17. Barron, A. R., Birgé, L. and Massart, P. (1999): Risk bounds for model selection via penalization. Probability Theory and Related Fields 113, 301-413.zbMATHMathSciNetCrossRefGoogle Scholar
  18. Bauer, P., Pötscher, B. M. and Hackl, P. (1988): Model selection by multiple test procedures. Statistics 19, 39-44.zbMATHMathSciNetCrossRefGoogle Scholar
  19. Beran, R. (1996): Confidence sets centered at C p-estimators. Annals of the Institute of Statistical Mathematics 48, 1-15.zbMATHMathSciNetCrossRefGoogle Scholar
  20. Beran, R. (2000): REACT scatterplot smoothers: Superefficiency through basis economy. Journal of the American Statistical Association 95, 155-171.zbMATHMathSciNetCrossRefGoogle Scholar
  21. Beran, R. and Dümbgen, L. (1998): Modulation of estimators and confidence sets. Annals of Statistics 26, 1826-1856.zbMATHMathSciNetCrossRefGoogle Scholar
  22. Berger, J. O. and Pericchi, L. R. (2001): Objective Bayesian methods for model selection: Introduction and comparison. In: Lahiri, P. (Ed.): Model Selection. IMS Lecture Notes Monograph Series 38 135–193.Google Scholar
  23. Bhansali, R. J. (1999): Parameter estimation and model selection for multistep prediction of a time series: A review Ghosh, S. (Ed.): Asymptotics, Nonparametrics and Time Series—A Tribute to Madan Lal Puri, 201-225. Dekker, New York.Google Scholar
  24. Bhansali, R. J. and Downham, D. Y. (1977): Some properties of the order of an autoregressive model selected by a generalization of Akaike’s FPE criterion. Biometrika 64, 547-551.zbMATHMathSciNetGoogle Scholar
  25. Birgé, L. (2004): Model selection for Gaussian regression with random design. Bernoulli 10, 1039-1051.zbMATHMathSciNetCrossRefGoogle Scholar
  26. Birgé, L. (2006): Model selection via testing: An alternative to (penalized) maximum likelihood estimators. Annales de l’Institute Henri Poincaré 42, 273-325.zbMATHCrossRefGoogle Scholar
  27. Birgé, L. and Massart, P. (2001): Gaussian model selection. Journal of the European Mathematical Society 3, 203-268.zbMATHCrossRefGoogle Scholar
  28. Breiman, L. (1995): Better subset regression using the nonnegative garrote. Technometrics 37, 373-384.zbMATHMathSciNetCrossRefGoogle Scholar
  29. Breiman, L. and Freedman, D. (1983): How many variables should be entered in a regression equation? Journal of the American Statistical Association 78, 131-136.zbMATHMathSciNetCrossRefGoogle Scholar
  30. Brook, R. J. (1976): On the use of a regret function to set significance points in prior tests of estimation. Journal of the American Statistical Association 71, 126-131. (Correction: Journal of the American Statistical Association 71, 1010.)zbMATHMathSciNetCrossRefGoogle Scholar
  31. Brown, P. J., Vannucci, M. and Fearn, T. (2002): Bayes model averaging with selection of regressors. Journal of the Royal Statistical Society Series B 64, 519-536.zbMATHMathSciNetCrossRefGoogle Scholar
  32. Boucheron, S., Bousquet, O. and Lugosi, G. (2005): Theory of classification: A survey of some recent advances. ESAIM Probability and Statistics 9, 323-375.zbMATHMathSciNetCrossRefGoogle Scholar
  33. Buckland S. T., Burnham, K. P. and Augustin, N. H. (1997): Model selection: An integral part of inference. Biometrics 53, 603-618.zbMATHCrossRefGoogle Scholar
  34. Bunea, F. (2004): Consistent covariate selection and post model selection inference in semiparametric regression. Annals of Statistics 32, 898-927.zbMATHMathSciNetCrossRefGoogle Scholar
  35. Bunea, F., Tsybakov, A. and Wegkamp, M. H. (2007): Aggregation for Gaussian regression. Annals of Statistics 35, 1674-1697.zbMATHMathSciNetCrossRefGoogle Scholar
  36. Bunea, F., Wegkamp, M. H. and Auguste, A. (2006): Consistent variable selection in high dimensional regression via multiple testing. Journal of Statistical Planning and Inference 136, 4349-4364.zbMATHMathSciNetCrossRefGoogle Scholar
  37. Burnham, K. P. and Anderson, D. R. (2002): Model Selection and Multimodal Inference (2nd edition). Springer, New York.Google Scholar
  38. Cesa-Bianchi, N. and Lugosi, G. (2006): Prediction, Learning, and Games. Cambridge University Press, Cambridge.zbMATHGoogle Scholar
  39. Chan, N.-H. (2008): Time series with roots on or near the unit circle Andersen, T.G., Davis, R.A., Kreiss, J.-P. and Mikosch, T. Handbook of Financial Time Series, 694-707. Springer, New York.Google Scholar
  40. Chen, S. S., Donoho, D. L. and Saunders, M. A. (1998): Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing 20, 33-61.MathSciNetCrossRefGoogle Scholar
  41. Chen, Z. G. and Ni, J. Y. (1989): Subset regression time series and its modeling procedures. Journal of Multivariate Analysis 31, 266-288.zbMATHMathSciNetCrossRefGoogle Scholar
  42. Choi, B. (1992): ARMA Model Identification. Springer, New York.zbMATHGoogle Scholar
  43. Claeskens, G. and Hjort, N. L. (2003): The focused information criterion. Journal of the American Statistical Association 98, 900-916.zbMATHMathSciNetCrossRefGoogle Scholar
  44. Craven, P. and Wahba, G. (1979): Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik 31, 377-403.zbMATHMathSciNetCrossRefGoogle Scholar
  45. Danilov, D. and Magnus, J. R. (2004): On the harm that ignoring pretesting can cause. Journal of Econometrics 122, 27-46.MathSciNetCrossRefGoogle Scholar
  46. Davisson, L. D. (1965): The prediction error of stationary Gaussian time series of unknown covariance. IEEE Transactions on Information Theory 11, 527-532.zbMATHMathSciNetCrossRefGoogle Scholar
  47. DeGooijer, J. G., Bovas, A., Gould, A. and Robinson, L. (1985): Methods for determining the order of an autoregressive-moving average process: A survey. International Statistical Review 53, 301-329.MathSciNetGoogle Scholar
  48. Dijkstra, T. K. and Veldkamp, J. H. (1988): Data-driven selection of regressors and the bootstrap Dijkstra, T. K. (Ed.): Lecture Notes in Economics and Mathematical Systems 307, 17-38, Springer, New York.Google Scholar
  49. Draper, N. R. and Smith, H. (1981): Applied Regression Analysis (2nd edition). Wiley, New York.zbMATHGoogle Scholar
  50. Droge, B. (1993): On finite-sample properties of adaptive least squares regression estimates. Statistics 24, 181-203.zbMATHMathSciNetCrossRefGoogle Scholar
  51. Droge, B. and Georg, T. (1995): On selecting the smoothing parameter of least squares regression estimates using the minimax regret approach. Statistics and Decisions 13, 1-20.zbMATHMathSciNetGoogle Scholar
  52. Dufour, J. M., Pelletier, D. and Renault, E. (2006): Short run and long run causality in time series: Inference. Journal of Econometrics 132, 337-362.MathSciNetCrossRefGoogle Scholar
  53. Fan, J. and Li, R. (2001): Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348-1360.zbMATHMathSciNetCrossRefGoogle Scholar
  54. Findley, D. F. (1985): On the unbiasedness property of AIC for exact or approximating linear stochastic time series models. Journal of Time Series Analysis 6, 229-252.zbMATHMathSciNetCrossRefGoogle Scholar
  55. Findley, D. F. (1991): Model selection for multistep-ahead forecasting. American Statistical Association Proceedings of the Business and Economic Statistics Section 243-247.Google Scholar
  56. Findley, D. F. and Wei, C. Z. (2002): AIC, overfitting principles, and the boundedness of moments of inverse matrices for vector autoregressions and related models. Journal of Multivariate Analysis 83, 415-450.zbMATHMathSciNetCrossRefGoogle Scholar
  57. Foster, D. P. and George, E. I. (1994): The risk inflation criterion for multiple regression. Annals of Statistics 22, 1947-1975.zbMATHMathSciNetCrossRefGoogle Scholar
  58. Frank, I. E. and Friedman, J. H. (1993): A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109-148.zbMATHCrossRefGoogle Scholar
  59. Francq, C., Roussignol, M. and Zakoïan, J. M. (2001): Conditional heteroskedasticity driven by hidden Markov chains. Journal of Time Series Analysis 22, 197-220.zbMATHMathSciNetCrossRefGoogle Scholar
  60. George, E. I. and Foster, D. P. (2000): Calibration and empirical Bayes variable selection. Biometrika 87, 731-747.zbMATHMathSciNetCrossRefGoogle Scholar
  61. Geweke, J. and Meese, R. (1981): Estimating regression models of finite but unknown order. International Economic Review 22, 55-70.zbMATHMathSciNetCrossRefGoogle Scholar
  62. Giles, J. A. and Giles, D. E. A. (1993): Pre-test estimation and testing in econometrics: recent developments. Journal of Economic Surveys 7, 145-197.CrossRefGoogle Scholar
  63. Guyon, X. and Yao, J. (1999): On the underfitting and overfitting sets of models chosen by order selection criteria. Journal of Multivariate Analysis 70, 221-249.zbMATHMathSciNetCrossRefGoogle Scholar
  64. Hall, A. R. and Peixe, F. P. M. (2003): A consistent method for the selection of relevant instruments. Econometric Reviews 22, 269-287.zbMATHMathSciNetCrossRefGoogle Scholar
  65. Hannan, E. J. (1980): The estimation of the order of an ARMA process. Annals of Statistics 8, 1071-1081.zbMATHMathSciNetCrossRefGoogle Scholar
  66. Hannan, E. J. (1981): Estimating the dimension of a linear system. Journal of Multivariate Analysis 11, 459-473.zbMATHMathSciNetCrossRefGoogle Scholar
  67. Hannan, E. J. and Deistler, M. (1988): The Statistical Theory of Linear Systems. Wiley, New York.zbMATHGoogle Scholar
  68. Hannan, E. J. and Quinn, B. G. (1979): The determination of the order of an autoregression. Journal of the Royal Statistical Society Series B 41, 190-195.zbMATHMathSciNetGoogle Scholar
  69. Hansen, M. H. and Yu, B. (2001): Model selection and the principle of minimum description length. Journal of the American Statistical Association 96, 746-774.zbMATHMathSciNetCrossRefGoogle Scholar
  70. Haughton, D. (1991): Consistency of a class of information criteria for model selection in nonlinear regression. Communications in Statistics. Theory and Methods 20, 1619-1629.zbMATHMathSciNetCrossRefGoogle Scholar
  71. Hemerly, E. M. and Davis, M.H.A. (1989): Strong consistency of the PLS criterion for order determination of autoregressive processes. Annals of Statistics 17, 941-946.zbMATHMathSciNetCrossRefGoogle Scholar
  72. Hidalgo, J. (2002): Consistent order selection with strongly dependent data and its application to efficient estimation. Journal of Econometrics 110, 213-239.zbMATHMathSciNetCrossRefGoogle Scholar
  73. Hjort, N. L. and Claeskens, G. (2003): Frequentist model average estimators. Journal of the American Statistical Association 98, 879-899.zbMATHMathSciNetCrossRefGoogle Scholar
  74. Hocking, R. R. (1976): The analysis and selection of variables in linear regression. Biometrics 32, 1-49.zbMATHMathSciNetCrossRefGoogle Scholar
  75. Hoerl, A. E. and Kennard, R. W. (1970): Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12, 55-67.zbMATHCrossRefGoogle Scholar
  76. Hoeting, J. A., Madigan, D., Raftery, A. and Volinsky, C. T. (1999): Bayesian model averaging: A tutorial. Statistical Science 14, 382-401. (Corrections: Statistical Science 15, 193-195.)zbMATHMathSciNetCrossRefGoogle Scholar
  77. Hosoya, Y. (1984): Information criteria and tests for time series models Anderson, O. D. (Ed.): Time Series Analysis: Theory and Practice 5, 39-52. North-Holland, Amsterdam.Google Scholar
  78. Hosoya, Y. (1986): A simultaneous test in the presence of nested alternative hypotheses. Journal of Applied Probability 23A, 187-200.MathSciNetCrossRefGoogle Scholar
  79. Hurvich, M. M. and Tsai, C. L. (1989): Regression and time series model selection in small samples. Biometrika 76, 297-307.zbMATHMathSciNetCrossRefGoogle Scholar
  80. Ing, C. K. (2004): Selecting optimal multistep predictors for autoregressive processes of unknown order. Annals of Statistics 32, 693-722.zbMATHMathSciNetCrossRefGoogle Scholar
  81. Ing, C. K. (2007): Accumulated prediction errors, information criteria and optimal forecasting for autoregressive time series. Annals of Statistics 35, 1238-1277.zbMATHMathSciNetCrossRefGoogle Scholar
  82. Ing, C. K. and Wei, C. Z. (2005): Order selection for same-realization predictions in autoregressive processes. Annals of Statistics 33, 2423-2474.zbMATHMathSciNetCrossRefGoogle Scholar
  83. Ing, C. K. and Wei, C. Z. (2006): A maximal moment inequality for long range dependent time series with applications to estimation and model selection. Statistica Sinica 16, 721-740.zbMATHMathSciNetGoogle Scholar
  84. James, W. and Stein, C. (1961): Estimation with quadratic loss Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, 361-379. California University Press, Berkeley.Google Scholar
  85. Judge, G. G. and Bock, M. E. (1978): The Statistical Implications of Pre-test and Stein-Rule Estimators in Econometrics. North-Holland, Amsterdam.zbMATHGoogle Scholar
  86. Juditsky, A. and Nemirovski, A. (2000): Functional aggregation for nonparametric regression. Annals of Statistics 28, 681-712.zbMATHMathSciNetCrossRefGoogle Scholar
  87. Kabaila, P. (1995): The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537-549.MathSciNetCrossRefGoogle Scholar
  88. Kabaila, P. (1996): The evaluation of model selection criteria: Pointwise limits in the parameter space Dowe, D. L., Korb, K. B. and Oliver, J. J. Information, Statistics and Induction in Science, 114-118. World Scientific, Singapore.Google Scholar
  89. Kabaila, P. (1998): Valid confidence intervals in regression after variable selection. Econometric Theory 14, 463-482.MathSciNetCrossRefGoogle Scholar
  90. Kabaila, P. (2002): On variable selection in linear regression. Econometric Theory 18, 913-925.zbMATHMathSciNetCrossRefGoogle Scholar
  91. Kabaila, P. and Leeb, H. (2006): On the large-sample minimal coverage probability of confidence intervals after model selection. Journal of the American Statistical Association 101, 619-629.zbMATHMathSciNetCrossRefGoogle Scholar
  92. Kempthorne, P. J. (1984): Admissible variable-selection procedures when fitting regression models by least squares for prediction. Biometrika 71, 593-597.zbMATHMathSciNetCrossRefGoogle Scholar
  93. Kennedy, W. J. and Bancroft, T. A. (1971): Model building for prediction in regression based upon repeated significance tests. Annals of Mathematical Statistics 42, 1273-1284.zbMATHMathSciNetCrossRefGoogle Scholar
  94. Keribin, C. and Haughton, D. (2003): Asymptotic probabilities of over-estimating and under-estimating the order of a model in general regular families. Communications in Statistics. Theory and Methods 32, 1373-1404.zbMATHMathSciNetCrossRefGoogle Scholar
  95. Kneip, A. (1994): Ordered linear smoothers. Annals of Statistics 22, 835-866.zbMATHMathSciNetCrossRefGoogle Scholar
  96. Knight, K. (1989): Consistency of Akaike's information criterion for infinite variance autoregressive processes. Annals of Statistics 17, 824-840.zbMATHMathSciNetCrossRefGoogle Scholar
  97. Knight, K. and Fu, W. (2000): Asymptotics for lasso-type estimators. Annals of Statistics 28, 1356-1378.zbMATHMathSciNetCrossRefGoogle Scholar
  98. Kohn, R. (1983): Consistent estimation of minimal subset dimension. Econometrica 51, 367-376.zbMATHMathSciNetCrossRefGoogle Scholar
  99. Konishi, S. and Kitagawa, G. (1996): Generalized information criteria in model selection. Biometrika 83, 875-890.zbMATHMathSciNetCrossRefGoogle Scholar
  100. Konishi, S. and Kitagawa, G. (2008): Information Criteria and Statistical Modeling. Springer, New York.zbMATHGoogle Scholar
  101. Leeb, H. (2005): The distribution of a linear predictor after model selection: Conditional finite-sample distributions and asymptotic approximations. Journal of Statistical Planning and Inference 134, 64-89.zbMATHMathSciNetCrossRefGoogle Scholar
  102. Leeb, H. (2006a): The distribution of a linear predictor after model selection: Unconditional finite-sample distributions and asymptotic approximations Rojo, J. (Ed.): IMS Lecture Notes Monograph Series 49, 291-311. Institute of Mathematical Statistics, Beachwood.Google Scholar
  103. Leeb, H. (2006b): Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process. Bernoulli, forthcoming.Google Scholar
  104. Leeb, H. (2007): Conditional predictive inference post model selection. Manuscript, Department of Statistics, Yale University.Google Scholar
  105. Leeb, H. and Pötscher, B. M. (2003): The finite-sample distribution of post-model-selection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100-142.zbMATHMathSciNetCrossRefGoogle Scholar
  106. Leeb, H. and Pötscher, B. M. (2005): Model selection and inference: Facts and fiction. Econometric Theory 21, 21-59.zbMATHMathSciNetCrossRefGoogle Scholar
  107. Leeb, H. and Pötscher, B. M. (2006a): Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results. Econometric Theory 22, 69-97. (Corrigendum. Econometric Theory 24, 581-583.)zbMATHCrossRefGoogle Scholar
  108. Leeb, H. and Pötscher, B. M. (2006b): Can one estimate the conditional distribution of post-model-selection estimators? Annals of Statistics 34, 2554-2591.zbMATHCrossRefGoogle Scholar
  109. Leeb, H. and Pötscher, B. M. (2008a): Sparse estimators and the oracle property, or the return of Hodges' estimator. Journal of Econometrics 142, 201-211.CrossRefGoogle Scholar
  110. Leeb, H. and Pötscher, B. M. (2008b): Can one estimate the unconditional distribution of post-model-selection estimators? Econometric Theory 24, 338-376.zbMATHGoogle Scholar
  111. Leung, G. and Barron, A. R. (2006): Information theory and mixing least-squares regressions. IEEE Transactions on Information Theory 52, 3396-3410.MathSciNetCrossRefGoogle Scholar
  112. Li, K. C. (1987): Asymptotic optimality for C p, C L, cross-validation and generalized cross-validation: Discrete index set. Annals of Statistics 15, 958-975.zbMATHMathSciNetCrossRefGoogle Scholar
  113. Linhart, H. and Zucchini, W. (1986): Model Selection. Springer, New York.zbMATHGoogle Scholar
  114. Lütkepohl, H. (1990): Asymptotic distributions of impulse response functions and forecast error variance decompositions of vector autoregressive models. Review of Economics and Statistics 72, 116-125.CrossRefGoogle Scholar
  115. Magnus, J. R. (1999): The traditional pretest estimator. Teoriya Veroyatnostei i Ee Primeneniya 44, 401-418; translation in Theory of Probability and Its Applications 44, (2000), 293-308.MathSciNetGoogle Scholar
  116. Magnus, J. R. (2002): Estimation of the mean of a univariate normal distribution with known variance. The Econometrics Journal 5, 225-236.zbMATHMathSciNetCrossRefGoogle Scholar
  117. Mallows, C. L. (1965): Some approaches to regression problems. Unpublished manuscript.Google Scholar
  118. Mallows, C. L. (1967): Choosing a subset regression. Bell Telephone Laboratories unpublished report.Google Scholar
  119. Mallows, C. L. (1973): Some comments on C p. Technometrics 15, 661-675.zbMATHCrossRefGoogle Scholar
  120. Mallows, C. L. (1995): More comments on C p. Technometrics 37, 362-372.zbMATHMathSciNetCrossRefGoogle Scholar
  121. McKay, R. J. (1977): Variable selection in multivariate regression: An application of simultaneous test procedures. Journal of the Royal Statistical Society Series B 39, 371-380.zbMATHMathSciNetGoogle Scholar
  122. McQuarrie, A. D. R. and Tsai, C. L. (1998): Regression and time series model selection. World Scientific, River Edge.zbMATHGoogle Scholar
  123. Miller, A. (2002): Subset Selection in Regression (2nd edition). Chapman and Hall, Boca Raton.zbMATHGoogle Scholar
  124. Nishii, R. (1984): Asymptotic properties of criteria for selection of variables in multiple regression. Annals of Statistics 12, 758-765.zbMATHMathSciNetCrossRefGoogle Scholar
  125. Nishii, R. (1988): Maximum likelihood principle and model selection when the true model is unspecified. Journal of Multivariate Analysis 27, 392-403.zbMATHMathSciNetCrossRefGoogle Scholar
  126. Paulsen, J. (1984): Order determination of multivariate autoregressive time series with unit roots. Journal of Time Series Analysis 5, 115-127.zbMATHMathSciNetCrossRefGoogle Scholar
  127. Phillips, P. C. B. (2005): Automated discovery in econometrics. Econometric Theory 21, 3-20.zbMATHMathSciNetGoogle Scholar
  128. Polyak, B. T. and Tsybakov, A. B. (1990): Asymptotic optimality of the C p -test for the orthogonal series estimation of regression. Theory of Probability and Its Applications 35, 293-306.zbMATHMathSciNetCrossRefGoogle Scholar
  129. Pötscher, B. M. (1983): Order estimation in ARMA-models by Lagrangian multiplier tests. Annals of Statistics 11, 872-885.zbMATHMathSciNetCrossRefGoogle Scholar
  130. Pötscher, B. M. (1985): The behaviour of the Lagrangian multiplier test in testing the orders of an ARMA-model. Metrika 32, 129-150.zbMATHMathSciNetCrossRefGoogle Scholar
  131. Pötscher, B. M. (1989): Model selection under nonstationarity: Autoregressive models and stochastic linear regression models. Annals of Statistics 17, 1257-1274.zbMATHMathSciNetCrossRefGoogle Scholar
  132. Pötscher, B. M. (1990): Estimation of autoregressive moving average order given an infinite number of models and approximation of spectral densities. Journal of Time Series Analysis 11, 165-179.zbMATHMathSciNetCrossRefGoogle Scholar
  133. Pötscher, B. M. (1991): Effects of model selection on inference. Econometric Theory 7, 163-185.MathSciNetCrossRefGoogle Scholar
  134. Pötscher, B. M. (1995): Comment on ‘The effect of model selection on confidence regions and prediction regions’ by P. Kabaila. Econometric Theory 11, 550-559.CrossRefGoogle Scholar
  135. Pötscher, B. M. (2006): The distribution of model averaging estimators and an impossibility result regarding its estimation Ho, H.-C., Ing, C.-K. and Lai, T.-L. Time Series and Related Topics: In Memory of Ching-Zong Wei. IMS Lecture Notes and Monograph Series 52, 113-129. Institute of Mathematical Statistics, Beachwood.CrossRefGoogle Scholar
  136. Pötscher, B. M. (2007): Confidence sets based on sparse estimators are necessarily large. Working paper, Department of Statistics, University of Vienna. arXiv:0711.1036.Google Scholar
  137. Pötscher, B. M. and Leeb, H. (2007): On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding. Working paper, Department of Statistics, University of Vienna. arXiv:0711.0660.Google Scholar
  138. Pötscher, B. M. and Novak, A. J. (1998): The distribution of estimators after model selection: Large and small sample results. Journal of Statistical Computation and Simulation 60, 19-56.zbMATHMathSciNetCrossRefGoogle Scholar
  139. Pötscher, B. M. and Schneider, U. (2007): On the distribution of the adaptive LASSO estimator. Working paper, Department of Statistics, University of Vienna. arXiv:0801.4627.Google Scholar
  140. Pötscher, B. M. and Srinivasan, S. (1994): A comparison of order estimation procedures for ARMA models. Statistica Sinica 4, 29-50.zbMATHMathSciNetGoogle Scholar
  141. Quinn, B. G. (1980): Order determination for a multivariate autoregression. Journal of the Royal Statistical Society Series B 42, 182-185.zbMATHMathSciNetGoogle Scholar
  142. Quinn, B. G. (1988): A note on AIC order determination for multivariate autoregressions. Journal of Time Series Analysis 9, 241-245.zbMATHMathSciNetCrossRefGoogle Scholar
  143. Rao, C. R. and Wu, Y. (1989): A strongly consistent procedure for model selection in a regression problem. Biometrika 76, 369-374.zbMATHMathSciNetCrossRefGoogle Scholar
  144. Rao, C. R. and Wu, Y. (2001): On model selection Lahiri, P. (Ed.): Model Selection. IMS Lecture Notes Monograph Series 38, 1-57. Institute of Mathematical Statistics, Beachwood.Google Scholar
  145. Rao, C. R. and Wu, Y. (2005): Linear model selection by cross-validation. Journal of Statistical Planning and Inference 128, 231-240.zbMATHMathSciNetCrossRefGoogle Scholar
  146. Reschenhofer, E. (1999): Improved estimation of the expected Kullback-Leibler discrepancy in case of misspecification. Econometric Theory 15, 377-387.zbMATHMathSciNetCrossRefGoogle Scholar
  147. Rissanen, J. (1978): Modeling by shortest data description. Automatica 14, 465-471.zbMATHCrossRefGoogle Scholar
  148. Rissanen, J. (1983): A universal prior for integers and estimation by minimum description length. Annals of Statistics 11, 416-431.zbMATHMathSciNetCrossRefGoogle Scholar
  149. Rissanen, J. (1986a): Stochastic complexity and modeling. Annals of Statistics 14, 1080-1100.zbMATHMathSciNetCrossRefGoogle Scholar
  150. Rissanen, J. (1986b): A predictive least squares principle. IMA Journal of Mathematical Control and Information 3, 211-222.zbMATHCrossRefGoogle Scholar
  151. Rissanen, J. (1987): Stochastic complexity (with discussion). Journal of the Royal Statistical Society Series B 49, 223-265.zbMATHMathSciNetGoogle Scholar
  152. Rissanen, J. (1989): Stochastic Complexity and Statistical Inquiry. World Scientific, Tea-neck.Google Scholar
  153. Sakai, H. (1981): Asymptotic distribution of the order selected by AIC in multivariate autoregressive model fitting. International Journal of Control 33, 175-180.zbMATHMathSciNetCrossRefGoogle Scholar
  154. Saleh, A. K. M. E. (2006): Theory of Preliminary Test and Stein-Type Estimation with Applications. Wiley, Hoboken.zbMATHCrossRefGoogle Scholar
  155. Saleh, A. K. M. E. and Sen, P. K. (1983): Asymptotic properties of tests of hypothesis following a preliminary test. Statistics and Decisions 1, 455-477.zbMATHMathSciNetGoogle Scholar
  156. Sawa, T. and Hiromatsu, T. (1973): Minimax regret significance points for a preliminary test in regression analysis. Econometrica 41, 1093-1101.zbMATHCrossRefGoogle Scholar
  157. Schwarz, G. (1978): Estimating the dimension of a model. Annals of Statistics 6, 461-464.zbMATHMathSciNetCrossRefGoogle Scholar
  158. Sclove, S. L., Morris, C. and Radhakrishnan, R. (1972): Non-optimality of preliminary-test estimators for the mean of a multivariate normal distribution. Annals of Mathematical Statistics 43, 1481-1490.zbMATHMathSciNetCrossRefGoogle Scholar
  159. Sen, P. K (1979): Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 1019-1033.zbMATHMathSciNetCrossRefGoogle Scholar
  160. Sen, P. K and Saleh, A. K. M. E. (1987): On preliminary test and shrinkage M-estimation in linear models. Annals of Statistics 15, 1580-1592.zbMATHMathSciNetCrossRefGoogle Scholar
  161. Shao, J. (1993): Linear model selection by cross-validation. Journal of the American Statistical Association 88, 486-494.zbMATHMathSciNetCrossRefGoogle Scholar
  162. Shao, J. (1997): An asymptotic theory for linear model selection (with discussion) Statistica Sinica 7, 221-264.zbMATHMathSciNetGoogle Scholar
  163. Shao, J. (1998): Convergence rates of the generalized information criterion. Journal of Nonparametric Statistics 9, 217-225.zbMATHMathSciNetCrossRefGoogle Scholar
  164. Shen, X., Huang, H. C. and Ye, J. (2004): Inference after model selection. Journal of the American Statistical Association 99, 751-762.zbMATHMathSciNetCrossRefGoogle Scholar
  165. Shibata, R. (1976): Selection of the order of an autoregressive model by Akaike’s information criterion. Biometrika 63, 117-126.zbMATHMathSciNetCrossRefGoogle Scholar
  166. Shibata, R. (1980): Asymptotically efficient selection of the order of the model for estimating parameters of a linear process. Annals of Statistics 8, 147-164.zbMATHMathSciNetCrossRefGoogle Scholar
  167. Shibata, R. (1981a): An optimal autoregressive spectral estimate. Annals of Statistics 9, 300-306.zbMATHMathSciNetCrossRefGoogle Scholar
  168. Shibata, R. (1981b): An optimal selection of regression variables. Biometrika 68, 45-54. (Correction: Biometrika 69, 492).zbMATHMathSciNetCrossRefGoogle Scholar
  169. Shibata, R. (1984): Approximate efficiency of a selection procedure for the number of regression variables. Biometrika 71, 43-49.zbMATHMathSciNetCrossRefGoogle Scholar
  170. Shibata, R. (1986a): Consistency of model selection and parameter estimation. Journal of Applied Probability 23A, 127-141.MathSciNetCrossRefGoogle Scholar
  171. Shibata, R. (1986b): Selection of the number of regression variables; a minimax choice of generalized FPE. Annals of the Institute of Statistical Mathematics 38, 459-474.zbMATHMathSciNetCrossRefGoogle Scholar
  172. Shibata, R. (1989): Statistical aspects of model selection. In: J. C. Willems (Ed.): From Data to Model, 215-240. Springer, New York.Google Scholar
  173. Shibata, R. (1997): Bootstrap estimate of Kullback-Leibler information for model selection. Statistica Sinica 7, 375-394.zbMATHMathSciNetGoogle Scholar
  174. Söderström, T. (1977): On model structure testing in system identification. International Journal of Control 26, 1-18.zbMATHMathSciNetCrossRefGoogle Scholar
  175. Stone, C. (1981): Admissible selection of an accurate and parsimonious normal linear regression model. Annals of Statistics 9, 475-485.zbMATHMathSciNetCrossRefGoogle Scholar
  176. Stone, C. (1982): Local asymptotic admissibility of a generalization of Akaike’s model selection rule. Annals of the Institute of Statistical Mathematics 34, 123-133.zbMATHMathSciNetCrossRefGoogle Scholar
  177. Stone, M. (1974): Cross-validatory choice and assessment of statistical prediction. Journal of the Royal Statistical Society Series B 36, 111-133.zbMATHGoogle Scholar
  178. Stone, M. (1977): An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. Journal of the Royal Statistical Society Series B 39, 44-47.zbMATHGoogle Scholar
  179. Strawderman, W. E. (1971): Proper Bayes minimax estimators of the multivariate normal mean. Annals of Mathematical Statistics 42, 385-388.zbMATHMathSciNetCrossRefGoogle Scholar
  180. Sugiura, N. (1978): Further analysis of the data by Akaike’s information criterion and the finite corrections. Communications in Statistics A7, 13-26.MathSciNetCrossRefGoogle Scholar
  181. Takada, Y. (1982): Admissibility of some variable selection rules in linear regression model. Journal of the Japanese Statistical Society 12, 45-49.zbMATHMathSciNetGoogle Scholar
  182. Takeuchi, K. (1976): Distribution of informational statistics and a criterion of model fitting. Suri-Kagaku 153, 12-18. (In Japanese.)Google Scholar
  183. Teräsvirta, T. and Mellin, I. (1986): Model selection criteria and model selection tests in regression models. Scandinavian Journal of Statistics 13, 159-171.zbMATHGoogle Scholar
  184. Theil, H. (1961): Economic Forecasts and Policy (2nd edition). North-Holland, Amsterdam.Google Scholar
  185. Thompson, M. L. (1978a): Selection of variables in multiple regression: part I. A review and evaluation. International Statistical Review 46, 1-19.zbMATHGoogle Scholar
  186. Thompson, M. L. (1978b): Selection of variables in multiple regression: part II. Chosen procedures, computations and examples. International Statistical Review 46, 129-146.zbMATHCrossRefGoogle Scholar
  187. Tibshirani, R. (1996): Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B 58, 267-288.zbMATHMathSciNetGoogle Scholar
  188. Toro-Vizcarrondo, C. and Wallace, T. D. (1968): A test of the mean square error criterion for restrictions in linear regression. Journal of the American Statistical Association 63, 558-572.zbMATHMathSciNetCrossRefGoogle Scholar
  189. Toyoda, T. and Wallace, T. D. (1976): Optimal critical values for pre-testing in regression. Econometrica 44, 365-375.zbMATHMathSciNetCrossRefGoogle Scholar
  190. Tsay, R. S. (1984): Order selection in nonstationary autoregressive models. Annals of Statistics 12, 1425-1433.zbMATHMathSciNetCrossRefGoogle Scholar
  191. Venter, J. H. and Steele, S. J. (1992): Some contributions to selection and estimation in the normal linear model. Annals of the Institute of Statistical Mathematics 44, 281-297.zbMATHMathSciNetCrossRefGoogle Scholar
  192. Vuong, Q. H. (1989): Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica 57, 307-333.zbMATHMathSciNetCrossRefGoogle Scholar
  193. Wallace, T. D. (1972): Weaker criteria and tests for linear restrictions in regression. Econometrica 40, 689-698.zbMATHMathSciNetCrossRefGoogle Scholar
  194. Wei, C. Z. (1992): On predictive least squares principles. Annals of Statistics 20, 1-42.zbMATHMathSciNetCrossRefGoogle Scholar
  195. Wegkamp, M. (2003): Model selection in nonparametric regression. Annals of Statistics 31, 252-273.zbMATHMathSciNetCrossRefGoogle Scholar
  196. Yang, Y. (1999): Model selection for nonparametric regression. Statistica Sinica 9, 475-499.zbMATHMathSciNetGoogle Scholar
  197. Yang, Y. (2001): Adaptive regression by mixing. Journal of the American Statistical Association 96, 574-588.zbMATHMathSciNetCrossRefGoogle Scholar
  198. Yang, Y. (2003): Regression with multiple candidate models: Selecting or mixing? Statistica Sinica 13, 783-809.zbMATHMathSciNetGoogle Scholar
  199. Yang, Y. (2005): Can the strengths of AIC and BIC be shared? A conflict between model identification and regression estimation. Biometrika 92, 937-950.zbMATHMathSciNetCrossRefGoogle Scholar
  200. Yang, Y. (2007): Prediction/estimation with simple linear models: Is it really that simple? Econometric Theory 23, 1-36.MathSciNetCrossRefGoogle Scholar
  201. Yang, Y. and Barron, A. R. (1998): An asymptotic property of model selection criteria. IEEE Transactions on Information Theory 44, 95-116.zbMATHMathSciNetCrossRefGoogle Scholar
  202. Yang, Y. and Barron, A. R. (1999): Information-theoretic determination of minimax rates of convergence. Annals of Statistics 27, 1564-1599.zbMATHMathSciNetCrossRefGoogle Scholar
  203. Zhang, P. (1992): Inference after variable selection in linear regression models. Biometrika 79, 741-746.zbMATHMathSciNetCrossRefGoogle Scholar
  204. Zhang, P. (1993a): Model selection via multifold cross validation. Annals of Statistics 21, 299-313.CrossRefGoogle Scholar
  205. Zhang, P. (1993b): On the convergence rate of model selection criteria. Communications in Statistics. Theory and Methods 22, 2765-2775.zbMATHCrossRefGoogle Scholar
  206. Zheng, X. and Loh, W. Y. (1995): Consistent variable selection in linear models. Journal of the American Statistical Association 90, 151-156.zbMATHMathSciNetCrossRefGoogle Scholar
  207. Zheng, X. and Loh, W. Y. (1997): A consistent variable selection criterion for linear models with high-dimensional covariates. Statistica Sinica 7, 311-325.zbMATHMathSciNetGoogle Scholar
  208. Zou, H. (2006): The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 1418-1429.zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  1. 1.Department of StatisticsYale UniversityNew HavenUSA
  2. 2.Department of StatisticsUniversity of ViennaViennaAustria

Personalised recommendations

Cite chapter

Buy options