Skip to main content
SpringerLink
Log in

Least Squares Model Averaging Based on Generalized Cross Validation

  • Published:
Acta Mathematicae Applicatae Sinica, English Series Aims and scope Submit manuscript

Abstract

Frequentist model averaging has received much attention from econometricians and statisticians in recent years. A key problem with frequentist model average estimators is the choice of weights. This paper develops a new approach of choosing weights based on an approximation of generalized cross validation. The resultant least squares model average estimators are proved to be asymptotically optimal in the sense of achieving the lowest possible squared errors. Especially, the optimality is built under both discrete and continuous weigh sets. Compared with the existing approach based on Mallows criterion, the conditions required for the asymptotic optimality of the proposed method are more reasonable. Simulation studies and real data application show good performance of the proposed estimators.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ando, T., Li, K.C. A weight-relaxed model averaging approach for high dimensional generalized linear models. Annals of Statistics, 45: 2654–2679 (2007)

    MathSciNet  MATH  Google Scholar 

  2. Bates, J.M., Granger, C.W.J. The combination of forecasts. Operations Research Quarterly, 20: 451–468 (1969)

    Article  Google Scholar 

  3. Buckland, S.T., Burnham, K.P., Augustin, N.H. Model selection: an integral part of inference. Biometrics, 53: 603–618 (1997)

    Article  Google Scholar 

  4. Burnham, K.P., Andreson, D.R. Model Selection and Multimodel Inference: A Practical Infromation-Theoretic Approach. Springer, New York, 2002

    Google Scholar 

  5. Clyde, M.A. and George, E. Model uncertainty. Statistical Science, 19: 81–94 (2004)

    Article  MathSciNet  Google Scholar 

  6. Craven, P., Wahba, G. Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik, 31: 377–403 (1979)

    Article  MathSciNet  Google Scholar 

  7. Danilov, D., Magnus, J.R. Forecast accuracy after pretesting with an application to the stock market. Journal of Forecasting, 23: 251–274 (2004)

    Article  Google Scholar 

  8. Draper, D. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society: Series B, 57: 45–70 (1995)

    MathSciNet  MATH  Google Scholar 

  9. Gao, Y., Zhang, X., Wang, S., Chong, T.T.L., Zou, G. Frequentist model averaging for threshold models. Annals of the Institute of Statistical Mathematics, 71: 275–306 (2019)

    Article  MathSciNet  Google Scholar 

  10. Gao, Y., Zhang, X., Wang, S., Zou, G. Model averaging based on leave-subject-out cross validation. Journal of Econometrics, 192: 139–151 (2016)

    Article  MathSciNet  Google Scholar 

  11. Hansen, B.E. Least squares model averaging. Econometrica, 75: 1175–1189 (2007)

    Article  MathSciNet  Google Scholar 

  12. Hansen, B.E. Least squares forecast averaging. Journal of Econometrics, 146: 342–350 (2008)

    Article  MathSciNet  Google Scholar 

  13. Hansen, B.E. Averaging estimators for fegressions with a possible structural break. Econometric Theory, 25: 1498–1514 (2009)

    Article  MathSciNet  Google Scholar 

  14. Hansen, B.E. Averaging estimators for autoregressions with a near unit root. Journal of Econometrics, 158: 142–155 (2010)

    Article  MathSciNet  Google Scholar 

  15. Hjort, N.L., Claeskens, G. Frequentist model average estimators. Journal of the American Statistical Association, 98: 879–899 (2003)

    Article  MathSciNet  Google Scholar 

  16. Hoeting, J.A., Madigan, D., Raftery, A.E., Volinsky, C.T. Bayesian model averaging: A tutorial. Statistical Science, 14: 382–417 (1999)

    Article  MathSciNet  Google Scholar 

  17. Koop, G., Potter, S. Forecasting in dynamic factor models using Bayesian model averaging. Econometrics Journal, 7: 550–565 (2004)

    Article  MathSciNet  Google Scholar 

  18. Leung, G., Barron, A.R. Infromation theory and mixing least-squares regressions. IEEE Transactions on Information Theory, 52: 3396–3410 (2006)

    Article  MathSciNet  Google Scholar 

  19. Li, K.C. Asymptotic optimality for Cp, Cl, cross-validation and generalized cross-validation: Discrete index set. The Annals of Statistics, 15: 958–975 (1987)

    Article  MathSciNet  Google Scholar 

  20. Mallows, C.L. Some comments on Cp. Technometrics, 15: 661–675 (1973)

    MATH  Google Scholar 

  21. Newey, W. Convergence rates and asymptotic normality for series estimators. Journal of Econometrics, 79: 147–168 (1997)

    Article  MathSciNet  Google Scholar 

  22. Pearson, M.H., Timmermann, A. Forecasting stock returns: An examination of market trading in the presence of transaction costs. Journal of Forecasting, 13: 335–367 (1994)

    Article  Google Scholar 

  23. Qiu, Y., Zhang, X., Xie, T., Zhao, S. Versatile HAR model for realized volatility: A least square model averaging perspective. Journal of Management Science and Engineering, 4: 55–73 (2019)

    Article  Google Scholar 

  24. Reftery, A.E., Madigen, D., Hoeting, J.A. Bayesian model averaging for regression models. Journal of the American Statistical Association, 92: 179–191 (1997)

    Article  MathSciNet  Google Scholar 

  25. Raftery, A.E. and Zheng, Y. Discussion: Performance of Bayesian model averaging. Journal of the American Statistical Association, 98: 931–938 (2003)

    Article  Google Scholar 

  26. Shen, X. and Huang, H.C. Optimal model assessment, selection and combination. Journal of the American Statistical Association, 101: 554–568 (2006).

    Article  MathSciNet  Google Scholar 

  27. Sun, Z. Extended focused information criterion for censored quantile regression model and averaging estimation. Sci Sin Math, 44(8): 859–857 (2014) (in Chinese)

    Article  Google Scholar 

  28. Wan, A.T.K., Zhang, X., Zou, G. Least squares model combining by Mallows criterion. Journal of Econometrics, 156: 277–283 (2010)

    Article  MathSciNet  Google Scholar 

  29. Wang, H., Zhang, X., Zou, G. Frequentist model averaging estimation: a review. Journal of Systems Science and Complexity, 22: 732–748 (2009)

    Article  MathSciNet  Google Scholar 

  30. Whittle, P. Bounds for the moments of linear and quadratic forms in independent variables. Theory of Probability and Its Applications, 5: 302–305 (1960)

    Article  MathSciNet  Google Scholar 

  31. Yang, Y. Adaptive regression by mixing. Journal of the American Statistical Association, 96: 574–586 (2001)

    Article  MathSciNet  Google Scholar 

  32. Yuan, Z. and Yang, Y. Combining linear regression models: When and how? Journal of the American Statistical Association, 100: 1202–1214 (2005)

    Article  MathSciNet  Google Scholar 

  33. Zhang, X. Model averaging and its applications. Ph.D. Thesis. Academy of Mathematics and Systems Science, Chinese Academy of Sciences, 2010

  34. Zhang, X., Liang, H. Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics, 39: 174–200 (2011)

    MathSciNet  MATH  Google Scholar 

  35. Zhang, X., Lu, Z., Zou, G. Adaptively combined forecasting for discrete response time series. Journal of Econometrics, 176: 80–91 (2013)

    Article  MathSciNet  Google Scholar 

  36. Zhang, X., Yu, D., Zou, G., Liang, H. Optimal model averaging estimation for generalized linear models and generalized linear mixed-effects models. Journal of American Statistical Association, 111: 1775–1790 (2016)

    Article  MathSciNet  Google Scholar 

  37. Zhang, X., Zheng, Y., Wang, S. A demand forecasting method based on stochastic frontier analysis and model average: An application in air travel demand forecasting. Journal of Systems Science and Complexity, 32: 615–633 (2019)

    Article  MathSciNet  Google Scholar 

  38. Zhang, X., Zou, G., Liang, H. Model averaging and weight choice in linear mixed-effects models. Biometrika, 101: 205–218 (2014)

    Article  MathSciNet  Google Scholar 

  39. Zhao, S. Model averaging based on James-Stein estimator. Metrika, 77: 1013–1022 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors thank the editor, the associate editor and the two referees for their constructive comments and suggestions that have substantially improved the original manuscript.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Qingdao University, Qingdao, 266071, China

    Xin-min Li

  2. School of Mathematical Sciences, Capital Normal University, Beijing, 100048, China

    Guo-hua Zou

  3. Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

    Xin-yu Zhang

  4. Beijing Academy of Artificial Intelligence, Beijing, 100084, China

    Xin-yu Zhang

  5. College of Science, Minzu University of China, Beijing, 100081, China

    Shang-wei Zhao

Authors
  1. Xin-min Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guo-hua Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin-yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shang-wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin-yu Zhang.

Additional information

This work was supported by National Key R&D Program of China (2020AAA0105200), the Ministry of Science and Technology of China (Grant no. 2016YFB0502301) and the National Natural Science Foundation of China (Grant nos. 11871294, 12031016, 11971323, 71925007, 72042019, 72091212 and 12001559), and a joint grant from the Academy for Multidisciplinary Studies, Capital Normal University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Xm., Zou, Gh., Zhang, Xy. et al. Least Squares Model Averaging Based on Generalized Cross Validation. Acta Math. Appl. Sin. Engl. Ser. 37, 495–509 (2021). https://doi.org/10.1007/s10255-021-1024-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10255-021-1024-x

Keywords

2000 MR Subject Classification

Search

Navigation