Skip to main content
SpringerLink
Log in

Least Squares Model Averaging for Two Non-Nested Linear Models

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

This paper studies the least squares model averaging methods for two non-nested linear models. It is proved that the Mallows model averaging weight of the true model is root-n consistent. Then the authors develop a penalized Mallows criterion which ensures that the weight of the true model equals 1 with probability tending to 1 and thus the averaging estimator is asymptotically normal. If neither candidate model is true, the penalized Mallows averaging estimator is asymptotically optimal. Simulation results show the selection consistency of the penalized Mallows method and the superiority of the model averaging approach compared with the model selection estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Zhang X and Zou G, Model averaging method and its application in forecast, Statistical Research, 2011, 28(6): 97–102 (in Chinese).

    Google Scholar 

  2. Gao Y, Zhang X, Wang S, et al., Model averaging based on leave-subject-out cross-validation, Journal of Econometrics, 2016, 192: 139–151.

    Article  MathSciNet  MATH  Google Scholar 

  3. Gao Y, Luo M, and Zou G, Forecasting with model selection or model averaging: A case study for monthly container port throughput, Transportmetrica A: Transport Science, 2016, 12(4): 366–384.

    Article  Google Scholar 

  4. Zhao S and Zhou J, Forecasting Chinese ports container throughput: A combining time series, Journal of Systems Science and Mathematical Sciences, 2018, 38(2): 210–219 (in Chinese).

    Google Scholar 

  5. Gao Y, Zhou J, Wang H, et al., Forecasting Chinese ports container throughput based on the jackknife model averaging method, Journal of Systems Science and Mathematical Sciences, 2020, 40(4): 729–737 (in Chinese).

    MATH  Google Scholar 

  6. Zhang X, Zheng Y, and Wang S, A demand forecasting method based on stochastic frontier analysis and model average: An application in air travel demand forecasting, Journal of Systems Science and Complexity, 2019, 32(2): 167–185.

    Article  MathSciNet  Google Scholar 

  7. Hoeting J A, Madigan D, Raftery A E, et al., Bayesian model averaging: A tutorial, Statistical Science, 1999, 14: 382–417.

    MathSciNet  MATH  Google Scholar 

  8. Raftery A E and Zheng Y, Long-run performance of Bayesian model averaging, Journal of the American Statistical Association, 2003, 98: 931–938.

    Article  Google Scholar 

  9. Buckland S T, Burnhamn K P, and Augustin N H, Model selection: An integral part of inference, Biometrics, 1997, 53: 603–618.

    Article  MATH  Google Scholar 

  10. Yang Y, Mixing strategies for density estimation, Annals of Statistics, 1999, 28(1): 76–87.

    MathSciNet  Google Scholar 

  11. Hjort N L and Claeskens G, Frequentist model average estimators, Journal of the American Statistical Association, 2003, 98: 879–899.

    Article  MathSciNet  MATH  Google Scholar 

  12. Yuan Z and Yang Y, Combining linear regression models: When and how, Journal of the American Statistical Association, 2005, 100: 1202–1214.

    Article  MathSciNet  MATH  Google Scholar 

  13. Hansen B E, Least squares model averaging, Econometrica, 2007, 75: 1175–1189.

    Article  MathSciNet  MATH  Google Scholar 

  14. Hansen B E, Averaging estimators for regressions with a possible structural break, Econometric Theory, 2009, 25: 1498–1514.

    Article  MathSciNet  MATH  Google Scholar 

  15. Hansen B E, Averaging estimators for autoregressions with a near unit root, Journal of Econometrics, 2010, 158: 142–155.

    Article  MathSciNet  MATH  Google Scholar 

  16. Liu Q and Okui R, Heteroskedasticity-robust Cp model averaging, Econometrics Journal, 2013, 16: 463–472.

    Article  MathSciNet  MATH  Google Scholar 

  17. Ando T and Li K, A model-averaging approach for high-dimensional regression, Journal of the American Statistical Association, 2014, 109: 254–265.

    Article  MathSciNet  MATH  Google Scholar 

  18. Ando T and Li K, A weight-relaxed model averaging approach for high-dimensional generalized linear models, Annals of Statistics, 2017, 45: 2654–2679.

    Article  MathSciNet  MATH  Google Scholar 

  19. Wan A T K, Zhang X, and Zou G, Least squares model averaging by Mallows criterion, Journal of Econometrics, 2010, 156: 277–283.

    Article  MathSciNet  MATH  Google Scholar 

  20. Zhang X, A new study on asymptotic optimality of least squares model averaging, Econometric Theory, 2021, 37: 388–407.

    Article  MathSciNet  MATH  Google Scholar 

  21. Liang H, Zou G, Wan A T K, et al., On optimal weight choice for Frequentist model average estimators, Journal of the American Statistical Association, 2011, 106(495): 1053–1066.

    Article  MathSciNet  MATH  Google Scholar 

  22. Hansen B E and Racine J S, Jackknife model averaging, Journal of Econometrics, 2012, 167: 38–46.

    Article  MathSciNet  MATH  Google Scholar 

  23. Zhang X, Wan A T K, and Zou G, Model averaging by jackknife criterion in models with dependent data, Journal of Econometrics, 2013, 174: 82–94.

    Article  MathSciNet  MATH  Google Scholar 

  24. Zhang X, Zou G, and Carroll R J, Model averaging based on Kullback-Leibler distance, Statistica Sinica, 2015, 25: 1583–1598.

    MathSciNet  MATH  Google Scholar 

  25. Zhou J and Zhao S, Frequentist model averaging for high dimensional poisson models, Journal of Systems Science and Mathematical Sciences, 2018, 38(6): 679–687 (in Chinese).

    MathSciNet  MATH  Google Scholar 

  26. Zhao Z and Zou G, Average estimation of semiparametric models for high-dimensional longitudinal data, Journal of Systems Science and Complexity, 2020, 33(6): 2013–2047.

    Article  MathSciNet  MATH  Google Scholar 

  27. Chen X and Zhao Z, Model average for high-dimensional longitudinal data, Journal of Systems Science and Mathematical Sciences, 2020, 40(7): 1297–1324 (in Chinese).

    MATH  Google Scholar 

  28. Zhang X, Consistency of model averaging estimators, Economics Letters, 2015, 130: 120–123.

    Article  MathSciNet  MATH  Google Scholar 

  29. Alhorn K, Dette H, and Schorning K, Optimal designs for model averaging in non-nested models, 2019, Working paper.

    MATH  Google Scholar 

  30. Fang F and Liu M, Limit of the optimal weight in least squares model averaging with non-nested models, Economics Letters, 2020, 196: 109586.

    Article  MathSciNet  MATH  Google Scholar 

  31. Royston P and Thompson S G, Comparing non-nested regression models, Biometrics, 1995, 51(1): 114–127.

    Article  MATH  Google Scholar 

  32. Zhang X, Zou G, Liang H, et al., Parsimonious model averaging with a diverging number of parameters, Journal of the American Statistical Association, 2020, 115(530): 972–984.

    Article  MathSciNet  MATH  Google Scholar 

  33. Akaike H, Information theory and an extension of the maximum likelihood principle, Proceedings of the Second International Symposium on Information Theory, Eds. by Petrov B N and Csaki F, Budapest, 1973, 267–281.

    Google Scholar 

  34. Schwarz G, Estimating the dimension of a model, Annals of Statistics, 1978, 6(2): 461–464.

    Article  MathSciNet  MATH  Google Scholar 

  35. Mallows C L, Some comments on Cp, Technometrics, 1973, 15(4): 661–675.

    MATH  Google Scholar 

  36. Claeskens G and Hjort H L, Model Selection and Model Averaging, Cambridge University Press, New York, 2008. sai][37]_McAleer M, The significance of testing empirical non-nested models, Journal of Econometrics, 1995, 67(1): 149–171.

    Google Scholar 

  37. Davidson R and MacKinnon J G, Several tests for model specification in the presence of alternative hypotheses, Econometrica, 1981, 49: 781–793.

    Article  MathSciNet  MATH  Google Scholar 

  38. Watnik M R, Johnson W O, and Bedrick E J, Nonnested linear model selection revisited, Communications in Statistics-Theory and Methods, 2001, 30: 1–20.

    Article  MathSciNet  MATH  Google Scholar 

  39. Chen X, Fan Y, Wan A, et al., Post-J test inference in non-nested linear regression models, Science China Mathematics, 2015, 58(6): 1203–1216.

    Article  MathSciNet  MATH  Google Scholar 

  40. Bera A K and McAleer M, On exact and asymptotic tests of non-nested models, Statistics & Probability Letters, 1987, 5: 19–22.

    Article  MathSciNet  MATH  Google Scholar 

  41. Fan J and Li R, Variable selection via penalized likelihood and its oracle properties, Journal of the American Statistical Association, 2001, 96: 1348–1360.

    Article  MathSciNet  MATH  Google Scholar 

  42. Wahba G, Spline Models for Observational Data, Society for Industrial and Applied Mathematics, Philadelphia, 1990.

    Book  MATH  Google Scholar 

  43. Green P J and Silverman B W, Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach, Volume 58 of Monographs on Statistics and Applied Probability, Chapman & Hall, London, 1994.

    Book  MATH  Google Scholar 

  44. Tibshirani R, Regression shrinkage and selection via lasso, Journal of the Royal Statistical Society, Series B, 1996, 58: 267–288.

    MathSciNet  MATH  Google Scholar 

  45. Breiman L, Better subset regression using the nonnegative garrote, Technometrics, 1995, 37: 373–384.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We thank Prof. Xinyu Zhang for his constructive comments and suggestions which greatly improved the original manuscript.

Author information

Authors and Affiliations

  1. Department of Statistics, College of Science, Minzu University of China, Beijing, 100081, China

    Yan Gao

  2. Faculty of Science, Beijing University of Technology, Beijing, 100124, China

    Tianfa Xie

  3. School of Mathematical Sciences, Capital Normal University, Beijing, 100048, China

    Guohua Zou

Authors
  1. Yan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianfa Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guohua Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianfa Xie.

Additional information

This research was supported by the National Natural Science Foundation of China under Grant Nos. 11801598, 12031016 and 11971323, the National Statistical Research Program under Grant No. 2018LY96, the Beijing Natural Science Foundation under Grant No. 1202001, and NQI Project under Grant No. 2022YFF0609903.

This paper was recommended for publication by Editor HE Xu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, Y., Xie, T. & Zou, G. Least Squares Model Averaging for Two Non-Nested Linear Models. J Syst Sci Complex 36, 412–432 (2023). https://doi.org/10.1007/s11424-023-1172-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-023-1172-6

Keywords

Search

Navigation