Skip to main content
SpringerLink
Log in

Semiparametric Model Averaging for Ultrahigh-Dimensional Conditional Quantile Prediction

  • Published:
Acta Mathematica Sinica, English Series Aims and scope Submit manuscript

Abstract

In this paper, we develop a flexible semiparametric model averaging marginal regression procedure to forecast the joint conditional quantile function of the response variable for ultrahigh-dimensional data. First, we approximate the joint conditional quantile function by a weighted average of one-dimensional marginal conditional quantile functions that have varying coefficient structures. Then, a local linear regression technique is employed to derive the consistent estimates of marginal conditional quantile functions. Second, based on estimated marginal conditional quantile functions, we estimate and select the significant model weights involved in the approximation by a nonconvex penalized quantile regression. Under some relaxed conditions, we establish the asymptotic properties for the nonparametric kernel estimators and oracle estimators of the model averaging weights. We further derive the oracle property for the proposed nonconvex penalized model averaging procedure. Finally, simulation studies and a real data analysis are conducted to illustrate the merits of our proposed model averaging method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ando, T., Li, K.-C.: A model-averaging approach for high-dimensional regression. J. Amer. Statist. Assoc., 109, 1–46 (2014)

    MathSciNet  MATH  Google Scholar 

  2. Ando, T., Li, K.-C.: A weight-relaxed model averaging approach for high-dimensional generalized linear models. Ann. Statist., 45, 2654–2679 (2017)

    MathSciNet  MATH  Google Scholar 

  3. Bradic, J., Fan, J., Wang, W.: Penalized composite quasi-likelihood for ultrahigh dimensional variable selection. J. R. Stat. Soc. Ser. B Stat. Methodol., 73, 325–349 (2011)

    MathSciNet  MATH  Google Scholar 

  4. Chen, J., Li, D., Linton, O., et al.: Semiparametric dynamic portfolio choice with multiple conditioning variables. J. Econometrics, 194, 309–318 (2016)

    MathSciNet  MATH  Google Scholar 

  5. Chen, J., Li, D., Linton, O., et al.: Semiparametric ultra-high dimensional model averaging of nonlinear dynamic time series. J. Amer. Statist. Assoc., 113, 919–932 (2018)

    MathSciNet  MATH  Google Scholar 

  6. Cheng, M., Honda, T., Zhang, J.: Forward variable selection for sparse ultra-high dimensional varying coefficient models. J. Amer. Statist. Assoc., 111, 1209–1221 (2016)

    MathSciNet  Google Scholar 

  7. Fan, J., Fan, Y., Barut, E.: Adaptive robust variable selection. Ann. Statist., 42, 324–351 (2014)

    MathSciNet  MATH  Google Scholar 

  8. Fan, J., Gijbels, I.: Local Polynomial Modelling and Its Applications, Chapman & Hall, London, 1996

    MATH  Google Scholar 

  9. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc., 96, 1348–1360 (2001)

    MathSciNet  MATH  Google Scholar 

  10. Fan, J., Lv, J.: A selective overview of variable selection in high dimensional feature space. Statist. Sinica, 20, 101–148 (2010)

    MathSciNet  MATH  Google Scholar 

  11. Fan, J., Lv, J.: Nonconcave penalized likelihood with NP-dimensionality. IEEE T. Inform. Theory, 57, 5467–5484 (2011)

    MathSciNet  MATH  Google Scholar 

  12. Fan, J., Ma, Y., Dai, W.: Nonparametric independence screening in sparse ultra-high-dimensional varying coefficient models. J. Amer. Statist. Assoc., 109, 1270–1284 (2014)

    MathSciNet  MATH  Google Scholar 

  13. Fan, J., Peng, H.: On non-concave penalized likelihood with diverging number of parameters. Ann. Statist., 32, 928–961 (2004)

    MathSciNet  MATH  Google Scholar 

  14. Frumento, P., Bottai, M.: Parametric modeling of quantile regression coefficient functions. Biometrics, 72, 74–84 (2016)

    MathSciNet  MATH  Google Scholar 

  15. Hansen, B. E.: Least squares model averaging. Econometrica, 75, 1175–1189 (2007)

    MathSciNet  MATH  Google Scholar 

  16. Hastie, T., Tibshirani, R.: Varying-coefficient model. J. R. Stat. Soc. Ser. B Stat. Methodol., 55, 757–796 (1993)

    MathSciNet  MATH  Google Scholar 

  17. Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity: the Lasso and Generalizations, Chemical Rubber Company Press, Taylor & Francis Group, 2015

  18. He, X., Wang, L., Hong, H. G.: Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. Ann. Statist., 41, 342–369 (2013)

    MathSciNet  MATH  Google Scholar 

  19. Hu, T., Xia, Y.: Adaptive semi-varying coefficient model selection. Statist. Sinica, 22, 575–599 (2012)

    MathSciNet  MATH  Google Scholar 

  20. Hunter, D., Lange, K.: Quantile regression via an MM algorithm. J. Comput. Graph. Statist., 9, 60–77 (2000)

    MathSciNet  Google Scholar 

  21. Jiang, J., Zhao, Q., Hui, Y. V.: Robust modelling of ARCH models. J. Forecasting, 20, 111–133 (2001)

    Google Scholar 

  22. Kai, B., Li, R., Zou, H.: New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models. Ann. Statist., 39, 305–332 (2011)

    MathSciNet  MATH  Google Scholar 

  23. Kim, Y., Choi, H., Oh, H. S.: Smoothly clipped absolute deviation on high dimensions. J. Amer. Statist. Assoc., 103, 1665–1673 (2008)

    MathSciNet  MATH  Google Scholar 

  24. Knight, K.: Limiting distributions for L1 regression estimators under general conditions. Ann. Statist., 26, 755–770 (1998)

    MathSciNet  MATH  Google Scholar 

  25. Koenker, R.: Quantile Regression, Cambridge University Press, Cambridge, 2005

    MATH  Google Scholar 

  26. Lee, E. R., Noh, H., Park, B. U.: Model selection via Bayesian information criterion for quantile regression models. J. Amer. Statist. Assoc., 109, 216–229 (2014)

    MathSciNet  MATH  Google Scholar 

  27. Li, C., Li, Q., Racine, J., et al.: Optimal model averaging of varying coefficient models. Statist. Sinica, 28, 2795–2809 (2018)

    MathSciNet  MATH  Google Scholar 

  28. Li, D., Linton, O., Lu, Z.: A flexible semiparametric forecasting model for time series. J. Econometrics, 187, 345–357 (2015)

    MathSciNet  MATH  Google Scholar 

  29. Li, J., Xia, X., Wong, W., et al.: Varying-coefficient semiparametric model averaging prediction. Biometrics, 74, 1417–1426 (2018)

    MathSciNet  Google Scholar 

  30. Li, X., Ma, X., Zhang, J.: Conditional quantile correlation screening procedure for ultrahigh-dimensional varying coefficient models. J. Statist. Plann. Inference, 197, 69–92 (2018)

    MathSciNet  MATH  Google Scholar 

  31. Liang, H., Zou, G., Wan, A.T.K., et al.: Optimal weight choice for frequentist model average estimators. J. Amer. Statist. Assoc., 106, 1053–1066 (2011)

    MathSciNet  MATH  Google Scholar 

  32. Lin, B, Wang, Q., Zhang, J., et al.: Stable prediction in high-dimensional linear models. Stat. Comput., 27, 1401–1412 (2017)

    MathSciNet  MATH  Google Scholar 

  33. Lu, X., Su, L.: Jackknife model averaging for quantile regressions. J. Econometrics, 188, 40–58 (2015)

    MathSciNet  MATH  Google Scholar 

  34. Ma, S., He, X.: Inference for single-index quantile regression models with profile optimization. Ann. Statist., 44, 1234–1268 (2016)

    MathSciNet  MATH  Google Scholar 

  35. Ma, S., Li, R., Tsai, C.-L.: Variable screening via quantile partial correlation. J. Amer. Statist. Assoc., 112, 650–663 (2017)

    MathSciNet  Google Scholar 

  36. Noh, H., Chung, K., Keilegom, I.: Variable selection of varying coefficient models in quantile regression. Electron. J. Stat., 6, 1220–1238 (2012)

    MathSciNet  MATH  Google Scholar 

  37. Parzen, E.: On estimation of a probability density function and model. Ann. Statist., 33, 1065–1076 (1962)

    MathSciNet  MATH  Google Scholar 

  38. Pollard, D.: Asymptotics for least absolute deviation regression estimators. Economet. Theor., 7, 186–199 (1991)

    MathSciNet  Google Scholar 

  39. Sherwood, B., Wang, L.: Partially linear additive quantile regression in ultra-high dimension. Ann. Statist., 44, 288–317 (2016)

    MathSciNet  MATH  Google Scholar 

  40. Tang, Y., Song, X., Wang, H., et al.: Variable selection in high-dimensional quantile varying coefficient models. J. Multivariate Anal., 122, 115–132 (2013)

    MathSciNet  MATH  Google Scholar 

  41. Tao, P. D., An, L. T. H.: Convex analysis approach to d.c. programming: Theory, algorithms and applications. Acta Math. Vietnam., 22, 289–355 (1997)

    MathSciNet  MATH  Google Scholar 

  42. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol., 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  43. Wang, H., Xia, Y.: Shrinkage estimation of the varying coefficient model. J. Amer. Statist. Assoc., 104, 747–757 (2009)

    MathSciNet  MATH  Google Scholar 

  44. Wang, L., Wu, Y., Li, R.: Quantile regression for analyzing heterogeneity in ultra-high dimension. J. Amer. Statist. Assoc., 107, 214–222 (2012)

    MathSciNet  MATH  Google Scholar 

  45. Welsh, A. H.: On M-processes and M-estimation. Ann. Statist., 17, 337–361 (1989)

    MathSciNet  MATH  Google Scholar 

  46. Xia, X., Li, J., Fu, B.: Conditional quantile correlation learning for ultrahigh dimensional varying coefficient models and its application in survival analysis. Statist. Sinica, 29, 645–669 (2019)

    MathSciNet  MATH  Google Scholar 

  47. Xue, L., Qu, A.: Variable selection in high-dimensional varying-coefficient models with global optimality. J. Mach. Learn. Res., 13, 1973–1998 (2012)

    MathSciNet  MATH  Google Scholar 

  48. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol., 68, 49–67 (2006)

    MathSciNet  MATH  Google Scholar 

  49. Zhang, C. H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Statist., 38, 894–942 (2010)

    MathSciNet  MATH  Google Scholar 

  50. Zhao, W., Jiang, X., Lian, H.: A principal varying-coefficient model for quantile regression: Joint variable selection and dimension reduction. Comput. Statist. Data Anal., 127, 269–280 (2018)

    MathSciNet  MATH  Google Scholar 

  51. Zhu, L. P., Li, L., Li, R., et al.: Model-free feature screening for ultrahigh-dimensional data. J. Amer. Statist. Assoc., 106, 1464–1475 (2011)

    MathSciNet  MATH  Google Scholar 

  52. Zhu, R., Wan, T. K., Zhang, X., et al.: A Mallows-type model averaging estimator for the varying-coefficient partially linear model. J. Amer. Statist. Assoc., 114, 882–892 (2019)

    MathSciNet  MATH  Google Scholar 

  53. Zou, H.: The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc., 101, 1418–1429 (2006)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We appreciate the constructive suggestions from the referees and the editors.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Chongqing Normal University, Chongqing, 401331, P. R. China

    Chao Hui Guo

  2. School of Mathematics and Statistics, Southwest University, Chongqing, 400715, P. R. China

    Jing Lv

  3. College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, P. R. China

    Hu Yang & Jing Wen Tu

  4. College of Finance, University of International Business and Economics, Beijing, 100020, P. R. China

    Chen Xiao Tian

Authors
  1. Chao Hui Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Wen Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chen Xiao Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Lv.

Additional information

Supported by the National Natural Science Foundation of China Grant (Grant No. 12201091), Natural Science Foundation of Chongqing Grant (Grant Nos. CSTB2022NSCQ-MSX0852, cstc2021jcyj-msxmX0502), Innovation Support Program for Chongqing Overseas Returnees (Grant No. cx2020025), Science and Technology Research Program of Chongqing Municipal Education Commission (Grant Nos. KJQN202100526, KJQN201900511), the National Statistical Science Research Program (Grant No. 2022LY019) and Chongqing University Innovation Research Group Project: Nonlinear Optimization Method and Its Application (Grant No. CXQT20014)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, C.H., Lv, J., Yang, H. et al. Semiparametric Model Averaging for Ultrahigh-Dimensional Conditional Quantile Prediction. Acta. Math. Sin.-English Ser. 39, 1171–1202 (2023). https://doi.org/10.1007/s10114-023-0346-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10114-023-0346-4

Keywords

MR(2010) Subject Classification

Search

Navigation