Advertisement

Automatic Component Selection in Additive Modeling of French National Electricity Load Forecasting

  • Anestis Antoniadis
  • Xavier Brossat
  • Yannig Goude
  • Jean-Michel Poggi
  • Vincent ThouvenotEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 175)

Abstract

We consider estimation and model selection in sparse high-dimensional linear additive models when multiple covariates need to be modeled nonparametrically, and propose some multi-step estimators based on B-splines approximations of the additive components. In such models, the overall number of regressors d can be large, possibly much larger than the sample size n. However, we assume that there is a smaller than n number of regressors that capture most of the impact of all covariates on the response variable. Our estimation and model selection results are valid without assuming the conventional “separation condition”—namely, without assuming that the norm of each of the true nonzero components is bounded away from zero. Instead, we relax this assumption by allowing the norms of nonzero components to converge to zero at a certain rate. The approaches investigated in this paper consist of two steps. The first step implements the variable selection, typically by the Group Lasso, and the second step applies a penalized P-splines estimation to the selected additive components. Regarding the model selection task we discuss, the application of several criteria such as Akaike information criterion (AIC), Bayesian information criterion (BIC), and generalized cross validation (GCV) and study the consistency of BIC, i.e. its ability to select the true model with probability converging to 1. We then study post-model estimation consistency of the selected components. We end the paper by applying the proposed procedure on some real data related to electricity load consumption forecasting: the EDF (Électricité de France) portfolio.

Keywords

B-splines approximation BIC Consistency Group LASSO Multi-step estimator Load forecasting P-splines Sparse additive model Variable selection 

Notes

Acknowledgments

The authors thank the Editor and a reviewer for their comments on the original manuscript.

References

  1. 1.
    Antoniadis, A., Fan, J.: Regularization of wavelet approximations. J. Am. Stat. Assoc. 96, 939–967 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Antoniadis, A., Gijbels, I., Verhasselt, A.: Variable selection in additive models using P-splines. Technometrics 54(4), 425–438 (2012)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Antoniadis, A., Goude, Y., Poggi, J-M., Thouvenot, V.: Sélection de variables dans les modèles additifs avec des estimateurs en plusieurs étapes. Technical report, 2015. https://hal.archives-ouvertes.fr/hal-01116100
  4. 4.
    Bach, F.R.: Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res. 9, 1179–1225 (2008)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Belloni, A., Chernozhukov, V.: Least squares after model selection in high-dimensional sparse models. Bernoulli 19(2), 521–547 (2013)Google Scholar
  6. 6.
    Taieb, S.B., Hyndman, R.J.: A gradient boosting approach to the kaggle load forecasting competition. Int. J. Forecast. 30(2), 382–394 (2014)CrossRefGoogle Scholar
  7. 7.
    Buhlmann, P., Van de Geer, S.: Statistics for High-Dimensional Data: Methods. Theory and Applications, 1st edn. Springer Publishing Company, Incorporated (2011)Google Scholar
  8. 8.
    Candes, E., Tao, T.: The Dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35(6), 2313–2351 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Cantoni, E., Mills Flemming, J., Ronchetti, E.: Variable selection in additive models by nonnegative garrote. Stat. Modell. 11(3):237–252 (2006)Google Scholar
  10. 10.
    Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–121 (1996)Google Scholar
  11. 11.
    Fan, J., Feng, Y., Song, R.: Nonparametric independence screening in sparse ultra-high-dimensional additive models. J. Am. Stat. Assoc. 106(494), 544–557 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Fan, J., Jiang, J.: Generalized likelihood ratio tests for additive models. J. Am. Stat. Assoc. 100, 890–907 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Fan, S., Hyndman, R.J.: Short-term load forecasting based on a semi-parametric additive model. IEEE Trans. Power Syst. 27(1), 134–141 (2012)CrossRefGoogle Scholar
  14. 14.
    Goude, Y., Nedellec, R., Kong, N.: Local short and middle term electricity load forecasting with semi-parametric additive models. IEEE Trans. Smart Grid 5(1), 440–446 (2014)CrossRefGoogle Scholar
  15. 15.
    Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models. Chapman & Hall, London (1990)zbMATHGoogle Scholar
  16. 16.
    Hong, T., Pinson, P., Fan, S.: Global energy forecasting competition 2012. Int. J. Forecast. 30(2), 357–363 (2014)CrossRefGoogle Scholar
  17. 17.
    Horowitz, J., Klemela, J., Mammen, E.: Optimal estimation in additive regression models. Bernoulli 12(2), 271–298 (2006)Google Scholar
  18. 18.
    Huang, J., Horowitz, J.L., Wei, F.: Variable selection in nonparametric additive models. Ann. Stat. 38(4), 2282–2313 (2010)Google Scholar
  19. 19.
    Kato, K.: Two-step estimation of high dimensional additive models. Technical report, July 2012. http://adsabs.harvard.edu/abs/2012arXiv1207.5313K
  20. 20.
    Koltchinskii, V., Yuan, M.: Sparsity in multiple kernel learning. Ann. Stat. 38(6), 3660–3695 (2010)Google Scholar
  21. 21.
    Lin, Y., Zhang, H.H.: Component selection and smoothing in multivariate nonparametric regression. Ann. Stat. 34(5), 2272–2297 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Marra, G., Wood, S.: Practical variable selection for generalized additive models. Comput. Stat. Data Anal. 55(7), 2372–2387 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Nedellec, R., Cugliari, J., Goude, Y.: Gefcom 2012: electric load forecasting and backcasting with semi-parametric models. Int. J. Forecast. 30(2), 375–381 (2014)CrossRefGoogle Scholar
  24. 24.
    Pierrot, A., Goude, Y.: Short-term electricity load forecasting with generalized additive models. In: Proceedings of ISAP power, pp. 593–600 (2011)Google Scholar
  25. 25.
    Raskutti, G., Wainwright, M.J., Yu, B.: Minimax-optimal rates for sparse additive models over kernel classes via convex programming. J. Mach. Learn. Res. 13(1), 389–427 (2012)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Ravikumar, P., Lafferty, Jo., Liu, H., Wasserman, L.: Sparse additive models. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 71(5), 1009–1030 (2009)Google Scholar
  27. 27.
    Stone, C.J.: Additive regression and other nonparametric models. Ann. Stat. 13(2), 689–705 (1985)Google Scholar
  28. 28.
    Suzuki, T., Tomioka, R., Sugiyama, M.: Fast convergence rate of multiple kernel learning with elastic-net regularization. arXiv:1103.0431 (2011)
  29. 29.
    Thouvenot, V., Pichavant, A., Goude, Y., Antoniadis, A., Poggi, J.-M.: Electricity forecasting using multi-stage estimators of nonlinear additive models. IEEE Trans. Power Syst. (2015)Google Scholar
  30. 30.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1994)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Wang, H., Li, B., Leng, C.: Shrinkage tuning parameter selection with a diverging number of parameters. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 71(3), 671–683 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Wood, S.: Generalized Additive Models: An Introduction with R. Chapman and Hall/CRC (2006)Google Scholar
  33. 33.
    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B 68, 49–67 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Zhou, S.: Restricted eigenvalue conditions on subgaussian random matrices. Technical report, Dec 2009. http://adsabs.harvard.edu/abs/2009arXiv0912.4045Z

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Anestis Antoniadis
    • 1
    • 2
  • Xavier Brossat
    • 3
  • Yannig Goude
    • 3
    • 5
  • Jean-Michel Poggi
    • 5
    • 6
  • Vincent Thouvenot
    • 4
    • 5
    Email author
  1. 1.University Cape TownCape TownSouth Africa
  2. 2.University Joseph FourierGrenobleFrance
  3. 3.EDF R&DClamartFrance
  4. 4.Thales Communication & SecurityGennevilliersFrance
  5. 5.University Paris-SudOrsayFrance
  6. 6.University Paris DescartesParisFrance

Personalised recommendations