Boosting techniques for nonlinear time series models

Abstract

Many of the popular nonlinear time series models require a priori the choice of parametric functions which are assumed to be appropriate in specific applications. This approach is mainly used in financial applications, when sufficient knowledge is available about the nonlinear structure between the covariates and the response. One principal strategy to investigate a broader class on nonlinear time series is the Nonlinear Additive AutoRegressive (NAAR) model. The NAAR model estimates the lags of a time series as flexible functions in order to detect non-monotone relationships between current and past observations. We consider linear and additive models for identifying nonlinear relationships. A componentwise boosting algorithm is applied for simultaneous model fitting, variable selection, and model choice. Thus, with the application of boosting for fitting potentially nonlinear models we address the major issues in time series modelling: lag selection and nonlinearity. By means of simulation we compare boosting to alternative nonparametric methods. Boosting shows a strong overall performance in terms of precise estimations of highly nonlinear lag functions. The forecasting potential of boosting is examined on the German industrial production (IP); to improve the model’s forecasting quality we include additional exogenous variables. Thus we address the second major aspect in this paper which concerns the issue of high dimensionality in models. Allowing additional inputs in the model extends the NAAR model to a broader class of models, namely the NAARX model. We show that boosting can cope with large models which have many covariates compared to the number of observations.

This is a preview of subscription content, access via your institution.

References

  1. Audrino, F.: What drives short rate dynamics? A functional gradient descent approach. Working Paper 640, University of St. Gallen (2010)

  2. Audrino, F., Barone-Adesi, G.: A dynamic model of expected bond returns: A functional gradient descent approach. Comput. Stat. Data Anal. 51(4), 2267–2277 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  3. Audrino, F., Bühlmann, P.: Volatility estimation with functional gradient descent for very highdimensional financial time series. J. Comput. Finance 6(3), 65–89 (2003)

    Google Scholar 

  4. Audrino, F., Bühlmann, P.: Splines for financial volatility. J. R. Stat. Soc., Ser. B, Stat. Methodol. 71(3), 655–670 (2009)

    Article  MathSciNet  Google Scholar 

  5. Benner, J., Meier, C.: Prognosegüte alternativer Frühindikatoren für die Konjunktur in Deutschland. Jahrb. Natl.ökon. Stat. 224(6), 637–652 (2004)

    Google Scholar 

  6. Breitung, J., Jagodzinski, D.: Prognoseeigenschaften alternativer Indikatoren der konjunkturellen Entwicklung in Deutschland. Konjunkturpolitik 47(4), 292–314 (2001)

    Google Scholar 

  7. Bühlmann, P.: Boosting for high-dimensional linear models. Ann. Stat. 34(2), 559–583 (2006)

    Article  MATH  Google Scholar 

  8. Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting. Stat. Sci. 22(4), 477–505 (2007)

    Article  Google Scholar 

  9. Bühlmann, P., Yu, B.: Boosting with the L 2 loss: regression and classification. J. Am. Stat. Assoc. 98(462), 324–339 (2003)

    Article  MATH  Google Scholar 

  10. Chen, R., Tsay, R.: Nonlinear additive ARX models. J. Am. Stat. Assoc. 88(423), 955–967 (1993)

    Article  MathSciNet  Google Scholar 

  11. Chevillon, G., Hendry, D.: Non-parametric direct multi-step estimation for forecasting economic processes. Int. J. Forecast. 21(2), 201–218 (2005)

    Article  Google Scholar 

  12. Claveria, O., Pons, E., Ramos, R.: Business and consumer expectations and macroeconomic forecasts. Int. J. Forecast. 23(1), 47–69 (2007)

    Article  Google Scholar 

  13. Clements, M., Franses, P., Swanson, N.: Forecasting economic and financial time-series with non-linear models. Int. J. Forecast. 20(2), 169–183 (2004)

    Article  Google Scholar 

  14. Diebold, F.X., Mariano, R.S.: Comparing predictive accuracy. J. Bus. Econ. Stat. 13(3), 253–263 (1995)

    Article  Google Scholar 

  15. Dreger, C., Schumacher, C.: Out-of-sample performance of leading indicators for the German business cycle. Single vs combined forecasts. J. Bus. Cycle Meas. Anal. 2(1), 71–88 (2005)

    Article  Google Scholar 

  16. Eilers, P., Marx, B.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–102 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  17. Elliot, G., Timmermann, A.: Economic forecasting. J. Econ. Lit. 66(1), 3–56 (2008)

    Article  Google Scholar 

  18. Fenske, N., Kneib, T., Hothorn, T.: Identifying risk factors for severe childhood malnutrition by boosting additive quantile regression. Technical Report 52, Institut für Statistik, Ludwig-Maximilians-Universität München (2009). http://epub.ub.uni-muenchen.de/10510/

  19. Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156. Morgan Kaufmann, San Francisco (1996)

    Google Scholar 

  20. Friedman, J.: Multivariate adaptive regression splines. Ann. Stat. 19(1), 1–67 (1991)

    Article  MATH  Google Scholar 

  21. Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)

    Article  MATH  Google Scholar 

  22. Fritsche, U., Stephan, S.: Leading indicators of German business cycles—an assessment of properties. Jahrb. Natl.ökon. Stat. 222(3), 289–311 (2002)

    Google Scholar 

  23. Goldrian, G.: Handbook of Survey-Based Business Cycle Analysis. Edward Elgar, Cheltenham Glos (2007)

    Google Scholar 

  24. Hamilton, J.: Time Series Analysis. Princeton University Press, Princeton (1994)

    Google Scholar 

  25. Hansen, P.R., Lunde, A., Nason, J.M.: The model confidence set. CREATES Research Papers 2010-76, School of Economics and Management, University of Aarhus (2010). http://ideas.repec.org/p/aah/create/2010-76.html

  26. Harvey, D., Leybourne, S., Newbold, P.: Testing the equality of prediction mean squared errors. Int. J. Forecast. 13(2), 281–291 (1997)

    Article  Google Scholar 

  27. Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall/CRC, London (1990)

    Google Scholar 

  28. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Berlin (2001)

    Google Scholar 

  29. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, Berlin (2009a)

    Google Scholar 

  30. Hastie, T., Tibshirani, R., Leisch, F., Hornik, K., Ripley, B.D.: mda: mixture and flexible discriminant analysis. R package version 0.3-4 (2009b)

  31. Hothorn, T., Buehlmann, P., Kneib, T., Schmid, M., Hofner, B.: mboost: model-based boosting, R package, version 1.0-7 (2009). http://CRAN.R-project.org/package=mboost

  32. Huang, J., Yang, L.: Identification of non-linear additive autoregressive models. J. R. Stat. Soc., Ser. B, Stat. Methodol. 66(2), 463–477 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  33. Hüfner, F., Schröder, M.: Prognosegehalt von ifo-Geschäftserwartungen und ZEW-Konjunkturerwartungen: Ein ökonometrischer Vergleich. Jahrb. Natl.ökon. Stat. 222(3), 316–336 (2002)

    Google Scholar 

  34. Hurvich, C.M., Simonoff, J.S., Tsai, C.-L.: Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc., Ser. B, Stat. Methodol. 60, 271–293 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  35. Hyndman, R., Koehler, A.: Another look at measures of forecast accuracy. Int. J. Forecast. 22(4), 679–688 (2006)

    Article  Google Scholar 

  36. Kneib, T., Hothorn, T., Tutz, G.: Variable selection and model choice in geoadditive regression models. Biometrics 65, 626–634 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  37. Lewis, P., Stevens, J.: Nonlinear modeling of time series using multivariate adaptive regression splines (MARS). J. Am. Stat. Assoc. 86(416), 864–877 (1991)

    Article  MATH  Google Scholar 

  38. Lütkepohl, H.: Introduction to Multiple Time Series Analysis. Springer, Berlin (1991)

    Google Scholar 

  39. Lütkepohl, H.: New Introduction to Multiple Time Series Analysis. Springer, Berlin (2006)

    Google Scholar 

  40. Lutz, R.W., Kalisch, M., Bühlmann, P.: Robustified L 2 boosting. Comput. Stat. Data Anal. 52(7), 3331–3341 (2008)

    Article  MATH  Google Scholar 

  41. Marcellino, M., Schumacher, C.: Factor nowcasting of German GDP with ragged-edge data. A model comparison using MIDAS projections. Technical report, Bundesbank Discussion Paper, Series 1, 34/2007 (2007)

  42. Marcellino, M., Stock, J., Watson, M.: A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. J. Econom. 135(1–2), 499–526 (2006)

    Article  MathSciNet  Google Scholar 

  43. Matías, J., Febrero-Bande, M., González-Manteiga, W., Reboredo, J.: Boosting GARCH and neural networks for the prediction of heteroskedastic time series. Math. Comput. Model. 51(3–4), 256–271 (2010)

    Article  MATH  Google Scholar 

  44. Pfaff, B.: VAR, SVAR and SVEC models: Implementation within R package vars. J. Stat. Softw. 27(4) (2008). http://www.jstatsoft.org/v27/i04/

  45. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2009). ISBN 3-900051-07-0. http://www.R-project.org

    Google Scholar 

  46. Robinzonov, N., Wohlrabe, K.: Freedom of choice in macroeconomic forecasting. CESifo Econ. Stud. 56(2), 192–220 (2010)

    Article  Google Scholar 

  47. Schmid, M., Hothorn, T.: Boosting additive models using component-wise P-splines. Comput. Stat. Data Anal. 53(2), 298–311 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  48. Shafik, N., Tutz, G.: Boosting nonlinear additive autoregressive time series. Comput. Stat. Data Anal. 53(7), 2453–2464 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  49. Stock, J., Watson, M.: Forecasting output and inflation: The role of asset prices. J. Econ. Lit. 41(3), 788–829 (2003)

    Article  Google Scholar 

  50. Teräsvirta, T., van Dijk, D., Medeiros, M.: Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A reexamination. Int. J. Forecast. 21(4), 755–774 (2005)

    Article  Google Scholar 

  51. Tschernig, R., Yang, L.: Nonparametric lag selection for time series. J. Time Ser. Anal. 21(4), 457–487 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  52. Wood, S.: Generalized Additive Models: An Introduction with R. Chapman & Hall/CRC, London (2006)

    Google Scholar 

  53. Wood, S.: mgcv: GAMs with GCV smoothness estimation and GAMMs by REML/PQL. R package, version 1.4-1.1 (2009)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Nikolay Robinzonov.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Robinzonov, N., Tutz, G. & Hothorn, T. Boosting techniques for nonlinear time series models. AStA Adv Stat Anal 96, 99–122 (2012). https://doi.org/10.1007/s10182-011-0163-4

Download citation

Keywords

  • Componentwise boosting
  • Forecasting
  • Nonlinear times series
  • Autoregressive additive models
  • Lag selection