AStA Advances in Statistical Analysis

, Volume 96, Issue 1, pp 99–122 | Cite as

Boosting techniques for nonlinear time series models

  • Nikolay Robinzonov
  • Gerhard Tutz
  • Torsten Hothorn
Original Paper


Many of the popular nonlinear time series models require a priori the choice of parametric functions which are assumed to be appropriate in specific applications. This approach is mainly used in financial applications, when sufficient knowledge is available about the nonlinear structure between the covariates and the response. One principal strategy to investigate a broader class on nonlinear time series is the Nonlinear Additive AutoRegressive (NAAR) model. The NAAR model estimates the lags of a time series as flexible functions in order to detect non-monotone relationships between current and past observations. We consider linear and additive models for identifying nonlinear relationships. A componentwise boosting algorithm is applied for simultaneous model fitting, variable selection, and model choice. Thus, with the application of boosting for fitting potentially nonlinear models we address the major issues in time series modelling: lag selection and nonlinearity. By means of simulation we compare boosting to alternative nonparametric methods. Boosting shows a strong overall performance in terms of precise estimations of highly nonlinear lag functions. The forecasting potential of boosting is examined on the German industrial production (IP); to improve the model’s forecasting quality we include additional exogenous variables. Thus we address the second major aspect in this paper which concerns the issue of high dimensionality in models. Allowing additional inputs in the model extends the NAAR model to a broader class of models, namely the NAARX model. We show that boosting can cope with large models which have many covariates compared to the number of observations.


Componentwise boosting Forecasting Nonlinear times series Autoregressive additive models Lag selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Audrino, F.: What drives short rate dynamics? A functional gradient descent approach. Working Paper 640, University of St. Gallen (2010) Google Scholar
  2. Audrino, F., Barone-Adesi, G.: A dynamic model of expected bond returns: A functional gradient descent approach. Comput. Stat. Data Anal. 51(4), 2267–2277 (2006) CrossRefzbMATHMathSciNetGoogle Scholar
  3. Audrino, F., Bühlmann, P.: Volatility estimation with functional gradient descent for very highdimensional financial time series. J. Comput. Finance 6(3), 65–89 (2003) Google Scholar
  4. Audrino, F., Bühlmann, P.: Splines for financial volatility. J. R. Stat. Soc., Ser. B, Stat. Methodol. 71(3), 655–670 (2009) CrossRefMathSciNetGoogle Scholar
  5. Benner, J., Meier, C.: Prognosegüte alternativer Frühindikatoren für die Konjunktur in Deutschland. Jahrb. Natl.ökon. Stat. 224(6), 637–652 (2004) Google Scholar
  6. Breitung, J., Jagodzinski, D.: Prognoseeigenschaften alternativer Indikatoren der konjunkturellen Entwicklung in Deutschland. Konjunkturpolitik 47(4), 292–314 (2001) Google Scholar
  7. Bühlmann, P.: Boosting for high-dimensional linear models. Ann. Stat. 34(2), 559–583 (2006) CrossRefzbMATHGoogle Scholar
  8. Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting. Stat. Sci. 22(4), 477–505 (2007) CrossRefGoogle Scholar
  9. Bühlmann, P., Yu, B.: Boosting with the L 2 loss: regression and classification. J. Am. Stat. Assoc. 98(462), 324–339 (2003) CrossRefzbMATHGoogle Scholar
  10. Chen, R., Tsay, R.: Nonlinear additive ARX models. J. Am. Stat. Assoc. 88(423), 955–967 (1993) CrossRefMathSciNetGoogle Scholar
  11. Chevillon, G., Hendry, D.: Non-parametric direct multi-step estimation for forecasting economic processes. Int. J. Forecast. 21(2), 201–218 (2005) CrossRefGoogle Scholar
  12. Claveria, O., Pons, E., Ramos, R.: Business and consumer expectations and macroeconomic forecasts. Int. J. Forecast. 23(1), 47–69 (2007) CrossRefGoogle Scholar
  13. Clements, M., Franses, P., Swanson, N.: Forecasting economic and financial time-series with non-linear models. Int. J. Forecast. 20(2), 169–183 (2004) CrossRefGoogle Scholar
  14. Diebold, F.X., Mariano, R.S.: Comparing predictive accuracy. J. Bus. Econ. Stat. 13(3), 253–263 (1995) CrossRefGoogle Scholar
  15. Dreger, C., Schumacher, C.: Out-of-sample performance of leading indicators for the German business cycle. Single vs combined forecasts. J. Bus. Cycle Meas. Anal. 2(1), 71–88 (2005) CrossRefGoogle Scholar
  16. Eilers, P., Marx, B.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–102 (1996) CrossRefzbMATHMathSciNetGoogle Scholar
  17. Elliot, G., Timmermann, A.: Economic forecasting. J. Econ. Lit. 66(1), 3–56 (2008) CrossRefGoogle Scholar
  18. Fenske, N., Kneib, T., Hothorn, T.: Identifying risk factors for severe childhood malnutrition by boosting additive quantile regression. Technical Report 52, Institut für Statistik, Ludwig-Maximilians-Universität München (2009).
  19. Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156. Morgan Kaufmann, San Francisco (1996) Google Scholar
  20. Friedman, J.: Multivariate adaptive regression splines. Ann. Stat. 19(1), 1–67 (1991) CrossRefzbMATHGoogle Scholar
  21. Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001) CrossRefzbMATHGoogle Scholar
  22. Fritsche, U., Stephan, S.: Leading indicators of German business cycles—an assessment of properties. Jahrb. Natl.ökon. Stat. 222(3), 289–311 (2002) Google Scholar
  23. Goldrian, G.: Handbook of Survey-Based Business Cycle Analysis. Edward Elgar, Cheltenham Glos (2007) Google Scholar
  24. Hamilton, J.: Time Series Analysis. Princeton University Press, Princeton (1994) zbMATHGoogle Scholar
  25. Hansen, P.R., Lunde, A., Nason, J.M.: The model confidence set. CREATES Research Papers 2010-76, School of Economics and Management, University of Aarhus (2010).
  26. Harvey, D., Leybourne, S., Newbold, P.: Testing the equality of prediction mean squared errors. Int. J. Forecast. 13(2), 281–291 (1997) CrossRefGoogle Scholar
  27. Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall/CRC, London (1990) zbMATHGoogle Scholar
  28. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Berlin (2001) zbMATHGoogle Scholar
  29. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, Berlin (2009a) zbMATHGoogle Scholar
  30. Hastie, T., Tibshirani, R., Leisch, F., Hornik, K., Ripley, B.D.: mda: mixture and flexible discriminant analysis. R package version 0.3-4 (2009b) Google Scholar
  31. Hothorn, T., Buehlmann, P., Kneib, T., Schmid, M., Hofner, B.: mboost: model-based boosting, R package, version 1.0-7 (2009).
  32. Huang, J., Yang, L.: Identification of non-linear additive autoregressive models. J. R. Stat. Soc., Ser. B, Stat. Methodol. 66(2), 463–477 (2004) CrossRefzbMATHMathSciNetGoogle Scholar
  33. Hüfner, F., Schröder, M.: Prognosegehalt von ifo-Geschäftserwartungen und ZEW-Konjunkturerwartungen: Ein ökonometrischer Vergleich. Jahrb. Natl.ökon. Stat. 222(3), 316–336 (2002) Google Scholar
  34. Hurvich, C.M., Simonoff, J.S., Tsai, C.-L.: Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc., Ser. B, Stat. Methodol. 60, 271–293 (1998) CrossRefzbMATHMathSciNetGoogle Scholar
  35. Hyndman, R., Koehler, A.: Another look at measures of forecast accuracy. Int. J. Forecast. 22(4), 679–688 (2006) CrossRefGoogle Scholar
  36. Kneib, T., Hothorn, T., Tutz, G.: Variable selection and model choice in geoadditive regression models. Biometrics 65, 626–634 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  37. Lewis, P., Stevens, J.: Nonlinear modeling of time series using multivariate adaptive regression splines (MARS). J. Am. Stat. Assoc. 86(416), 864–877 (1991) CrossRefzbMATHGoogle Scholar
  38. Lütkepohl, H.: Introduction to Multiple Time Series Analysis. Springer, Berlin (1991) zbMATHGoogle Scholar
  39. Lütkepohl, H.: New Introduction to Multiple Time Series Analysis. Springer, Berlin (2006) zbMATHGoogle Scholar
  40. Lutz, R.W., Kalisch, M., Bühlmann, P.: Robustified L 2 boosting. Comput. Stat. Data Anal. 52(7), 3331–3341 (2008) CrossRefzbMATHGoogle Scholar
  41. Marcellino, M., Schumacher, C.: Factor nowcasting of German GDP with ragged-edge data. A model comparison using MIDAS projections. Technical report, Bundesbank Discussion Paper, Series 1, 34/2007 (2007) Google Scholar
  42. Marcellino, M., Stock, J., Watson, M.: A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. J. Econom. 135(1–2), 499–526 (2006) CrossRefMathSciNetGoogle Scholar
  43. Matías, J., Febrero-Bande, M., González-Manteiga, W., Reboredo, J.: Boosting GARCH and neural networks for the prediction of heteroskedastic time series. Math. Comput. Model. 51(3–4), 256–271 (2010) CrossRefzbMATHGoogle Scholar
  44. Pfaff, B.: VAR, SVAR and SVEC models: Implementation within R package vars. J. Stat. Softw. 27(4) (2008).
  45. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2009). ISBN 3-900051-07-0. Google Scholar
  46. Robinzonov, N., Wohlrabe, K.: Freedom of choice in macroeconomic forecasting. CESifo Econ. Stud. 56(2), 192–220 (2010) CrossRefGoogle Scholar
  47. Schmid, M., Hothorn, T.: Boosting additive models using component-wise P-splines. Comput. Stat. Data Anal. 53(2), 298–311 (2008) CrossRefzbMATHMathSciNetGoogle Scholar
  48. Shafik, N., Tutz, G.: Boosting nonlinear additive autoregressive time series. Comput. Stat. Data Anal. 53(7), 2453–2464 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  49. Stock, J., Watson, M.: Forecasting output and inflation: The role of asset prices. J. Econ. Lit. 41(3), 788–829 (2003) CrossRefGoogle Scholar
  50. Teräsvirta, T., van Dijk, D., Medeiros, M.: Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A reexamination. Int. J. Forecast. 21(4), 755–774 (2005) CrossRefGoogle Scholar
  51. Tschernig, R., Yang, L.: Nonparametric lag selection for time series. J. Time Ser. Anal. 21(4), 457–487 (2000) CrossRefzbMATHMathSciNetGoogle Scholar
  52. Wood, S.: Generalized Additive Models: An Introduction with R. Chapman & Hall/CRC, London (2006) zbMATHGoogle Scholar
  53. Wood, S.: mgcv: GAMs with GCV smoothness estimation and GAMMs by REML/PQL. R package, version 1.4-1.1 (2009) Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Nikolay Robinzonov
    • 1
  • Gerhard Tutz
    • 2
  • Torsten Hothorn
    • 1
  1. 1.Department of StatisticsLudwig-Maximilians-Universität MünchenMunichGermany
  2. 2.Department of StatisticsLudwig-Maximilians-Universität MünchenMunichGermany

Personalised recommendations