Abstract
The forecasting of time-series data plays an important role in various domains. It is of significance in theory and application to improve prediction accuracy of the time-series data. With the progress in the study of time-series, time-series forecasting model becomes more complicated, and consequently great concern has been drawn to the techniques in designing the forecasting model. A modeling method which is easy to use by engineers and may generate good results is in urgent need. In this paper, a gradient-boost AR ensemble learning algorithm (AREL) is put forward. The effectiveness of AREL is assessed by theoretical analyses, and it is demonstrated that this method can build a strong predictive model by assembling a set of AR models. In order to avoid fitting exactly any single training example, an insensitive loss function is introduced in the AREL algorithm, and accordingly the influence of random noise is reduced. To further enhance the capability of AREL algorithm for non-stationary time-series, improve the robustness of algorithm, discourage overfitting, and reduce sensitivity of algorithm to parameter settings, a weighted kNN prediction method based on AREL algorithm is presented. The results of numerical testing on real data demonstrate that the proposed modeling method and prediction method are effective.
Similar content being viewed by others
References
Chen S Y, Song S F, Li L X, et al. Survey on smart grid technology. Power Syst Technol, 2009, 33(8): 1–7
Guan X H, Zhai Q Z, Feng Y H, et al. Optimization based scheduling for a class of production systems with integral constraints. Sci China Ser E-Tech Sci, 2009, 52(12): 3533–3544
Gao Y L, Gao F, Zhai Q Z, et al. Balance analysis of large-scale power consuming corporation. Proc CSEE, 2009, 28(19): 71–77
Gao Y L, Gao F, Pan J Y, et al. Self-scheduling for electrical energy balance and output power control of energy-intensive enterprises. Proc CSEE, 2010, 30(19): 76–83
Alexiadis M C, Dokopoulos P S, Sahsamanoglou H S. Short term forecasting of wind speed and related electrical power. Sol Energy, 1998, 63(1): 61–68
Yang X Y, Xiao Y, Chen S Y. Wind Speed and generated power forecasting in wind farm. Proc CSEE, 2005, 25(11): 1–5
Zhang L, Xiong G L, Liu H S, et al. TVAR and SVM based fault diagnosis method for rotation machine. Proc CSEE, 2007, 27(9): 99–103
Xu F, Wang Z F, Wang B S. Research on AR model applied to forecast trend of vibration signals. J Tsinghua Univ (Science and Technology), 1999, 29(4): 57–59
Wang Y Q, Liu W Q. Probabilistic properties of functional coefficient auto-regression models with regularly varying tailed. J Shanxi Univ (Natural Science Edition), 2008, 31(3): 318–322
Brown B G, Katz R W, Murphy A H. Time-series models to simulate and forecast wind speed and wind power. J Appl Meteor, 1984, 23(8): 1184–1195
Sun C S, Wang Y N, Li X R. A vector autoregression model of hourly wind speed and its applications in hourly wind speed forecasting. Proc CSEE, 2008, 28(14): 112–117
Franses P H, Paap R. Model selection in periodic autoregressions. Oxford Bull Econom Statist, 1994, 56(4): 421–439
Zhao H W, Ren Z, Huang W Y. A short term load forecasting method based on PAR model. Proc CSEE, 1997, 17(5): 348–351
Lütkepohl H. New Introduction to Multiple Time-Series Analysis. Berlin: Springer, 2005
Allen P G, Morzuch B J. Twenty-five years of progress, problems, and conflicting evidence in econometric forecasting. What about the next 25 years? Int J Forecast, 2006, 22(3): 475–492
Greene W H. Econometric Analysis. New Jersey: Prentice Hall, 2003
Ewing B T, Kruse J B, Schroeder J L, et al. Time-series analysis of wind speed using VAR and the generalized impulse response technique. J Wind Eng Ind Aerodyn, 2007, 95(3): 209–219
Jin Y, An H Z. Nonlinear autoregressive models with heavy-tailed innovation. Sci China Ser A-Math, 2005, 48(3): 333–340
Wu Q F, Zhao Y M, Li Y, et al. Testing heteroscedasticity by wavelets in a nonparametric autoregressive model. Acta Math Appl Sin, 2009, 32(4): 595–607
Fei W C, Bai L. Time-varying parameter auto-regressive models for autocovariance nonstationary time-series. Sci China Ser A-Math, 2009, 52(3): 577–584
Du X L, Wang F Q. Modal parameter identification under non-stationary ambient excitation based on continuous time AR Model. Sci China Ser E-Tech Sci, 2009, 52(11): 3180–3187
Du X L, Wang F Q. Modal identification of system driven by lévy random excitation based on continuous time AR model. Sci China Ser E-Tech Sci, 2009, 52(12): 3649–3653
Granger C W J, Joyeux R. An introduction to long-memory time-series models and fractional differencing. J Time-Ser Anal, 1980, 1(1): 15–29
Hannan E J, Deistler M. The Statistical Theory of Linear Systems. New York: Springer, 1988
Chen R, Tsay R S. Nonlinear additive ARX models. J Am Statist Assoc, 1993, 88(423): 955–967
Liu C H. Theories and Methods in Power Load Forecasting. Haerbin: Harbin Institute of Technology Press, 1987
Zhao H W, Ren Z, Huang W Y. Short term load forecasting considering weekly period based on PAR. Proc CSEE, 1997, 17(3): 211–216
Schapire R E, Singer Y. Improved Boosting algorithms using confidence-rated predictions. Mach Learn, 1999, 37(3): 297–336
Drucker H, Cortes C, Jackl L D, et al. Boosting and other ensemble methods. Neural Comput, 1994, 6(6): 1289–1301
Schapire R E, Freund Y, Bartlett P L, et al. Boosting the margin: A new explanation for the effectiveness of voting methods. Ann Stat, 1998, 26(5): 1651–1686
Wang R X. Stochastic Process. Xi’an: Xi’an Jiaotong University Press, 1995
Hu F. Hierarchical identification for paramters of autoregressive models. ACTA Autom Sin, 1994, 20(4): 464–469
Lin Z H, Feng R Z. A least square estimation of autoregressive processes. Acta Scientiarium Naturalium Universitatis Jilinensis, 2001, (2): 1–4
Meng Z W. Maximum liklihood estimation for vector autoression models. J Shandong Inst Technol, 2001, 15(2): 25–28
Gu L. The Application of Time-Series Analysis Technique in Economy. Beijing: Urban Statistical Yearbook of China, 1994
Koutsoyiannis A. Theory of Econometrics, an Introductory Exposition of Econometric Method. London and Basingstoke: The Macmillan Press LTD, 1977
Valiant L G. A theory of the learnable. Commun ACM, 1984, 27(11): 1134–1142
Kearns M, Valiant L G. Cryptographic limitations on learning Boolean formulae and finite automata. J ACM, 1994, 41(1): 67–95
Schapire R E. The strength of weak learnability. Mach Learn, 1990, 5(2): 197–227
Freund Y. Boosting a weak algorithm by majority. Inf Comput, 1995, 121(2): 256–285
Freund Y, Schapire R E. A decision theoretic generation of online learning and an application to boosting. J Comput Syst Sci, 1997, 55(1): 119–139
Friedman J H. Greedy function approximation: A gradient boosting machine. Ann Stat, 2001, 29(5): 1189–1232
Gao Y L, Gao F. Edited AdaBoost by weighted kNN. Neurocomputing, 2010, 73(16–18): 3079–3088
Vapnic V N. The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995
Lange M. On the uncertainty of wind power predictions—Analysis of the forecast accuracy and statistical distribution of errors. J Sol Energy Eng, 2005, 127(2): 177–184
Möhrlen C. Uncertainty in Wind Energy Forecasting. Cork: National University of Ireland, 2004
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gao, Y., Pan, J., Ji, G. et al. A time-series modeling method based on the boosting gradient-descent theory. Sci. China Technol. Sci. 54, 1325–1337 (2011). https://doi.org/10.1007/s11431-011-4340-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11431-011-4340-1