Abstract
Prediction Interval (PI) is a powerful technique for quantifying the uncertainty of regression tasks. However, research on PI for data streams has not received much attention. Moreover, traditional PI-generating approaches are not directly applicable due to the dynamic and evolving nature of data streams. This paper presents AdaPI (ADAptive Prediction Interval), a novel method that can automatically adjust the interval width by an appropriate amount according to historical information to converge the coverage to a user-defined percentage. AdaPI can be applied to any streaming PI technique as a postprocessing step. This paper develops an incremental variant of the pervasive Mean and Variance Estimation (MVE) method for use with AdaPI. An empirical evaluation over a set of standard streaming regression tasks demonstrates AdaPI’s ability to generate compact prediction intervals with a coverage close to the desired level, outperforming alternative methods.
Y. Sun—I would like to acknowledge the support from TAIAO project.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barber, R.F., Candes, E.J., Ramdas, A., Tibshirani, R.J.: Predictive inference with the jackknife+ (2021)
Bifet, A., et al.: Moa: massive online analysis, a framework for stream classification and clustering. PMLR (2010)
Breiman, L.: Bagging predictors. Mach. Learn. 24, 123–140 (1996)
Fanaee-T, H., Gama, J.: Event labeling combining ensemble detectors and background knowledge. Progr. Artif. Intell. 2, 113–127 (2014)
Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11(1), 86–92 (1940)
Gomes, H.M., Barddal, J.P., Ferreira, L.E.B., Bifet, A.: Adaptive random forests for data stream regression. In: ESANN (2018)
Gomes, H.M., Montiel, J., Mastelini, S.M., Pfahringer, B., Bifet, A.: On ensemble techniques for data stream regression. In: 2020 IJCNN. IEEE (2020)
Hadjicharalambous, M., Polycarpou, M.M., Panayiotou, C.G.: Neural network-based construction of online prediction intervals. Neural Comput. Appl. 32(11), 6715–6733 (2020)
Hahn, G.J., Factors for calculating two-sided prediction intervals for samples from a normal distribution. J. Am. Stat. Assoc. (1969)
Hahn, G.J., Nelson, W.: A survey of prediction intervals and their applications. J. Qual. Technol. 5(4), 178–188 (1973)
Ikonomovska, E., Gama, J., Džeroski, S.: Learning model trees from evolving data streams. Data Min. Knowl. Disc. 23, 128–168 (2011)
Khosravi, A., Nahavandi, S., Creighton, D., Atiya, A.F.: Lower upper bound estimation method for construction of neural network-based prediction intervals. IEEE Trans. Neural Netw. 22(3), 337–346 (2010)
Liu, Y., Zhao, J., Wang, W., Pedrycz, W.: Prediction intervals for granular data streams based on evolving type-2 fuzzy granular neural network dynamic ensemble. IEEE Trans. Fuzzy Syst. 29(4), 874–888 (2020)
Nix, D.A., Weigend, A.S.: Estimating the mean and variance of the target probability distribution. In: ICNN’94. IEEE (1994)
Shafer, G., Vovk, V.: A tutorial on conformal prediction. J. Mach. Learn. Res. 9(3) (2008)
Shrestha, D.L., Solomatine, D.P.: Machine learning approaches for estimation of prediction interval for the model output. Neural Netw. 19(2), 225–235 (2006)
Sun, Y., Pfahringer, B., Gomes, H.M., Bifet, A.: SOKNL: a novel way of integrating k-nearest neighbours with adaptive random forest regression for data streams. Data Min. Knowl. Discov. (2022)
Waugh, S.G.: Extending and benchmarking cascade-correlation: extensions to the cascade-correlation architecture and benchmarking of feed-forward supervised artificial neural networks. Ph.D. thesis, University of Tasmania (1995)
Xu, C., Xie, Y.: Conformal prediction interval for dynamic time-series. In: International Conference on Machine Learning, pp. 11559–11569. PMLR (2021)
Zhao, J., Wang, W., Sheng, C., Zhao, J., Wang, W., Sheng, C.: Industrial prediction intervals with data uncertainty. In: Data-Driven Prediction for Industrial Processes and Their Applications, pp. 159–222 (2018)
Zhao, X., Barber, S., Taylor, C.C., Milan, Z.: Interval forecasts based on regression trees for streaming data. Adv. Data Anal. Classif. 15, 5–36 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, Y., Pfahringer, B., Murilo Gomes, H., Bifet, A. (2024). Adaptive Prediction Interval for Data Stream Regression. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14647. Springer, Singapore. https://doi.org/10.1007/978-981-97-2259-4_10
Download citation
DOI: https://doi.org/10.1007/978-981-97-2259-4_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2261-7
Online ISBN: 978-981-97-2259-4
eBook Packages: Computer ScienceComputer Science (R0)