Skip to main content
SpringerLink
Log in

Kalman recursions Aggregated Online

  • Regular Article
  • Published:
Statistical Papers Aims and scope Submit manuscript

Abstract

In this article, we aim to improve the prediction from experts’ aggregation by using the underlying properties of the models that provide the experts involved in the aggregation procedure. We restrict ourselves to the case where experts perform their predictions by fitting state-space models to the data using Kalman recursions. Using exponential weights, we construct different Kalman recursions Aggregated Online (KAO) algorithms that compete with the best expert or the best convex combination of experts in a more or less adaptive way. When the experts are Kalman recursions, we improve the existing results on experts’ aggregation literature, taking advantage of the second-order properties of the Kalman recursions. We apply our approach to Kalman recursions and extend it to the general adversarial expert setting by state-space modeling the experts’ errors. We apply these new algorithms to a real-data set of electricity consumption and show how they can improve forecast performances compared to other exponentially weighted average procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Amat C, Michalski T, Stoltz G (2018) Fundamentals and exchange rate forecastability with simple machine learning methods. J Int Money Financ 88:1–24

    Article  Google Scholar 

  • Audibert JY (2007) No fast exponential deviation inequalities for the progressive mixture rule. Adv Neural Inf Process Syst 2041–2048

  • Auger F, Hilairet M, Guerrero JM et al (2013) Industrial applications of the Kalman filter: a review. IEEE Trans Ind Electron 60(12):5458–5471

    Article  Google Scholar 

  • Ba A, Sinn M, Goude Y et al (2012) Adaptive learning of smoothing functions: application to electricity load forecasting. Neurips, pp 2519–2527. http://books.nips.cc/papers/files/nips25/NIPS2012_1205.pdf

  • Carlson NA, Berarducci MP (1994) Federated Kalman filter simulation results. Navig J Inst Navig 41(3):297–322

    Article  Google Scholar 

  • Carrassi A, Bocquet M, Bertino L et al (2018) Data assimilation in the geosciences: an overview of methods, issues, and perspectives. WIREs Clim Change 9(5):e535. https://doi.org/10.1002/wcc.535

    Article  Google Scholar 

  • Cassola F, Burlando M (2012) Wind speed and wind energy forecast through Kalman filtering of numerical weather prediction model output. Appl Energy 99:154–166. https://doi.org/10.1016/j.apenergy.2012.03.054

    Article  ADS  Google Scholar 

  • Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games, vol 2. Cambridge University Press, London. https://doi.org/10.1017/cbo9780511546921

    Book  Google Scholar 

  • Cesa-Bianchi N, Mansour Y, Stoltz G (2007) Improved second-order bounds for prediction with expert advice. Mach Learn 66(2–3):321–352

    Article  Google Scholar 

  • Chen S (2011) Kalman filter for robot vision: a survey. IEEE Trans Ind Electron 59(11):4409–4420

    Article  Google Scholar 

  • Clark S, Hyndman RJ, Pagendam D et al (2020) Modern strategies for time series regression. Int Stat Rev 88(S1):S179–S204

    Article  Google Scholar 

  • De Franco C, Geissler C, Margot V et al (2020) ESG investments: filtering versus machine learning approaches. arXiv preprint. arXiv:2002.07477

  • De Vilmarest J, Goude Y (2022) State-space models for online post-covid electricity load forecasting competition. IEEE Open Access J Power Energy. https://doi.org/10.1109/OAJPE.2022.3141883

    Article  Google Scholar 

  • de Vilmarest J, Wintenberger O (2021) Stochastic online optimization using Kalman recursion. JMLR 22:1–55

    MathSciNet  Google Scholar 

  • Devaine M, Gaillard P, Goude Y et al (2013) Forecasting electricity consumption by aggregating specialized experts. Mach Learn 90(2):231–260

    Article  MathSciNet  Google Scholar 

  • Diderrich GT (1985) The Kalman filter from the perspective of Goldberger—Theil estimators. Am Stat 39(3):193–198

    Google Scholar 

  • Dordonnat V, Koopman SJ, Ooms M (2012) Dynamic factors in periodic time-varying regressions with an application to hourly electricity load modelling. Comput Stat Data Anal 56(11):3134–3152

    Article  MathSciNet  Google Scholar 

  • Durbin J, Koopman SJ (2012) Time series analysis by state space methods. Oxford University Press, London. https://doi.org/10.1093/acprof:oso/9780199641178.001.0001

    Book  Google Scholar 

  • Evensen G (2002) Sequential data assimilation for nonlinear dynamics: the ensemble Kalman filter. Springer, Cham, pp 97–116. https://doi.org/10.1007/978-3-662-22648-3_6

    Book  Google Scholar 

  • Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MathSciNet  Google Scholar 

  • Gaillard P, Goude Y (2014) Forecasting electricity consumption by aggregating experts; how to design a good set of experts. Lecture notes in statistics: modeling and stochastic learning for forecasting in high dimension

  • Gaillard P, Goude Y (2016) Opera: online prediction by expert aggregation, r package version 1.0. https://CRAN.R-project.org/package=opera

  • Gaillard P, Wintenberger O (2017) Sparse accelerated exponential weights. In: Artificial intelligence and statistics, PMLR, pp 75–82

  • Gaillard P, Wintenberger O (2018) Efficient online algorithms for fast-rate regret bounds under sparsity. In: Advances in neural information processing systems, pp 7026–7036

  • Gaillard P, Stoltz G, Van Erven T (2014) A second-order bound with excess losses. In: Conference on learning theory, pp 176–196

  • Gaillard P, Goude Y, Nedellec R (2016) Additive models and robust aggregation for GEFCom2014 probabilistic electric load and electricity price forecasting. Int J Forecast 32(3):1038–1050

    Article  Google Scholar 

  • Geysen D, De Somer O, Johansson C et al (2018) Operational thermal load forecasting in district heating networks using machine learning and expert advice. Energy Build 162:144–153

    Article  Google Scholar 

  • Giraud C (2008) Mixing least-squares estimators when the variance is unknown. Bernoulli 14(4):1089–1107

    Article  MathSciNet  Google Scholar 

  • Goehry B, Goude Y, Massart P et al (2019) Aggregation of multi-scale experts for bottom-up load forecasting. IEEE Trans Smart Grid 11(3):1895–1904

    Article  Google Scholar 

  • Govaers F, Koch W (2012) An exact solution to track-to-track-fusion at arbitrary communication rates. IEEE Trans Aerosp Electron Syst 48(3):2718–2729

    Article  ADS  Google Scholar 

  • Guo L (1994) Stability of recursive stochastic tracking algorithms. SIAM J Control Optim 32(5):1195–1225

    Article  MathSciNet  Google Scholar 

  • Hand DJ (2008) Forecasting with exponential smoothing: the state space approach by Rob J. Hyndman, Anne B. Koehler, J. Keith Ord, Ralph D. Snyder. Int Stat Rev 77(2):315–316. https://doi.org/10.1111/j.1751-5823.2009.00085_17.x

    Article  Google Scholar 

  • Hazan E (2016) Introduction to online convex optimization. Found Trends® Optim 2(3–4):157–325

  • Herbster M, Warmuth MK (1998) Tracking the best expert. Mach Learn 32(2):151–178

    Article  Google Scholar 

  • Huard M, Garnier R, Stoltz G (2020) Hierarchical robust aggregation of sales forecasts at aggregated levels in e-commerce, based on exponential smoothing and Holt’s linear trend method. Working paper or preprint. https://hal.archives-ouvertes.fr/hal-02794320

  • Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(1):35–45. https://doi.org/10.1115/1.3662552

    Article  MathSciNet  Google Scholar 

  • Littlestone N, Warmuth MK (1994) The weighted majority algorithm. Inf Comput 108(2):212–261

    Article  MathSciNet  Google Scholar 

  • Mackenzie D (2003) Ensemble Kalman filters bring weather models up to date. SIAM News 36(8):10–03

    Google Scholar 

  • McGee LA, Schmidt SF (1985) Discovery of the Kalman filter as a practical tool for aerospace and industry. Vol. 86847. National Aeronautics and Space Administration

  • R Core Team (2019) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. https://www.R-project.org/

  • Ridder KD, Kumar U, Lauwaet D et al (2012) Kalman filter-based air quality forecast adjustment. Atmos Environ 50:381–384. https://doi.org/10.1016/j.atmosenv.2012.01.032

    Article  ADS  CAS  Google Scholar 

  • Rigby RA, Stasinopoulos DM (2005) Generalized additive models for location, scale and shape. J R Stat Soc Ser C (Appl Stat) 54(3):507–554

    Article  MathSciNet  Google Scholar 

  • Schneider W (1988) Analytical uses of Kalman filtering in econometrics—a survey. Stat Pap 29(1):3–33

    Article  MathSciNet  Google Scholar 

  • Thorey J, Chaussin C, Mallet V (2018) Ensemble forecast of photovoltaic power with online CRPS learning. Int J Forecast 34(4):762–773

    Article  Google Scholar 

  • Tsay RS (2005) Analysis of financial time series, 2nd edn. Wiley, Hoboken

    Book  Google Scholar 

  • Vovk VG (1990) Aggregating strategies. In: Proceedings of computational learning theory

  • Vovk V (1998) A game of prediction with expert advice. J Comput Syst Sci 56(2):153–173

    Article  MathSciNet  Google Scholar 

  • Wintenberger O (2017) Optimal learning with Bernstein online aggregation. Mach Learn 106(1):119–141

    Article  MathSciNet  Google Scholar 

  • Wood S (2015) Package ‘mgcv’. R package version 1(29):729

Download references

Acknowledgements

We would like to thank Joseph de Vilmarest for fruitful discussions and the share of codes.

Author information

Authors and Affiliations

  1. MIA Paris-Saclay (UMR 518), Paris-Saclay University, AgroParisTech Campus, 22 Place de l’Agronomie, 91120, Palaiseau, France

    Eric Adjakossa

  2. OSIRIS Department, EDF Lab, 7 Boulevard Gaspard Monge, 91120, Palaiseau, France

    Yannig Goude

  3. Laboratoire de Probabilités, Statistique et Modélisation (LPSM, UMR 8001), Sorbonne Université, 4 place Jussieu, 75005, Paris, France

    Olivier Wintenberger

  4. Wolfgang Pauli Institut, c/o Fakultät für Mathematik, Universität Wien, 1090, Vienna, Austria

    Olivier Wintenberger

Authors
  1. Eric Adjakossa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannig Goude
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Olivier Wintenberger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eric Adjakossa.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (rdata 335 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adjakossa, E., Goude, Y. & Wintenberger, O. Kalman recursions Aggregated Online. Stat Papers 65, 909–944 (2024). https://doi.org/10.1007/s00362-023-01410-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00362-023-01410-7

Keywords

Search

Navigation