Change Detection with Kalman Filter and CUSUM

  • Milton Severo
  • João Gama
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4265)


In most challenging applications learning algorithms acts in dynamic environments where the data is collected over time. A desirable property of these algorithms is the ability of incremental incorporating new data in the actual decision model. Several incremental learning algorithms have been proposed. However most of them make the assumption that the examples are drawn from a stationary distribution [13]. The aim of this study is to present a detection system (DSKC) for regression problems. The system is modular and works as a post-processor of a regressor. It is composed by a regression predictor, a Kalman filter and a Cumulative Sum of Recursive Residual (CUSUM) change detector. The system continuously monitors the error of the regression model. A significant increase of the error is interpreted as a change in the distribution that generates the examples over time. When a change is detected, the actual regression model is deleted and a new one is constructed. In this paper we tested DSKC with a set of three artificial experiments, and two real-world datasets: a Physiological dataset and a clinic dataset of Sleep Apnoea. Sleep Apnoea is a common disorder characterized by periods of breathing cessation (apnoea) and periods of reduced breathing (hypopnea) [7]. This is a real-application where the goal is to detect changes in the signals that monitor breathing. The experimental results showed that the system detected changes fast and with high probability. The results also showed that the system is robust to false alarms and can be applied with efficiency to problems where the information is available over time.


False Alarm Learning Algorithm Kalman Filter Change Detection Regression Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andre, D., Stone, P.: Physiological data modeling contest. Technical report, University of Texas at Austin (2004)Google Scholar
  2. 2.
    Basseville, M., Nikiforov, I.: Detection of Abrupt Changes: Theory and Applications. Prentice-Hall, Englewood Cliffs (1993)Google Scholar
  3. 3.
    Bhattacharyya, G., Johnson, R.: Statistical Concepts and Methods. John Willey & Sons, New York (1977)Google Scholar
  4. 4.
    Bianchi, G., Tinnirello, I.: Kalman filter estimation of the number of competing terminals in ieee. In: The 22nd Annual Joint Conference of IEEE Computer and Communications (2003)Google Scholar
  5. 5.
    Cauwenberghs, Gert, Poggio, Tomaso: Incremental and decremental support vector machine learning. Advances in Neural Information Processing Systems, 13 (2001)Google Scholar
  6. 6.
    Domingos, P., Hulten, G.: Mining high-speed data streams. In: Knowledge Discovery and Data Mining, pp. 71–80 (2000)Google Scholar
  7. 7.
    Flemons, W.W., Littner, M.R., Rowley, J.A., Gay, W.M.A.P., Hudgel, D.W., McEvoy, R.D., Loube, D.I.: Home diagnosis of sleep apnoeas: A systematic review of the literature. In: Chest, vol. 1543-1579, pp. 211–237 (2003)Google Scholar
  8. 8.
    Friedman, J.: Multivariate adaptive regression splines. Annals of Statistics 19(1), 1–141 (1991)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Gama, J., Medas, P., Castillo, G.: Learning with drift detection. In: Brazilian AI Conference, pp. 286–295. Springer, Heidelberg (2004)Google Scholar
  10. 10.
    Grant, E., Leavenworth, R.: Statistical Quality Control. McGraw-Hill, New York (1996)Google Scholar
  11. 11.
    Guimarães, G., Peter, J.H., Penzel, T., Ultsch, A.: A method for automated temporal knowledge acquisition applied to sleep-related breathing disorders. Artificial Intelligence in Medicine 23, 211–237 (2001)CrossRefGoogle Scholar
  12. 12.
    Higgins, C.M., Goodman, R.M.: Incremental learning using rule-based neural networks. In: International Joint Conference on Neural Networks, Seattle, WA, pp. 875–880 (1991)Google Scholar
  13. 13.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proceedings of Knowledge Discovery and Data Mining. ACM Press, New York (2001)Google Scholar
  14. 14.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. In: Transaction of ASME - Journal of Basic Engineering, 35–45 (1960)Google Scholar
  15. 15.
    Pang, K.P., Ting, K.M.: Improving the centered CUSUMs statistic for structural break detection in time series. In: Proc. 17th Australian Join Conference on Artificial Intelligence. Springer, Heidelberg (2004)Google Scholar
  16. 16.
    R Development Core Team. R: A language and environment for statistical computing. In: R Foundation for Statistical Computing, Vienna, Austria (2005) ISBN 3-900051-07-0Google Scholar
  17. 17.
    Rojas, R.: The Kalman filter. Technical report, Freie University of Berlin (2003)Google Scholar
  18. 18.
    Welch, G., Bishop, G.: An introduction to the Kalman filter. Technical report, 95-041, Department of Computer Science, University of North Caroline at Chapel Hill (April 2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Milton Severo
    • 1
    • 2
  • João Gama
    • 1
    • 3
  1. 1.Fac. of EconomyUniversity of PortoPortugal
  2. 2.Department of Hygiene and Epidemiology, Fac. of MedicineUniversity of PortoPortugal
  3. 3.LIACCUniversity of Porto 

Personalised recommendations