Soft Computing

, Volume 17, Issue 11, pp 1971–1981 | Cite as

Ensembles of change-point methods to estimate the change point in residual sequences

  • Cesare Alippi
  • Giacomo Boracchi
  • Manuel Roveri


Change-point methods (CPMs) are statistical tests design to assess whether a given sequence comes from an unique, stationary, data-generating process. CPMs eventually estimate the change-point location, i.e., the point where the data-generating process shifted. While there exists a large literature concerning CPMs meant for sequences of independent and identically distributed (i.i.d.) random variables, their use on time-dependent signals has not been properly investigated. In this case, a straightforward solution consists in computing at first the residuals between the observed signal and the output of a suitable approximation model, and then applying the CPM on the residual sequence. Unfortunately, in practical applications, such residuals are seldom i.i.d., and this may prevent the CPMs to operate properly. To counteract this problem, we introduce the ensemble of CPMs, which aggregates several estimates obtained from CPMs executed on different subsequences of residuals, obtained from random sampling. Experiments show that the ensemble of CPMs improves the change-point estimates when the residuals are not i.i.d., as it is often the case in real-world scenarios.


Ensemble of methods Change-point methods Changes in processes Residual sequences 


  1. Akaike H (1974) A new look at the statistical model identification. IEEE Transactions on Automatic Control, vol 19, no 6, pp 716–723Google Scholar
  2. Alippi C, Boracchi G, Roveri M (2011) A just-in-time adaptive classification system based on the intersection of confidence intervals rule. Neural Netw 24(8):791–800CrossRefGoogle Scholar
  3. Alippi C, Boracchi G, Puig V, Roveri M (2013a) A hierarchy of change-point methods for estimating the time instant of leakages in water distribution networks. In: Proceedings of LEAPS, the 1st workshop on learning strategies and data processing in nonstationary environments, in 9th AIAI conference, pp 1–10Google Scholar
  4. Alippi C, Boracchi G, Puig V, Roveri M (2013b) An ensemble approach to estimate the fault-time instant. In: Proceedings of the 4th international conference on intelligent control and information processing (ICICIP 2013)Google Scholar
  5. Alippi C, Boracchi G, Roveri M (2013c) Just-in-time classifiers for recurrent concepts. IEEE Transactions on Neural Networks and Learning Systems, vol 24, no 4, pp 620–634Google Scholar
  6. Alippi C, Liu D, Zhao D, Bu L (2013d) Detecting and reacting to changes in sensing units: the active classifier case. IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol PP, no 99, p 1Google Scholar
  7. Alippi C, Ntalampiras S, Roveri M (2013e) Model ensemble for an effective on-line reconstruction of missing data in sensor networks. In: International joint conference on neural networks (IJCNN 2013)Google Scholar
  8. Anderson TW (1962) On the distribution of the two-sample Cramer–von Mises criterion. Ann Math Stat 33(3):1148–1159CrossRefzbMATHGoogle Scholar
  9. Bai J (November 1997) Estimation of a change point in multiple regression models. Rev Econ Stat 79(4):551–563Google Scholar
  10. Basseville M, Nikiforov IV (1993) Detection of abrupt changes: theory and application. Prentice-Hall, Upper Saddle RiverGoogle Scholar
  11. Chen J, Gupta AK (2000) Parametric statistical change point analysis. Birkhauser, BaselGoogle Scholar
  12. Dietterich T (2000) Ensemble methods in machine learning. Multiple classifier systems, pp 1–15Google Scholar
  13. Gustafsson F (2000) Adaptive filtering and change detection. Wiley, New York (online).,descCd-description.html
  14. Hawkins DM (1977) English testing a sequence of observations for a shift in location. English J Am Stat Assoc 72(357):180–186 (online).
  15. Hawkins DM, Qiu P, Kang CW (2003) The changepoint model for statistical process control. J Qual Technol 35(4):355–366Google Scholar
  16. Hawkins DM, Zamba KD (2005) A change-point model for a shift in variance. J Qual Technol 37(1):21–31Google Scholar
  17. Isermann R (2006) Fault-diagnosis systems: an introduction from fault detection to fault tolerance. Springer, New YorkGoogle Scholar
  18. Krogh A, Sollich P (1997) Statistical mechanics of ensemble learning. Phys Rev E 55(1):811CrossRefGoogle Scholar
  19. Lepage Y (April 1974) A combination of Wilcoxon’s and Ansari–Bradley’s statistics. Biometrika 58(1):213–217Google Scholar
  20. Ljung L (1986) System identification: theory for the user. Prentice-Hall, Inc., Upper Saddle RiverGoogle Scholar
  21. Ljung L (1999) System identification. Wiley, New YorkGoogle Scholar
  22. Mangalova E, Agafonov E (2012) Time series forecasting using ensemble of ar models with time-varying structure. In: 2012 IEEE conference on evolving and adaptive intelligent systems (EAIS), pp 198–203Google Scholar
  23. Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60 (online). Google Scholar
  24. Mood AM (1954) On the asymptotic efficiency of certain nonparametric two-sample tests. Ann Math Stat 25(3):514–522Google Scholar
  25. Perry MB, Pignatiello JJ (2010) Identifying the time of step change in the mean of autocorrelated processes. J Appl Stat 37(1):119–136MathSciNetCrossRefGoogle Scholar
  26. Pettitt AN (1979) A non-parametric approach to the change-point problem. Appl Stat 28(2):126–135 (online). Google Scholar
  27. Reeves J, Chen J, Wang XL, Lund R, QiQi L (2007) A review and comparison of changepoint detection techniques for climate data. J Appl Meteorol Climatol 46(6):900–915 (online). Google Scholar
  28. Ross GJ (2013) Parametric and nonparametric sequential change detection in R: The cpm package. J Stat Softw (forthcoming)Google Scholar
  29. Ross GJ, Tasoulis DK, Adams NM (2011) Nonparametric monitoring of data streams for changes in location and scale. Technometrics 53(4):379–389MathSciNetCrossRefGoogle Scholar
  30. Ross G, Adams NM (2012) Two nonparametric control charts for detecting arbitrary distribution changes. J Qual Technol 44(22):102–116Google Scholar
  31. Wichard J, Ogorzalek M (2004) Time series prediction with ensemble models. In: Proceedings of 2004 IEEE international joint conference on neural networks, vol 2, pp 1625–1630 Google Scholar
  32. Zamba KD, Hawkins DM (2006) English a multivariate change-point model for statistical process control. English Technometrics 48(4):539–549 (online). Google Scholar
  33. Zhou Z-H (2012) Ensemble methods foundations and algorithms. Chapman & Hall, LondonGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Cesare Alippi
    • 1
  • Giacomo Boracchi
    • 1
  • Manuel Roveri
    • 1
  1. 1.Dipartimento di Elettronica, Informazione e BioingegneriaPolitecnico di MilanoMilanItaly

Personalised recommendations