A New Combination of Diversity Techniques in Ensemble Classifiers for Handling Complex Concept Drift

  • Imen KhamassiEmail author
  • Moamar Sayed-Mouchaweh
  • Moez Hammami
  • Khaled Ghédira
Part of the Studies in Big Data book series (SBD, volume 41)


Recent advances in Computational Intelligent Systems have focused on addressing complex problems related to the dynamicity of the environments. Generally in dynamic environments, data are presented as streams that may evolve over time and this is known by concept drift. Handling concept drift through ensemble classifiers has received a great interest in last decades. The success of these ensemble methods relies on their diversity. Accordingly, various diversity techniques can be used like block-based data, weighting-data or filtering-data. Each of these diversity techniques is efficient to handle certain characteristics of drift. However, when the drift is complex, they fail to efficiently handle it. Complex drifts may present a mixture of several characteristics (speed, severity, influence zones in the feature space, etc.) which may vary over time. In this case, drift handling is more complicated and requires new detection and updating tools. For this purpose, a new ensemble approach, namely EnsembleEDIST2, is presented. It combines the three diversity techniques in order to take benefit from their advantages and outperform their limits. Additionally, it makes use of EDIST2, as drift detection mechanism, in order to monitor the ensemble’s performance and detect changes. EnsembleEDIST2 was tested through different scenarios of complex drift generated from synthetic and real datasets. This diversity combination allows EnsembleEDIST2 to outperform similar ensemble approaches in terms of accuracy rate, and present stable behaviors in handling different scenarios of complex drift.


Concept Drift Complex Drift Drift Detection Mechanism Block-based Data Drift Handling 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bifet, A., Frank, E., Holmes, G., Pfahringer, B., Sugiyama, M., Yang, Q.: Accurate ensembles for data streams: combining restricted hoeffding trees using stacking. In: 2nd Asian Conference on Machine Learning (ACML2010), pp. 225–240 (2010)Google Scholar
  2. 2.
    Bifet, A., Holmes, G., Pfahringer, B.: Leveraging bagging for evolving data streams. In: Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases: Part I. ECML PKDD’10, pp. 135–150. Springer, Berlin, Heidelberg (2010)Google Scholar
  3. 3.
    Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010)Google Scholar
  4. 4.
    Brzezinski, D., Stefanowski, J.: Accuracy updated ensemble for data streams with concept drift. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) Hybrid Artificial Intelligent Systems. Lecture Notes in Computer Science, vol. 6679, pp. 155–163. Springer, Berlin, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Brzezinski, D., Stefanowski, J.: Reacting to different types of concept drift: the accuracy updated ensemble algorithm. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 81–94 (2014)CrossRefGoogle Scholar
  6. 6.
    Domingos, P., Hulten, G.: Mining high-speed data streams. In: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 71–80. KDD00. ACM, New York (2000)Google Scholar
  7. 7.
    Gama, J., Sebastião, R., Rodrigues, P.: On evaluating stream learning algorithms. Mach. Learn. 90(3), 317–346 (2013)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Harries, M.: Splice-2 comparative evaluation: electricity pricing. Technical Report, The University of South Wales (1999)Google Scholar
  9. 9.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, August 26–29, 2001, pp. 97–106 (2001)Google Scholar
  10. 10.
    Katakis, I., Tsoumakas, G., Vlahavas, I.: Tracking recurring contexts using ensemble classifiers: an application to email filtering. Knowl. Inform. Syst. 22(3), 371–391 (2010)CrossRefGoogle Scholar
  11. 11.
    Khamassi, I., Sayed-Mouchaweh, M.: Drift detection and monitoring in non-stationary environments. In: Evolving and Adaptive Intelligent Systems (EAIS), Linz, pp. 1–6 (2014)Google Scholar
  12. 12.
    Khamassi, I., Sayed-Mouchaweh, M.: Self-adaptive ensemble classifier for handling complex concept drift. In: 2nd ECML/PKDD 2017 Workshop on Large-scale Learning from Data Streams in Evolving Environments, Skopje, pp. 52–72 (2017)Google Scholar
  13. 13.
    Khamassi, I., Sayed-Mouchaweh, M., Hammami, M., Ghédira, K.: Ensemble classifiers for drift detection and monitoring in dynamical environments. In: Annual Conference of the Prognostics and Health Management Society, New Orlean (2013)Google Scholar
  14. 14.
    Khamassi, I., Sayed-Mouchaweh, M., Hammami, M., Ghédira, K.: Self-adaptive windowing approach for handling complex concept drift. Cogn. Comput. 7(6), 772–790 (2015)CrossRefGoogle Scholar
  15. 15.
    Khamassi, I., Sayed-Mouchaweh, M., Hammami, M., Ghédira, K.: Discussion and review on evolving data streams and concept drift adapting. Evol. Syst. 9(1), 1–23 (2018)CrossRefGoogle Scholar
  16. 16.
    Kolter, J.Z., Maloof, M.A.: Dynamic weighted majority: an ensemble method for drifting concepts. J. Mach. Learn. Res. 8, 2755–2790 (2007)zbMATHGoogle Scholar
  17. 17.
    Minku, L., White, A., Yao, X.: The impact of diversity on online ensemble learning in the presence of concept drift. IEEE Trans. Knowl. Data Eng. 22(5), 730–742 (2010)CrossRefGoogle Scholar
  18. 18.
    Minku, L., Yao, X.: Ddd: a new ensemble approach for dealing with concept drift. IEEE Trans. Knowl. Data Eng. 24(4), 619–633 (2012)CrossRefGoogle Scholar
  19. 19.
    Oza, N.C., Russell, S.: Online bagging and boosting. In: Artificial Intelligence and Statistics 2001, pp. 105–112. Morgan Kaufmann, Boston (2001)Google Scholar
  20. 20.
    Polikar, R., Upda, L., Upda, S., Honavar, V.: Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 31(4), 497–508 (2001)CrossRefGoogle Scholar
  21. 21.
    Ren, Y., Zhang, L., Suganthan, P.N.: Ensemble classification and regression-recent developments, applications and future directions. IEEE Comput. Intell. Mag. 11(1), 41–53 (2016)CrossRefGoogle Scholar
  22. 22.
    Sayed-Mouchaweh, M.: Handling Concept Drift. In: Learning from Data Streams in Dynamic Environments, pp. 33–59. Springer International Publishing, Cham (2016)Google Scholar
  23. 23.
    Schlimmer, J.C., Granger Jr., R.H.: Incremental learning from noisy data. Mach. Learn. 1(3), 317–354 (1986)Google Scholar
  24. 24.
    Street, W.N., Kim, Y.: A streaming ensemble algorithm (sea) for large-scale classification. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD01, pp. 377–382. ACM, New York (2001)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  • Imen Khamassi
    • 1
    Email author
  • Moamar Sayed-Mouchaweh
    • 2
  • Moez Hammami
    • 1
  • Khaled Ghédira
    • 1
  1. 1.Université de TunisInstitut Supérieur de Gestion de TunisTunisTunisia
  2. 2.Institute Mines-Telecom Lille DouaiDouaiFrance

Personalised recommendations