Random Forest Based Approach for Concept Drift Handling

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 661)


Concept drift has potential in smart grid analysis because the socio-economic behaviour of consumers is not governed by the laws of physics. Likewise there are also applications in wind power forecasting. In this paper we present decision tree ensemble classification method based on the Random Forest algorithm for concept drift. The weighted majority voting ensemble aggregation rule is employed based on the ideas of Accuracy Weighted Ensemble (AWE) method. Base learner weight in our case is computed for each sample evaluation using base learners accuracy and intrinsic proximity measure of Random Forest. Our algorithm exploits ensemble pruning as a forgetting strategy. We present results of empirical comparison of our method and other state-of-the-art concept-drfit classifiers.


Machine learning Decision tree Concept drift Ensemble learning Classification Random forest 



This work is funded by the RSF grant No. 14-19-00054 and by the International science and technology cooperation program of China, project 2015DFR70850, NSFC Grant No. 61673398.


  1. 1.
    Zhukov, A., Kurbatsky, V., Tomin, N., Sidorov, D., Panasetsky, D., Foley, A.: Ensemble methods of classification for power systems security assessment. arXiv, Artificial Intelligence (cs.AI), pp. 1–6. arXiv:1601.01675 (2016)
  2. 2.
    Tomin, N., Zhukov, A., Sidorov, D., Kurbatsky, V., Panasetsky, D., Spiryaev, V.: Random forest based model for preventing large-scale emergencies in power systems. Int. J. Artif. Intell. 13, 211–228 (2015)Google Scholar
  3. 3.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)zbMATHGoogle Scholar
  4. 4.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)CrossRefGoogle Scholar
  5. 5.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  6. 6.
    Saffari, A., Leistner, C., Santner, J., Godec, M., Bischof, H.: On-line random forests. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), 1393–1400. IEEE (2009)Google Scholar
  7. 7.
    Sidorov, D.: Modelling of non-linear dynamic systems by Volterra series. In: Attractors, Signals, and Synergetics Workshopp, vol. 2000, pp. 276–282. Pabst Science Publication, USA-Germany (2002)Google Scholar
  8. 8.
    Sidorov, D.: Integral Dynamical Models: Singularities, Signals and Control. World Scientific Publishing, Singapore (2015)zbMATHGoogle Scholar
  9. 9.
    Wang, H., Fan, W., Yu, P.S., Han, J.: Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of SIGKDD, 24–27 August 2003, Washington, DC, USA, pp. 226–235 (2003)Google Scholar
  10. 10.
    Gama, J.: Knowledge discovery from data streams. CRC Press Publishing, Singapore (2010)CrossRefzbMATHGoogle Scholar
  11. 11.
    Kuncheva, L.: Classier ensembles for changing environment. In: Roli, F., Kittler, J., Windeatt, T. (eds.) 2004 5th International Workshop on Multiple Classier Systems, pp. 1–15. Springer, Heidelberg (2004)Google Scholar
  12. 12.
    Turkov, P., Krasotkina, O., Mottl, V.: Dynamic programming for Bayesian logistic regression learning under concept drift. In: Maji, P., Ghosh, A., Murty, M.N., Ghosh, K., Pal, S.K. (eds.) PReMI 2013. LNCS, vol. 8251, pp. 190–195. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Wang, H., Fan, W., Yu, P.S., Han, J.: Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 226–235. ACM (2003)Google Scholar
  14. 14.
    Zhukov, A., Kurbatsky, V., Tomin, N., Sidorov, D., Panasetsky, D., Spiryaev, V.: Random forest based model for emergency state monitoring in power systems. In: Mathematical Method for Pattern Recognition: Book of abstract of the 17th All-Russian Conference with Interneational Participation, p. 274. TORUS PRESS, Svetlogorsk (2015)Google Scholar
  15. 15.
    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)CrossRefzbMATHGoogle Scholar
  16. 16.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  17. 17.
    Brzezinski, D.: Mining data streams with concept drift. Dissertion MS thesis. Department of Computing Science and Management, Poznan University of Technology (2010)Google Scholar
  18. 18.
    Brzezinski, D., Stefanowski, J.: Reacting to different types of concept drift: the accuracy updated ensemble algorithm. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 81–94 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Energy Systems Institute SB RASIrkutskRussia
  2. 2.Irkutsk State UniversityIrkutskRussia
  3. 3.Queens University BelfastBelfastUK

Personalised recommendations