Advertisement

Weighted Aging Classifier Ensemble for the Incremental Drifted Data Streams

  • Michał Woźniak
  • Andrzej Kasprzak
  • Piotr Cal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8132)

Abstract

Evolving systems are recently focus of intense research because for most of the real problems we can observe that the parameters of the decision tasks should adapt to new conditions. In classification such a problem is usually called concept drift. The paper deals with the data stream classification where we assume that the concept drift is sudden but its rapidity is limited. To deal with this problem we propose a new algorithm called Weighted Aging Ensemble (WAE), which is able to adapt to changes of classification model parameters. The method is inspired by well-known algorithm Accuracy Weighted Ensemble (AWE) which allows to change the line-up of a classifier ensemble, but the proposed method incudes two important modifications: (i) classifier weights depend on the individual classifier accuracies and time they have been spending in the ensemble, (ii) individual classifier are chosen to the ensemble on the basis on the non-pairwise diversity measure. The proposed method was evaluated on the basis of computer experiments which were carried out on SEA dataset. The obtained results encourage us to continue the work on the proposed concept.

Keywords

machine learning classifier ensemble data stream concept drift incremental learning forgetting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alpaydin, E.: Introduction to Machine Learning, 2nd edn. The MIT Press (2010)Google Scholar
  2. 2.
    Bifet, A., Holmes, G., Pfahringer, B., Read, J., Kranen, P., Kremer, H., Jansen, T., Seidl, T.: Moa: a real-time analytics open source framework. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part III. LNCS, vol. 6913, pp. 617–620. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  4. 4.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 97–106 (2001)Google Scholar
  5. 5.
    Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3, 79–87 (1991)CrossRefGoogle Scholar
  6. 6.
    Klinkenberg, R., Renz, I.: Adaptive information filtering: Learning in the presence of concept drifts, pp. 33–40 (1998)Google Scholar
  7. 7.
    Kolter, J.Z., Maloof, M.A.: Dynamic weighted majority: a new ensemble method for tracking concept drift. In: Third IEEE International Conference on Data Mining, ICDM 2003, pp. 123–130 (November 2003)Google Scholar
  8. 8.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience (2004)Google Scholar
  9. 9.
    Lazarescu, M.M., Venkatesh, S., Bui, H.H.: Using multiple windows to track concept drift. Intell. Data Anal. 8(1), 29–59 (2004)Google Scholar
  10. 10.
    Muhlbaier, M.D., Topalis, A., Polikar, R.: Learn + + .nc: Combining ensemble of classifiers with dynamically weighted consult-and-vote for efficient incremental learning of new classes. IEEE Transactions on Neural Networks 20(1), 152–168 (2009)CrossRefGoogle Scholar
  11. 11.
    Partridge, D., Krzanowski, W.: Software diversity: practical statistics for its measurement and exploitation. Information and Software Technology 39(10), 707–717 (1997)CrossRefGoogle Scholar
  12. 12.
    Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  13. 13.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Series in Machine Learning. Morgan Kaufmann Publishers (1993)Google Scholar
  14. 14.
    Shipp, C.A., Kuncheva, L.: Relationships between combination methods and measures of diversity in combining classifiers. Information Fusion 3(2), 135–148 (2002)CrossRefGoogle Scholar
  15. 15.
    Nick Street, W., Kim, Y.: A streaming ensemble algorithm (sea) for large-scale classification. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2001, pp. 377–382. ACM, New York (2001)CrossRefGoogle Scholar
  16. 16.
    Wang, H., Fan, W., Yu, P.S., Han, J.: Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2003, pp. 226–235. ACM, New York (2003)CrossRefGoogle Scholar
  17. 17.
    Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Mach. Learn. 23(1), 69–101 (1996)Google Scholar
  18. 18.
    Wolpert, D.H.: The supervised learning no-free-lunch theorems. In: Proc. 6th Online World Conference on Soft Computing in Industrial Applications, pp. 25–42 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Michał Woźniak
    • 1
  • Andrzej Kasprzak
    • 1
  • Piotr Cal
    • 1
  1. 1.Department of Systems and Computer NetworksWroclaw University of TechnologyWroclawPoland

Personalised recommendations