Neural Processing Letters

, Volume 45, Issue 1, pp 341–363 | Cite as

Evolving Fuzzy Min–Max Neural Network Based Decision Trees for Data Stream Classification

Article

Abstract

Learning from data streams is a challenging task which demands a learning algorithm with several high quality features. In addition to space complexity and speed requirements needed for processing the huge volume of data which arrives at high speed, the learning algorithm must have a good balance between stability and plasticity. This paper presents a new approach to induce incremental decision trees on streaming data. In this approach, the internal nodes contain trainable split tests. In contrast with traditional decision trees in which a single attribute is selected as the split test, each internal node of the proposed approach contains a trainable function based on multiple attributes, which not only provides the flexibility needed in the stream context, but also improves stability. Based on this approach, we propose evolving fuzzy min–max decision tree (EFMMDT) learning algorithm in which each internal node of the decision tree contains an evolving fuzzy min–max neural network. EFMMDT splits the instance space non-linearly based on multiple attributes which results in much smaller and shallower decision trees. The extensive experiments reveal that the proposed algorithm achieves much better precision in comparison with the state-of-the-art decision tree learning algorithms on the benchmark data streams, especially in the presence of concept drift.

Keywords

Pattern recognition Data stream classification Decision tree Min–max neural network Stability 

References

  1. 1.
    Bifet A, Gavaldá R (2009) Adaptive learning from evolving data streams, vol 5772., Advances in intelligent data analysis VIII, Lecture notes in computer scienceSpringer, Berlin, pp 249–260Google Scholar
  2. 2.
    Bifet A, Holmes G, Kirkby R, Pfahringer B (2010) Moa: massive online analysis. J Mach Learn Res 11:1601–1604Google Scholar
  3. 3.
    Breiman L (1996) Heuristics of instability and stabilization in model selection. Ann Stat 24(6):2350–2383MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Davtalab R, Dezfoulian MH, Mansoorizadeh M (2014) Multi-level fuzzy min-max neural network classifier. IEEE Trans Neural Netw Learn Syst 25(3):470–482CrossRefGoogle Scholar
  5. 5.
    Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATHGoogle Scholar
  6. 6.
    Domingos P, Hulten G (2000) Mining high-speed data streams. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining, KDD, pp 71–80. doi:10.1145/347090.347107
  7. 7.
    Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRefMATHGoogle Scholar
  8. 8.
    Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Gabrys B, Bargiela A (2000) General fuzzy min-max neural network for clustering and classification. Trans Neural Netw 11(3):769–783CrossRefGoogle Scholar
  10. 10.
    Gama J, Rocha R, Medas P (2003) Accurate decision trees for mining high-speed data streams. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, KDD’03, New York, USA, pp 523–528Google Scholar
  11. 11.
    Gama J, Rodrigues P, Sebastião R (2009) Evaluating algorithms that learn from data streams. In: Proceedings of the 2009 ACM symposium on applied computing, SAC’09, pp 1496–1500Google Scholar
  12. 12.
    Gama J (2004) Functional trees. Mach Learn 55(3):219–250CrossRefMATHGoogle Scholar
  13. 13.
    Gama J, Sebastião R, Rodrigues P (2013) On evaluating stream learning algorithms. Mach Learn 90(3):317–346MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Hashemi S, Yang Y, Mirzamomen Z, Kangavari M (2009) Adapted one-versus-all decision trees for data stream classification. IEEE Trans Knowl Data Eng 21(5):624–637CrossRefGoogle Scholar
  15. 15.
    Hashemi S, Yang Y (2009) Flexible decision tree for data stream classification in the presence of concept change, noise and missing values. Data Mining Knowl Discov 19:95–131MathSciNetCrossRefGoogle Scholar
  16. 16.
    Hulten G, Spencer L, Domingos P (2001) Mining time-changing data streams. In: Proceedings of the 2001 ACM SIGKDD international conference on knowledge discovery and data mining, pp 97–106Google Scholar
  17. 17.
    Ikonomovska E, Gama J, Dz̆eroski S (2011) Learning model trees from evolving data streams. Data Mining Knowl Discov 23(1):128–168MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Kirkby R (2007) Improving hoeffding trees. Dissertation, University of WaikatoGoogle Scholar
  19. 19.
    Kohavi R (1996) Scaling up the accuracy of naive-bayes classifiers: a decision-tree hybrid. In: Proceedings of the second international conference on knowledge discovery and data mining, AAAI Press, pp 202–207Google Scholar
  20. 20.
    Last M, Maimon O, Minkov E (2002) Improving stability of decision trees. Int J Pattern Recogn Artif Intell 16(02):145–159CrossRefGoogle Scholar
  21. 21.
    Nandedkar AV, Biswas PK (2007) A fuzzy min-max neural network classifier with compensatory neuron architecture. IEEE Trans Neural Netw 18(1):42–54CrossRefGoogle Scholar
  22. 22.
    Blake C, Keogh E, Merz C (1998) UCI repository of machine learning databases. http://archive.ics.uci.edu/ml. Accessed 20 Sep 2014
  23. 23.
    Pfahringer B, Holmes G, Kirkby R (2007) New options for hoeffding trees. In: Orgun MA, Thornton J (eds) AI 2007: advances in artificial intelligence, lecture notes in computer science, vol 4830, Springer, Berlin, pp 90–99Google Scholar
  24. 24.
    Simpson PK (1992) Fuzzy min-max neural networks, i, classification. IEEE Trans Neural Netw 3(5):776–786CrossRefGoogle Scholar
  25. 25.
    Street WN, Kim Y (2001) A streaming ensemble algorithm (sea) for large-scale classification. In: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, KDD’01, pp 377–382Google Scholar
  26. 26.
    Wang X, Fan W, Yu PS, Han J (2003) Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD’03, pp 226–235Google Scholar
  27. 27.
    Yang H, Fong S (2012) Incrementally optimized decision tree for noisy big data. In: Proceedings of the 1st international workshop on big data, streams and heterogeneous source mining: algorithms, systems, programming models and applications, BigMine’12, pp 36–44Google Scholar
  28. 28.
    Yang Y, Wu X, Zhu X. (2005) Combining proactive and reactive predictions for data streams. In: Grossman R, Bayardo R, Bennett KP (eds), KDD, pp 710–715Google Scholar
  29. 29.
    Zhang H, Liu J, Ma D, Wang Z (2011) Data-core-based fuzzy min-max neural network for pattern classification. IEEE Trans Neural Netw 22(12):2339–2352CrossRefGoogle Scholar
  30. 30.
    Zhao Q (2005) Learning with data streams: an nntree based approach. In: Enokido T, Yan L, Xiao B, Kim D, Dai Y, Yang L (eds) Embedded and ubiquitous computing EUC 2005 workshops, vol 3823., Lecture notes in computer scienceSpringer, Berlin, pp 519–528CrossRefGoogle Scholar
  31. 31.
    Zimmermann A (2008) Ensemble-trees: leveraging ensemble power inside decision trees. Discovery science, vol 5255., Lecture notes in computer scienceSpringer, Berlin, pp 76–87CrossRefGoogle Scholar
  32. 32.
    Mirzamomen Z, Kangavari M (2016) Fuzzy min-max neural network based decision trees. Intell Data Anal 20(4)Google Scholar
  33. 33.
    Mohammed MF, Lim CP (2015) An enhanced fuzzy min-max neural network for pattern classification. IEEE Trans Neural Netw Learn Syst 26(3):417–429MathSciNetCrossRefGoogle Scholar
  34. 34.
    Shinde SV, Kulkarni UV (2016) Extracting classification rules from modified fuzzy min max neural network for data with mixed attributes. Appl Soft Comput 40:364–378CrossRefGoogle Scholar
  35. 35.
    Shinde SV, Kulkarni UV, Chaudhary AN (2015) Extracting the classification rules from general fuzzy min-max neural network. Int J Comput Appl 121(23):1–7Google Scholar
  36. 36.
    Kulkarni SU, Shetty BS (2015) Data mining using modified GFMM neural network. Int J Comput Appl 116(15):18–22Google Scholar
  37. 37.
    Forghani Y, Yazdi HS (2015) Fuzzy min-max neural network for learning a classifier with symmetric margin. Neural Process Lett 42(2):317–353CrossRefGoogle Scholar
  38. 38.
    Seera M, Lim CP, Loo CK, Jain LC (2015) Data clustering using a modified fuzzy min-max neural network, soft computing applications. In: Proceedings of the 6th international workshop soft computing applications (SOFA 2014), Vol 1, Springer, pp 413–422Google Scholar
  39. 39.
    Quteishat A, Lim CP (2008) A modified fuzzy min-max neural network with rule extraction and its application to fault detection and classification. Appl Soft Comput 8(2):985–995CrossRefGoogle Scholar
  40. 40.
    Ma D, Liu J, Wang Z (2012) The pattern classification based on fuzzy min-max neural network with new algorithm. In: Wang J, Yen GG, Polycarpou MM (eds) Proceedings of the 9th international symposium on advances in neural networks, Springer, Berlin, pp 1–9Google Scholar
  41. 41.
    Rizzi A, Panella M, Massimo F, Mascioli F (2002) Adaptive resolution min-max classifiers. IEEE Trans Neural Netw 13(2):402–414CrossRefGoogle Scholar
  42. 42.
    Leite D, Costa P, Gomide F (2013) Evolving granular neural networks from fuzzy data streams. Neural Netw 38:1–16CrossRefMATHGoogle Scholar
  43. 43.
    Pfahringer B, Holmes G, Kirkby R (2007) New options for hoeffding trees. In: Orgun MA, Thornton J (eds) Proceedings of the 20th Australian joint conference on advances in artificial intelligence, AI07, Springer, Berlin, pp 90–99Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.School of Computer EngineeringIran University of Science and TechnologyTehranIran

Personalised recommendations