Evolving Systems

, Volume 8, Issue 4, pp 303–315 | Cite as

A novel online multi-label classifier for high-speed streaming data applications

  • Rajasekar Venkatesan
  • Meng Joo Er
  • Mihika Dave
  • Mahardhika Pratama
  • Shiqian Wu
Original Paper


In this paper, a high-speed online neural network classifier based on extreme learning machines for multi-label classification is proposed. In multi-label classification, each of the input data sample belongs to one or more than one of the target labels. The traditional binary and multi-class classification where each sample belongs to only one target class forms the subset of multi-label classification. Multi-label classification problems are far more complex than binary and multi-class classification problems, as both the number of target labels and each of the target labels corresponding to each of the input samples are to be identified. The proposed work exploits the high-speed nature of the extreme learning machines to achieve real-time multi-label classification of streaming data. A new threshold-based online sequential learning algorithm is proposed for high speed and streaming data classification of multi-label problems. The proposed method is experimented with six different datasets from different application domains such as multimedia, text, and biology. The hamming loss, accuracy, training time and testing time of the proposed technique is compared with nine different state-of-the-art methods. Experimental studies shows that the proposed technique outperforms the existing multi-label classifiers in terms of performance and speed.


Classification Multi-label Extreme learning machines High speed Real-time 



The authors would like to acknowledge the funding support from the Ministry of Education, Singapore (Tier 1 AcRF, RG29/15) and the Research student scholarship awarded to the first author.


  1. Angelov P (2012) Autonomous learning systems: from data streams to knowledge in real-time. Wiley, New YorkCrossRefGoogle Scholar
  2. Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159:3160–3182CrossRefzbMATHMathSciNetGoogle Scholar
  3. Bernardini FC, da Silva RB, Rodovalho RM, Meza EBM (2014) Cardinality and density measures and their influence to multi-label learning methods (Submitted to Learning and nonlinear models)Google Scholar
  4. Bouchachia A (2010) An evolving classification cascade with self-learning. Evol Syst 1:143–160CrossRefGoogle Scholar
  5. Boutell M, Shen X, Luo J, Brown C (2003) Multi-label semantic scene classification. Technical report, Dept. Comp. Sci. U. RochesterGoogle Scholar
  6. Crammer YS (2004) Online learning of complex categorical problems. Hebrew University of JerusalemGoogle Scholar
  7. de Carvalho ACPLF, Freitas AA (2009) A tutorial on multi-label classification techniques. In: Abraham A, Hassanien A-E, Snášel V (eds) Foundations of computational intelligence volume 5. Studies in computational intelligence, vol 205. Springer, Berlin, Heidelberg, pp 177–195CrossRefGoogle Scholar
  8. Ding S, Zhao H, Zhang Y, Xu X, Nie R (2015) Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 44:103–115CrossRefGoogle Scholar
  9. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, vol 14. MIT Press, Massachusetts, USA, pp 681–687Google Scholar
  10. Gama J (2010) Knowledge discovery from data streams. CRC Press, Boca RatonCrossRefzbMATHGoogle Scholar
  11. Gonçalves T, Quaresma P (2003) A preliminary approach to the multilabel classification problem of Portuguese juridical documents. In: Pires F, Abreu S (eds) Progress in artificial intelligence, vol 2902. Lecture notes in computer science. Springer, Berlin, pp 435–444Google Scholar
  12. Hua X-S, Qi G-J (2008) Online multi-label active learning for large-scale multimedia annotation. TechReport MSR-TR-2008-103Google Scholar
  13. Huang G-B (2015) What are extreme learning machines? Filling the gap between Frank Rosenblatt’s dream and John von Neumann’s puzzle. Cogn Comput 7:263–278CrossRefGoogle Scholar
  14. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRefGoogle Scholar
  15. Huang G-B, Wang D, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122CrossRefGoogle Scholar
  16. Iglesias JA, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agents’ behaviors: a general approach. Evol Syst 1:161–171CrossRefGoogle Scholar
  17. Joachims T (1998) Text categorization with support vector machines: learning with many relevant features. In: Nédellec C, Rouveirol C (eds) Machine learning: ECML-98, vol 1398. Lecture notes in computer science. Springer, Berlin, pp 137–142Google Scholar
  18. Karali A, Pirnat V (1991) Significance level based multiple tree classification. Informatica 15(5)Google Scholar
  19. Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach. Springer Science & Business Media, BerlinzbMATHGoogle Scholar
  20. Lemos A, Caminhas W, Gomide F (2013) Adaptive fault detection and diagnosis using an evolving fuzzy classifier. Inf Sci 220:64–85CrossRefGoogle Scholar
  21. Li B, Wang J, Li Y, Song Y (2007) An improved on-line sequential learning algorithm for extreme learning machine. In: Liu D, Fei S, Hou Z-G, Zhang H, Sun C (eds) Advances in neural networks—ISNN 2007: 4th international symposium on neural networks, ISNN 2007, Nanjing, China, June 3–7, 2007, Proceedings, Part I. Springer, Berlin, Heidelberg, pp 1087–1093CrossRefGoogle Scholar
  22. Liang N-Y, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17:1411–1423CrossRefGoogle Scholar
  23. Lughofer E, Buchtala O (2013) Reliable all-pairs evolving fuzzy classifiers. IEEE Trans Fuzzy Syst 21:625–641CrossRefGoogle Scholar
  24. Luo X, Zincir-Heywood AN (2005) Evaluation of two systems on multi-class multi-label document classification. In: Hacid M-S, Murray N, Raś Z, Tsumoto S (eds) Foundations of intelligent systems, vol 3488. Lecture notes in computer science. Springer, Berlin, pp 161–169Google Scholar
  25. Madjarov G, Kocev D, Gjorgjevikj D, Džeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recognit 45:3084–3104CrossRefGoogle Scholar
  26. Mohiuddin K, Mao J (2014) A comparative study of different classifiers for handprinted character recognition. In: Pattern recognition in practice IV. Elsevier, Amsterdam, pp 437–448Google Scholar
  27. Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern Part C 31:497–508CrossRefGoogle Scholar
  28. Pratama M, Anavatti S, Lu J (2015a) Recurrent classifier based on an incremental meta-cognitive-based scaffolding algorithm. IEEE Trans Fuzzy Syst 23(6):2048–2066CrossRefGoogle Scholar
  29. Pratama M, Anavatti SG, Meng J, Lughofer ED (2015b) pClass: an effective classifier for streaming examples. IEEE Trans Fuzzy Syst 23:369–386CrossRefGoogle Scholar
  30. Pratama M, Lu J, Anavatti S, Lughofer E, Lim C-P (2015c) An incremental meta-cognitive-based scaffolding fuzzy neural network. Neurocomputing 171:89–105CrossRefGoogle Scholar
  31. Pratama M, Lu J, Zhang G (2015d) Evolving type-2 fuzzy classifier. IEEE Trans Fuzzy Syst 24(3):574–589CrossRefGoogle Scholar
  32. Read J, Bifet A, Holmes G, Pfahringer B (2011) Streaming multi-label classification. In: WAPA, pp 19–25Google Scholar
  33. Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments: methods and applications. Springer Science & Business Media, BerlinCrossRefzbMATHGoogle Scholar
  34. Shen X, Boutell M, Luo J, Brown C (2003) Multilabel machine learning and its application to semantic scene classification. In: Electronic imaging, pp 188–199Google Scholar
  35. Song Y, Ben Salem M, Hershkop S, Stolfo SJ (2013) System level user behavior biometrics using Fisher features and Gaussian mixture models. In: Security and privacy workshops (SPW), 2013 IEEE. IEEE, pp 52–59Google Scholar
  36. Sorower MS (2010) A literature survey on algorithms for multi-label learning. Oregon State University, CorvallisGoogle Scholar
  37. Srivastava N, Agrawal U, Roy SK, Tiwary U (2015) Human identification using linear multiclass SVM and eye movement biometrics. In: International conference on contemporary computing (IC3). IEEE, Noida, pp 365–369Google Scholar
  38. Tikk D, Biró G (2003) Experiments with multi-label text classifier on the Reuters collection. In: Proceedings of the international conference on computational cybernetics (ICCC 03), pp 33–38Google Scholar
  39. Tsoumakas G, Katakis I (2006) Multi-label classification: an overview. Dept of Informatics, Aristotle University of Thessaloniki, GreeceGoogle Scholar
  40. Tsoumakas G, Katakis I, Vlahavas I (2010) Mining multi-label data. In: Maimon O, Rokach L (eds) Data mining and knowledge discovery handbook. Springer, US, pp 667–685Google Scholar
  41. Wang Y, Cao F, Yuan Y (2011) A study on effectiveness of extreme learning machine. Neurocomputing 74:2483–2490CrossRefGoogle Scholar
  42. Wang N, Han M, Dong N, Er MJ (2014a) Constructive multi-output extreme learning machine with application to large tanker motion dynamics identification. Neurocomputing 128:59–72CrossRefGoogle Scholar
  43. Wang N, Er MJ, Han M (2014b) Parsimonious extreme learning machine using recursive orthogonal least squares. IEEE Trans Neural Netw Learn Syst 25:1828–1841CrossRefGoogle Scholar
  44. Wang N, Sun JC, Er MJ, Liu YC (2015a) A novel extreme learning control framework of unmanned surface vehicles. IEEE Transa Cybern 46(5):1106–1117CrossRefGoogle Scholar
  45. Wang N, Er MJ, Han M (2015b) Generalized single-hidden layer feedforward networks for regression problems. IEEE Trans Neural Netw Learn Syst 26:1161–1176CrossRefMathSciNetGoogle Scholar
  46. Xydeas C, Angelov P, Chiao S-Y, Reoullas M (2006) Advances in classification of EEG signals via evolving fuzzy classifiers and dependant multiple HMMs. Comput Biol Med 36:1064–1083CrossRefGoogle Scholar
  47. Yu K, Yu S, Tresp V (2005) Multi-label informed latent semantic indexing. Paper presented at the Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval, Salvador, BrazilGoogle Scholar
  48. Zhang M-L, Zhou Z-H (2005) A k-nearest neighbor based algorithm for multi-label classification. In: 2005 IEEE international conference on granular computing, vol 2. IEEE, pp 718–721, 25–27 July 2005Google Scholar
  49. Zhang M-L, Zhou Z-H (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40:2038–2048CrossRefzbMATHGoogle Scholar
  50. Zhang X, Graepel T, Herbrich R (2010) Bayesian online learning for multi-label and multi-variate performance measures. In: International conference on artificial intelligence and statistics, pp 956–963Google Scholar
  51. Zhu B, Poon CK (1999) Efficient approximation algorithms for multi-label map labeling. In: International symposium on algorithms and computation. Springer, Berlin, Heidelberg, pp 143–152Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.School of Electrical and Electronic EngineeringNanyang Technological UniversitySingaporeSingapore
  2. 2.School of Machinery and AutomationWuhan University of Science and TechnologyWuhanChina
  3. 3.Department of Computer Science and ITLa Trobe UniversityMelbourneAustralia

Personalised recommendations