Neural Processing Letters

, Volume 38, Issue 3, pp 465–486 | Cite as

Weighted Online Sequential Extreme Learning Machine for Class Imbalance Learning

  • Bilal Mirza
  • Zhiping Lin
  • Kar-Ann Toh


Most of the existing sequential learning methods for class imbalance learn data in chunks. In this paper, we propose a weighted online sequential extreme learning machine (WOS-ELM) algorithm for class imbalance learning (CIL). WOS-ELM is a general online learning method that alleviates the class imbalance problem in both chunk-by-chunk and one-by-one learning. One of the new features of WOS-ELM is that an appropriate weight setting for CIL is selected in a computationally efficient manner. In one-by-one learning of WOS-ELM, a new sample can update the classification model without waiting for a chunk to be completed. Extensive empirical evaluations on 15 imbalanced datasets show that WOS-ELM obtains comparable or better classification performance than competing methods. The computational time of WOS-ELM is also found to be lower than that of the competing CIL methods.


Class imbalance Online sequential learning Extreme learning machine (ELM) Weighted least squares Total error rate 



The authors would like to thank the anonymous reviewers whose insightful and helpful comments greatly improved this paper.


  1. 1.
    Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRefGoogle Scholar
  2. 2.
    Haibo H, Garcia E (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRefGoogle Scholar
  3. 3.
    Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. Int Conf Mach Learn, Nashville, TNGoogle Scholar
  4. 4.
    Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357zbMATHGoogle Scholar
  5. 5.
    Ling CX, Sheng VS (2008) Cost-sensitive learning and the class imbalance problem. In: Sammut C (ed) Encyclopedia of machine learning. Springer, BerlinGoogle Scholar
  6. 6.
    Ozawa S, Pang S, Kasabov N (2008) Incremental learning of chunk data for online pattern classification. IEEE Trans Neural Netw 19(6):1061–1074CrossRefGoogle Scholar
  7. 7.
    Lian NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423CrossRefGoogle Scholar
  8. 8.
    Huang GB, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern B Cybern 34(6):2284–2292CrossRefGoogle Scholar
  9. 9.
    Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans Neural Netw 16(1):57–67CrossRefGoogle Scholar
  10. 10.
    Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern C Hum Syst 31(4):497–508CrossRefGoogle Scholar
  11. 11.
    Shilton A, Palaniswami M, Ralph D, Tsoi AC (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131CrossRefGoogle Scholar
  12. 12.
    Muhlbaier M, Polikar R (2007) Multiple classifiers based incremental learning algorithm for learning nonstationary environments. International Conference on Machine Learning and Cybernetics, pp 3618–3623, Hong Kong, China.Google Scholar
  13. 13.
    Kim Y, Toh KA, Teoh ABJ, Eng HL, Yau WY (2013) An online learning network for biometric scores fusion. Neurocomputing 102:65–77Google Scholar
  14. 14.
    Ditzler G, Polikar R, Chawla NV (2010) An incremental learning algorithm for non-stationary environments and class imbalance. International Conference on, Pattern Recognition, pp 2997–3000.Google Scholar
  15. 15.
    Ditzler G, Muhlbaier M, Polikar R (2010) Incremental learning of new classes in unbalanced datasets: learn++. UDNC. MCS Lect Notes Comput Sci 5997:33–42CrossRefGoogle Scholar
  16. 16.
    Gao J, Fan W, Han J, Yu PS (2007) A general framework for mining concept-drifting streams with skewed distribution. SIAM International Conference on Data Mining, vol. 7.Google Scholar
  17. 17.
    Chen S, He H (2009) SERA: selectively recursive approach towards nonstationary imbalanced stream data mining. International Joint Conference on Neural Networks, AtlantaGoogle Scholar
  18. 18.
    Chen S, He H (2011) Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach. Evolv Syst 2(1):35–50CrossRefGoogle Scholar
  19. 19.
    Wang Y, Zhang Y, Wang Y (2009) Mining data streams with skewed distribution by static classifier ensemble. Stud Comput Intell 214:65–71CrossRefGoogle Scholar
  20. 20.
    Nguyen H, Cooper E, Kamei K (2011) Online learning from imbalanced sata dtreams. International Conference of Soft Computing and Pattern Recognition (SoCPaR), pp 347–352.Google Scholar
  21. 21.
    Toh KA, Eng HL (2008) Between classification-error approximation and weighted least-squares learning. IEEE Trans Pattern Anal Mach Intell 30(4):658–669CrossRefGoogle Scholar
  22. 22.
    Toh KA (2008) Deterministic neural classification. Neural Comput 20(6):1565–1595MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Tang Y, Zhang Y, Chawla NV, Krasser S (2009) SVMs modeling for highly imbalanced classification. IEEE Trans Syst Man Cybern B Cybern 39(1):281–288CrossRefGoogle Scholar
  24. 24.
    Akbani JR, Kwek S, Japkowicz N (2004) Applying support vector machines to imbalanced datasets. 15th European Conference on Machine Learning, Pisa, Italy, pp 39–50.Google Scholar
  25. 25.
    Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571CrossRefGoogle Scholar
  26. 26.
    Batuwita R, Palade V (2012) Adjusted geometric-mean: a novel performance measure for imbalanced bioinformatics datasets learning. J Bioinf Comput Biol 10(4):23. doi: 10.1142/S0219720012500035 CrossRefGoogle Scholar
  27. 27.
    Batuwita R, Palade V (2012) Class imbalance learning methods for support vector machines. In: Haibo He, Yunqian Ma (eds) Imbalanced learning: foundations, algorithms, and applications, Wiley, (in press).Google Scholar
  28. 28.
    Batuwita R, Palade V (2009) AGm: a New performance measure for class imbalance learning. Application to bioinformatics problems. The 8th IEEE International Conference on Machine Learning and Applications, Miami, USA, pp 545–550.Google Scholar
  29. 29.
    Hoens TR, Chawla NV (2012) Learning in non-stationary environments with class imbalance. International conference on Knowledge discovery and data mining, pp 168–176.Google Scholar
  30. 30.
    Frank A, Asuncion A (2010) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. Accessed on 10 June 2012
  31. 31.
    Kubat M, Holte RC, Matwin S (1998) Machine learning for the detection of oil spills in satellite radar images. Mach Learn 30:195–215CrossRefGoogle Scholar
  32. 32.
    Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.School of Electrical and Electronic EngineeringNanyang Technological UniversitySingaporeSingapore
  2. 2.School of Electrical and Electronic EngineeringYonsei UniversitySeodaemun-guSouth Korea

Personalised recommendations