Abstract
Most of the existing sequential learning methods for class imbalance learn data in chunks. In this paper, we propose a weighted online sequential extreme learning machine (WOS-ELM) algorithm for class imbalance learning (CIL). WOS-ELM is a general online learning method that alleviates the class imbalance problem in both chunk-by-chunk and one-by-one learning. One of the new features of WOS-ELM is that an appropriate weight setting for CIL is selected in a computationally efficient manner. In one-by-one learning of WOS-ELM, a new sample can update the classification model without waiting for a chunk to be completed. Extensive empirical evaluations on 15 imbalanced datasets show that WOS-ELM obtains comparable or better classification performance than competing methods. The computational time of WOS-ELM is also found to be lower than that of the competing CIL methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501
Haibo H, Garcia E (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284
Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. Int Conf Mach Learn, Nashville, TN
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357
Ling CX, Sheng VS (2008) Cost-sensitive learning and the class imbalance problem. In: Sammut C (ed) Encyclopedia of machine learning. Springer, Berlin
Ozawa S, Pang S, Kasabov N (2008) Incremental learning of chunk data for online pattern classification. IEEE Trans Neural Netw 19(6):1061–1074
Lian NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Huang GB, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern B Cybern 34(6):2284–2292
Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans Neural Netw 16(1):57–67
Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern C Hum Syst 31(4):497–508
Shilton A, Palaniswami M, Ralph D, Tsoi AC (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131
Muhlbaier M, Polikar R (2007) Multiple classifiers based incremental learning algorithm for learning nonstationary environments. International Conference on Machine Learning and Cybernetics, pp 3618–3623, Hong Kong, China.
Kim Y, Toh KA, Teoh ABJ, Eng HL, Yau WY (2013) An online learning network for biometric scores fusion. Neurocomputing 102:65–77
Ditzler G, Polikar R, Chawla NV (2010) An incremental learning algorithm for non-stationary environments and class imbalance. International Conference on, Pattern Recognition, pp 2997–3000.
Ditzler G, Muhlbaier M, Polikar R (2010) Incremental learning of new classes in unbalanced datasets: learn++. UDNC. MCS Lect Notes Comput Sci 5997:33–42
Gao J, Fan W, Han J, Yu PS (2007) A general framework for mining concept-drifting streams with skewed distribution. SIAM International Conference on Data Mining, vol. 7.
Chen S, He H (2009) SERA: selectively recursive approach towards nonstationary imbalanced stream data mining. International Joint Conference on Neural Networks, Atlanta
Chen S, He H (2011) Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach. Evolv Syst 2(1):35–50
Wang Y, Zhang Y, Wang Y (2009) Mining data streams with skewed distribution by static classifier ensemble. Stud Comput Intell 214:65–71
Nguyen H, Cooper E, Kamei K (2011) Online learning from imbalanced sata dtreams. International Conference of Soft Computing and Pattern Recognition (SoCPaR), pp 347–352.
Toh KA, Eng HL (2008) Between classification-error approximation and weighted least-squares learning. IEEE Trans Pattern Anal Mach Intell 30(4):658–669
Toh KA (2008) Deterministic neural classification. Neural Comput 20(6):1565–1595
Tang Y, Zhang Y, Chawla NV, Krasser S (2009) SVMs modeling for highly imbalanced classification. IEEE Trans Syst Man Cybern B Cybern 39(1):281–288
Akbani JR, Kwek S, Japkowicz N (2004) Applying support vector machines to imbalanced datasets. 15th European Conference on Machine Learning, Pisa, Italy, pp 39–50.
Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571
Batuwita R, Palade V (2012) Adjusted geometric-mean: a novel performance measure for imbalanced bioinformatics datasets learning. J Bioinf Comput Biol 10(4):23. doi:10.1142/S0219720012500035
Batuwita R, Palade V (2012) Class imbalance learning methods for support vector machines. In: Haibo He, Yunqian Ma (eds) Imbalanced learning: foundations, algorithms, and applications, Wiley, (in press).
Batuwita R, Palade V (2009) AGm: a New performance measure for class imbalance learning. Application to bioinformatics problems. The 8th IEEE International Conference on Machine Learning and Applications, Miami, USA, pp 545–550.
Hoens TR, Chawla NV (2012) Learning in non-stationary environments with class imbalance. International conference on Knowledge discovery and data mining, pp 168–176.
Frank A, Asuncion A (2010) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml. Accessed on 10 June 2012
Kubat M, Holte RC, Matwin S (1998) Machine learning for the detection of oil spills in satellite radar images. Mach Learn 30:195–215
Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Acknowledgments
The authors would like to thank the anonymous reviewers whose insightful and helpful comments greatly improved this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mirza, B., Lin, Z. & Toh, KA. Weighted Online Sequential Extreme Learning Machine for Class Imbalance Learning. Neural Process Lett 38, 465–486 (2013). https://doi.org/10.1007/s11063-013-9286-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-013-9286-9