Abstract
In imbalanced classification tasks, the training datasets may suffer from other problems like class overlapping, small disjuncts, classes of low density, etc. In such a situation, the learning for the minority class is imprecise. Data complexity metrics help us to identify the relationship between classifier’s learning accuracy and dataset characteristics. This paper presents an experimental study for imbalanced datasets wherein dwCM complexity metric is used to group the datasets based on the complexity level, thereafter the behavior of under-sampling based pre-processing techniques are analyzed for these different groups of datasets. Experiments are conducted on 22 real life datasets with different levels of imbalance, class overlapping and density of the classes. The experimental results show that these groups formed using dwCM metric can better explain the difficulty of imbalanced datasets and help in predicting the response of the classifiers to the under-sampling algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Branco, P., Torgo, L., & Ribeiro, R. P. (2016). A survey of predictive modeling on imbalanced do- mains. ACM Computing Surveys, 49(2), 1–50.
Gosain A, Saha A, & Singh, D. (2016). Analysis of sampling based classification techniques to overcome class imbalancing. In Proceedings 3rd international conference on computing for sustainable global development (INDIACom) IEEE pp. (7320–7326).
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). Smote: Synthetic minority over-sampling technique. The Journal of Artificial Intelligence Research, 16, 321–357.
Estabrooks, A., & Jo, T., Japkowicz, N. (2004). A multiple resampling method for learning from imbalanced data sets. Journal Computational intelligence, 20(1).
Gracia, S., & Herrera, F. (2009). Evolutionary undersampling for classification with imbal- anced datasets: Proposals and taxonomy. Journal Evolutionary computation, 17, 275–306.
Anand, R., Mehrotra, K., Mohan, C., & Ranka, S. (1993). An improved algorithm for neural net- work classification of imbalanced training sets, IEEE Trans. Neural Networks, 4, 962–969.
Bruzzone, L., & Serpico, S. (1997). Classification of imbalanced remote-sensing data by neural networks. Pattern Recognition Letters, 18, 1323–1328.
Domingos, P. (1999). Metacost: A general method for making classifiers cost sensitive. In Proceedings of fifth ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’99 (pp. 155–164). ACM, New York.
Zhou, Z.-H., & Liu, X.-Y. (2006). Training cost-sensitive neural networks with methods ad- dressing the class imbalance problem. IEEE Transactions on knowledge and data engineering, 18, 63–77.
Basu, M., & Ho, T.K. (2006). Data complexity in pattern recognition. In Advance information and knowledge processing. Springer.
Bernado-Manshilla, E., & Ho, T. K. (2005). Domain of competence of XCS classifier system in complexity measurement space. IEEE Transactions on Evolutionary Computation, 9(1), 82–104.
Li, Y., Member, S., & Dong, M. (2005). Classificability-based omnivariate decision trees. IEEE Transactions on Neural Networks, 16(6), 1547–1560.
Baumgartner, R., & Somorjai, R. L. (2006). Data complexity assessment in undersampled classification of high-dimensional biomedical data. Pattern Recognition Letters, 12, 1383–1389.
Yu, H., Ni, J., Xu, S., Qin, B., & Jv, H. (2014). Estimating harmfulness of class imbalance by scatter matrix based class separability measure. Intelligent Data Analysis, 18, 203–216.
Gracia, S., Cano, J. R., Bernado-Mansilla, E., & Herrera, F. (2009). Diagnose of effective evolutionary prototype selection using an overlapping measure. International Journal of Pattern Recognition and Artificial Intelligence, 23(8), 2378–2398.
Anwar, N., Jones, G., & Ganesh, S. (2014). Measurement of data complexity for classification problems with unbalanced data. Statistical Analysis and Data Mining, 7(3), 194–211.
Fernandez, L.M., Canedo, V.B., & Betanzos, A.A. (2016). Data complexity measures for analyzing the effect of SMOTE over microarrays. In Proceedings European Symposium on artificial neural networks, computational intelligence and machine learning (pp. 289–294).
Fernandez, L. M., Canedo, V. B., & Betanzos, A. A. (2017). Can classification performance be predicted by complexity measures? A study using microarray data. International Journal Knowledge and Information Systems, Springer, 51(3), 1067–1090.
Singh, D., Gosain, A., & Saha, A. (2020). Weighted k-nearest neighbor data complexity metrics for imbalanced datasets. Journal of Statistical Analysis and Data Mining. https://doi.org/10.1002/sam.11463
Jo, T., & Japkowicz, N. (2004). Class Imbalances versus small disjuncts. ACM SIGKDD Ex- plorations Newsletter, 6(1), 40–49.
Denil, M., Trappenberg, T.P. (2010). Overlap versus imbalance. In Canadian conference on AI (pp. 220–231).
Barella, V. H., Garcia, L.P.F., De Souto, M.P., Lorena, A.C., & De Carvalho, A. (2018). Data complexity measures for imbalanced classification tasks. In Proceedings international joint conference on neural networks (IJCNN) (pp. 1–8). Rio de Janeiro. https://doi.org/10.1109/IJCNN.2018.8489661
Brun, A. L., Britto, A. S., Jr., Oliveira, L. S., Enembreck, F., & Sabourin, R. (2018). A framework for dynamic classifier selection oriented by the classification problem difficulty. Pattern Recognition, 76, 175–190.
Xing, Y., Cai, H., Cai, Y., Hejlesen, O., & Toft, E. (2013) Preliminary evaluation of classification complexity measures on imbalanced data. Proceedings Chinese intelligent automation conference (pp. 189–196).
Yu, H., Ni, J., Xu, S., Qin, B., & Jv, H. (2014). Estimating harmfulness of class imbalance by scatter matrix based class separability measure. Journal Intelligent Data Analysis, 18, 203–216.
Diez-Pastor, J. F., Rodriguez, J. J., Garcia-Osorio, C. I., & Kuncheva, L. I. (2015). Diversity tech- niques improve the performance of the best imbalance learning ensembles. Information Sciences, 325, 98–117.
Tomek, I. (1976). Two modifications of CNN. IEEE transactions on systems man and communication SMC-6 (pp. 769–772).
Hart, P.E. (1968). The condensed nearest neighbour rule. IEEE transactions on information theory IT-14 (pp. 515–516).
Kubat, M., & Matwin, S. (1997). Addressing the curse of imbalanced datasets: one sided sampling. In Proceedings of 14th international conference on machine learning (pp. 179–186). Nashville, TN.
Laurikkala, J. (2001). Improving identification of difficult small classes by balancing class distribution. Technical Report A-2001-2, University of Tampere.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Singh, D., Saha, A., Gosain, A. (2022). Predicting Classifiers Efficacy in Relation with Data Complexity Metric Using Under-Sampling Techniques. In: Gupta, D., Khanna, A., Kansal, V., Fortino, G., Hassanien, A.E. (eds) Proceedings of Second Doctoral Symposium on Computational Intelligence . Advances in Intelligent Systems and Computing, vol 1374. Springer, Singapore. https://doi.org/10.1007/978-981-16-3346-1_7
Download citation
DOI: https://doi.org/10.1007/978-981-16-3346-1_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-3345-4
Online ISBN: 978-981-16-3346-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)