Skip to main content
SpringerLink
Log in

Hybrid density-based adaptive weighted collaborative representation for imbalanced learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Collaborative representation-based classification (CRC) has been extensively applied to various recognition fields due to its effectiveness and efficiency. Nevertheless, it is generally suboptimal for imbalanced learning. Previous studies have revealed that a class-imbalance distribution can lead CRC, and even most conventional classification methods, to ignore the minority class and prioritize the majority class. To address this deficiency, this paper proposes a hybrid density-based adaptive weighted collaborative representation model that incorporates a regularization technique and an adaptive weight generation mechanism into the CRC framework. A new regularization term, based on class-specific representation, is introduced to decrease the correlation between classes and enhance CRC’s discriminative ability. The sample distribution and density information within and between classes are employed to assign greater weights to minority samples, thereby strengthening the representation capabilities of minority samples and reducing the bias towards the majority class. Furthermore, it is theoretically demonstrated that this model has a closed-form solution. Its complexity is comparable to that of CRC, ensuring its efficiency. Extensive experiments on diverse data sets from the KEEL repository show the superiority of the proposed method compared to other state-of-the-art imbalanced classification methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability and access

All data sets in this study are available in the KEEL repository (https://sci2s.ugr.es/keel/imbalanced.php)

Notes

  1. https://sci2s.ugr.es/keel/imbalanced.php

References

  1. Li F, Wang B, Wang P, Jiang M, Li Y (2023) An imbalanced ensemble learning method based on dual clustering and stage-wise hybrid sampling. Appl Intell 53(18):21167–21191. https://doi.org/10.1007/s10489-023-04650-0

    Article  Google Scholar 

  2. Huang K, Wang X (2023) CCR-GSVM: A boundary data generation algorithm for support vector machine in imbalanced majority noise problem. Appl Intell 53(1):1192–1204. https://doi.org/10.1007/s10489-022-03408-4

    Article  Google Scholar 

  3. Jin J, Qin Z, Yu D, Li Y, Liang J, Chen CLP (2022) Regularized discriminative broad learning system for image classification. Knowl Based Syst 251:109306. https://doi.org/10.1016/j.knosys.2022.109306

    Article  Google Scholar 

  4. Dai Q, Liu J, Yang J (2023) SWSEL: sliding window-based selective ensemble learning for class-imbalance problems. Eng Appl Artif Intell 121:105959. https://doi.org/10.1016/j.engappai.2023.105959

    Article  Google Scholar 

  5. Jin J, Geng B, Li Y, Liang J, Xiao Y, Chen CLP (2023) Flexible label-induced manifold broad learning system for multiclass recognition. IEEE Trans Neural Netw Learn Syst 1–15. https://doi.org/10.1109/TNNLS.2023.3291793

  6. Roy S, Roy U, Sinha D, Pal RK (2023) Imbalanced ensemble learning in determining parkinson’s disease using keystroke dynamics. Expert Syst Appl 217:119522. https://doi.org/10.1016/j.eswa.2023.119522

    Article  Google Scholar 

  7. Liu G, Shen W, Gao L, Kusiak A (2023) Active broad-transfer learning algorithm for class-imbalanced fault diagnosis. IEEE Trans Instrum Meas 72:1–16. https://doi.org/10.1109/TIM.2022.3227995

    Article  Google Scholar 

  8. Yang K, Yu Z, Wen X, Cao W, Chen CLP, Wong H, You J (2020) Hybrid classifier ensemble for imbalanced data. IEEE Trans Neural Netw Learn Syst 31(4):1387–1400. https://doi.org/10.1109/TNNLS.2019.2920246

    Article  MathSciNet  Google Scholar 

  9. Jin J, Li Y, Yang T, Zhao L, Duan J, Chen CLP (2021) Discriminative group-sparsity constrained broad learning system for visual recognition. Inf Sci 576:800–818. https://doi.org/10.1016/j.ins.2021.06.008

    Article  MathSciNet  Google Scholar 

  10. Ng WWY, Xu S, Zhang J, Tian X, Rong T, Kwong S (2022) Hashing-based undersampling ensemble for imbalanced pattern classification problems. IEEE Trans Cybern 52(2):1269–1279. https://doi.org/10.1109/TCYB.2020.3000754

    Article  Google Scholar 

  11. Yang K, Yu Z, Chen CLP, Cao W, You J, Wong H (2022) Incremental weighted ensemble broad learning system for imbalanced data. IEEE Trans Knowl Data Eng 34(12):5809–5824. https://doi.org/10.1109/TKDE.2021.3061428

    Article  Google Scholar 

  12. Gao X, Jia X, Liu J, Xue B, Huang Z, Fu S, Zhang G, Li K (2022) An ensemble contrastive classification framework for imbalanced learning with sample-neighbors pair construction. Knowl Based Syst 249:109007. https://doi.org/10.1016/j.knosys.2022.109007

    Article  Google Scholar 

  13. Abbaszadeh Shahri A, Chunling S, Larsson S (2023) A hybrid ensemble-based automated deep learning approach to generate 3d geo-models and uncertainty analysis. Eng Comput 1–16. https://doi.org/10.1007/s00366-023-01852-5

  14. Naik DL, Kiran R (2021) A novel sensitivity-based method for feature selection. J Big Data 8(1):128. https://doi.org/10.1186/s40537-021-00515-w

    Article  Google Scholar 

  15. Jin J, Li Y, Chen CP (2022) Pattern classification with corrupted labeling via robust broad learning system. IEEE Trans Knowl Data Eng 34(10):4959–4971. https://doi.org/10.1109/TKDE.2021.3049540

    Article  Google Scholar 

  16. Zhang X, Peng H, Zhang J, Wang Y (2023) A cost-sensitive attention temporal convolutional network based on adaptive top-k differential evolution for imbalanced time-series classification. Expert Syst Appl 213(Part C):119073. https://doi.org/10.1016/j.eswa.2022.119073

  17. Wang Z, Jusup M, Shi L, Lee J-H, Iwasa Y, Boccaletti S (2018) Exploiting a cognitive bias promotes cooperation in social dilemma experiments. Nat Commun 9(1):2954. https://doi.org/10.1038/s41467-018-05259-5

    Article  Google Scholar 

  18. Du G, Zhang J, Jiang M, Long J, Lin Y, Li S, Tan KC (2023) Graph-based class-imbalance learning with label enhancement. IEEE Trans Neural Netw Learn Syst 34(9):6081–6095. https://doi.org/10.1109/TNNLS.2021.3133262

    Article  MathSciNet  Google Scholar 

  19. Du G, Zhang J, Ma F, Zhao M, Lin Y, Li S (2021) Towards graph-based class-imbalance learning for hospital readmission. Expert Syst Appl 176:114791. https://doi.org/10.1016/j.eswa.2021.114791

    Article  Google Scholar 

  20. Li Y, Jin J, Ma J, Zhu F, Jin B, Liang J, Chen CLP (2023) Imbalanced least squares regression with adaptive weight learning. Inf Sci 648:119541. https://doi.org/10.1016/j.ins.2023.119541

    Article  Google Scholar 

  21. Shu T, Zhang B, Tang YY (2020) Sparse supervised representation-based classifier for uncontrolled and imbalanced classification. IEEE Trans Neural Netw Learn Syst 31(8):2847–2856. https://doi.org/10.1109/TNNLS.2018.2884444

    Article  MathSciNet  Google Scholar 

  22. Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition?. In. Metaxas DN, Quan L, Sanfeliu A, and Gool LV (eds) IEEE international conference on computer vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, IEEE Computer Society, pp 471–478. https://doi.org/10.1109/ICCV.2011.6126277

  23. Li Y, Jin J, Chen CLP (2021) A real-time classification model based on joint sparse-collaborative representation. J. Real Time Image Process 18(5):1837–1849. https://doi.org/10.1007/s11554-021-01167-y

    Article  Google Scholar 

  24. Jin J, Li Y, Sun L, Miao J, Chen CLP (2020) A new local knowledge-based collaborative representation for image recognition. IEEE Access 8:81 069-81 079. https://doi.org/10.1109/ACCESS.2020.2989452

  25. Liu R (2023) A novel synthetic minority oversampling technique based on relative and absolute densities for imbalanced classification. Appl Intell 53(1):786–803. https://doi.org/10.1007/s10489-022-03512-5

    Article  Google Scholar 

  26. Wang Z, Jusup M, Guo H, Shi L, Geček S, Anand M, Perc M, Bauch CT, Kurths J, Boccaletti S et al (2020) Communicating sentiment and outlook reverses inaction against collective risks. Proc Natl Acad Sci 117(30):17 650-17 655. https://www.pnas.org/doi/abs/10.1073/pnas.1922345117

  27. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357. https://doi.org/10.1613/jair.953

    Article  Google Scholar 

  28. Han H, Wang W, Mao B (2005) Borderline-smote: A new over-sampling method in imbalanced data sets learning. In. Huang D, Zhang XS, Huang G (eds) Advances in intelligent computing, international conference on intelligent computing, ICIC 2005, Hefei, China, August 23-26, 2005, Proceedings, Part I, ser. Lecture Notes in Computer Science, vol 3644. Springer, pp 878–887. https://doi.org/10.1007/11538059_91

  29. Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C (2009) Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In. Theeramunkong T, Kijsirikul B, Cercone N, Ho TB (eds) Advances in knowledge discovery and data mining, 13th pacific-asia conference, PAKDD 2009, Bangkok, Thailand, April 27-30, 2009, Proceedings, ser. Lecture Notes in Computer Science, vol 5476. Springer, pp 475–482. https://doi.org/10.1007/978-3-642-01307-2_43

  30. Koto F, (2014) Smote-out, smote-cosine, and selected-smote: An enhancement strategy to handle imbalance in data level. In 2014 international conference on advanced computer science and information system, pp 280–284. IEEE. https://doi.org/10.1109/ICACSIS.2014.7065849

  31. Douzas G, Bação F, Last F (2018) Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE. Inf Sci 465:1–20. https://doi.org/10.1016/j.ins.2018.06.056

    Article  Google Scholar 

  32. Huang K, Wang X (2022) ADA-INCVAE: improved data generation using variational autoencoder for imbalanced classification. Appl Intell 52(3):2838–2853. https://doi.org/10.1007/s10489-021-02566-1

    Article  Google Scholar 

  33. Iranmehr A, Masnadi-Shirazi H, Vasconcelos N (2019) Cost-sensitive support vector machines. Neurocomputing 343:50–64. https://doi.org/10.1016/j.neucom.2018.11.099

    Article  Google Scholar 

  34. Sun J, Lang J, Fujita H, Li H (2018) Imbalanced enterprise credit evaluation with DTE-SBD: decision tree ensemble based on SMOTE and bagging with differentiated sampling rates. Inf Sci 425:76–91. https://doi.org/10.1016/j.ins.2017.10.017

    Article  MathSciNet  Google Scholar 

  35. Gao X, Ren B, Zhang H, Sun B, Li J, Xu J, He Y, Li K (2020) An ensemble imbalanced classification method based on model dynamic selection driven by data partition hybrid sampling. Expert Syst Appl 160:113660. https://doi.org/10.1016/j.eswa.2020.113660

    Article  Google Scholar 

  36. Asheghi R, Hosseini SA, Saneie M, Shahri AA (2020) Updating the neural network sediment load models using different sensitivity analysis methods: a regional application. J Hydroinformatics 22(3):562–577. https://doi.org/10.2166/hydro.2020.098

    Article  Google Scholar 

  37. Zhang P (2019) A novel feature selection method based on global sensitivity analysis with application in machine learning-based prediction model. Appl Soft Comput 85. https://doi.org/10.1016/j.asoc.2019.105859

  38. Liu Z, Jin W, Mu Y (2020) Variances-constrained weighted extreme learning machine for imbalanced classification. Neurocomputing 403:45–52. https://doi.org/10.1016/j.neucom.2020.04.052

    Article  Google Scholar 

  39. Yang R, Kan J (2023) Euclidean distance-based adaptive collaborative representation with tikhonov regularization for hyperspectral image classification. Multim Tools Appl 82(4):5823–5838. https://doi.org/10.1007/s11042-022-13597-2

    Article  Google Scholar 

  40. Cai S, Zhang L, Zuo W, Feng X (2016) A probabilistic collaborative representation based approach for pattern classification. In 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, pp 2950–2959. https://doi.org/10.1109/CVPR.2016.322

  41. Li Y, Jin J, Zhao L, Wu H, Sun L, Chen CLP (2021) A neighborhood prior constrained collaborative representation for classification. Int J Wavelets Multiresolution Inf Process 19(2):2 050 073:1-2:050:0732.2. https://doi.org/10.1142/S0219691320500733

  42. Li Y, Wang S, Jin J, Chen CLP (2022) Weighted competitive-collaborative representation based classifier for imbalanced data classification. In. Fang L, Povey D, Zhai G, Mei T, Wang R (eds) Artificial intelligence - Second CAAI International Conference, CICAI 2022, Beijing, China, August 27-28, 2022, Revised Selected Papers, Part II, ser. Lecture Notes in Computer Science, vol 13605. Springer, pp 462–472. https://doi.org/10.1007/978-3-031-20500-2_38

  43. Wang X, Zhang M, Chen B, Wei D, Shao Y (2023) Dynamic weighted multitask learning and contrastive learning for multimodal sentiment analysis. Electronics 12(13):2986. https://www.mdpi.com/2079-9292/12/13/2986

  44. Li J, Zhang H, Zhang L (2015) A nonlinear multiple feature learning classifier for hyperspectral images with limited training samples. IEEE J Sel Top Appl Earth Obs Remote Sens 8(6):2728–2738. https://doi.org/10.1109/JSTARS.2015.2400634

    Article  Google Scholar 

  45. Mirfallah Lialestani SP, Parcerisa D, Himi M, Abbaszadeh Shahri A (2022) Generating 3d geothermal maps in catalonia, spain using a hybrid adaptive multitask deep learning procedure. Energies 15(13). https://www.mdpi.com/1996-1073/15/13/4602

  46. Li J, Zhang H, Zhang L, Huang X, Zhang L (2014) Joint collaborative representation with multitask learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 52(9):5923–5936. https://doi.org/10.1109/TGRS.2013.2293732

    Article  Google Scholar 

  47. Han B, Wei Y, Wang Q, Wan S (2023) Dual adaptive learning multi-task multi-view for graph network representation learning. Neural Netw 162:297–308. https://doi.org/10.1016/j.neunet.2023.02.026

    Article  Google Scholar 

  48. Mao Y, Wang Z, Liu W, Lin X, Xie P (2022) Meta Weighting: Learning to weight tasks in multi-task learning. In. Muresan S, Nakov P, Villavicencio A (eds) Findings of the association for computational linguistics: ACL 2022, Dublin, Ireland: Association for Computational Linguistics, pp 3436–3448. https://aclanthology.org/2022.findings-acl.271

  49. Gong T, Lee T, Stephenson C, Renduchintala V, Padhy S, Ndirango A, Keskin G, Elibol OH (2019) A comparison of loss weighting strategies for multi task learning in deep neural networks. IEEE Access 7:141 627-141 632. https://doi.org/10.1109/ACCESS.2019.2943604

Download references

Acknowledgements

This work was funded in part by the National Natural Science Foundation of China under Grant 62106068, 62106233, and 62303427, in part by the Science and Technology Research Project of Henan Province under Grant 242102211057, 242102211018, 232102210062, 222102210096, and 232102210014, in part by the Science and Technology Innovation Project of Zhengzhou University of Light Industry under Grant 23XNKJTD0205, and in part by the Grant 2022BSJJZK13, 202310463038, 2020BSJJ027, 22ZZRDZX29, 2020BSJJ067, 202310463060, 202310463002, and PX-38233882.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhengzhou University of Light Industry, 450001, Zhengzhou, China

    Yanting Li, Shuai Wang, Hongwei Tao & Chuang Han

  2. College of Artificial Intelligence and Big Data, Henan University of Technology, 450001, Zhengzhou, China

    Junwei Jin

  3. School of Computer Science and Engineering South, China University of Technology, 510641, Guangzhou, China

    C. L. Philip Chen

Authors
  1. Yanting Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junwei Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongwei Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chuang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. C. L. Philip Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yanting Li: Conceptualization, Methodology; Shuai Wang: Software, Writing- Original draft preparation; Junwei Jin: Data curation, Validation; Hongwei Tao: Visualization, Investigation; Chuang Han: Editing, Supervision; C. L. Philip Chen: Supervision, Writing- Reviewing.

Corresponding author

Correspondence to Junwei Jin.

Ethics declarations

Ethical and informed consent for data used

This study has not been submitted elsewhere for publication, in whole or in part, and all the authors listed have approved the manuscript that is enclosed. The authors are aware that all data used are public and non-confidential

Competing Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Wang, S., Jin, J. et al. Hybrid density-based adaptive weighted collaborative representation for imbalanced learning. Appl Intell 54, 4334–4351 (2024). https://doi.org/10.1007/s10489-024-05393-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05393-2

Keywords

Search

Navigation