Skip to main content
SpringerLink
Log in

Coarse-to-fine knowledge transfer based long-tailed classification via bilateral-sampling network

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Long-tailed classification faces a considerable challenge from the imbalanced distribution of head and tail data. Re-sampling is a traditional Single Branch Sampling (SBS) method used to adjust data imbalances that effectively improves the performance of long-tailed classification models. Most existing SBS models assume that the classes are independent of each other and ignore the hierarchical relations among the classes. However, the hierarchical structure is exhibited as coarse- and fine-grained semantic relations, a significant knowledge to guide long-tailed classification. In this paper, we propose a Coarse-to-Fine knowledge transfer based Bilateral-Sampling Network (CFBSNet) for long-tailed classification that alleviates the effects of imbalances in long-tailed data and considers coarse- and fine-grained semantic relationships. First, we present a Bilateral-Branch Sampling Network consisting of two sampling branches. The two sampling branches perform reverse sampling and uniform sampling, respectively. Second, we design a Coarse-to-Fine Knowledge Transfer strategy that regulates different learning stages by adjusting loss weight in each task progressively. CFBSNet pays attention to the semantic relationship between tail data and granularity. The experimental results demonstrated the effectiveness of CFBSNet for long-tailed classification tasks. For instance, the classification accuracy of CFBSNet is 3.16\(\%\) and 2.62\(\%\) better than that of baseline models on the CIFAR-100-LT and the SUN datasets, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Some or all data, models, or code generated or used during the study are available from the Data availability corresponding author by request (Hong Zhao).

References

  1. Cao Y, Kuang J, Gao M, Zhou A, Wen Y, Chua T (2023) Learning relation prototype from unlabeled texts for long-tail relation extraction. IEEE Trans Knowl Data Eng 35(2):1761–1774

  2. Wu T, Liu Z, Huang Q, Wang Y, Lin D (2021) Adversarial robustness under long-tailed distribution. In: IEEE/CVF conference on computer vision and pattern recognition, pp 8659–8668

  3. Goecks J, Jalili V, Heiser LM, Gray JW (2020) How machine learning will transform biomedicine. Cell 181(1):92–101

    Article  Google Scholar 

  4. Kulkarni R, Di Minin E (2021) Automated retrieval of information on threatened species from online sources using machine learning. Methods Ecol Evol 12(7):1226–1239

    Article  Google Scholar 

  5. Zeng D, Veldhuis R, Spreeuwers L (2021) A survey of face recognition techniques under occlusion. IET Biom 10(6):581–606

    Article  Google Scholar 

  6. Haggag M, Siam AS, El-Dakhakhni W, Coulibaly P, Hassini E (2021) A deep learning model for predicting climate-induced disasters. Nat Hazards 107(1):1009–1034

    Article  Google Scholar 

  7. Lin T, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: IEEE international conference on computer vision, pp 2980–2988

  8. Cao K, Wei C, Gaidon A, Arechiga N, Ma T (2019) Learning imbalanced datasets with label-distribution-aware margin loss. Adv Neural Inf Process Syst 32:1–8

    Google Scholar 

  9. Jamal MA, Brown M, Yang M, Wang L, Gong B (2020) Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective. In: IEEE/CVF conference on computer vision and pattern recognition, pp 7610–7619

  10. Huang C, Li Y, Loy CC, Tang X (2019) Deep imbalanced learning for face recognition and attribute prediction. IEEE Trans Pattern Anal Mach Intell 42(11):2781–2794

    Article  Google Scholar 

  11. Xiang L, Ding G, Han J (2020) Learning from multiple experts: self-paced knowledge distillation for long-tailed classification. In: European conference on computer vision, pp 247–263

  12. Chu P, Bian X, Liu S, Ling H (2020) Feature space augmentation for long-tailed data. In: European conference on computer vision, pp 694–710

  13. Yin X, Yu X, Sohn K, Liu X, Chandraker M (2019) Feature transfer learning for face recognition with under-represented data. In: IEEE/CVF conference on computer vision and pattern recognition, pp 5704–5713

  14. Jiang Z, Pan T, Zhang C, Yang J (2021) A new oversampling method based on the classification contribution degree. Symmetry 13(2):194

    Article  Google Scholar 

  15. Barua S, Islam MM, Yao X, Murase K (2012) MWMOTE-majority weighted minority oversampling technique for imbalanced data set learning. IEEE Trans Knowl Data Eng 26(2):405–425

    Article  Google Scholar 

  16. Han H, Wang W, Mao B (2005) Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International conference on intelligent computing, pp 878–887

  17. Kim J, Jeong J, Shin J (2020) M2m: imbalanced classification via major-to-minor translation. In: IEEE/CVF conference on computer vision and pattern recognition, pp 13896–13905

  18. Ng WW, Hu J, Yeung DS, Yin S, Roli F (2014) Diversified sensitivity-based undersampling for imbalance classification problems. IEEE Trans Cybern 45(11):2402–2412

    Article  Google Scholar 

  19. Deng X, Zhong W, Ren J, Zeng D, Zhang H (2016) An imbalanced data classification method based on automatic clustering under-sampling. In: IEEE International performance computing and communications conference, pp 1–8

  20. Kang Q, Chen X, Li S (2016) A noise-filtered under-sampling scheme for imbalanced classification. IEEE Trans Cybern 47(12):4263–4274

    Article  Google Scholar 

  21. Rekha G, Reddy VK, Tyagi AK (2020) Critical instances removal based under-sampling (CIRUS): a solution for class imbalance problem. Int J Hybrid Intell Syst 16(2):55–66

    Google Scholar 

  22. Xu H, Zhang X, Li H, Xie L, Dai W, Xiong H, Tian Q (2022) Seed the views: hierarchical semantic alignment for contrastive representation learning. IEEE Trans Pattern Anal Mach Intell 45(3):3753–3767

  23. Li S, Gong K, Liu CH, Wang Y, Qiao F, Cheng X (2021) Metasaug: meta semantic augmentation for long-tailed visual recognition. In: IEEE/CVF conference on computer vision and pattern recognition, pp 5212–5221

  24. Xu W, Yuan K, Li W, Ding W (2023) An emerging fuzzy feature selection method using composite entropy-based uncertainty measure and data distribution. IEEE Trans Emerg Top Comput Intell 7(1):76–88

    Article  Google Scholar 

  25. Li J, Li Y, Mi Y, Wu W (2020) Meso-granularity labeled method for multi-granularity formal concept analysis. J Comput Res Dev 57(2):447–458

    Google Scholar 

  26. Liu R (2022) A novel synthetic minority oversampling technique based on relative and absolute densities for imbalanced classification. Appl Intell 53(1):768–803

  27. Buda M, Maki A, Mazurowski MA (2018) A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw 106:249–259

    Article  Google Scholar 

  28. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  MATH  Google Scholar 

  29. Tahir MA, Kittler J, Yan F (2012) Inverse random under sampling for class imbalance problem and its application to multi-label classification. Pattern Recognit 45(10):3738–3750

    Article  Google Scholar 

  30. Zhou B, Cui Q, Wei X, Chen Z (2020) Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In: IEEE/CVF conference on computer vision and pattern recognition, pp 9719–9728

  31. Yao Y (2004) A partition model of granular computing. LNCS Trans Rough Sets I, LNCS 3100:232–253

  32. Yao Y (2008) Granular computing: past, present and future. In: IEEE international conference on granular computing, pp 80–85

  33. Chen Q, Liu Q, Lin E (2021) A knowledge-guide hierarchical learning method for long-tailed image classification. Neurocomputing 459:408–418

    Article  Google Scholar 

  34. Xu W, Guo D, Qian Y, Ding W (2022) Two-way concept-cognitive learning method: a fuzzy-based progressive learning. IEEE Trans Fuzzy Syst 1–15

  35. Xu W, Pan Y, Chen X, Ding W, Qian Y (2022) A novel dynamic fusion approach using information entropy for interval-valued ordered datasets. IEEE Trans Big Data 1–15

  36. Kendall A, Gal Y, Cipolla R (2018) Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: IEEE conference on computer vision and pattern recognition, pp 7482–7491

  37. Li T, Wang L, Wu G (2021) Self supervision to distillation for long-tailed visual recognition. In: IEEE/CVF international conference on computer vision, pp 630–639

  38. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, pp 630–645

  39. Bottou L (2012) Stochastic gradient descent tricks. In: Montavon G, Orr GB, Müller KR (eds) Neural networks: tricks of the trade. Springer, Berlin, pp 421–436

  40. Xu W, Guo D, Mi J, Qian Y, Zheng K, Ding W (2023) Two-way concept-cognitive learning via concept movement viewpoint. IEEE Trans Neural Netw Learn Syst 1–15

  41. Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41

    Article  Google Scholar 

  42. Yuan K, Xu W, Li W, Ding W (2022) An incremental learning mechanism for object classification based on progressive fuzzy three-way concept. Inf Sci 584:127–147

    Article  Google Scholar 

  43. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770–778

  44. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  45. Zhou B, Lapedriza A, Torralba A, Oliva A (2017) Places: an image database for deep scene understanding. J Vis 17(10):296–296

    Article  Google Scholar 

  46. Cui Y, Song Y, Sun C, Howard A, Belongie S (2018) Large scale fine-grained categorization and domain-specific transfer learning. In: IEEE conference on computer vision and pattern recognition, pp 4109–4118

  47. Huang C, Li Y, Loy CC, Tang X (2016) Learning deep representation for imbalanced classification. In: IEEE conference on computer vision and pattern recognition, pp 5375–5384

  48. De Boer P, Kroese DP, Mannor S, Rubinstein RY (2005) A tutorial on the cross-entropy method. Ann Oper Res 134(1):19–67

    Article  MathSciNet  MATH  Google Scholar 

  49. Li Z, Zhao H, Lin Y (2022) Multi-task convolutional neural network with coarse-to-fine knowledge transfer for long-tailed classification. Inf Sci 608:900–916

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Fujian Province under Grant no. 2021J011003 and the National Natural Science Foundation of China under Grant no. 62141602.

Author information

Authors and Affiliations

  1. School of Computer Science, Minnan Normal University, Zhangzhou, 363000, Fujian, China

    Junyan Xu, Wei Zhao & Hong Zhao

  2. Key Laboratory of Data Science and Intelligence Application, Fujian Province University, Zhangzhou, 363000, Fujian, China

    Junyan Xu, Wei Zhao & Hong Zhao

Authors
  1. Junyan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, J., Zhao, W. & Zhao, H. Coarse-to-fine knowledge transfer based long-tailed classification via bilateral-sampling network. Int. J. Mach. Learn. & Cyber. 14, 3323–3336 (2023). https://doi.org/10.1007/s13042-023-01835-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01835-4

Keywords

Search

Navigation