Skip to main content
SpringerLink
Log in

Label contrastive learning for image classification

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Image classification is one of the most important research tasks in computer vision. Current image classification methods with supervised learning have achieved good classification accuracy. However, supervised image classification methods mainly focus on the semantic differences at the class level, while lacking attention to the instance level. The core idea of contrastive learning is to compare positive and negative samples in the feature space to learn the feature representation, and the focus on instance-level information can make up for the lack of supervised learning. To this end, in this paper, we combine supervised learning and contrastive learning to propose labeled contrastive learning (LCL). Here, the supervised learning component ensures the distinguishability of different classes, the contrastive learning component enhances the compactness within classes and the separability between classes. In the contrastive learning component, instances with the same label are set as positive samples and instances with different labels are set as negative samples, which avoids the problem of false negative samples (positive samples are mislabeled as negative samples). Also, we applied a dynamic label memory bank and a momentum updated encoder. The experimental results show that LCL can further improve the accuracy of image classification compared with some supervised learning method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  • An F (2020) Image classification algorithm based on stacked sparse coding deep learning model-optimized kernel function nonnegative sparse representation. Soft Comput 24(22):16967–16981

    Article  Google Scholar 

  • Caron M, Misra I, Mairal J et al (2020) Unsupervised learning of visual features by contrasting cluster assignments. Adv Neural Inform Proces Syst 33:9912–9924

    Google Scholar 

  • Chen T, Kornblith S, Norouzi M, et al (2020a) A simple framework for contrastive learning of visual representations. In: International conference on machine learning, PMLR, pp 1597–1607

  • Chen T, Kornblith S, Swersky K et al (2020) Big self-supervised models are strong semi-supervised learners. Adv Neural Inform Proces Syst 33:22243–22255

    Google Scholar 

  • Chen X, He K (2021) Exploring simple siamese representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 15,750–15,758

  • Chen X, Fan H, Girshick R, et al (2020c) Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297

  • Chen X, Xie S, He K (2021) An empirical study of training self-supervised visual transformers. arXiv e-prints pp arXiv–2104

  • Ding X, Zhang X, Ma N, et al (2021) Repvgg: Making vgg-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13,733–13,742

  • Elsayed G, Krishnan D, Mobahi H, et al (2018) Large margin deep networks for classification. Advances in neural information processing systems 31

  • Goodfellow I, PougeAbadie J, Mirza M, et al (2014) Generative adversarial nets. Advances in neural information processing systems 27

  • Elsayed G, Krishnan D, Mobahi H, et al (2018) Large margin deep networks for classification. Adv Neural Inform Proces Syst 31

  • He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • He K, Fan H, Wu Y, et al (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9729–9738

  • Huang G, Liu Z, Van Der Maaten L, et al (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  • Iandola FN, Han S, Moskewicz MW, et al (2016) Squeezenet: alexnet-level accuracy with 50x fewer parameters and 0.5 mb model size. arXiv preprint arXiv:1602.07360

  • Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Proces Syst 25

  • LeCun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  • Van den Oord A, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding. arXiv e-prints pp arXiv–1807

  • Qin Z, Zhang Z, Chen X, et al (2018) Fd-mobilenet: Improved mobilenet with a fast downsampling strategy. In: 2018 25th IEEE International Conference on Image Processing (ICIP), IEEE, pp 1363–1367

  • Radosavovic I, Kosaraju RP, Girshick R, et al (2020) Designing network design spaces. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10428–10436

  • Ren Y, Bai J, Zhang J (2021) Label contrastive coding based graph neural network for graph classification. database systems for advanced applications

  • Rezende DJ, Mohamed S, Wierstra D (2014) Stochastic backpropagation and variational inference in deep latent gaussian models. In: International conference on machine learning, Citeseer, p 2

  • Shah A, Sra S, Chellappa R, et al (2022) Max-margin contrastive learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 8220–8230

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  • Szegedy C, Liu W, Jia Y, et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  • Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning, PMLR, pp 6105–6114

  • Thirumaladevi S, Veera Swamy K, Sailaja M (2022) Improved transfer learning of cnn through fine-tuning and classifier ensemble for scene classification. Soft Comput 26(12):5617–5636

    Article  Google Scholar 

  • Touvron H, Vedaldi A, Douze M, et al (2019) Fixing the train-test resolution discrepancy. Adv Neural Inform Proces Syst 32

  • Vincent P, Larochelle H, Bengio Y, et al (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning, pp 1096–1103

  • Wu Z, Xiong Y, Yu SX, et al (2018) Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3733–3742

  • Yang C, An Z, Cai L, et al (2022) Mutual contrastive learning for visual representation learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 3045–3053

  • Zbontar J, Jing L, Misra I, et al (2021) Barlow twins: Self-supervised learning via redundancy reduction. In: International Conference on Machine Learning, PMLR, pp 12310–12320

  • Zhang X, Zhou X, Lin M, et al (2018) Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6848–6856

  • Zheng L, Xiong J, Zhu Y, et al (2022) Contrastive learning with complex heterogeneity. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp 2594–2604

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Qingdao University, No.308 Ningxia Road, Qingdao, 266000, Shandong, China

    Han Yang & Jun Li

Authors
  1. Han Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Li.

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Ethical approval.

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by Oscar Castillo.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Li, J. Label contrastive learning for image classification. Soft Comput 27, 13477–13486 (2023). https://doi.org/10.1007/s00500-022-07808-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-07808-z

Keywords

Search

Navigation