Abstract
Compared to supervised learning, semi-supervised learning reduces the dependence of deep learning on a large number of labeled samples. In this work, we use a small number of labeled samples and perform data augmentation on unlabeled samples to achieve image classification. Our method constrains all samples to the predefined evenly-distributed class centroids (PEDCC) by the corresponding loss function. Specifically, the PEDCC-Loss for labeled samples, and the maximum mean discrepancy loss for unlabeled samples are used to make the feature distribution closer to the distribution of PEDCC. Our method ensures that the inter-class distance is large and the intra-class distance is small enough to make the classification boundaries between different classes clearer. Meanwhile, for unlabeled samples, we also use KL divergence to constrain the consistency of the network predictions between unlabeled and augmented samples. Our semi-supervised learning method achieves the state-of-the-art results, with 4000 labeled samples on CIFAR10 and 1000 labeled samples on SVHN, and the accuracy is 95.10% and 97.58% respectively. Code is available in https://github.com/sweetTT/semi-supervised-method-based-on-PEDCC.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 70–778
Lee DH (2013) Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: ICML workshop on challenges in representation learning, 3: 2
Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge
Cireşan DC, Meier U, Gambardella LM, Schmidhuber J (2010) Deep, big, simple neural nets for handwritten digit recognition. Neural Computation 22(12):3207–3220
Rasmus A, Berglund M, Honkala M, Valpola H, Raiko T (2015) Semi-supervised learning with ladder networks. In: Advances in neural information processing systems, pp 3546–3554
Laine S, Aila T (2017) Temporal ensembling for semi-supervised learning. In: 5th international conference on learning representations
Sajjadi M, Javanmardi M, Tasdizen T (2016) Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In: Advances in Neural Information Processing Systems, pp 1163–1171
Tarvainen A, Valpola H (2017) Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: Advances in neural information processing systems, pp 1195–1204
Miyato T, Maeda S, Koyama M, Ishii S (2018) Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell 41(8):1979–1993
Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2017) mixup: Beyond empirical risk minimization. arXiv:1710.09412
Berthelot D, Carlini N, Goodfellow I, Oliver A, Papernot N, Raffel C (2019) Mixmatch: A holistic approach to semi-supervised learning. arXiv. arXiv:1905.02249
Xie Q, Dai Z, Hovy E, Luong MT, Le QV (2019) Unsupervised data augmentation. arXiv:1904.12848
Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2018). arXiv:1805.09501
Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823
Liu W, Wen Y, Yu Z, Yang M (2016) Large-margin softmax loss for convolutional neural networks. In: International Conference on Machine Learning 2(3): 7
Wang F, Cheng J, Liu W, Liu H (2018) Additive margin softmax for face verification. IEEE Signal Process Lett 25(7):926– 930
Zhu QY, Zhang PJ, Ye X (2019) A New Loss Function for CNN Classifier Based on Pre-defined Evenly-Distributed Class Centroids. arXiv:1904.06008
Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Scholkopf B, Smola AJ (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14):e49– e57
Zhu QY, Zhang RX (2019) A classification supervised auto-encoder based on predefined evenly-distributed class Centroids. arXiv:1902.00220
Zagoruyko S, Komodakis N (2016) Wide residual networks. In: Proceedings of the British machine vision conference (BMVC)
Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. University of Toronto, Technical report
Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: NIPS workshop on deep learning and unsupervised feature learning
Oliver A, Odena A, Raffel CA, Cubuk ED, Goodfellow I (2018) Realistic evaluation of deep semi-supervised learning algorithms. In: Advances in neural information processing systems, pp 3235–3246
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhu, Qy., Li, Tt. Semi-supervised learning method based on predefined evenly-distributed class centroids. Appl Intell 50, 2770–2778 (2020). https://doi.org/10.1007/s10489-020-01689-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-01689-1