Skip to main content
SpringerLink
Log in

Discriminative Feature Learning via Sparse Autoencoders with Label Consistency Constraints

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Autoencoders have been successfully used to build deep hierarchical models of data. However, a deep architecture usually needs further supervised fine-tuning to obtain better discriminative capacity. To improve the discriminative capacity of deep hierarchical features, this paper proposes a new deterministic autoencoder, trained by a label consistency constraints algorithm that injects discriminative information to the network. We introduce the center loss as label consistency constraints to learn the hidden features of data and add it to the Sparse AutoEncoder to form a new autoencoder, namely Label Consistency Constrained Sparse AutoEncoders (LCCSAE). Specifically, the center loss learns the center of each class, and simultaneously penalizes the distances between the features and their corresponding class centers. In the end, autoencoders are stacked to form a deep architecture of LCCSAE for image classification tasks. To validate the effectiveness of LCCSAE, we compare it with other autoencoders in terms of the deeply learned features and the subsequent classification tasks on MNIST and CIFAR-bw datasets. Experimental results demonstrate the superiority of LCCSAE over other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MathSciNet  Google Scholar 

  2. Xu J, Xiang L, Liu Q, Gilmore H, Wu J, Tang J, Madabhushi A (2016) Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging 35(1):119–130

    Article  Google Scholar 

  3. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1794–1801

  4. Chen Y, Luo W, Yang J (2015) Facial landmark detection via pose-induced auto-encoder networks. In: IEEE international conference on image processing (ICIP), pp 2115–2119

  5. Luo W, Yang J, Xu W, Fu T (2015) Locality-constrained sparse auto-encoder for image classification. IEEE Signal Process Lett 22(8):1070–1073

    Article  Google Scholar 

  6. Yu J, Rui Y, Tang YY et al (2014) High-order distance-based multiview stochastic learning in image classification. IEEE Trans Cybern 44(12):2431–2442

    Article  Google Scholar 

  7. Erhan D, Szegedy C, Toshev A, Anguelov D (2014) Scalable object detection using deep neural networks. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2147–2154

  8. Li LJ, Su H, Fei-Fei L, Xing EP (2010) Object bank: a high-level image representation for scene classification and semantic feature sparsification. In: Advances in neural information processing systems, pp 1378–1386

  9. Zuo Z, Wang G, Shuai B, Zhao L, Yang Q (2015) Exemplar based deep discriminative and shareable feature learning for scene image classification. Pattern Recognit 48(10):3004–3015

    Article  Google Scholar 

  10. Feng Z, Huber P, Kittler J, Christmas W, Wu X (2015) Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Process Lett 1(22):76–80

    Article  Google Scholar 

  11. Shu Z, Zhao C, Huang P (2015) Local regularization concept factorization and its semi-supervised extension for image representation. Neurocomputing 158:1–12

    Article  Google Scholar 

  12. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3:71–86

    Article  Google Scholar 

  13. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  14. He X, Ji M, Bao H (2009) Graph embedding with constraints. Int Joint Conf Artif Intell (IJCAI) 9:1065–1070

    Google Scholar 

  15. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  16. Song X, Feng Z, Hu G, Kittler J, Christmas W, Wu X (2016) Dictionary integration using 3D morphable face models for pose-invariant collaborative-representation-based classification. ArXiv preprint arXiv:1611.00284

  17. Song X, Feng Z, Hu G, Wu X (2017) Half-face dictionary integration for representation-based classification. IEEE Trans Cybern 47(1):142–152

    Article  Google Scholar 

  18. Lu X, Wang Y, Yuan Y (2013) Graph-regularized low-rank representation for destriping of hyperspectral images. IEEE Trans Geosci Remote Sens 51(7):4009–4018

    Article  Google Scholar 

  19. Feng Z, Kittler J, Christmas W, Wu X, Pfeiffer S (2012) Automatic face annotation by multilinear AAM with missing values. In: 2012 21st international conference on pattern recognition (ICPR), pp 2586–2589

  20. Feng Z, Kittler J, Christmas W, Wu X (2016) A unified tensor-based active appearance face model. ArXiv preprint arXiv:1612.09548

  21. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788

    Article  Google Scholar 

  22. Shu Z, Wu X, Fan H, Huang P, Wu D, Hu C, Ye F (2017) Parameter-less auto-weighted multiple graph regularized nonnegative matrix factorization for data representation. Knowl Based Syst 131:1–194

    Article  Google Scholar 

  23. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  24. Erhan D, Bengio Y, Courville A, Manzagol PA, Vincent P, Bengio S (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660

    MathSciNet  MATH  Google Scholar 

  25. Poultney C, Chopra S, Cun YL (2006) Efficient learning of sparse representations with an energy-based model. In: Advances in neural information processing systems, pp 1137–1144

  26. Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 513–520

  27. Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on machine learning, pp 1096–1103

  28. Hosseini-Asl E, Zurada JM, Nasraoui O (2016) Deep learning of part-based representation of data using sparse autoencoders with nonnegativity constraints. IEEE Trans Neural Netw Learn Syst 27:2486–2498

    Article  Google Scholar 

  29. Hu C, Wu XJ (2016) Autoencoders with drop strategy. In: The 8th international conference on advances in brain inspired cognitive systems (BICS), pp 80–89

  30. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Advances in neural information processing systems, 19(153)

  31. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. ArXiv preprint arXiv:1207.0580

  32. Bourlard H, Kamp Y (1988) Auto-association by multilayer perceptrons and singular value decomposition. Biol Cybern 59(4–5):291–294

    Article  MathSciNet  Google Scholar 

  33. Riedmiller M (1994) Advanced supervised learning in multi-layer perceptrons-from backpropagation to adaptive learning algorithms. Comput Stand Interfaces 16(3):265–278

    Article  Google Scholar 

  34. Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision, pp 499–515

  35. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  36. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images Vol 1, No 4. Technical report, University of Toronto, p 7

Download references

Author information

Authors and Affiliations

  1. School of Internet of Things Engineering, Jiangnan University, Wuxi, 214122, Jiangsu Province, China

    Cong Hu & Xiao-Jun Wu

  2. School of Computer Engineering, Jiangsu University of Technology, Changzhou, 231001, Jiangsu Province, China

    Zhen-Qiu Shu

Authors
  1. Cong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao-Jun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen-Qiu Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Jun Wu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the 111 Project of Chinese Ministry of Education under Grant B12018, the Grants of the National Natural Science Foundation of China (Grant 61373055, 61672265 and 61603159) and Natural Science Foundation of Jiangsu Province of China under Grant BK20160293.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, C., Wu, XJ. & Shu, ZQ. Discriminative Feature Learning via Sparse Autoencoders with Label Consistency Constraints. Neural Process Lett 50, 1079–1091 (2019). https://doi.org/10.1007/s11063-018-9898-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-9898-1

Keywords

Search

Navigation