Skip to main content

Semi Supervised Autoencoder

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 9948)

Abstract

Autoencoders are self-supervised learning tools, but are unsupervised in the sense that class information is not required for training; but almost invariably they are used for supervised classification tasks. We propose to learn the autoencoder for a semi-supervised paradigm, i.e. with both labeled and unlabeled samples available. Given labeled and unlabeled data, our proposed autoencoder automatically adjusts – for unlabeled data it acts as a standard autoencoder (unsupervised) and for labeled data it additionally learns a linear classifier. We use our proposed semi-supervised autoencoder to (greedily) construct a stacked architecture. We demonstrate the efficacy our design in terms of both accuracy and run time requirements for the case of image classification. Our model is able to provide high classification accuracy with even simple classification schemes as compared to existing models for deep architectures.

Keywords

  • Autoencoder
  • Feature extraction
  • Classification
  • Semi-supervised learning

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-46672-9_10
  • Chapter length: 8 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-46672-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.

References

  1. Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length, and Helmholtz free energy. Adv. Neural Inf. Process. Syst. 6, 3–10 (1994)

    Google Scholar 

  2. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P-A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)

    Google Scholar 

  3. Chen, Y., Lin, Z., Zhao, X., Wang, G., Gu, Y.: Deep learning-based classification of hyperspectral data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(6), 2094–2107 (2014)

    CrossRef  Google Scholar 

  4. Maldonado, S., Weber, R., Basak, J.: Simultaneous feature selection and classification using kernel-penalized support vector machines. Inf. Sci. 181(1), 115–128 (2011)

    CrossRef  Google Scholar 

  5. Kong, H., Li, X., Wang, L., Teoh, E.K., Wang, J.-G., Venkateswarlu, R.: Generalized 2D principal component analysis. In: Proceedings of 2005 IEEE International Joint Conference on Neural Networks, IJCNN 2005, vol. 1, pp. 108–113. IEEE (2005)

    Google Scholar 

  6. Bengio, Y.: Learning deep architectures for AI. Found. Trends® Mach. Learn. 2(1), 1–127 (2009)

    MathSciNet  CrossRef  MATH  Google Scholar 

  7. Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    MathSciNet  CrossRef  MATH  Google Scholar 

  8. Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649. IEEE (2012)

    Google Scholar 

  9. Gao, S., Zhang, Y., Jia, K., Lu, J., Zhang, Y.: Single sample face recognition via learning deep supervised autoencoders. IEEE Trans. Inf. Forensics Secur. 10(10), 2108–2118 (2015)

    CrossRef  Google Scholar 

  10. Huang, G., Song, S., Gupta, J.N.D., Wu, C.: Semi-supervised and unsupervised extreme learning machines. IEEE Trans. Cybern. 44(12), 2405–2417 (2014)

    CrossRef  Google Scholar 

  11. Ranzato, M., Szummer, M.: Semi-supervised learning of compact document representations with deep networks. In: Proceedings of the 25th International Conference on Machine Learning, pp. 792–799. ACM (2008)

    Google Scholar 

  12. Larochelle, H., Mandel, M., Pascanu, R., Bengio, Y.: Learning algorithm for the classification restricted boltzmann machine. J. Mach. Learn. Res. 13(1), 643–669 (2012)

    MathSciNet  MATH  Google Scholar 

  13. Almousli, H., Vincent, P.: Semi supervised autoencoders: better focusing model capacity during feature extraction. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8226, pp. 328–335. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  14. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.-A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)

    Google Scholar 

  15. Lemme, A., Reinhart, R.F., Steil, J.J.: Online learning and generalization of parts-based image representations by non-negative sparse autoencoders. Neural Netw. 33, 194–203 (2012)

    CrossRef  Google Scholar 

  16. Jiang, Z., Lin, Z., Davis, L.S.: Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1697–1704. IEEE (2011)

    Google Scholar 

  17. Shrivastava, A., Pillai, J.K., Patel, V.M., Chellappa, R.: Learning discriminative dictionaries with partially labeled data. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 3113–3116. IEEE (2012)

    Google Scholar 

  18. http://www.iro.umontreal.ca/~lisa/twiki/bin/view.cgi/Public/DeepVsShallowComparisonICML2007

  19. https://www.cs.toronto.edu/~kriz/cifar.html

  20. http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html

  21. Lawson, C.L., Hanson, R.J.: Solving least squares problems, vol. 161. Prentice-hall, Englewood Cliffs (1974)

    MATH  Google Scholar 

  22. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    MathSciNet  CrossRef  MATH  Google Scholar 

  23. http://ceit.aut.ac.ir/~keyvanrad/DeeBNet%20Toolbox.html

  24. Ng, A.: Sparse autoencoder. CS294A Lecture notes 72 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anupriya Gogna .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Gogna, A., Majumdar, A. (2016). Semi Supervised Autoencoder. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9948. Springer, Cham. https://doi.org/10.1007/978-3-319-46672-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46672-9_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46671-2

  • Online ISBN: 978-3-319-46672-9

  • eBook Packages: Computer ScienceComputer Science (R0)