Skip to main content
SpringerLink
Log in

Discriminative Autoencoder for Feature Extraction: Application to Character Recognition

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Conventionally, autoencoders are unsupervised representation learning tools. In this work, we propose a novel discriminative autoencoder. Use of supervised discriminative learning ensures that the learned representation is robust to variations commonly encountered in image datasets. Using the basic discriminating autoencoder as a unit, we build a stacked architecture aimed at extracting relevant representation from the training data. The efficiency of our feature extraction algorithm ensures a high classification accuracy with even simple classification schemes like KNN (K-nearest neighbor). We demonstrate the superiority of our model for representation learning by conducting experiments on standard datasets for character/image recognition and subsequent comparison with existing supervised deep architectures like class sparse stacked autoencoder and discriminative deep belief network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. Qian Y, Ye M, Zhou J (2013) Hyperspectral image classification based on structured sparse logistic regression and three-dimensional wavelet texture features. IEEE Trans Geosci Remote Sens 51(4):2276–2291

    Article  Google Scholar 

  2. Vigdor B, Lerner B (2006) Accurate and fast off and online fuzzy ARTMAP-based image classification with application to genetic abnormality diagnosis. IEEE Trans Neural Netw 17(5):1288–1300

    Article  Google Scholar 

  3. Tao H, Hou C, Nie F, Jiao Y, Yi D (2016) Effective discriminative feature selection with nontrivial solution. IEEE Trans Neural Netw Learn Syst 27(4):796–808

    Article  MathSciNet  Google Scholar 

  4. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1. IEEE, pp 886–893

  5. Cheung W, Hamarneh G (2007) N-sift: N-dimensional scale invariant feature transform for matching medical images. In: 2007 4th IEEE international symposium on biomedical imaging: from nano to macro. IEEE, pp 720–723

  6. Ahonen T, Matas J, He C, Pietikäinen M (2009) Rotation invariant image description with local binary pattern histogram fourier features. In: Scandinavian conference on image analysis. Springer, Berlin, pp 61–70

  7. Gunturk BK, Batur AU, Altunbasak Y, Hayes MH, Mersereau RM (2003) Eigenface-domain super-resolution for face recognition. IEEE Trans Image Process 12(5):597–606

    Article  Google Scholar 

  8. Jing X-Y, Wong H-S, Zhang D (2006) Face recognition based on 2D Fisherface approach. Pattern Recogn 39(4):707–710

    Article  MATH  Google Scholar 

  9. Zhang B, Fu M, Yan H (1998) Handwritten digit recognition by a mixture of local principal component analysis. Proc Neural Process Lett 8(3):241–252

    Article  Google Scholar 

  10. Maria Joao, Amaro Joao, Falcao Gabriel, Alexandre Luís A (2016) Stacked autoencoders using low-power accelerated architectures for object recognition in autonomous systems. Neural Process Lett 43(2):445–458

    Article  Google Scholar 

  11. Mohamed A-R, Dahl GE, Hinton G (2012) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Process 20(1):14–22

    Article  Google Scholar 

  12. Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MATH  Google Scholar 

  13. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  14. Yu D, Deng L (2011) Deep learning and its applications to signal and information processing [exploratory dsp]. IEEE Signal Process Mag 28(1):145–154

    Article  Google Scholar 

  15. Zhou S, Chen Q, Wang X (2013) Convolutional deep networks for visual data classification. Neural Process Lett 38(1):17–27

    Article  Google Scholar 

  16. Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8):1771–1800

    Article  MATH  Google Scholar 

  17. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153

    Google Scholar 

  18. Abbas HM (2004) Analysis and pruning of nonlinear auto-association networks. IEEE Proc Vis Image Signal Process 151(1):44–50

    Article  Google Scholar 

  19. Bourlard H, Kamp Y (1988) Auto-association by multilayer perceptrons and singular value decomposition. Biol Cybern 59(4–5):291–294

    Article  MathSciNet  MATH  Google Scholar 

  20. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408

    MathSciNet  MATH  Google Scholar 

  21. Olshausen BA (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583):607–609

    Article  Google Scholar 

  22. Längkvist M, Loutfi A (2012) Learning representations with a dynamic objective sparse autoencoder. In: Neural information processing systems

  23. Lemme A, Reinhart RF, Steil JJ (2012) Online learning and generalization of parts-based image representations by non-negative sparse autoencoders. Neural Netw 33:194–203

    Article  Google Scholar 

  24. Chen M, Weinberger KQ, Sha F, Bengio Y (2014) Marginalized denoising auto-encoders for nonlinear representations. In: ICML, pp 1476–1484

  25. Razakarivony S, Jurie F (2014) Discriminative autoencoders for small targets detection. In: IAPR international conference on pattern recognition, pp 3528–3533

  26. Wang J, Gao X (2015) Max–min distance nonnegative matrix factorization. Neural Netw 61:75–84

    Article  MATH  Google Scholar 

  27. Zhang Q, Li B (2010) Discriminative K-SVD for dictionary learning in face recognition. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2691–2698

  28. Jiang Z, Lin Z, Davis LS (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35(11):2651–2664

    Article  Google Scholar 

  29. Larochelle H, Bengio Y (2008) Classification using discriminative restricted Boltzmann machines. In: Proceedings of the 25th international conference on machine learning. ACM, pp 536–543

  30. Goldstein T, Osher S (2009) The split Bregman method for L1-regularized problems. SIAM J Imaging Sci 2(2):323–343

    Article  MathSciNet  MATH  Google Scholar 

  31. http://www.iro.umontreal.ca/~lisa/twiki/bin/view.cgi/Public/DeepVsShallowComparisonICML2007

  32. http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html

  33. http://www.isical.ac.in/~ujjwal/download/database.html

  34. Lawson CL, Hanson RJ (1995) Solving least squares problems, vol 15. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  35. Ng A (2011) Sparse autoencoder. CS294A lecture notes 72:1–19

    Google Scholar 

  36. Majumdar A, Vatsa M, Singh R (2017) Face recognition via class sparsity based supervised encoding. IEEE Trans Pattern Anal Mach Intell 39(6):1273–1280

    Article  Google Scholar 

  37. Liu Y, Zhoub S, Chen Q (2011) Discriminative deep belief networks for visual data classification. Pattern Recogn 44(10–11):2287–2296

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Indraprastha Institute of Information Technology, New Delhi, India

    Anupriya Gogna & Angshul Majumdar

Authors
  1. Anupriya Gogna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angshul Majumdar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angshul Majumdar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gogna, A., Majumdar, A. Discriminative Autoencoder for Feature Extraction: Application to Character Recognition. Neural Process Lett 49, 1723–1735 (2019). https://doi.org/10.1007/s11063-018-9894-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-9894-5

Keywords

Search

Navigation