A Hierarchical Approach for Handwritten Digit Recognition Using Sparse Autoencoder

  • An T. Duong
  • Hai T. Phan
  • Nam Do-Hoang Le
  • Son T. Tran
Part of the Studies in Computational Intelligence book series (SCI, volume 530)


Higher level features learning algorithms have been applied on handwritten digit recognition and got more promising results than just using raw intensity values with classification algorithms. However, the approaches of these algorithms still not take the advantage of specific characteristics of data. We propose a new method to learn higher level features from specific characteristics of data using sparse autoencoder. The main key of our appoarch is to divide the handwritten digits into subsets corresponding to specific characteristics. The experimental results show that the proposed method achieves lower error rates and time complexity than the original approach of sparse autoencoder. The results also show that the more correlated characteristics we define, the better higher level features we learn.


Higher level features Sparse autoencoder Handwritten digit recognition Specific characteristics 


  1. 1.
    Bagnell, J.A., Bradley, D.M.: Differentiable sparse coding. In: NIPS. pp. 113–120 (2008)Google Scholar
  2. 2.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR. vol. 1, pp. 886–893 (2005)Google Scholar
  3. 3.
    DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Mach. Learn. 46(1–3), 161–190 (2002)Google Scholar
  4. 4.
    Grosse, R., Raina, R., Kwong, H., Andrew, Y.Ng.: Shift-invariance sparse coding for audio classification. CoRR, abs/1206.5241, 2012.Google Scholar
  5. 5.
    Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Lennig M., Hunt, M., Mermelstein, P.: Experiments in syllable-based recognition of continuous speech. In: Proceedings of International Conference on Acoustics, Speech and, Signal Processing, pp. 880–883 (1996)Google Scholar
  8. 8.
    Huth, A.G., Gallant, J.L., Vu, A.T., Nishimoto, S.: A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76(6), 1210–1224 (2012)CrossRefGoogle Scholar
  9. 9.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, H.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE. vol. 86, pp. 2278–2324 (1998)Google Scholar
  10. 10.
    Lee, H., Battle, A., Raina, R., Andrew, Y.Ng.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems. vol. 19, pp. 801–808 (2007)Google Scholar
  11. 11.
    Lee, H., Grosse, R., Ranganath, R., Andrew, Y.Ng.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML. p. 77 (2009)Google Scholar
  12. 12.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)Google Scholar
  13. 13.
    Martens, J.: Deep learning via hessian-free optimization. In: Fürnkranz, J., Joachims, T.,(eds.) Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, pp. 735–742 June 2010Google Scholar
  14. 14.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)CrossRefGoogle Scholar
  15. 15.
    Raina, R., Battle, A., Lee, H., Packer, B., Andrew Y.Ng.: Self-taught learning: Transfer learning from unlabeled data. In: ICML ’07: Proceedings of the 24th International Conference on, Machine learning (2007)Google Scholar
  16. 16.
    Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, pp. 958–962 (2003)Google Scholar
  17. 17.
    Socher, R., Eric, H., Pennin, J., Andrew, Y.Ng, Manning, C.D.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q., (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, pp. 801–809 (2011)Google Scholar
  18. 18.
    Kai. Y., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Proceeding of 24th Advances in Neural Information Processing Systems, NIPS, pp. 2223–2231 (2009) (Curran Associates Inc.)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • An T. Duong
    • 1
  • Hai T. Phan
    • 1
    • 2
  • Nam Do-Hoang Le
    • 1
    • 3
  • Son T. Tran
    • 1
  1. 1.University of Science, HCMCHo Chi MinhVietnam
  2. 2.Advanced Program in Computer ScienceUniversity of Science, HCMCHo Chi MinhVietnam
  3. 3.John von Neumann InstituteVietnam National University HCMCHo Chi MinhVietnam

Personalised recommendations