Handwritten Digit Recognition Using GIST Descriptors and Random Oblique Decision Trees

  • Thanh-Nghi DoEmail author
  • Nguyen-Khang Pham
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 341)


Our investigation aims at constructing random oblique decision trees to recognize handwritten digits. At the pre-processing step, we propose to use the GIST descriptor to represent digit images in large number of dimensions. And then we propose a multi-class version of random oblique decision trees based on the linear discriminant analysis and the Kolmogorov-Smirnov splitting criterion that is suited for classifying high dimensional datasets. The experimental results on USPS, MNIST datasets show that our proposal has very high accuracy compared to state-of-the-art algorithms.


Handwritten digit recognition Random oblique decision trees GIST descriptors 


  1. 1.
    LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., Jackel, L.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRefGoogle Scholar
  2. 2.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  3. 3.
    Decoste, D., Schölkopf, B.: Training invariant support vector machines. Mach. Learn. 46(1–3), 161–190 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Kégl, B., Busa-Fekete, R.: Boosting products of base classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09), pp. 497–504. ACM (2009)Google Scholar
  5. 5.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)CrossRefzbMATHGoogle Scholar
  6. 6.
    Abou-zeid, H., El-ghazal, A., Al-khatib, A.: Computer recognition of unconstrained handwritten numerals. In: 2003 IEEE 46th Midwest Symposium on Circuits and Systems, vol. 2, pp. 969–973 (2003)Google Scholar
  7. 7.
    Ranzato, M., Boureau, Y.L., Chopra, S., LeCun, Y.: A unified energy-based framework for unsupervised learning. In: AISTATS, pp. 371–379 (2007)Google Scholar
  8. 8.
    Lauer, F., Suen, C.Y., Bloch, G.: A trainable feature extractor for handwritten digit recognition. Pattern Recogn. 40(6), 1816–1824 (2007)CrossRefzbMATHGoogle Scholar
  9. 9.
    Labusch, K., Barth, E., Martinetz, T.: Simple method for high-performance digit recognition based on sparse coding. Trans. Neural Netw. 19(11), 1985–1989 (2008)CrossRefGoogle Scholar
  10. 10.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)CrossRefGoogle Scholar
  11. 11.
    Cireşan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Computer Vision and Pattern Recognition (CVPR 2012), pp. 3642–3649 (2012). Arxiv preprint: arXiv:1202.2745
  12. 12.
    Lowe, D.: Object recognition from local scale invariant features. In: Proceedings of the 7th International Conference on Computer Vision, pp. 1150–1157 (1999)Google Scholar
  13. 13.
    Lowe, D.: Distinctive image features from scale invariant keypoints. Int. J. Comput. Vis. 91–110 (2004)Google Scholar
  14. 14.
    Bosch, A., Zisserman, A., Muñoz, X.: Scene classification via pLSA. In: Proceedings of the European Conference on Computer Vision, pp. 517–530 (2006)Google Scholar
  15. 15.
    Li, X., Wu, C., Zach, C., Lazebnik, S., Frahm, J.: Modeling and recognition of landmark image collections using iconic scene graphs. In: Proceedings of the 10th European Conference on Computer Vision: Part I, pp. 427–440 (2008)Google Scholar
  16. 16.
    Douze, M., Jégou, H., Sandhawalia, H., Amsaleg, L., Schmid, C.: Evaluation of gist descriptors for web-scale image search. In: Proceedings of the ACM International Conference on Image and Video Retrieval, pp. 1–8 (2009)Google Scholar
  17. 17.
    Dietterich, T., Kong, E.B.: Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report (1995)Google Scholar
  18. 18.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  19. 19.
    Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Computational Learning Theory: Proceedings of the Second European Conference, pp. 23–37 (1995)Google Scholar
  20. 20.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer (1995)Google Scholar
  21. 21.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.: Classification and Regression Trees. Wadsworth International (1984)Google Scholar
  22. 22.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
  23. 23.
    Murthy, S., Kasif, S., Salzberg, S., Beigel, R.: OC1: Randomized induction of oblique decision trees. In: Proceedings of the Eleventh National Conference on Artificial Intelligence, pp. 322–327 (1993)Google Scholar
  24. 24.
    Do, T.N., Lenca, P., Lallich, S., Pham, N.K.: Classifying very-high-dimensional data with random forests of oblique decision trees. In: Advances in Knowledge Discovery and Management. Studies in Computational Intelligence, vol. 292, pp. 39–55. Springer, Berlin (2010)Google Scholar
  25. 25.
    Fisher, R.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)CrossRefGoogle Scholar
  26. 26.
    Lemmond, T.D., Chen, B.Y., Hatch, A.O., Hanley, W.G.: Discriminant random forests. In: DMIN, pp. 55–61 (2008)Google Scholar
  27. 27.
    Menze, B.H., Kelm, B.M., Splitthoff, D.N., Koethe, U., Hamprecht, F.A.: On oblique random forests. In: Proceedings of the 2011 European Conference on Machine Learning and Knowledge Discovery in Databases—Volume Part II (ECML PKDD’11), pp. 453–469 Springer (2011)Google Scholar
  28. 28.
    Do, T.N., Lenca, P., Lallich, S.: Enhancing network intrusion classification through the kolmogorov-smirnov splitting criterion. In: Proceedings of the 3rd International Conference on Theories and Applications of Computer Science, pp. 50–61 (2010)Google Scholar
  29. 29.
    Friedman, J.H.: A recursive partitioning decision rule for nonparametric classification. IEEE Trans. Comput. 26(4), 404–408 (1977)CrossRefzbMATHGoogle Scholar
  30. 30.
    KreBel, U.: Pairwise classification and support vector machines, Advances in Kernel Methods: Support Vector Learning, pp. 255–268 (1999)Google Scholar
  31. 31.
    Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin DAGs for multiclass classification. Adv. Neural Inf. Process. Syst. 12, 547–553 (2000)Google Scholar
  32. 32.
    Vural, V., Dy, J.: A hierarchical method for multi-class support vector machines. In: Proceedings of the Twenty-First International Conference on Machine Learning, pp. 831–838 (2004)Google Scholar
  33. 33.
    Benabdeslem, K., Bennani, Y.: Dendogram-based SVM for multi-class classification. J. Comput. Inf. Technol. 14(4), 283–289 (2006)Google Scholar
  34. 34.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Berkeley Symposium on Mathematical Statistics and Probability. Univ. Calif. Press 1, 281–297 (1967)MathSciNetGoogle Scholar
  35. 35.
    Whaley, R., Dongarra, J.: Automatically tuned linear algebra software. In: Ninth SIAM Conference on Parallel Processing for Scientific Computing CD-ROM Proceedings (1999)Google Scholar
  36. 36.
    Chang, C.C., Lin, C.J.: LIBSVM—a library for support vector machines. (2003)
  37. 37.
    Bromley, J., Sackinger, E.: Neural-network and k-nearest-neighbor classifiers (1991)Google Scholar
  38. 38.
    Simard, P., LeCun, Y., Denker, J.: Efficient pattern recognition using a new transformation distance. In: Advances in Neural Information Processing Systems 5, [NIPS Conference], pp. 50–58 (1993)Google Scholar
  39. 39.
    Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of the Seventh International Conference on Document Analysis and Recognition, vol. 2, pp. 958–963 (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.College of Information TechnologyCan Tho UniversityCan ThoVietnam

Personalised recommendations