Comparative Study of Preprocessing and Classification Methods in Character Recognition of Natural Scene Images

  • Yash SinhaEmail author
  • Prateek Jain
  • Nirant Kasliwal
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 390)


This paper presents an approach to character recognition in natural scene images. Recognizing such text is a challenging problem in the field of Computer Vision, more than the recognition of scanned documents due to several reasons. We propose a classification technique for classifying characters based on a pipeline of image processing operations and ensemble machine learning techniques. This pipeline tackles problems where Optical Character Recognition (OCR) fails. We present a framework that comprises a sequence of operations such as resizing, grey scaling, thresholding, morphological opening and median filtering on the images to handle background clutter, noise, multi-sized and multi-oriented characters and variance in illumination. We used image pixels and HOG (Histogram of Oriented Gradients) as features to train three different models based on Nearest-Neighbour, Random Forest and Extra Tree classifiers. When the input images were pre-processed, HOG features were extracted and fed into extra tree classifier, and the model classified the characters with maximum accuracy, among the other models that we tested. The proposed steps have been experimentally proven to yield better accuracy than the present state-of-the-art classification techniques on the Chars74k dataset. In addition, the paper includes a comparative study elaborating on various image processing operations, feature extraction methods and classification techniques.


Camera-based character recognition Histogram of oriented gradients Feature extraction Scene text recognition Ensemble classifiers 


  1. 1.
    Fraz, M., Sarfraz, M., Edirisinghe, E.: Exploiting colour information for better scene text recognition. In: Proceedings of the British Machine Vision Conference. BMVA Press (2014)Google Scholar
  2. 2.
    de Campos, T.E., Babu, B.R., Varma, M.: Character recognition in natural images. In: Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal (2009)Google Scholar
  3. 3.
    Sheshadri, K., Divvala, S.K.: Exemplar driven character recognition in the wild. In: BMVC, pp. 1–10 (2012)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  5. 5.
    Zhang, Z., Sturgess, P., Sengupta, S., Crook, N., Torr, P.: Efficient discriminative learning of parametric nearest neighbor classifiers. (2012)Google Scholar
  6. 6.
    Kumar, D., Ramakrishnan, A.: Recognition of kannada characters extracted from scene images. In: Proceeding of the Workshop on Document Analysis and Recognition, pp. 15–21. ACM (2012)Google Scholar
  7. 7.
    Neumann, L., Matas, J.: A method for text localization and recognition in real-world images. In: Computer Vision-ACCV 2010, pp. 770–783. Springer (2011)Google Scholar
  8. 8.
    deCampos, T.: The Chars74k Dataset. (2009). Accessed 5 Dec 2014]
  9. 9.
    8.3, M., Release, C.V.S.T., 2014a, I.P.T.R.: version 8.3 (R2014a). The MathWorks Inc., Natick, Massachusetts (2014)Google Scholar
  10. 10.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)zbMATHMathSciNetGoogle Scholar
  11. 11.
    Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11, 23–27 (1975)Google Scholar
  12. 12.
    Gonzalez, R.C., Woods, R.E.: Morphological image processing. Digital Image Processing. Addison-Wesley Longman Publishing Co., Inc., Boston (2001)Google Scholar
  13. 13.
    Gonzalez, R.C., Woods, R.E.: Order-statistic (nonlinear) filters, example 3.14. In: Digital Image Processing, pp. 157. Pearson Education (2008)Google Scholar
  14. 14.
    Lim, J.S.: Two-Dimensional Signal and Image Processing, vol. 710, p. 1. Prentice Hall, Englewood Cliffs (1990)Google Scholar
  15. 15.
    Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13, 21–27 (1967)zbMATHCrossRefGoogle Scholar
  16. 16.
    Scikit-learn: sklearn.neighbors.KNeighborsClassifier Scikit-learn. ( Accessed 5 Nov 2014
  17. 17.
    Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)zbMATHCrossRefGoogle Scholar
  18. 18.
    Scikit-learn: sklearn.ensemble.RandomForestClassifier. ( Accessed 5 Nov 2014
  19. 19.
    Scikit-learn: Grid Search: Searching for estimator parameters. ( Accessed 5 Nov 2014
  20. 20.
    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63, 3–42 (2006)zbMATHCrossRefGoogle Scholar
  21. 21.
    Scikit-learn: sklearn.ensemble.ExtraTreesClassifier - Scikit-learn. ( Accessed 5 Nov 2014
  22. 22.
    Shi, C.Z., Wang, C.H., Xiao, B.H., Zhang, Y., Gao, S.: Multi-scale graph-matching based kernel for character recognition from natural scenes. Acta Autom. Sinica 40, 751–756 (2014)CrossRefGoogle Scholar
  23. 23.
    Newell, A.J., Griffin, L.D.: Natural image character recognition using oriented basic image features. In: Proceedings of the 2011 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp. 191–196. IEEE (2011)Google Scholar

Copyright information

© Springer India 2016

Authors and Affiliations

  1. 1.Department of Computer Science and Information SystemsBirla Institute of Technology and SciencePilaniIndia

Personalised recommendations