Advertisement

Sādhanā

, 43:15 | Cite as

Efficient document-image super-resolution using convolutional neural network

  • Ram Krishna Pandey
  • A G Ramakrishnan
Article

Abstract

Experiments performed by us using optical character recognizers (OCRs) show that the character level accuracy of the OCR reduces significantly with decrease in the spatial resolution of document images. There are real life scenarios, where high-resolution (HR) images are not available, where it is desirable to enhance the resolution of the low-resolution (LR) document image. In this paper, our objective is to construct a HR image, given a single LR binary image. The works reported in the literature mostly deal with super-resolution of natural images, whereas we try to overcome the spatial resolution problem in document images. We have trained and obtained a novel convolutional model based on neural networks, which achieves significant improvement in terms of the peak-signal-to-noise ratio (PSNR) of the reconstructed HR images. Using parametric rectified linear units, mean PSNR improvements of 2.32, 4.38, 6.43 and 8.92 dB have been achieved over those of LR input images of 50, 75, 100 and 150 dots per inch (dpi) resolution and average word level accuracy of almost 43%, 45% and 57% on 75 dpi Tamil, English and Kannada images, respectively.

Keywords

Document image spatial resolution convolutional neural network super-resolution document quality OCR PSNR recognition accuracy 

References

  1. 1.
    Aparna K G and Ramakrishnan A G 2002 A complete tamil optical character recognition system. In: Proceedings of the Fifth IAPR Workshop on Document Analysis Systems DAS-02, Princeton, NJ, August 19–21, pp. 53–57Google Scholar
  2. 2.
    Kumar V and Ramakrishnan A G 2002 Machine recognition of printed Kannada text. In: Proceedings of the Fifth IAPR Workshop on Document Analysis Systems DAS-02, August 19–21, Springer, Berlin, pp. 37–48Google Scholar
  3. 3.
    Manthan award 2014 for the project Gift of new abilities. http://manthanaward.org/e-inclusion-accessibilty-winner-2014/
  4. 4.
    Madhavaraj A, Ramakrishnan A G, Shiva Kumar H R and Bhat N 2014 Improved recognition of aged Kannada documents by effective segmentation of merged characters. In: Proceedings of the Tenth International Conference on Signal Processing and Communication (SPCOM 2014), Bangalore, July 22–24Google Scholar
  5. 5.
    Project Madurai for ancient Tamil literary works. http://www.projectmadurai.org/
  6. 6.
    Pandey R K and Ramakrishnan A G 2017 Language independent single document image super-resolution using CNN for improved recognition. arXiv preprint arXiv:1701.08835
  7. 7.
    Parker J A, Kenyon R V and Troxel D E 1983 Comparison of interpolating methods for image resampling. IEEE Trans. Med. Imaging 2(1): 31–39CrossRefGoogle Scholar
  8. 8.
    Yang C Y, Ma C and Yang M H 2014 Single-image super-resolution: a benchmark. In: Proceedings of the European Conference on Computer Vision. Springer International Publishing, Cham, SwitzerlandGoogle Scholar
  9. 9.
    Nasrollahi K and Moeslund T B 2014 Super-resolution: a comprehensive survey. Machine Vis. Appl. 25(6): 1423–1468CrossRefGoogle Scholar
  10. 10.
    Yang J, Wright J, Huang T S and Ma Y 2010 Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11): 2861–2873MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Yang J, Wang Z, Lin Z, Cohen S and Huang T 2012 Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8): 3467–3478MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Yang J, Wright J, Huang T and Ma Y 2008 Image super-resolution as sparse representation of raw image patches. In: Proceedings of Computer Vision and Pattern Recognition Google Scholar
  13. 13.
    Timofte R, Smet V D and Gool L V 2014 A+: adjusted anchored neighborhood regression for fast super-resolution. In: Proceedings of the Asian Conference on Computer Vision. Springer International Publishing, Cham, Switzerland, pp. 111–126Google Scholar
  14. 14.
    Timofte R, Rothe R and Gool L V 2016 Seven ways to improve example-based single image super resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Google Scholar
  15. 15.
    Timofte R, Smet V D and Gool L V 2013 Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision Google Scholar
  16. 16.
    Huang J B, Singh A and Ahuja N 2015 Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Google Scholar
  17. 17.
    Dong C, Loy C C, He K and Tang X 2014 Learning a deep convolutional network for image super-resolution. In: Proceedings of the European Conference on Computer Vision. Springer International Publishing, Cham, Switzerland, pp. 184–199Google Scholar
  18. 18.
    Dong C, Loy C C, He K and Tang X 2016 Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2): 295–307CrossRefGoogle Scholar
  19. 19.
    Johnson J, Alahi A and Fei-Fei L 2016 Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of the European Conference on Computer Vision. Springer International Publishing, Cham, Switzerland, pp. 694–711Google Scholar
  20. 20.
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y 2014 Generative adversarial nets. In: Advances in neural information processing systems Google Scholar
  21. 21.
    Ledig C, Theis L, Huszr F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z and Shi W 2016 Photo-realistic single image super-resolution using a generative adversarial network. arXiv:1609.04802
  22. 22.
    Schmidhuber J 2015 Deep learning in neural networks: an overview. Neural Netw. 61: 85–117CrossRefGoogle Scholar
  23. 23.
    Nielsen M A Neural networks and deep learning. URL: http://neuralnetworksanddeeplearning.com/
  24. 24.
    Hecht-Nielsen R 1988 Theory of the backpropagation neural network. In: Proceedings of Neural Networks, pp. 445–448Google Scholar
  25. 25.
    Ruder S 2016 An overview of gradient descent optimization algorithms. arXiv:1609.04747
  26. 26.
    Shivakumar H R and Ramakrishnan A G 2013 A tool that converted 200 Tamil books for use by blind students. In: Proceedings of the 12th International Tamil Internet Conference, Kuala Lumpur, MalaysiaGoogle Scholar
  27. 27.
    Smith R 2007 An overview of the Tesseract OCR engine. In: Proceedings of the Ninth International Conference on Document Analysis and Recognition, ICDAR 2007, IEEE, vol. 2Google Scholar
  28. 28.
    Wang Z,Liu D, Yang J, Han W and Huang T 2015 Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE International Conference on Computer Vision Google Scholar
  29. 29.
    Kim J, Lee J K and Lee K M 2016 Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Google Scholar
  30. 30.
    Glorot X and Bengio Y 2010 Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of AISTATS, vol. 9Google Scholar
  31. 31.
    He K, Zhang X, Ren S and Sun J 2015 Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings the IEEE International Conference on Computer Vision Google Scholar
  32. 32.
    Nair V and Hinton G E 2010 Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning Google Scholar
  33. 33.
    Karpathy A, Li F and Johnson J 2016 CS231n: Convolutional neural network for visual recognition. Online CourseGoogle Scholar
  34. 34.
    Team, The Theano Development, et al 2016 Theano: a python framework for fast computation of mathematical expressions. arXiv:1605.02688
  35. 35.
    Chollet F 2015 Keras. https://github.com/fchollet/keras
  36. 36.

Copyright information

© Indian Academy of Sciences 2018

Authors and Affiliations

  1. 1.Department of Electrical EngineeringIndian Institute of ScienceBangaloreIndia

Personalised recommendations