Handling noise in textual image resolution enhancement using online and offline learned dictionaries

  • Rim Walha
  • Fadoua Drira
  • Frank Lebourgeois
  • Christophe Garcia
  • Adel M. Alimi
Original Paper


The resolution enhancement of textual images poses a significant challenge mainly in the presence of noise. The inherent difficulties are twofold. First is the reconstruction of an upscaled version of the input low-resolution image without amplifying the effect of noise. Second is the achievement of an improved visual image quality and a better OCR accuracy. Classically, the issue is addressed by the application of a denoising step used as a preprocessing or a post-processing to the magnification process. Starting by a denoising process could be more promising to avoid any magnified artifacts while proceeding otherwise. However, the state of the art underlines the limitations of denoising approaches faced with the low spatial resolution of textual images. Recently, sparse coding has attracted increasing interest due to its effectiveness in different reconstruction tasks. This study proves that the application of an efficient sparse coding-based denoising process followed by the magnification process can achieve good restoration results even if the input image is highly noisy. The main specificities of the proposed sparse coding-based framework are: (1) cascading denoising and magnification of each image patch, (2) the use of sparsity stemmed from the non-local self-similarity given in textual images and (3) the use of dual dictionary learning involving both online and offline dictionaries that are selected adaptively for each local region of the input degraded image to recover its corresponding noise-free high-resolution version. Extensive experiments on synthetic and real low-resolution noisy textual images are carried out to validate visually and quantitatively the effectiveness of the proposed system. Promising results, in terms of image visual quality as well as character recognition rates, are achieved when compared it with the state-of-the-art approaches.


Resolution enhancement Denoising Textual image Sparse coding Online and offline learned dictionaries 


  1. 1.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). CrossRefzbMATHGoogle Scholar
  2. 2.
    Alaei, A., Conte, D., Blumenstein, M., Raveaux, R.: Document image quality assessment based on texture similarity index. In: Proceedings of 12th IAPR Workshop on Document Analysis Systems (DAS), Santorini, Greece, pp. 132–137 (2016).
  3. 3.
    Alaei, A., Conte, D., Raveaux, R.: Document image quality assessment based on improved gradient magnitude similarity deviation. In: Proceedings of 13th International Conference on Document Analysis and Recognition (ICDAR), Nancy, France, pp. 176–180 (2015).
  4. 4.
    Banerjee, J., Jawahar, C.V.: Super-resolution of text images using edge-directed tangent field. In: Proceedings of DAS, pp. 76–83. Washington, DC, USA (2008).
  5. 5.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proceedings of CVPR, pp. 60–65 (2005).
  6. 6.
    Caner, G., Haritaoglu, I.: Shape-dna: effective character restoration and enhancement for Arabic text documents. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 2053–2056. Washington, DC, USA (2010).
  7. 7.
    Chatterjee, P., Milanfar, P.: Clustering-based denoising with locally learned dictionaries. IEEE Trans. Image Process. 18(7), 1438–1451 (2009). MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chen, G., Zhu, F., Ann Heng, P.: An efficient statistical method for image noise level estimation. In: The IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  9. 9.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001). MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Chou, S.L., Yu, S.S.: Sorting qualities of handwritten Chinese characters for setting up a research database. In: International Conference on Document Analysis and Recognition, pp. 474–477 (1993)Google Scholar
  11. 11.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.O.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007). MathSciNetCrossRefGoogle Scholar
  12. 12.
    Dai, S., Han, M., Wu, Y., Gong, Y.: Bilateral back-projection for single image super resolution. In: Proceedings of ICME, pp. 1039–1042 (2007).
  13. 13.
    Dalley, G., Freeman, W.T., Marks, J.: Single-frame text super-resolution: a Bayesian approach. In: Proceedings of ICIP, pp. 3295–3298 (2004)Google Scholar
  14. 14.
    Davis, G., Mallat, S., Avellaneda, M.: Adaptive greedy approximations. Constr. Approx. 13(1), 57–98 (1997). MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Dong, W., Zhang, L., Lukac, R., Shi, G.: Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans. Image Process. 22(4), 1382–1394 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Dong, W., Zhang, L., Shi, G., Wu, X.: Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. 20(7), 1838–1857 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Drira, F., Lebourgeois, F.: Denoising textual images using local/non-local smoothing filters: a comparative study. In: Proceedings of ICFHR, pp. 521–526 (2012).
  18. 18.
    Drira, F., Lebourgeois, F., Emptoz, H.: Document images restoration by a new tensor based diffusion process: application to the recognition of old printed documents. In: Proceedings of ICDAR, pp. 321–325 (2009)Google Scholar
  19. 19.
    Drira, F., Lebourgeois, F., Emptoz, H.: A new pde-based approach for singularity-preserving regularization: application to degraded characters restoration. IJDAR 15(3), 183–212 (2012)CrossRefGoogle Scholar
  20. 20.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006). MathSciNetCrossRefGoogle Scholar
  21. 21.
    Fadili, M.J., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2007)CrossRefGoogle Scholar
  22. 22.
    Freeman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 28(3), 1–10 (2010)Google Scholar
  23. 23.
    Hale, C., Smith, E.H.B.: Human image preference and document degradation models. In: Proceedings of ICDAR, Curitiba, Brazil, pp. 257–261 (2007).
  24. 24.
    Hartley, R.T., Crumpton, K.: Quality of OCR for degraded text images. In: ACM Conference on Digital Library, pp. 228–229 (1999)Google Scholar
  25. 25.
    Hoang, T.V., Smith, E.H.B., Tabbone, S.: Sparsity-based edge noise removal from bilevel graphical document images. IJDAR 17(2), 161–179 (2014)CrossRefGoogle Scholar
  26. 26.
    Kulesh, V., Schaffer, K., Sethi, I.K., Schwartz, M.: Handwriting quality evaluation. In: Proceedings of Second International Conference on Advances in Pattern Recognition (ICAPR), Brazil, pp. 157–165 (2001)Google Scholar
  27. 27.
    Kumar, D., Ramakrishnan, A.G.: Quad: quality assessment of documents. In: International Workshop on Camera-based Document Analysis and Recognition, pp. 79–84 (2011)Google Scholar
  28. 28.
    Kumar, J., Chen, F., Doermann, D.S.: Sharpness estimation for document and scene images. In: Proceedings of ICPR, Tsukuba, Japan, pp. 3292–3295 (2012)Google Scholar
  29. 29.
    Kumar, V., Bansal, A., Tulsiyan, G.H., Mishra, A., Namboodiri, A.M., Jawahar, C.V.: Sparse document image coding for restoration. In: Proceedings of ICDAR, pp. 713–717 (2013)Google Scholar
  30. 30.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 801–808 (2007)Google Scholar
  31. 31.
    Lelore, T., Bouchara, F.: FAIR: a fast algorithm for document image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 2039–2048 (2013). CrossRefGoogle Scholar
  32. 32.
    Li, R., Zhang, Y.J.: A hybrid filter for the cancellation of mixed Gaussian noise and impulse noise. In: IEEE International Conference on Information Communications and Signal Processing, pp. 508–512 (2003)Google Scholar
  33. 33.
    Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001). CrossRefGoogle Scholar
  34. 34.
    Liu, X., Tanaka, M., Okutomi, M.: Single-image noise level estimation for blind denoising. IEEE Trans. Image Process. 22(12), 5226–5237 (2013). CrossRefGoogle Scholar
  35. 35.
    Lopresti, D.P., Zhou, J., Nagy, G., Sarkar, P.: Spatial sampling effects in optical character recognition. In: Proceedings of ICDAR, pp. 309–314 (1995)Google Scholar
  36. 36.
    Lu, H., Kot, A.C., Shi, Y.Q.: Distance-reciprocal distortion measure for binary document images. IEEE Signal Process. Lett. 11(2), 228–231 (2004). CrossRefGoogle Scholar
  37. 37.
    Luong, H., Philips, W.: Robust reconstruction of low-resolution document images by exploiting repetitive character behaviour. IJDAR 11(1), 39–51 (2008). CrossRefGoogle Scholar
  38. 38.
    Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. In: IEEE Transactions on Image Processing, pp. 53–69. ITIP (2007)Google Scholar
  39. 39.
    Mirkin, B.: Clustering for Data Mining: A Data Recovery Approach, 1st edn. Chapman and Hall/CRC, Boca Raton (2005)CrossRefzbMATHGoogle Scholar
  40. 40.
    Namane, A., Sid-Ahmed, M.A.: Character scaling by contour method. IEEE Trans. Pattern Anal. Mach. Intell. 12(6), 600–606 (1990)CrossRefGoogle Scholar
  41. 41.
    Nayef, N., Chazalon, J., Gomez-Kramer, P., Ogier, J.M.: Efficient example-based super-resolution of single text images based on selective patch processing. In: Proceedings of DAS, pp. 227–231 (2014).
  42. 42.
    Obafemi-Ajayi, T., Agam, G.: Character-based automated human perception quality assessment in document images. IEEE Trans. Syst. Man Cybern. Part A 42(3), 584–595 (2012). CrossRefGoogle Scholar
  43. 43.
    Obafemi-Ajayi, T., Agam, G., Frieder, O.: Evaluation of human perception of degradation in document images. In: Proceedings of Document Recognition and Retrieval XVII, part of the IS&T-SPIE Electronic Imaging Symposium, San Jose, CA, USA, p. 75340T (2010).
  44. 44.
    Pan, J., Hu, Z., Su, Z., Yang, M.: Deblurring text images via L0-regularized intensity and gradient prior. In: Proceedings of CVPR, Columbus, OH, USA, pp. 2901–2908 (2014).
  45. 45.
    Pan, J., Hu, Z., Su, Z., Yang, M.: L\(_{\text{0 }}\)-regularized intensity and gradient prior for deblurring text images and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 342–355 (2017). CrossRefGoogle Scholar
  46. 46.
    Park, J., Kang, H., Lee, S.: Automatic quality measurement of gray-scale handwriting based on extended average entropy. In: Proceedings of ICPR, Barcelona, Spain, pp. 4426–4429 (2000).
  47. 47.
    Pati, Y.C., Rezaiifar, R., Rezaiifar, Y.C.P.R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of the 27th Annual Asilomar Conference on Signals, Systems, and Computers, pp. 40–44 (1993)Google Scholar
  48. 48.
    Peng, X., Cao, H., Subramanian, K., Prasad, R., Natarajan, P.: Automated image quality assessment for camera-captured OCR. In: Proceedings of ICIP, Brussels, Belgium, pp. 2621–2624 (2011).
  49. 49.
    Pham, T.A., Delalandre, M.: Effective decompression of JPEG document images. IEEE Trans. Image Process. 25(8), 3655–3670 (2016). MathSciNetCrossRefGoogle Scholar
  50. 50.
    Portilla, J.: Full blind denoising through noise covariance estimation using Gaussian scale mixtures in the wavelet domain. In: International Conference on Image Processing, vol. 2, pp. 1217–1220 (2004).
  51. 51.
    Pratt, W.K.: Digital Image Processing, 3rd edn. Wiley, New York (1978)zbMATHGoogle Scholar
  52. 52.
    Pratt, W.K.: Digital Image Processing: PIKS Inside, 3rd edn. Wiley, New York (2001)CrossRefzbMATHGoogle Scholar
  53. 53.
    Rice, S.V.: Measuring the accuracy of page-reading systems. Ph.D. thesis, UNLV, Las Vegas (1996)Google Scholar
  54. 54.
    Rubinstein, R., Bruckstein, A., Elad, M.: Dictionaries for sparse representation modeling. Proc. IEEE 98(6), 1045–1057 (2010). CrossRefGoogle Scholar
  55. 55.
    Rusiñol, M., Chazalon, J., Ogier, J.: Combining focus measure operators to predict OCR accuracy in mobile-captured document images. In: Proceedings of DAS, Tours, France, pp. 181–185 (2014).
  56. 56.
    Sarkar, P., Nagy, G., Zhou, J., Lopresti, D.P.: Spatial sampling of printed patterns. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 344–351 (1998)CrossRefGoogle Scholar
  57. 57.
    Smith, E.H.B., Andersen, T.L.: Text degradations and OCR training. In: Proceedings of ICDAR, Seoul, Korea, pp. 834–838 (2005).
  58. 58.
    Thouin, P., Du, Y., Chang, C.I.: Low resolution expansion of gray scale text images using Gibbs–Markov random field model. In: Proceedings of Symposium on Document Image Understanding Technologies, Columbia, MD, pp. 41–47 (2001)Google Scholar
  59. 59.
    Thouin, P.D., Chang, C.I.: A method for restoration of low-resolution document images. IJDAR 2(4), 200–210 (2000)CrossRefGoogle Scholar
  60. 60.
    Turkan, M.: Nouvelles méthodes de synthèse de texture; application à la prédiction et à l’inpainting d’images. Thesis, Université Rennes 1 (2011)Google Scholar
  61. 61.
    Walha, R., Drira, F., Lebourgeois, F., Alimi, A.M.: Super-resolution of single text image by sparse representation. In: Proceedings of the workshop on Document Analysis and Recognition (DAR), pp. 22–29. ACM, New York, NY, USA (2012).
  62. 62.
    Walha, R., Drira, F., Lebourgeois, F., Alimi, A.M., Garcia, C.: Resolution enhancement of textual images: a survey of single image-based methods. IET Image Process. 10(4), 325–337 (2016). CrossRefGoogle Scholar
  63. 63.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Multiple learned dictionaries based clustered sparse coding for the super-resolution of single text image. In: Proceedings of International Conference on Document Analysis and Recognition, ICDAR, pp. 484–488 (2013)Google Scholar
  64. 64.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Single textual image super-resolution using multiple learned dictionaries based sparse coding. In: Proceedings of International Conference on Image Analysis and Processing, ICIAP, Part 2, pp. 439–448 (2013)Google Scholar
  65. 65.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Histogram of structure tensors: application to pattern clustering. In: Proceedings of WSCG, Plzen, pp. 345–352 (2014)Google Scholar
  66. 66.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: A sparse coding based approach for the resolution enhancement and restoration of printed and handwritten textual images. In: Proceedings of ICFHR, Greece, pp. 696–701 (2014)Google Scholar
  67. 67.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Sparse coding with a coupled dictionary learning approach for textual image super-resolution. In: Proceedings of ICPR, Stockholm, pp. 4459–4464 (2014)Google Scholar
  68. 68.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Resolution enhancement of textual images via multiple coupled dictionaries and adaptive sparse representation selection. Int. J. Doc. Anal. Recognit. 18(1), 87–107 (2015). CrossRefGoogle Scholar
  69. 69.
    Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Joint denoising and magnification of noisy low-resolution textual images. In: Proceedings of ICDAR, Nancy, France, pp. 871–875 (2015)Google Scholar
  70. 70.
    Weickert, J.: A review of nonlinear diffusion filtering. In: Proceedings of International Conference on Scale-Space Theory in Computer Vision, pp. 3–28 (1997)Google Scholar
  71. 71.
    Xu, J., Ye, P., Li, Q., Liu, Y., Doermann, D.S.: No-reference document image quality assessment based on high order image statistics. In: Proceedings of ICIP, Phoenix, AZ, USA, pp. 3289–3293 (2016).
  72. 72.
    Yan, Z., Lu, Y., Li, J.: Super resolution of text image by pruning outlier. Proc. ICONIP 7064, 649–656 (2011)Google Scholar
  73. 73.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  74. 74.
    Yang, M.C., Wang, Y.C.F.: A self-learning approach to single image super-resolution. IEEE Trans. Multimed. 15(3), 498–508 (2013)MathSciNetCrossRefGoogle Scholar
  75. 75.
    Yang, S., Wang, M., Chen, Y., Sun, Y.: Single-image super-resolution reconstruction via learned geometric dictionaries and clustered sparse coding. IEEE Trans. Image Process. 21(9), 4016–4028 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  76. 76.
    Ye, P., Doermann, D.S.: Document image quality assessment: a brief survey. In: Proceedings of ICDAR, Washington, DC, USA, pp. 723–727 (2013).
  77. 77.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Proceedings of International conference on Curves and Surfaces, pp. 711–730 (2012).
  78. 78.
    Zhang, M., Desrosiers, C., Zhang, C., Cheriet, M.: Effective document image deblurring via gradient histogram preservation. In: Proceedings of ICIP, Quebec City, QC, Canada, pp. 779–783 (2015).
  79. 79.
    Zheng, Y., Kang, X., Li, S., He, Y., Sun, J.: Real-time document image super-resolution by fast matting. In: Proceedings of DAS, pp. 232–236 (2014).

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Rim Walha
    • 1
  • Fadoua Drira
    • 1
  • Frank Lebourgeois
    • 2
  • Christophe Garcia
    • 2
  • Adel M. Alimi
    • 1
  1. 1.REGIM-Lab, ENISUniversity of SfaxSfaxTunisia
  2. 2.LIRIS, UMR5205, CNRS, INSA-LyonUniversity of LyonLyonFrance

Personalised recommendations