Advertisement

Making scanned Arabic documents machine accessible using an ensemble of SVM classifiers

  • Randa Elanwar
  • Wenda Qin
  • Margrit Betke
Original Paper
  • 88 Downloads

Abstract

Raster-image PDF files originating from scanning or photographing paper documents are inaccessible to both text search engines and screen readers that people with visual impairments use. We here focus on the relatively less-researched problem of converting raster-image files with Arabic script into machine-accessible documents. Our method, called ECDP for “Ensemble-based classification of document patches,” segments the physical layout of the document, classifies image patches as containing text or graphics, assembles homogeneous document regions, and passes the text to an optical character recognition engine to convert into natural language. Classification is based on the majority voting of an ensemble of support vector machines. When tested on the dataset BCE-Arabic [Saad et al. in: ACM 9th annual international conference on pervasive technologies related to assistive environments (PETRA’16), Corfu, 2016], ECDP yielded an average patch classification accuracy of 97.3% and average \(F_1\) score of 95.26% for text patches and efficiently extracted text zones in both paragraphs and text-embedded graphics, even if the text is rotated by \(90^{\circ }\) or is in English. ECDP outperforms a classical layout analysis method (RLSA) and a state-of-the-art commercial product (RDI-CleverPage) on this dataset and maintains a relatively high level of performance on document images drawn from two other datasets (Hesham et al. in Pattern Anal Appl 20:1275–1287, 2017; Proprietary Dataset of 109 Arabic Documents. http://www.rdi-eg.com). The results suggest that the proposed method has the potential to generalize well to the analysis of documents with a broad range of content.

Keywords

Arabic document analysis Physical layout analysis Page layout analysis Optical character recognition (OCR)  Screen readers Classifier ensemble Page zone classification Creation of structured meta data 

Notes

Acknowledgements

The authors would like to thank the RDI team for sharing their datasets and especially Shaimaa Samir for testing our data on the RDI Clever Page software. The authors acknowledge partial funding from the National Science Foundation (1337866, 1421943) (to M.B.) and the Cairo Initiative Scholarship Program (to R.E.).

References

  1. 1.
    Cattoni, R., Coianiz, T., Messelodi, S., Modena, C.M.: Geometric layout analysis techniques for document image understanding: a review. Technical Report TR9703-09, ITC-IRST, Trento, January 1998, 68 pagesGoogle Scholar
  2. 2.
    Chen, K., Seuret, M., Wei, H., Liwicki, M., Hennebert, J., Ingold, R.: Ground truth model, tool, and dataset for layout analysis of historical documents. In: Proceedings of SPIE 9402, Document Recognition and Retrieval XXII, Feb. 2015, 10 pagesGoogle Scholar
  3. 3.
    Alshameri, A., Abdou, S., Mostafa, K.: A combined algorithm for layout analysis of Arabic document images and text lines extraction. Int. J. Comput. Appl. 49(23), 30–37 (2012)Google Scholar
  4. 4.
    Bukhari, S.S., Breuel, T.M., Asi, A., El-Sana, J.: Layout analysis for Arabic historical document images using machine learning. In: International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 639–644 (2012)Google Scholar
  5. 5.
    Hadjar, K., Ingold, R.: Physical layout analysis of complex structured Arabic documents using artificial neural nets. In: International Workshop on Document Analysis Systems, pp. 170–178 (2004)Google Scholar
  6. 6.
    Saad, R.S.M., Elanwar, R.I., Abdel Kader, N.S., Mashali, S., Betke, M.: BCE-Arabic-v1 dataset: a step towards interpreting Arabic document images for people with visual impairments. In: ACM 9th Annual International Conference on Pervasive Technologies Related to Assistive Environments (PETRA’16), pp. 25–32, Corfu, June (2016)Google Scholar
  7. 7.
    BCE-Arabic Dataset: Publicly Available. http://www.cs.bu.edu/faculty/betke/BCE June (2016)
  8. 8.
    Hesham, A.M., Rashwan, M.A., Al-Barhamtoshy, H.M., Abdou, S.M., Badr, A.A., Farag, I.: Arabic document layout analysis. Pattern Anal. Appl. 20, 1275–1287 (2017)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Proprietary Dataset of 109 Arabic Documents: Provided by RDI, The Engineering Compuany for Digital Systems Development. http://www.rdi-eg.com
  10. 10.
    Nagy, G., Seth, S., Viswanathan, M.: A prototype document image analysis system for technical journals. Computer 7(25), 10–22 (1992)CrossRefGoogle Scholar
  11. 11.
    Baird, H.: Background structure in document images. Int. J. Pattern Recognit. Artif. Intell. 8(5), 1013–1030 (1994)CrossRefGoogle Scholar
  12. 12.
    O’Gorman, L.: The document spectrum for page layout analysis. IEEE Trans. Pattern Recognit. Mach. Learn. (TPAMI) 15(11), 1162–1173 (1993)CrossRefGoogle Scholar
  13. 13.
    Kise, K., Sato, A., Iwata, M.: Segmentation of page images using the area Voronoi diagram. Comput. Vis. Image Underst. 70(3), 370–382 (1998)CrossRefGoogle Scholar
  14. 14.
    Breuel, T.M.: Two geometric algorithms for layout analysis. In: Workshop on Document Analysis Systems, pp. 188–199, Princeton (2002)Google Scholar
  15. 15.
    Wong, K., Casey, R., Wahl, F.: Document analysis system. IBM J. Res. Dev. 26(6), 647–656 (1982)CrossRefGoogle Scholar
  16. 16.
    Jain, A.K., Yu, B.: Document representation and its application to page decomposition. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 20(3), 294–308 (1988)CrossRefGoogle Scholar
  17. 17.
    Fletcher, L.A., Kasturi, R.: A robust algorithm for text string separation from mixed text/graphics images. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 10, 910–918 (1988)CrossRefGoogle Scholar
  18. 18.
    Shafait, F., Hasan, A., Keysers, D., Breuel, T.M.: Layout analysis of Urdu document images. In: 10th International Multitopic Conference, pp. 293–298, Islamabad (2006)Google Scholar
  19. 19.
    Won, C.S.: Image extraction in digital documents. J. Electron. Imaging 17(3), 033016 (2008)CrossRefGoogle Scholar
  20. 20.
    Bukhari, S.S., Shafait, F., Breuel, T.M.: The IUPR dataset of camera-captured document images. In: International Workshop on Camera-Based Document Analysis and Recognition, pp. 164–171 (2012)Google Scholar
  21. 21.
    Lin, M.W., Tapamo, J.R., Ndovie, B.: A texture-based method for document segmentation and classification. S. Afr. Comput. J. 36(1), 49–56 (2006)Google Scholar
  22. 22.
    Bukhari, S.S., Azawi, A., Ali, M.I., Shafait, F., Breuel, T.M.: Document image segmentation using discriminative learning over connected components. In: DAS’10 Proceedings of the 9th IAPR International Workshop on Document Analysis Systems, pp. 183–190, Boston, June (2010)Google Scholar
  23. 23.
    Ye, X., Cheriet, M., Suen, C.Y.: A generic system to extract and clean handwritten data from business forms. In: 7th International Workshop on Frontiers in Handwriting Recognition, pp. 63–72 (2000)Google Scholar
  24. 24.
    Fan, W., Sun, J., Naoi, S.: Separation of text and background regions for high performance document image compression. In: Proceedings SPIE 9402, Document Recognition and Retrieval XXII, pp. 94020K1–9420K12, February (2015)Google Scholar
  25. 25.
    Le, V.P., Nayef, N., Visani, M., Ogier, J.M., De Tran, C.: Text and non-text segmentation based on connected component features. In: IEEE 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1096–1100, August (2015)Google Scholar
  26. 26.
    Rahman, A.F.R., Fairhurst, M.C.: Multiple classifier decision combination strategies for character recognition: a review. Int. J. Doc. Anal. Recognit. (IJDAR) 5, 166–194 (2003)CrossRefGoogle Scholar
  27. 27.
    Guyon, I., Haralick, R.M., Hull, J.J., Phillips, I.T.: Data sets for OCR and document image understanding research. In: Wang, H.B. (ed.) Handbook of Character Recognition and Document Image Analysis, pp. 779–799. World Scientific, Singapore (1997)CrossRefGoogle Scholar
  28. 28.
    Taghva, K., Nartker, T., Borsack, J., Condit, A.: UNLV-ISRI document collection for research in OCR and information retrieval. In: IS&T/SPIE Conference on Document Recognition and Retrieval VII, San Jose, pp. 157–164, January (2000)Google Scholar
  29. 29.
    Shafait, F., Breuel, T.M.: Document image dewarping contest. In: 2nd International Workshop on Camera-Based Document Analysis and Recognition, pp. 181–188, Curitiba (2007)Google Scholar
  30. 30.
    Zelenika, D., Povh, J., Ženko, B.: Text detection in document images by machine learning algorithms. In: 9th International Conference on Computer Recognition Systems CORES 2015, pp. 169–179. Springer (2016)Google Scholar
  31. 31.
    Belaïd, A., Ouwayed, N.: Segmentation of ancient Arabic documents. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 103–122. Springer, London (2012)CrossRefGoogle Scholar
  32. 32.
    Boussellaa, W., Zahour, A., Taconet, B., Alimi, A., Benabdelhafid, A.: PRAAD: preprocessing and analysis tool for Arabic ancient documents. In: 9th International Conference on Document Analysis and Recognition (ICDAR 2007), vol. 2, pp. 1058–1062 (2007)Google Scholar
  33. 33.
    Bukhari, S.S., Shafait, F., Breuel, T.M.: Layout analysis of Arabic script documents. In: Margner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 35–53. Springer, London (2012)Google Scholar
  34. 34.
    Bloomberg, D.: Multiresolution morphological approach to document image analysis. In: 1st International Conference on Document Analysis and Recognition, pp. 963–971 (1991)Google Scholar
  35. 35.
    Breuel, T.M.: An algorithm for finding maximal whitespace rectangles at arbitrary orientations for document layout analysis. In: 7th International Conference on Document Analysis and Recognition, pp. 66–70 (2003)Google Scholar
  36. 36.
    Hadjar, K., Ingold, R.: Arabic newspaper page segmentation. In: 7th International Conference on Document Analysis and Recognition, pp. 895–899 (2003)Google Scholar
  37. 37.
    Capobianco, S., Marinai, S.: Text line extraction in handwritten historical documents. In: Italian Research Conference on Digital Libraries, pp. 68–79 (2017)Google Scholar
  38. 38.
    Pastor-Pellicer, J., Afzal, M.Z., Liwicki, M., Castro-Bleda, M.J.: Complete system for text line extraction using convolutional neural networks and watershed transform. In: 12th IAPR Workshop on Document Analysis Systems (DAS), Santorini, pp. 30–35 (2016)Google Scholar
  39. 39.
    Wick, C., Puppe, F.: Fully convolutional neural networks for page segmentation of historical document images. https://arxiv.org/pdf/1711.07695.pdf (2017), 6 pages
  40. 40.
    Hao, L., Gao, L., Yi, X., Tang, Z.: A table detection method for PDF documents based on convolutional neural networks. In: 12th IAPR Workshop on Document Analysis Systems (DAS), Santorini (2016)Google Scholar
  41. 41.
    Meier, B., Stadelmann, T., Stampfli, J., Arnold, M., Cieliebak, M.: Fully convolutional neural networks for newspaper article segmentation. In: 14th International Conference on Document Analysis and Recognition (ICDAR), pp. 1–6 (2017)Google Scholar
  42. 42.
    Oliveira, D.A.B., Viana, P.M.: Fast CNN-based document layout analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1173–1180, Waikiki (2017)Google Scholar
  43. 43.
    Capobianco, S., Marinai, S.: Deep neural networks for record counting in historical handwritten documents. In: Pattern Recognition Letters (2017).  https://doi.org/10.1016/j.patrec.2017.10.023
  44. 44.
    Rahman, M.M., Finin, T.: Understanding the logical and semantic structure of large documents. https://arxiv.org/pdf/1709.00770.pdf (2017), 10 pages
  45. 45.
    Afzal, M.Z., Capobiancot, S., Malik, M.I., Marinait, S., Breuel, T.M., Dengel, A., Liwicki, M.: DeepDocClassifier: document classification with deep convolutional neural network. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1111–1115 (2015)Google Scholar
  46. 46.
    Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for document image classification and retrieval. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 991–995, August (2015)Google Scholar
  47. 47.
    Kölsch, A., Afzal, M.Z., Ebbecke, M., Liwicki, M.: Real-time document image classification using deep CNN and extreme learning machines. https://arxiv.org/pdf/1711.05862.pdf (2017), 6 pages
  48. 48.
    Seuret, M., Fischer, A., Garz, A., Liwicki, M., Ingold, R.: Clustering historical documents based on the reconstruction error of autoencoders. In: ACM 3rd International Workshop on Historical Document Imaging and Processing, pp. 85–91 (2015)Google Scholar
  49. 49.
    Chen, K., Seuret, M., Liwicki, M., Hennebert, J., Ingold, R.: Page segmentation of historical document images with convolutional autoencoders. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1011–1015 (2015)Google Scholar
  50. 50.
    Chen, K., Liu, C.L., Seuret, M., Liwicki, M., Hennebert, J., Ingold, R.: Page segmentation for historical document images based on superpixel classification with unsupervised feature learning. In: 12th IAPR Workshop on Document Analysis Systems (DAS), pp. 299–304 (2016)Google Scholar
  51. 51.
    Wei, H., Seuret, M., Chen, K., Fischer, A., Liwicki, M., Ingold, R.: Selecting autoencoder features for layout analysis of historical documents. In: ACM 3rd International Workshop on Historical Document Imaging and Processing, pp. 55–62 (2015)Google Scholar
  52. 52.
    Zhu, W., Chen, Q., Wei, C., Li, Z.: A segmentation algorithm based on image projection for complex text layout. In: American Institute of Physics (AIP) Conference Proceedings, vol. 1890, p. 1 (2017)Google Scholar
  53. 53.
    Ahn, B., Ryu, J., Koo, H.I., Cho, N.I.: Textline detection in degraded historical document images. EURASIP J. Image Video Process. 82, 1–13 (2017)Google Scholar
  54. 54.
    Zhang, X., Duan, L., Ma, L., Wu, J.: Text extraction for historical Tibetan document images based on connected component analysis and corner point detection. In: Yang, J., et al. (eds.) Computer Vision. CCCV 2017. Communications in Computer and Information Science, vol. 772, pp. 545–555. Springer, Singapore (2017).  https://doi.org/10.1007/978-981-10-7302-1_45
  55. 55.
    Clausner, C., Pletschacher, S., Antonacopoulos, A.: Aletheia—an advanced document layout and text ground-truthing system for production environments. In: IEEE International Conference on Document Analysis and Recognition (ICDAR), pp. 48–52, September (2011)Google Scholar
  56. 56.
    Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth elements) format framework. In: 20th International Conference on Pattern Recognition (ICPR), pp. 257–260 (2010)Google Scholar
  57. 57.
    Kavitha, A.S., Shivakumara, P., Kumar, G.H., Lu, T.: Text segmentation in degraded historical document images. Egypt. Inf. J. 17, 189–197 (2016)CrossRefGoogle Scholar
  58. 58.
    Wang, Y., Phillips, I.T., Haralick, R.M.: A study on the document zone content classification problem. In: International Workshop on Document Analysis Systems, pp. 212–223 (2002)Google Scholar
  59. 59.
    Baechler, M., Bloechle, J.-L., Ingold, R.L.: Semi-automatic annotation tool for medieval manuscripts. In: 2010 IEEE International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 182–187 (2010)Google Scholar
  60. 60.
    Deivalakshmi, S., Palanisamy, P., Vishwanathan, G.: A novel method for text and non-text segmentation in document images. In: 2013 IEEE International Conference on Communications and Signal Processing (ICCSP), pp. 255–259 (2013)Google Scholar
  61. 61.
    Tahmasbi, A., Saki, F., Shokouhi, S.B.: Classification of benign and malignant masses based on Zernike moments. Comput. Biol. Med. 41(8), 726–735 (2011)CrossRefGoogle Scholar
  62. 62.
    RDI CleverPage: Document layout analysis software by RDI, The Engineering Compuany for Digital Systems Development. http://www.rdi-eg.com
  63. 63.
    Shafait, F., Keysers, D., Breuel, T.M.: Performance evaluation and benchmarking of six-page segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 941–954 (2008)CrossRefGoogle Scholar
  64. 64.
    Mao, S., Kanungo, T.: Empirical performance evaluation methodology and its application to page segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 23(3), 242–256 (2001)CrossRefGoogle Scholar
  65. 65.
    Oyedotun, O.K., Khashman, A.: Document segmentation using textural features summarization and feedforward neural network. Appl. Intell. 45, 1–15 (2016)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Electronics Research InstituteCairoEgypt
  2. 2.Boston UniversityBostonUSA

Personalised recommendations