Advertisement

Recognizing handwritten Arabic words using grapheme segmentation and recurrent neural networks

  • Gheith A. AbandahEmail author
  • Fuad T. Jamour
  • Esam A. Qaralleh
Original Paper

Abstract

The Arabic alphabet is used in around 27 languages, including Arabic, Persian, Kurdish, Urdu, and Jawi. Many researchers have developed systems for recognizing cursive handwritten Arabic words, using both holistic and segmentation-based approaches. This paper introduces a system that achieves high accuracy using efficient segmentation, feature extraction, and recurrent neural network (RNN). We describe a robust rule-based segmentation algorithm that uses special feature points identified in the word skeleton to segment the cursive words into graphemes. We show that careful selection from a wide range of features extracted during and after the segmentation stage produces a feature set that significantly reduces the label error. We demonstrate that using same RNN recognition engine, the segmentation approach with efficient feature extraction gives better results than a holistic approach that extracts features from raw pixels. We evaluated this segmentation approach against an improved version of the holistic system MDLSTM that won the ICDAR 2009 Arabic handwritten word recognition competition. On the IfN/ENIT database of handwritten Arabic words, the segmentation approach reduces the average label error by 18.5 %, the sequence error by 22.3 %, and the execution time by 31 %, relative to MDLSTM. This approach also has the best published accuracies on two IfN/ENIT test sets.

Keywords

Optical character recognition Handwritten Arabic words Segmentation Feature evaluation and selection Recurrent neural networks 

Notes

Acknowledgments

This work was supported by the Deanship of the Scientific Research in the University of Jordan. Some of this research was completed when G. Abandah was in a sabbatical leave in Princess Sumaya University for Technology. We would like to thank Alex Graves for making the RNNLIB publically available [19], for giving us a copy of the latest RNNLIB version, and for his help in using it. We also thank him for providing parts of the RNN sequence transcriber description included in Sect. 4. We would also like to thank Hanchuan Peng for making mRMR tools publically available [49]. We would like also to thank Haikal El Abed for giving us copies of sets f and s of the IfN/ENIT database.

References

  1. 1.
    Abandah, G., Jamour, F.: Recognizing handwritten Arabic script through efficient skeleton-based grapheme segmentation algorithm. In: International Conference Intelligent Systems Design and Applications, pp. 977–982 (2010)Google Scholar
  2. 2.
    Abandah, G., Jamour, F.: A word matching algorithm in handwritten Arabic recognition using multiple-sequence weighted edit distances (2014). Submitted.Google Scholar
  3. 3.
    Abandah, G., Khedher, M.: Analysis of handwritten Arabic letters using selected feature extraction techniques. Int. J. Comput. Process. Lang. 22(1), 49–73 (2009)CrossRefGoogle Scholar
  4. 4.
    Abandah, G., Malas, T.: Feature selection for recognizing handwritten Arabic letters. Dirasat Eng. Sci. J. 37(2), 242–256 (2010)Google Scholar
  5. 5.
    Al-Hajj, R., Likforman-Sulem, L., Mokbel, C.: Arabic handwriting recognition using baseline dependant features and hidden Markov modeling. In: International Conference Document Analysis and Recognition, pp. 893–897 (2005)Google Scholar
  6. 6.
    Alginahi, Y.M.: A survey on Arabic character segmentation. Int. J. Doc. Anal. Recogn. 16(2), 105–126 (2013)CrossRefGoogle Scholar
  7. 7.
    Alkhoury, I., Giménez, A., Juan, A.: Arabic handwriting recognition using Bernoulli HMMs. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 255–272. Springer, London (2012)CrossRefGoogle Scholar
  8. 8.
    Amin, A.: Arabic character recognition. In: Bunke, H., Wang, P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 397–420. World Scientific, Singapore (1997)Google Scholar
  9. 9.
    Arica, N., Yarman-Vural, F.: Optical character recognition for cursive handwriting. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 801–813 (2002)CrossRefGoogle Scholar
  10. 10.
    Azeem, S.A., Ahmed, H.: Effective technique for the recognition of offline Arabic handwritten words using hidden Markov models. Int. J. Doc. Anal. Recogn. 16(4), 399–412 (2013)Google Scholar
  11. 11.
    Chang, F., Chen, C.J., Lu, C.J.: A linear-time component-labeling algorithm using contour tracing technique. Comput. Vis. Image Underst. 93(2), 206–220 (2004)CrossRefGoogle Scholar
  12. 12.
    Deutsch, E.: Thinning algorithms on rectangular, hexagonal, and triangular arrays. Commun. ACM 15(9), 827–837 (1972)CrossRefGoogle Scholar
  13. 13.
    Douglas, D., Peucker, T.: Algorithms for the reduction of the number of points required to represent a line or its caricature. Canad. Cartogr. 10(2), 112–122 (1973)CrossRefGoogle Scholar
  14. 14.
    Dreuw, P., Rybach, D., Heigold, G., Ney, H.: RWTH OCR: A large vocabulary optical character recognition system for Arabic scripts. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 215–254. Springer, London (2012)Google Scholar
  15. 15.
    El Abed, H., Märgner, V.: Comparison of different preprocessing and feature extraction methods for offline recognition of handwritten Arabic words. In: International Conference Document Analysis and Recognition, pp. 974–978 (2007)Google Scholar
  16. 16.
    El-Hajj, R., Likforman-Sulem, L., Mokbel, C.: Arabic handwriting recognition using baseline dependant features and hidden Markov modeling. In: International Conference Document Analysis and Recognition, pp. 893–897 (2005)Google Scholar
  17. 17.
    Freeman, H.: On the encoding of arbitrary geometric configurations. IRE Trans. Electron. Comput. 10(2), 260–268 (1961)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Gers, F.: Long short-term memory in recurrent neural networks. Ph.D. thesis, Ecole Polytechnique Fédérale de Lausanne (2001)Google Scholar
  19. 19.
    Graves, A.: RNNLIB: a recurrent neural network library for sequence learning problems. http://sourceforge.net/projects/rnnl/
  20. 20.
    Graves, A.: Supervised sequence labelling with recurrent neural networks. Ph.D. thesis, Technische Universität München (2008)Google Scholar
  21. 21.
    Graves, A.: Offline Arabic handwriting recognition with multidimensional recurrent neural networks. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 297–313. Springer, London (2012)CrossRefGoogle Scholar
  22. 22.
    Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, vol. 382. Springer, Berlin (2012)CrossRefGoogle Scholar
  23. 23.
    Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In: International Conference Machine Learning (2006)Google Scholar
  24. 24.
    Graves, A., Fernández, S., Liwicki, M., Bunke, H., Schmidhuber, J.: Unconstrained online handwriting recognition with recurrent neural networks. Adv. Neural Inf. Process. Syst. 20, 1–8 (2008)Google Scholar
  25. 25.
    Graves, A., Fernández, S., Schmidhuber, J.: Multi-dimensional recurrent neural networks. In: International Conference Artificial Neural Networks (2007)Google Scholar
  26. 26.
    Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009)CrossRefGoogle Scholar
  27. 27.
    Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5), 602–610 (2005)CrossRefGoogle Scholar
  28. 28.
    Graves, A., Schmidhuber, J.: Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks. In: Advances in Neural Information Processing Systems, NIPS’22, vol. 22, chap., pp. 545–552. MIT Press, Vancouver (2009)Google Scholar
  29. 29.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3(1), 1157–1182 (2003)zbMATHGoogle Scholar
  30. 30.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  31. 31.
    Kuhl, F., Giardina, C.: Elliptic Fourier features of a closed contour. Comput. Graph. Image Process. 18(3), 236–258 (1982)CrossRefGoogle Scholar
  32. 32.
    Kundu, A., Hines, T., Phillips, J., Huyck, B.D., Van Guilder, L.C.: Arabic handwriting recognition using variable duration HMM. In: International Conference Document Analysis and Recognition, pp. 644–648 (2007)Google Scholar
  33. 33.
    Lee, H., Verma, B.: Binary segmentation algorithm for English cursive handwriting recognition. Pattern Recogn. 45(4), 1306–1317 (2012)CrossRefGoogle Scholar
  34. 34.
    Lewis, M.P. (ed.): Ethnologue: Languages of the World. SIL International, Dallas (2009)Google Scholar
  35. 35.
    Likforman-Sulem, L., Mohammad, R.A.H., Mokbel, C., Menasri, F., Bianne-Bernard, A.L., Kermorvant, C.: Features for HMM-based Arabic handwritten word recognition systems. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 123–143. Springer, London (2012)CrossRefGoogle Scholar
  36. 36.
    Liu, C.L.: Handwritten Chinese character recognition: effects of shape normalization and feature extraction. In: Doermann, D., Jaeger, S. (eds.) Arabic and Chinese Handwriting Recognition, vol. LNCS 4768, pp. 104–128. Springer, Berlin (2008)CrossRefGoogle Scholar
  37. 37.
    Lorigo, L., Govindaraju, V.: Segmentation and pre-recognition of Arabic handwriting. In: International Conference Document Analysis and Recognition, pp. 605–609 (2005)Google Scholar
  38. 38.
    Lorigo, L., Govindaraju, V.: Offline Arabic handwriting recognition: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 712–724 (2006)CrossRefGoogle Scholar
  39. 39.
    Maddouri, S.S., El-Abed, H., Samoud, F.B., Bouriel, K., Ellouze, N.: Baseline extraction: comparison of six methods on IFN/ENIT database. In: International Conference Frontiers in Handwriting Recognition (2008)Google Scholar
  40. 40.
    Märgner, V., El Abed, H.: ICDAR 2005—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 70–74 (2005)Google Scholar
  41. 41.
    Märgner, V., El Abed, H.: ICDAR 2007—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 1274–1278 (2007) Google Scholar
  42. 42.
    Märgner, V., El Abed, H.: ICDAR 2009—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 1383–1387 (2009)Google Scholar
  43. 43.
    Märgner, V., El Abed, H.: ICFHR 2010—Arabic handwriting recognition competition. In: International Conference Frontiers in Handwriting Recognition, pp. 709–714 (2010)Google Scholar
  44. 44.
    Märgner, V., El Abed, H.: ICDAR 2011—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 1444–1448 (2011)Google Scholar
  45. 45.
    Motawa, D., Amin, A., Sabourin, R.: Segmentation of Arabic cursive script. In: International Conference Document Analysis and Recognition, pp. 625–628 (1997)Google Scholar
  46. 46.
    Murray, A., Edwards, P.: Synaptic weight noise during multilayer perceptron training: fault tolerance and training improvements. IEEE Trans. Neural Netw. 4(4), 722–725 (1993)Google Scholar
  47. 47.
    Pechwitz, M., El Abed, H., Märgner, V.: Handwritten Arabic word recognition using the IFN/ENIT-database. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 169–213. Springer, London (2012)CrossRefGoogle Scholar
  48. 48.
    Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H.: IFN/ENIT - database of handwritten Arabic words. In: Colloque Int’l Francophone sur l’Ecrit et le Document, pp. 129–136 (2002)Google Scholar
  49. 49.
    Peng, H.: mRMR (minimum redundancy maximum relevance feature selection). http://penglab.janelia.org/proj/mRMR/
  50. 50.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  51. 51.
    Qaralleh, M., Abandah, G., Jamour, F.: Tuning recurrent neural networks for recognizing handwritten Arabic words. J. Softw. Eng. Appl. 6(10), 533–542 (2013)CrossRefGoogle Scholar
  52. 52.
    Ratcliff, J., Metzener, D.: Pattern matching: the Gestalt approach. Dr. Dobb’s J. 13(7), 46–72 (1988)Google Scholar
  53. 53.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, Chap., pp. 318–362. MIT, Cambridge (1986)Google Scholar
  54. 54.
    Safabakhsh, R., Adibi, P.: Nastaaligh handwritten word recognition using a continuous-density variable-duration HMM. Arab. J. Sci. Eng. 30(1B), 95–118 (2005)Google Scholar
  55. 55.
    Sari, T., Souici, L., Sellami, M.: Off-line handwritten Arabic character segmentation algorithm: ACSA. In: International Workshop on Frontiers in Handwriting Recognition, pp. 452–457 (2002)Google Scholar
  56. 56.
    Schambach, M.P., Rottland, J., Alary, T.: How to convert a Latin handwriting recognition system to Arabic. In: International Conference Frontiers in Handwriting Recognition (2008)Google Scholar
  57. 57.
    Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997)CrossRefGoogle Scholar
  58. 58.
    Smith, R.: An overview of the Tesseract OCR engine. In: International Conference Document Analysis and Recognition, pp. 629–633 (2007)Google Scholar
  59. 59.
    Wshah, S., Shi, Z., Govindaraju, V.: Segmentation of Arabic handwriting based on both contour and skeleton segmentation. In: International Conference Document Analysis and Recognition, pp. 793–797 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Gheith A. Abandah
    • 1
    Email author
  • Fuad T. Jamour
    • 2
  • Esam A. Qaralleh
    • 3
  1. 1.Computer Engineering DepartmentThe University of JordanAmmanJordan
  2. 2.King Abdullah University of Science and TechnologyThuwalSaudi Arabia
  3. 3.Princess Sumaya University for TechnologyAmmanJordan

Personalised recommendations