Skip to main content
Log in

Worddeepnet: handwritten gurumukhi word recognition using convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep learning models are considered a revolutionary learning paradigm in artificial intelligence and machine learning, piquing the interest of image recognition and computer vision experts. Because deep learning models have gained popularity and improved outcomes in the literature, this work provides a deep learning method based on a holistic approach to recognize offline handwritten Gurumukhi words. The holistic approach to word recognition treats a word as a separate entity rather than its component letters. Three characteristics are extracted from word pictures to train a Convolutional Neural Network (CNN), namely, zoning, diagonal, and centroid. Five performance measures are used to assess trained CNN performance, namely, Accuracy, True Positive Rate (TPR), False Positive Rate (FPR), Root Mean Square Error (RMSE), and Area Under Curve (AUC). The proposed model is trained and assessed using a 40,000 words benchmark dataset based on 70:30 partitioning technique, in which 70% of the data is used to train the model and 30% of the data is used to test the trained model. To assess the efficacy of the suggested technique, a fivefold cross validation process is performed. Using the partitioning method and cross-validation approach, the best accuracy rates of 95.11% and 94.96% are obtained after 30 epochs, respectively which surpassed the existing state-of-the-art offline handwritten Gurumukhi word recognition systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The datasets generated during the current study are available from the corresponding author on reasonable request.

References

  1. Abdurahman F, Sisay E, Fante KA (2021) AHWR-Net: Offline Handwritten Amharic Word Recognition using Convolutional Recurrent Neural Network. SN Appl. Sci. 3(8):1–11

    Google Scholar 

  2. Acharyya A, Rakshit S, Sarkar R, Basu S, Nasipuri M (2013) Handwritten Word Recognition Using MLP Based Classifier: A Holistic Approach. Int. J. Comput. Sci Issues 10(2):422–427

    Google Scholar 

  3. Akbarpour S (2011) Improved feature extraction and lexicon reduction methods classified by support vector machine for Farsi handwritten word recognition system. Dissertation, University Putra Malaysia

  4. AlbawiS, Mohammed TA and Al-Zawi S (2017) Understanding of a Convolutional Neural Network. Proceedings of IEEE Int. Conf. Eng. Technol (ICET), 1–6

  5. Alkhawaldeh RS (2021) Arabic (Indian) digit handwritten recognition using recurrent transfer deep architecture. Soft Comput 25(4):3131–3141

    Google Scholar 

  6. Almaza´n J, Gordo A, Forne´s A and Valveny E (2014) Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mac, 36(12):2552–2566

  7. Almodfer R, Xiong S, Mudhsh M and Duan P (2017) Multi-column deep neural network for offline Arabic handwriting recognition. Proc Int Conf Artificial Neural Netw 260–267

  8. Amrouch M and Rabi M (2017) Deep neural networks features for Arabic handwriting recognition. Proc Int Conf Adv. Inf. Syst. 138–149

  9. Arani SA, Kabir E, Ebrahimpour R (2019) Handwritten Farsi word recognition using NN-based fusion of HMM classifiers with different types of features. Int J Image Graph 19(01):1950001

    Google Scholar 

  10. Awni M, Khalil MI and Abbas HM (2019) Deep-learning ensemble for offline Arabic handwritten words recognition. Proceedings of IEEE 14th International Conf Comput Eng Syst (ICCES), 40–45

  11. Bhowmik S, Polley S, Roushan MG, Malakar S, Sarkar R, Nasipuri M (2015) A holistic word recognition technique for handwritten Bangla words. Int. J. Pattern Recognit (IJAPR) 2(2):142–159

    Google Scholar 

  12. Bhowmik S, Malakar S, Sarkar R, Basu S, Kundu M, Nasipuri M (2019) Off-Line Bangla Handwritten Word Recognition: A Holistic Approach. Neural Comput Appl 31:5783–5798

    Google Scholar 

  13. Bianne-Bernard AL, Menasri F, Mohamad RAH, Mokbel C, Kermorvant C, Likforman-Sulem L (2011) Dynamic and contextual information in HMM modeling for handwritten word recognition. IEEE Trans Pattern Anal Mach Intell 33(10):2066–2080

    Google Scholar 

  14. BlucheT and Messina R (2017) Gated Convolutional Recurrent Neural Networks for Multilingual Handwriting Recognition. Proceedings of IEEE 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) 1:646–651

  15. BlucheT, Ney H and Kermorvant C (2013) Tandem HMM with Convolutional Neural Network for handwritten word recognition. Proc IEEE Int Conf Acoust, Speech and Signal Processing, 2390–2394

  16. Bluche T, Ney H and Kermorvant C (2014) A Comparison of Sequence-Trained Deep Neural Networks and Recurrent Neural Networks Optical Modeling for Handwriting Recognition. Proc Int Conf Statistical Language Speech Process 199–210

  17. Blumenstein M, Verma B (1999) A New Segmentation Algorithm for Handwritten Word Recognition. Proc. Int. Jt. Conf. Neural Netw’99(IJCNN’99) 4:2893–2898

    Google Scholar 

  18. Bonyani M, Jahangard S, Daneshmand M (2021) Persian handwritten digit, character and word recognition using deep learning. Int J Doc Anal Recognit (IJDAR) 24(1):133–143

    Google Scholar 

  19. Boualam M, Elfakir Y, Khaissidi G, Mrabti M (2022) Arabic Handwriting Word Recognition Based on Convolutional Recurrent Neural Network. Proceedings of WITS 2020:877–885

    Google Scholar 

  20. BouazizS, Mezghani A and Kanoun S (2014) Arabic Handwritten Word Recognition with Large Vocabulary Based on Explicit Segmentation. Proc Int Conf Info Communication Technologies Innovation and Application (ICTIA), 1–4

  21. Bozinovic MR, Srihari NS (1989) Off-Line Cursive Script Word Recognition. IEEE Trans Pattern Anal Mach Intell 11(1):68–83

    Google Scholar 

  22. Chergui L, Kef M (2015) SIFT descriptors for Arabic handwriting recognition. Int J Comput Vis Robot 5(4):441–461

    Google Scholar 

  23. Cilia ND, De Stefano C, Fontanella F, Marrocco C, Molinara M, Scotto Di Freca A (2020) An end-to- end deep learning system for medieval writer identification. Pattern Recogn Lett 129:137–143

    Google Scholar 

  24. Dasgupta J, Bhattacharya K, Chanda B (2016) A Holistic Approach for Off-Line Handwritten Cursive Word Recognition using Directional Feature based on Arnold Transform. Pattern Recogn Lett 79:73–79

    Google Scholar 

  25. De Oliveira JJ, de A Freitas CO, de Carvalho JM and Sabourin R (2009) Handwritten Word Recognition Using Multi-view Analysis. In: Bayro-Corrochano E., Eklundh JO. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2009. Lecture Notes in Computer Science, vol 5856. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10268-4_44

  26. Dreuw P, Rybach D, Heigold G, Ney H (2012) RWTH OCR: A Large Vocabulary Optical Character Recognition System for Arabic Scripts. In: Märgner V, El Abed H (eds) Guide to OCR for Arabic Scripts. Springer, London, pp 215–254

    Google Scholar 

  27. Edelman S, Flash T, Ullman S (1990) Reading Cursive Handwriting by Alignment of Letter Prototypes. Int J Comput Vision 5(3):303–331

    Google Scholar 

  28. El-Sawy A, Hazem EB and Loey M (2016) CNN for handwritten Arabic digits recognition based on LeNet-5. Proc Inte Conf Adv. Intell. Syst Inform 566–575

  29. Elleuch M, Tagougui N and Kherallah M (2015) Towards unsupervised learning for Arabic handwritten recognition using deep architectures. Proc 22nd Int Conf Neural Info Proc Syst 363–372

  30. Elleuch M, Maalej R, Kherallah M (2016) A new design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition. Procedia Comput. Sci. 80:1712–1723

    Google Scholar 

  31. Espana-Boquera S, Castro-Bleda MJ, Gorbe-Moya J, Zamora-Martinez F (2011) Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN models. IEEE Trans Pattern Anal Mach Intell 33(4):767–779

    Google Scholar 

  32. Ghadikolaie MFY, Kabir E, Razzazi F (2016) Sub-word based offline handwritten Farsi word recognition using recurrent neural network. ETRI J 38(4):703–713

    Google Scholar 

  33. Ghosh M, Malakar S, Bhowmik S, Sarkar R and Nasipuri M (2019) Feature Selection for Handwritten Word Recognition Using Memetic Algorithm. In: Mandal J., Dutta P., Mukhopadhyay S. (eds) Advances in Intelligent Computing. Studies in Computational Intelligence, vol 687. Springer, Singapore. https://doi.org/10.1007/978-981-10-8974-9_6

  34. Gimenez A, Khoury I, Andres-Ferrer J, Juan A (2012) Handwriting word recognition using windowed Bernoulli HMMs. Pattern Recogn Lett 35(1):149–156

    Google Scholar 

  35. Golzari S, Khalili A, Sabzi R (2022) Combining convolutional neural networks with SVM classifier for recognizing Persian and Arabic handwritten words. Multimedia Tools and Applications 81:33785–33799

    Google Scholar 

  36. Haghighi F, Omranpour H (2021) Stacking ensemble model of deep learning and its application to Persian/ Arabic handwritten digits recognition. Knowledge Based Systems 220:106940

    Google Scholar 

  37. Hossain MT, Hasan MW and Das AK (2021) Bangla Handwritten Word Recognition System Using Convolutional Neural Network. Proc 15th Inte Conf Ubiquitous Info Manag Communication (IMCOM), 1–8

  38. Imani Z, Ahmadyfard A, Zohrevand A (2016) Holistic Farsi handwritten word recognition using gradient features. Journal of AI and Data Mining 4(1):19–25

    Google Scholar 

  39. JayadevanR, Kolhe SR, Patil PM and Pal U (2011) Database development and recognition of handwritten Devanagari legal amount words. Proc Inter Conf Doc. Anal. Recognit (ICDAR), 304–308.

  40. Jayech K, Mahjoub M, Amara NB (2016) Arabic Handwritten Word Recognition Based on Dynamic Bayesian Network. Inter Arab J Info Technol 13(6B):1024–1031

    Google Scholar 

  41. Jino PJ, Balakrishnan K (2017) Offline Handwritten Recognition of Malayalam District Name - A Holistic Approach. Int J Eng Technol 9(2):1–8

    Google Scholar 

  42. Jino PJ, Balakrishnan K and Bhattacharya U (2019) Offline Handwritten Malayalam Word Recognition Using a Deep Architecture. In: Bansal J., Das K., Nagar A., Deep K., Ojha A. (eds) Soft Computing for Problem Solving. Advances in Intelligent Systems and Computing, vol 816. Springer, Singapore. https://doi.org/10.1007/978-981-13-1592-3_73

  43. Kaur H and Kumar M (2019) Benchmark Dataset: Offline Handwritten Gurmukhi City Names for Postal Automation. In: Sundaram S., Harit G. (eds) Document Analysis and Recognition. DAR 2018. Communications in Computer and Information Science, vol 1020. Springer, Singapore. https://doi.org/10.1007/978-981-13-9361-7_14

  44. Kaur H, Kumar M (2021) Offline Handwritten Gurumukhi Word Recognition using eXtreme Gradient Boosting Methodology. Soft Comput 25:4451–4464

    Google Scholar 

  45. Kaur H, Kumar M (2021) On the Recognition of Offline Handwritten Word using Holistic Approach and AdaBoost Methodology. Multimedia Tools and Applications 80:11155–11175

    Google Scholar 

  46. Kaur H and Kumar M (2021c) Performance Evaluation ofVarious Feature Selection Techniques for Offline Handwritten Gurumukhi Place Name Recognition. In: Singh T.P., Tomar R., Choudhury T., Perumal T., Mahdi H.F. (eds) Data Driven Approach Towards Disruptive Technologies. Studies in Autonomic, Data- driven and Industrial Computing. Springer, Singapore. https://doi.org/10.1007/978-981-15-9873- 9_44

  47. Khosravi S, Chalechale A (2022) Chimp Optimization Algorithm to Optimize a Convolutional Neural Network for Recognizing Persian/Arabic Handwritten Words. Math Probl Eng. https://doi.org/10.1155/2022/4894922

    Article  Google Scholar 

  48. Kumar R and Sharma R (2013) An efficient post processing algorithm for online handwriting Gurmukhi character recognition using set theory. International Journal of Int. J. Pattern Recognit. Artif. 27(4): 1353002–1–1353002–17

  49. Kumar N, Gupta S (2018) Offline Handwritten Gurmukhi Word Recognition Using Deep Neural Networks. Int J Pure Appl Math 119(12):14749–14767

    Google Scholar 

  50. Kumar M, Jindal MKand Sharma RK (2011) k-nearest neighbor based offline handwritten Gurmukhi character recognition. Proc Int Conf Image Info Proc 1–4.

  51. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Google Scholar 

  52. Lee H, Verma B (2011) Binary Segmentation Algorithm for English Cursive Handwriting Recognition. Pattern Recogn 45(4):1306–1317

    Google Scholar 

  53. Leila C, Maamar K and Salim C (2011) Combining neural networks for Arabic handwriting recognition. Proc IEEE 10th Int Sympos Program Syst 74–79

  54. Loey M, El-Sawy A and EL-Bakry H (2017) Deep learning autoencoder approach for handwritten Arabic digits recognition. arXiv preprint arXiv:1706.067

  55. Maalej R and Kherallah M (2019) Maxout into MDLSTM for offline Arabic handwriting recognition. Proc Int Conf Neural Info Process 534-545

  56. Madhvanath S, Govindaraju V (2001) The Role of Holistic Paradigms in Handwritten Word Recognition. IEEE Trans Pattern Anal Mach Intell 23(2):149–164

    Google Scholar 

  57. Mhiri M, Desrosiers C, Cheriet M (2018) Convolutional pyramid of bidirectional character sequences for the recognition of handwritten words. Pattern Recogn Lett 111:87–93

    Google Scholar 

  58. Mondal R, Malakar S, Barney Smith EH, Sarkar R (2022) Handwritten English word recognition using a deep learning based object detection architecture. Multimedia Tools and Applications 81(1):975–1000

    Google Scholar 

  59. Mustafa ME, Elbashir MK (2020) A Deep Learning Approach for Handwritten Arabic Names Recognition. Int J Adv Comput Sci App (IJACSA) 11(1):678–682

    Google Scholar 

  60. Namane A, Guessoum A and Meyrueis P (2005) New Holistic Handwritten Word Recognition and Its Application to French Legal Amount. In: Singh S., Singh M., Apte C., Perner P. (eds) Pattern Recognition and Data Mining. ICAPR 2005. Lecture Notes in Computer Science, vol 3686, Springer, Berlin, Heidelberg. https://doi.org/10.1007/11551188_72

  61. Nanehkaran YA, Zhang D, Salimi S, Chen J, Tian Y, Al-Nabhan N (2021) Analysis and comparison of machine learning classifiers and deep neural networks techniques for recognition of Farsi handwritten digits. J Supercomput 77(4):3193–3222

    Google Scholar 

  62. Nurseitov D, Bostanbekov K, Kanatov M, Alimova A, Abdallah A, Abdimanap G (2020) Classification of Handwritten Names of Cities and Handwritten Text Recognition using Various Deep Learning Models. Adv Sci Technol Eng Syst J 5(5):934–943

    Google Scholar 

  63. Pal U, Roy K, Kimura F (2009) A Lexicon-Driven Handwritten City Name Recognition Scheme for Indian Postal Automation. IEICE Trans Inf Syst 92(5):1146–1158

    Google Scholar 

  64. Parseh M, Rahmanimanesh M, Keshavarzi P (2020) Persian handwritten digit recognition using combination of convolutional neural network and support vector machine methods. Int Arab J Info Technol 17(4):572–578

    Google Scholar 

  65. Patel MS, Reddy SL and Naik AJ (2015) An Efficient Way of Handwritten English Word Recognition. In: Satapathy S, Biswal B, Udgata S, Mandal J (eds) Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA) 2014. Advances in Intelligent Systems and Computing, vol 328. Springer, Cham. https://doi.org/10.1007/978-3-319-12012-6_62

  66. Pham V, Bluche T, Kermorvant C and Louradour J (2014)Dropout Improves Recurrent Neural Networks for Handwriting Recognition. Proc 14th Int Conf Frontiers in Handwriting Recognit 285–290

  67. Poznanski A and Wolf L (2016) CNN-N-gram for handwriting word recognition. Proc IEEE Conf Comput Vision Pattern Recognit (CVPR), 2305–2314.

  68. Pramanik R, Bag S (2020) Segmentation-Based Recognition System for Handwritten Bangla and Devanagari Words Using Conventional Classification and Transfer Learning. IET Image Proc 14(5):959–972

    Google Scholar 

  69. Pramanik R, Bag S (2021) Handwritten Bangla city name word recognition using CNN-based transfer learning and FCN. Neural Comput Appl 33:9329–9341

    Google Scholar 

  70. PuigcerverJ (2017) Are multidimensional recurrent layers really necessary for handwritten text recognition? Proc IEEE 14th IAPR Inter Conf Document Anal Recognit (ICDAR), 1:67–72

  71. Rahal N, Tounsi M, Hamdani TM and Alimi AM (2019)Handwritten words and digits recognition using Deep Learning based Bag of Features Framework. Proc Int Conf Doc Anal Recognit (ICDAR), 701–706.

  72. Rothacker L and Fink GA (2016) Robust Output Modeling in Bag-of-Features HMMs for Handwriting Recognition. Proc 15th Int Conf Frontiers in Handwriting Recognit (ICFHR), 199–204

  73. Roy A, Bhowmik KT, Parui KS and Roy U (2005a) A Novel Approach to Skew Detection and Character Segmentation for Handwritten Bangla Words. Proc Digital Image Computing: Techniques and Applications (DICTA), 1–8.

  74. RoyK, Vajda S, Pal U, Chaudhuri BB and Belaid A (2005b) A System for Indian Postal Automation. Proc 8th Int Conf Doc Anal Recognit (ICDAR'05), 1060–1064

  75. Saeed U, Muhammad T, Alghamdi AS, Alkatheiri MS (2020) Automatic recognition of handwritten Arabic using maximally stable extremal region features. Opt Eng 59(5):051405

    Google Scholar 

  76. Safarzadeh VM and Jafarzadeh P (2020) Offline Persian handwriting recognition with CNN and RNN-CTC. Proc IEEE 25th Int Comput Conf Comput Soc Iran (CSICC), 1–10

  77. ScheidlH (2018) Handwritten text recognition in historical documents, Master’s thesis, Technische Universitat¨ Wien, Vienna, 2018, diplom-Ingenieur in Visual Computing

  78. SharmaA, Kumar R and Sharma RK (2009) Rearrangement of recognized strokes in online handwritten Gurmukhi words recognition. Proc 10th Int Conf Doc Anal Recognit (ICDAR), 1241–1245

  79. Sharma S, Gupta S, Gupta D, Juneja S, Singal G, Dhiman G, Kautish S (2022) Recognition of Gurmukhi handwritten city names using deep learning and cloud computing. Sci Program. https://doi.org/10.1155/2022/5945117

    Article  Google Scholar 

  80. Shrestha A, Mahmood A (2019) Review of deep learning algorithms and architectures. IEEE Access 7:53040–53065

    Google Scholar 

  81. Singh S and Sharma A (2019) Online handwritten Gurmukhi words recognition: An inclusive study. ACM Transactions on Asian and Low-Resource Language Information Processing, 18(3):21:1–21:55

  82. Singh S, Sharma A, Chhabra I (2016) Online handwritten Gurmukhi strokes dataset based on minimal set of words. ACM Transactions on Asian and Low-Resource Language Information Processing 16:1–20

    Google Scholar 

  83. Singh S, Chauhan VK, Smith EHB (2020) A self- controlled RDP approach for feature extraction in online handwriting recognition using deep learning. Appl Intell 50(7):2093–2104

    Google Scholar 

  84. Singh S, Sharma A, Chauhan VK (2021) Online handwritten Gurmukhi word recognition using fine-tuned Deep Convolutional Neural Network on offline features. Machine Learning with Applications 5:100037. https://doi.org/10.1016/j.mlwa.2021.100037

    Article  Google Scholar 

  85. Sudholt S and Fink GA (2016) PHOCNet: a deep convolutional neural network for word spotting in handwritten documents. Proc 15th Int Conf Frontiers in Handwriting Recognit 277–282

  86. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J Mach Learn Res 15(56):1929–1958

    MathSciNet  MATH  Google Scholar 

  87. Talaat M, Wahbi and Musa MEM (2016) Holistic approach for Arabic word recognition. Int J Comput Appl. Res. Technol, 5(3):141–146

  88. WafaMohamed Musa MEM A (2009) Recognition of Arabic handwritten names using Probabilistic Neural Networks. Computer Studies Journal 1(1):1–12

    Google Scholar 

  89. Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9(4):611–629

    Google Scholar 

  90. Yan R, Peng L, Xiao S, Johnson MT, Wang S (2019) Dynamic temporal residual network for sequence modeling. International Journal on Document Analysis and Recognition (IJDAR) 22(3):235–246

    Google Scholar 

  91. Zamani Y, Souri Y, Rashidi H and Kasaei S (2015) Persian handwritten digit recognition by random Forest and convolutional neural networks. Proc 9th Iranian IEEE Conf Machine Vision and Image Process 37–40

  92. Zhang TY, Suen CY (1984) A Fast Parallel Algorithm for Thinning Digital Patterns. Commun ACM 27(3):236–239

    Google Scholar 

  93. Zohrevand A, Imani Z (2021) Holistic Persian Handwritten Word Recognition Using Convolutional Neural Network. Int J Eng 34(8):2028–2037

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Munish Kumar.

Ethics declarations

Ethical approval

The authors have created their own dataset for performing the experiments in the considered work.

Informed consent

All the authors are agreed for this submission.

Conflict of interest

The authors declare that they have no conflict of interest in this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaur, H., Bansal, S., Kumar, M. et al. Worddeepnet: handwritten gurumukhi word recognition using convolutional neural network. Multimed Tools Appl 82, 46763–46788 (2023). https://doi.org/10.1007/s11042-023-15527-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15527-2

Keywords

Navigation