Skip to main content
SpringerLink
Log in

HCR-Net: a deep learning based script independent handwritten character recognition network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Handwritten character recognition (HCR) remains a challenging pattern recognition problem despite decades of research, and lacks research on script independent recognition techniques. This is mainly because of similar character structures, different handwriting styles, diverse scripts, handcrafted feature extraction techniques, unavailability of data and code, and the development of script-specific deep learning techniques. To address these limitations, we have proposed a script independent deep learning network for HCR research, called HCR-Net, that sets a new research direction for the field. HCR-Net is based on a novel transfer learning approach for HCR, which partly utilizes feature extraction layers of a pre-trained network. Due to transfer learning and image augmentation, HCR-Net provides faster and computationally efficient training, better performance and generalizations, and can work with small datasets. HCR-Net is extensively evaluated on 40 publicly available datasets of Bangla, Punjabi, Hindi, English, Swedish, Urdu, Farsi, Tibetan, Kannada, Malayalam, Telugu, Marathi, Nepali and Arabic languages, and established 26 new benchmark results while performed close to the best results in the rest cases. HCR-Net showed performance improvements up to 11% against the existing results and achieved a fast convergence rate showing up to 99% of final performance in the very first epoch. HCR-Net significantly outperformed the state-of-the-art transfer learning techniques and also reduced the number of trainable parameters by 34% as compared with the corresponding pre-trained network. To facilitate reproducibility and further advancements of HCR research, the complete code is publicly released at https://github.com/jmdvinodjmd/HCR-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

All the datasets used in the paper are publicly available.

Notes

  1. http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf

  2. https://keras.io

References

  1. Acharya S, Pant AK, Gyawali PK (2015) Deep learning based large scale handwritten Devanagari character recognition. In: 2015 9th International conference on software, knowledge, information management and applications (SKIMA). IEEE, pp 1–6

  2. Akhlaghi M, Ghods V (2020) Farsi handwritten phone number recognition using deep learning. SN Appl Sci 2(3):1–10

    Article  Google Scholar 

  3. Al-wajih E, Ghazali R (2023) Threshold center-symmetric local binary convolutional neural networks for Bilingual handwritten digit recognition. Knowl-Based Syst 259:110079

    Article  Google Scholar 

  4. Ali H, Ullah A, Iqbal T, Khattak S (2020) Pioneer dataset and automatic recognition of Urdu handwritten characters using a deep autoencoder and convolutional neural network. SN Appl Sci 2(2):1–12

    Article  Google Scholar 

  5. Alkhawaldeh RS (2021) Arabic (Indian) digit handwritten recognition using recurrent transfer deep architecture. Soft Comput 25(4):3131–3141

    Article  Google Scholar 

  6. Basu S, Das N, Sarkar R, Kundu M, Nasipuri M, Basu DK (2010) A novel framework for automatic sorting of postal documents with multi-script address blocks. Pattern Recognit 43(10):3507–3521

    Article  ADS  Google Scholar 

  7. Bhattacharya U, Chaudhuri BB(2005) Databases for research on recognition of handwritten characters of Indian scripts. In: Eighth international conference on document analysis and recognition (ICDAR’05), vol 2. pp 789–793

  8. Bhattacharya U, Chaudhuri BB (2009) Handwritten numeral databases of Indian scripts and multistage recognition of mixed numerals. IEEE Trans Pattern Anal Mach Intell 31(3):444–457

    Article  PubMed  Google Scholar 

  9. Bhattacharya U, Shridhar M, Parui SK (2006) On recognition of handwritten Bangla characters. In: Computer vision, graphics and image processing. Springer, pp 817–828

  10. Biswas M, Islam R, Shom GK, Shopon M, Mohammed N, Momen S, Abedin A (2017) Banglalekha-isolated: a multi-purpose comprehensive dataset of handwritten Bangla isolated characters. Data in Brief 12:103–107

    Article  PubMed  PubMed Central  Google Scholar 

  11. Bonyani M, Jahangard S, Daneshmand M (2021) Persian handwritten digit, character and word recognition using deep learning. International Journal on document analysis and recognition (IJDAR), pp 1–11,

  12. Chauhan VK, Molaei S, Tania MH, Thakur A, Zhu T, Clifton DA (2023) Adversarial de-confounding in individualised treatment effects estimation. In: International conference on artificial intelligence and statistics. PMLR, vol 206, pp 837–849

  13. Chauhan VK, Dahiya K, Sharma A (2019) Problem formulations and solvers in linear SVM: a review. Artif Intell Rev 52(2):803–855

    Article  Google Scholar 

  14. Chauhan VK, Thakur A, O’Donoghue O, Clifton DA (2022) Coper: continuous patient state perceiver. In 2022 IEEE-EMBS international conference on biomedical and health informatics (BHI). IEEE, pp 1–4

  15. Chauhan VK, Thakur A, O’Donoghue O, Rohanian O, Clifton DA (2022) Continuous Patient State Attention Models. Medrxiv. https://doi.org/10.1101/2022.12.23.22283908

    Article  PubMed  PubMed Central  Google Scholar 

  16. Chauhan VK, Zhou J, Lu P, Molaei S, Clifton DA (2023) A brief review of hypernetworks in deep learning. arXiv:2306.06955

  17. Chauhan VK, Zhou J, Molaei S, Ghosheh G, Clifton DA (2023) Dynamic inter-treatment information sharing for individualized treatment effects estimation. arXiv:2305.15984

  18. Chen G, Chen P, Shi Y, Hsieh C-Y, Liao B, Zhang S (2019) Rethinking the usage of batch normalization and dropout in the training of deep neural networks. arXiv:1905.05928

  19. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1251–1258

  20. Chowdhury RR, Hossain MS, ul Islam R, Andersson K, Hossain S (2019) Bangla handwritten character recognition using convolutional neural network with data augmentation. In: 2019 Joint 8th international conference on informatics, electronics & vision (ICIEV) and 2019 3rd international conference on imaging, vision & pattern recognition (icIVPR). IEEE, pp 318–323

  21. Dargan S, Kumar M, Mittal A, Kumar K (2023) Handwriting-based gender classification using machine learning techniques. Multimed Tools Appl 1–25

  22. Das N, Acharya K, Sarkar R, Basu S, Kundu M, Nasipuri M (2014) A benchmark image database of isolated bangla handwritten compound characters. Int J Doc Anal Recognit (IJDAR) 17(4):413–431

    Article  Google Scholar 

  23. Das N, Basu S, Sarkar R, Kundu M, Nasipuri M et al (2015) An improved feature descriptor for recognition of handwritten bangla alphabet. arXiv:1501.05497

  24. Das N, Reddy JM, Sarkar R, Basu S, Kundu M, Nasipuri M, Basu DK (2012) A statistical-topological feature combination for recognition of handwritten numerals. Appl Soft Comput 12(8):2486–2495

    Article  Google Scholar 

  25. Das N, Sarkar R, Basu S, Kundu M, Nasipuri M, Basu DK (2012) A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application. Appl Soft Comput 12(5):1592–1606

    Article  Google Scholar 

  26. Das N, Sarkar R, Basu S, Saha PK, Kundu M, Nasipuri M (2015) Handwritten bangla character recognition using a soft computing paradigm embedded in two pass approach. Pattern Recognit 48(6):2054–2071

    Article  ADS  Google Scholar 

  27. Deore SP, Pravin A (2020) Devanagari handwritten character recognition using fine-tuned deep convolutional neural network on trivial dataset. Sādhanā 45(1):1–13

    Article  Google Scholar 

  28. Duerr B, Hättich W, Tropf H, Winkler G (1980) A combination of statistical and syntactical pattern recognition applied to classification of unconstrained handwritten numerals. Pattern Recognit 12(3):189–199

    Article  ADS  Google Scholar 

  29. Gan J, Chen Y, Hu B, Leng J, Wang W, Gao X (2023) Characters as graphs: interpretable handwritten Chinese character recognition via pyramid graph transformer. Pattern Recognit 109317

  30. Ghosh S, Chatterjee A, Singh PK, Bhowmik S, Sarkar R (2020) Language-invariant novel feature descriptors for handwritten numeral recognition. Vis Comput 1–23

  31. Granlund GH (1972) Fourier preprocessing for hand print character recognition. IEEE Trans Comput 100(2):195–201

    Article  MathSciNet  Google Scholar 

  32. Guha R, Das N, Kundu M, Nasipuri M, Santosh KC (2020) DevNet: an efficient CNN architecture for handwritten Devanagari character recognition. Int J Pattern Recognit Artif Intell 34(12):2052009

    Article  Google Scholar 

  33. Gupta A, Sarkhel R, Das N, Kundu M (2019) Multiobjective optimization for recognition of isolated handwritten Indic scripts. Pattern Recognit Lett 128:318–325

    Article  ADS  Google Scholar 

  34. Hamida S, Cherradi B, El Gannour O, Raihani A, Ouajji H (2023) Cursive Arabic handwritten word recognition system using majority voting and k-NN for feature descriptor selection. Multimed Tools Appl 1–25

  35. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 770–778

  36. Hijam D, Saharia S (2021) On developing complete character set Meitei Mayek handwritten character database. Vis Comput 1–15

  37. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708,

  38. Huang Z, Shivakumara P, Kaljahi MA, Kumar A, Pal U, Lu T, Blumenstein M (2023) Writer age estimation through handwriting. Multimed Tools Appl 82(11):16033–16055

    Article  Google Scholar 

  39. Inunganbi S (2023) A systematic review on handwritten document analysis and recognition. Multimed Tools Appl 1–27

  40. Inunganbi S, Choudhary P, Manglem K (2021) Handwritten Meitei Mayek recognition using three-channel convolution neural network of gradients and gray. Comput Intell 37(1):70–86

    Article  MathSciNet  Google Scholar 

  41. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

  42. Jiang W (2020) Mnist-mix: a multi-language handwritten digit recognition dataset. IOP SciNotes 1(2):025002

    Article  ADS  Google Scholar 

  43. Jinda SRl, Singh H (2019) Benchmark datasets for offline handwritten gurmukhi script recognition. In: Document analysis and recognition: 4th workshop, DAR 2018, Held in Conjunction with ICVGIP 2018, Hyderabad, India, December 18, 2018, Revised Selected Papers, vol 1020. Springer, pp 143

  44. Kaur S, Verma K (2020) Handwritten Devanagari character generation using deep convolutional generative adversarial network. In: Soft computing: theories and applications. Springer, pp 1243–1253

  45. Kavitha BR, Srimathi C (2019) Benchmarking on offline handwritten Tamil character recognition using convolutional neural networks. J King Saud Univ Comput Inf

  46. Keserwani P, Ali T, Roy PP (2019) Handwritten Bangla character and numeral recognition using convolutional neural network for low-memory Gpu. Int J Mach Learn Cybern 10(12):3485–3497

    Article  Google Scholar 

  47. Khosravi H, Kabir E (2007) Introducing a very large dataset of handwritten farsi digits and a study on their varieties. Pattern Recognit Lett 28(10):1133–1141

    Article  ADS  Google Scholar 

  48. Kim I-J, Xie X (2015) Handwritten hangul recognition using deep convolutional neural networks. Int J Doc Anal Recognit (IJDAR) 18(1):1–13

    Article  Google Scholar 

  49. Kong H, Tang D, Meng X, Lu T (2019) Garn: a novel generative adversarial recognition network for end-to-end scene character recognition. In: 2019 International conference on document analysis and recognition (ICDAR). pp 689–694

  50. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2:1097–1105

    Google Scholar 

  51. Kusetogullari H, Yavariabdi A, Cheddad A, Grahn H, Hall J (2019) Ardis: a Swedish historical handwritten digit dataset. Neural Comput Appl 1–14

  52. Lam L, Suen CY (1988) Structural classification and relaxation matching of totally unconstrained handwritten zip-code numbers. Pattern Recognit 21(1):19–31

    Article  ADS  Google Scholar 

  53. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11):2278–2324

    Article  Google Scholar 

  54. Li Z, Teng N, Jin M, Huaxiang L (2018) Building efficient CNN architecture for offline handwritten Chinese character recognition. Int J Doc Anal Recognit (IJDAR) 21(4):233–240

    Article  Google Scholar 

  55. Chi L, Asfandeyar A, Qu R, Yi W, Lei W, Wu G, Qiang L, Qiang Z (2023) A handwriting recognition system with wifi. IEEE Trans Mobile Comput 1–18

  56. Lincy RB, Gayathri R (2021) Optimally configured convolutional neural network for tamil handwritten character recognition by improved lion optimization model. Multimed Tools Appl 80(4):5917–5943

    Article  Google Scholar 

  57. D, Prat F, Marzal A, Vilar JM, Castro MJ, Amengual J-C, Barrachina S, Castellanos A, Boquera SE, Gómez JA, et al (2008) The ujipenchars database: a pen-based database of isolated handwritten characters. In LREC

  58. Mahapatra D, Choudhury C, Karsh RK (2020) Generator based methods for off-line handwritten character recognition. In: 2020 Advanced communication technologies and signal processing (ACTS). IEEE, pp 1–6

  59. Maitra DS, Bhattacharya U, Parui SK (2015) CNN based common approach to handwritten character recognition of multiple scripts. In: 2015 13th International conference on document analysis and recognition (ICDAR). IEEE, pp 1021–1025

  60. Majid N, Smith EHB (2022) Character spotting and autonomous tagging: offline handwriting recognition for Bangla, Korean and other alphabetic scripts. Int J Doc Anal Recognit (IJDAR) 25(4):245–263

    Article  Google Scholar 

  61. Manjusha K, Kumar MA, Soman KP (2018) Integrating scattering feature maps with convolutional neural networks for Malayalam handwritten character recognition. Int J Doc Anal Recognit (IJDAR) 21(3):187–198

    Article  Google Scholar 

  62. Manjusha K, Kumar MA, Soman KP (2019) On developing handwritten character image database for malayalam language script. Eng Sci Technol Int J 22(2):637–645

    Google Scholar 

  63. Melnyk P, You Z, Li K (2020) A high-performance cnn method for offline handwritten chinese character recognition and visualization. Soft Comput 24(11):7977–7987

    Article  Google Scholar 

  64. Mukhoti J, Dutta S, Sarkar R (2020) Handwritten digit classification in Bangla and Hindi using deep learning. Appl Artif Intell 34(14):1074–1099

    Article  Google Scholar 

  65. Muthureka K, Reddy US, Janet B (2023) An improved customized CNN model for adaptive recognition of cerebral palsy people’s handwritten digits in assessment. Int J Multimed Inf Retriev 12(2):23

    Article  Google Scholar 

  66. Pal U, Chaudhuri BB (2000) Automatic recognition of unconstrained off-line Bangla handwritten numerals. In: International conference on multimodal interfaces. Springer, pp 371–378

  67. Pant AK, Panday SP, Joshi SR (2012) Off-line Nepali handwritten character recognition using multilayer perceptron and radial basis function neural networks. In: 2012 Third Asian Himalayas international conference on internet. IEEE, pp 1–5

  68. Parseh MJ, Meftahi M (2017) A new combined feature extraction method for persian handwritten digit recognition. Int J Image Graph 17(02):1750012

    Article  Google Scholar 

  69. Porwal U, Fornés A, Shafait F (2022) Advances in handwriting recognition

  70. Prabhu VU (2019) Kannada-MNIST: A new handwritten digits dataset for the Kannada language. arXiv:1908.01242

  71. Pramanik R, Bag S (2018) Shape decomposition-based handwritten compound character recognition for Bangla OCR. J Vis Commun Image Represent 50:123–134

    Article  Google Scholar 

  72. Pramanik R, Dansena P, Bag S (2018) A study on the effect of CNN-based transfer learning on handwritten Indic and mixed numeral recognition. In: Workshop on document analysis and recognition. Springer, pp 41–51

  73. Prat F, Marzal A, Martın S, Ramos-Garijo R (2007) A two-stage template-based recognition engine for on-line handwritten characters. In: Proc. of the Asia-Pacific workshop, pp 77–82

  74. Prijatelj DS, Grieggs S, Yumoto F, Robertson E, Scheirer W (2023) Novelty in handwriting recognition. In: A unifying framework for formal theories of novelty: discussions, guidelines, and examples for artificial intelligence. Springer, pp 49–70

  75. Prince SJD (2023) Understanding Deep Learning. MIT press

  76. Ram S, Gupta S, Agarwal B (2018) Devanagri character recognition model using deep convolution neural network. J Stat Manage Syst 21(4):593–599

    Google Scholar 

  77. Rao Z, Zeng C, Wu M, Wang Z, Zhao N, Liu M, Wan X (2018) Research on a handwritten character recognition algorithm based on an extended nonlinear kernel residual network. KSII Trans Int Inf Syst 12(1):413–435

    Google Scholar 

  78. Roy A, Das N, Sarkar R, Basu S, Kundu M, Nasipuri M (2014) An axiomatic fuzzy set theory based feature selection methodology for handwritten numeral recognition. In: ICT and critical infrastructure: proceedings of the 48th annual convention of computer society of India-Vol I. Springer, pp 133–140

  79. Roy S, Das N, Kundu M, Nasipuri M (2017) Handwritten isolated Bangla compound character recognition: a new benchmark using a novel deep learning approach. Pattern Recognit Lett 90:15–21

    Article  ADS  Google Scholar 

  80. Saha P, Jaiswal A (2020) Handwriting recognition using active contour. In: Artificial intelligence and evolutionary computations in engineering systems. Springer, pp 505–514

  81. Saini A, Daniel S, Saini S, Mittal A (2021) Kannadares-next: a deep residual network for Kannada numeral recognition. In: Machine learning for intelligent multimedia analytics. Springer, pp 63–89

  82. Santosh KC, Iwata E (2012) Stroke-based cursive character recognition. Adv Character Recognit 175

  83. Santosh KC, Nattee C, Lamiroy B (2010) Spatial similarity based stroke number and order free clustering. In: 2010 12th International conference on frontiers in handwriting recognition. IEEE, pp 652–657

  84. Sarkar A, Singh K, Mukerjee A (2012) Handwritten Hindi numerals recognition system. Webpage: https://www.cse.iitk.ac.in/users/cs365/2012/submissions/aksarkar/cs365,CS365projectreport,

  85. Sarkhel R, Das N, Das A, Kundu M, Nasipuri M (2017) A multi-scale deep quad tree based feature extraction method for the recognition of isolated handwritten characters of popular indic scripts. Pattern Recognit 71:78–93

    Article  ADS  Google Scholar 

  86. Sarkhel R, Das N, Saha AK, Nasipuri M (2016) A multi-objective approach towards cost effective isolated handwritten Bangla character and digit recognition. Pattern Recognit 58:172–189

    Article  ADS  Google Scholar 

  87. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  88. Singh H, Sharma RK, Singh VP (2023) Language model based suggestions of next possible Gurmukhi character or word in online handwriting recognition system. Multimed Tools Appl 1–19

  89. Singh PK, Chatterjee I, Sarkar R, Smith EB, Nasipuri M (2021) A new feature extraction approach for script invariant handwritten numeral recognition. Expert Syst 38(6):e12699

    Article  Google Scholar 

  90. Singh PK, Sarkar R, Nasipuri M (2018) A comprehensive survey on Bangla handwritten numeral recognition. Int J Appl Pattern Recognit 5(1):55–71

    Article  Google Scholar 

  91. Singh S, Sharma A (2019) Online handwritten Gurmukhi words recognition: an inclusive study. ACM Trans Asian Low-Resour Lang Inf Process 18(3):21:1-21:55

    Article  Google Scholar 

  92. Singh S, Sharma A, Chauhan VK (2021) Online handwritten Gurmukhi word recognition using fine-tuned deep convolutional neural network on offline features. Mach Learn Appl 100037

  93. Singh S, Sharma A, Chauhan VK (2023) Indic script family and its offline handwriting recognition for characters/digits and words: a comprehensive survey. Artif Intell Rev 1–53

  94. Singh S, Sharma A, Chhabra I (2016) Online handwritten Gurmukhi strokes dataset based on minimal set of words. ACM Trans Asian Low-Res Lang Inf Process 16(1):1–20

    Google Scholar 

  95. Singh S, Sharma A, Chhabra I (2017) A dominant points-based feature extraction approach to recognize online handwritten strokes. Int J Doc Anal Recognit 20(1):37–58

    Article  Google Scholar 

  96. Sufian A, Ghosh A, Naskar A, Sultana F, Sil J, Rahman MMH (2020) BDNet: Bengali handwritten numeral digit recognition based on densely connected convolutional neural networks. J King Saud Univ-Comput Inf Sci

  97. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. InL Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1–9

  98. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2818–2826

  99. Wan L, Zeiler M, Zhang S, Cun YL, Fergus R (2013) Regularization of neural networks using dropconnect. In: Dasgupta S, McAllester D (eds) Proceedings of the 30th international conference on machine learning

  100. Zhao A, Li J (2023) A significantly enhanced neural network for handwriting assessment in parkinson’s disease detection. Multimed Tools Appl 1:1–21

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering Science, University of Oxford, Oxford, UK

    Vinod Kumar Chauhan

  2. D. M. College (affiliated to Panjab University Chandigarh), Moga, Punjab, India

    Sukhdeep Singh

  3. Department of Computer Science and Applications, Panjab University, Chandigarh, India

    Anuj Sharma

Authors
  1. Vinod Kumar Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sukhdeep Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anuj Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anuj Sharma.

Ethics declarations

Competing interest

The authors report there are no competing interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chauhan, V.K., Singh, S. & Sharma, A. HCR-Net: a deep learning based script independent handwritten character recognition network. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18655-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18655-5

Keywords

Search

Navigation