Abstract
For several decades, no satisfactory solutions have been provided to the problem of offline handwriting recognition. In the field of online recognition, researchers have had more successful performance, but the ability to extract dynamic information from static images has not been well explored yet. In this paper, we introduce a novel multi-lingual word handwriting recovery framework based on a convolutional denoising autoencoder with an attention model for pen up/down, velocity and temporal order recovery. The proposed framework consists of extracting robust features from a handwriting image using a stacked denoising autoencoder and an encoder Bidirectional Gated Recurrent Unit (BGRU) model. Then, the obtained vectors are decoded to produce an online script with dynamic characteristics using a BGRU with temporal attention. Evaluation is done on a Latin and Arabic Online and offline handwriting character / word databases and the proposed framework achieves high competitive results.
Similar content being viewed by others
References
Akouaydi H, Njah S, Ouarda W, Samet A, Zaied M, Alimi AM (2019) Convolutional neural networks for online arabic characters recognition with beta-elliptic knowledge domain. In: International Conference on Document Analysis and Recognition Workshops (ICDARW), pp 41–46
Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv: 1409.0473
Bhunia AK, Bhowmick A, Bhunia AK, Konwer A, Banerjee P, Roy PP, Pal U (2018) Handwriting trajectory recovery usingend-to-end deep encoder-decoder network. In: 2018 24thInternational Conference on Pattern Recognition (ICPR), pp. 3639–3644
Boubaker H, Elbaati A, Tagougui N, ElAbed H, Kherallah M, Alimi AM (2012) Online Arabic databases and applications. In: Guide to OCR for Arabic Scripts. Springer, London, pp 541–557
Chen Z, Yang D, Liang J, Liu X, Wang Y, Peng Z, Huang S (2022) Complex handwriting trajectory recovery: Evaluation metrics and algorithm. In: Proceedings of the Asian Conference on Computer Vision (pp. 1060–1076)
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555
Crispo G, Diaz M, Marcelli A, Ferrer MA (2018) Tracking the ballistic trajectory in complex and long handwritten signatures. In: 2018 16th International Conference on Frontiers in Handwriting Recognition(ICFHR), pp 351–356
Dhahri H, Rabhi B, Chelbi S, Almutiry O, Mahmood A, Alimi AM (2021) Automatic detection of COVID-19 using a stacked denoising convolutional autoencoder. Cmc-Comput Mat Contin:3259–3274
Dhieb T, Boubaker H, Ouarda W, Ayed MB, Alimi AM (2019) Deep bidirectional long short-term memory for online arabic writer identification based on beta-elliptic model. In International Conference on Document Analysis and Recognition Workshops (ICDARW), pp 35–40
Diaz M, Ferrer M, Parziale A (2017) Recovering Western On-line Signatures From Image-Based Specimens. Proc. International Conference on Document Analysis and Recognition (ICDAR)
Diaz M, Crispo G, Parziale A, Marcelli A, Ferrer MA (2022) Impact of Writing Order Recovery in Automatic Signature Verification. In: Carmona-Duarte C, Diaz M, Ferrer MA, Morales A (eds) Intertwining Graphonomics with Human Movements. IGS 2022, Lecture Notes in Computer Science, vol 13424. Springer, Cham. https://doi.org/10.1007/978-3-031-19745-1_2
Dinh M, Yang HJ, Lee GS, Kim SH, Do LN (2016) Recovery ofdrawing order from multi-stroke English handwritten images based ongraph models and ambiguous zone analysis. Expert Syst Appli 64:352–364
ElBaati A, Alimi AM, Charfi M, Ennaji A (2005) Recovery of temporalinformation from off-line arabic handwritten. In aiccsa, pp. 127-vii
Elbaati A, Kherallah M, Ennaji A, Alimi AM (2009) Temporal order recovery of the scanned handwriting. In: International Conference on Document Analysis and Recognition ICDAR'09, pp. 1116–1120
Hamdi Y, Boubaker H, Dhieb T, Elbaati A, Alimi AM (2019) Hybrid DBLSTM-SVM based Beta-elliptic-CNN Models for Online Arabic Characters Recognition,” In International Conference on Document Analysis and Recognition (ICDAR), pp. 545–550
Hamdi Y, Boubaker H, Alimi AM (2021) Data Augmentation using Geometric, Frequency, and Beta Modeling approaches for Improving Multi-lingual Online Handwriting Recognition. IJDAR 24:283–298. https://doi.org/10.1007/s10032-021-00376-2
Hassaïne A, Al Maadeed S, Bouridane A, (2013) Icdar 2013 competition onhandwriting stroke recovery from offline data. In Document Analysisand Recognition (ICDAR), 12th International Conference pp. 1412–1416
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778
Hinton E, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. science 313(5786):504–507
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput:1735–1780
Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167
Jentzen A, Von Wurstemberger P (2020) Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning rates. J Complex 57:101438
Y. Kato and M. Yasuhara, “Recovery of drawing order from single stroke handwriting images,” IEEE Trans Pattern Anal Mach Intell, vol. 22, no. 9, pp. 938–949, Sep 2000.
Kha VA, Kha HH, Blumenstein M (2016) Extraction of DynamicTrajectory on Multi-Stroke Static Handwriting Images Using LoopAnalysis and Skeletal Graph Model. REV J Electron Commun 6(1–2)
Kherallah M, Elbaati A, Elabed H, Alimi AM (2008) The on/of (LMCA) dual Arabic handwriting database. In: 11th international conference on frontiers in handwriting recognition (ICFHR)
Muramatsu D, Matsumoto T (2007) Effectiveness of pen pressure, azimuth, and altitude features for online signature verification. In: International Conference on Biometrics, pp. 503–512
Nakai M, Sudo T, Shimodaira H, Sagayama S, (2002) Pen pressure features for writer-independent on-line handwriting recognition based on substroke HMM. In: Object recognition supported by user interaction for service robots, pp. 220–223
Noubigh Z, Kherallah M (2017) A survey on handwriting recognition based on the trajectory recovery technique. In: Arabic Script Analysis and Recognition (ASAR), pp. 69–73
Qiao Y, Nishiara M, Yasuhara M (2006) A framework toward restoration of writing order from single-stroked handwriting image. IEEE Trans Pattern Anal Mach Intell 28(11):1724–1737
Rabhi B, Elbaati A, Hamdi Y, Alimi AM (2019) Handwriting Recognition Based on Temporal Order Restored by the End-to-End System. Int Conf Doc Analy Recog ICDAR:1231–1236. https://doi.org/10.1109/ICDAR.2019.00199
Rabhi B, Elbaati A, Boubaker H, Alimi AM (2021) Multi-lingual character handwriting framework based on an integrated deep learning based sequence-to-sequence attention model. Memetic Comp 13:459–475. https://doi.org/10.1007/s12293-021-00345-6
Rabhi B, Elbaati A, Hamdani TM, Alimi AM (2021) ASAR 2021 Competition on Online Signal Restoration Using Arabic Handwriting Dhad Dataset. In: Barney Smith E.H., Pal U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science, vol 12916. Springer, Cham. https://doi.org/10.1007/978-3-030-86198-8_26
Rousseau L, Anquetil E, Camillerapp J (2005) Recovery of a drawing orderfrom off-line isolated letters dedicated to on-line recognition. In: Document Analysis and Recognition, 2005. Proceedings. EighthInternational Conference, pp. 1121–1125
Sesa-Nogueras E, Faundez-Zanuy M (2012) Biometric recognition using online uppercase handwritten text. Pattern Recogn, pp. 128–144
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Steinherz T, Doermann D, Rivlin E, Intrator N (2009) Offline loopinvestigation for handwriting analysis. IEEE Trans Patt Analy Mach Intel 31(2):193–209
Vargas JF, Ferrer MA, Travieso CM, Alonso JB (2008) Off-line signature verification based on high pressure polar distribution,” In Procedeeins of the 11th International Conference on Frontiers in Handwriting Recognition, pp. 373–378
Viard-Gaudin C, Lallican PM, Knerr S, Binter P (1999) The ireste on/off (ironoff) dual handwriting database. In: Proceedings of the Fifth International Conference on Document Analysis and Recognition ICDAR, pp. 455–458
Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning, pp. 1096–1103
Wang Y, Sonogashira M, Hashimoto A, Iiyama M (2020) Two-stage Fully Convolutional Networks for Stroke Recovery of Handwritten Chinese Character
Yanikoglu B, Kholmatov A (2009) Online signature verification using Fourier descriptors. EURASIP J Adv Sign Proc, pp. 1–13
Zhang R, Chen J, Yang M (2019) Drawing Order Recovery based on deep learning. In: International Conference on Advanced Computational Intelligence (ICACI) (pp. 129–133)
Acknowledgements
The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES4.
Data Availability
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Funding
This study was funded by the Ministry of Higher Education and Scientific Research of Tunisia (grant number LR11ES4).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethics approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rabhi, B., Elbaati, A., Boubaker, H. et al. Multi-lingual handwriting recovery framework based on convolutional denoising autoencoder with attention model. Multimed Tools Appl 83, 22295–22326 (2024). https://doi.org/10.1007/s11042-023-16499-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-16499-z