Skip to main content
SpringerLink
Log in

Data Augmentation using Geometric, Frequency, and Beta Modeling approaches for Improving Multi-lingual Online Handwriting Recognition

  • Special Issue Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

The lack of large training data in the context of deep learning applications is a serious issue investigated by many studies that deal with the current challenge. In this paper, we introduce new data augmentation methods that generate more shape and dynamic variations to improve the performance of recognition systems using small datasets. Four data augmentation strategies are employed in our work. The first strategy employs the geometric methods that include: italicity angle, change of magnitude ratio, and baseline inclination angle. The second strategy applies a frequency treatment that attenuates or amplifies the trajectory high harmonics to generate handwriting modified styles. The third strategy employs the beta-elliptic model to extract a combined static and dynamic representation of the handwritten trajectory which undergoes limited random change around its parameters in order to generate more modified samples. The hybrid strategy consists of combining these strategies to maximize variations of the online handwriting trajectory (OHT). We evaluated our approach of data augmentation in the context of multi-lingual online handwriting recognition (OHR) tasks using end-to-end CNN architecture. Four databases; ADAB, ALTEC-OnDB, and Online_KHATT for Arabic script, and UNIPEN for Latin characters, are used to validate the proposed strategy. The obtained results show the effectiveness and the advantage of the adopted strategies compared with those registered before database extension or reported in the state-of-the-art systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Abdelaziz, I., Abdou, S.: “AltecOnDB: A Large-Vocabulary Arabic Online Handwriting Recognition Database”, arXiv preprint arXiv:1412.7626, (2014)

  2. Alex K., Ilya S., Geoffrey, E.: Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, (2012)

  3. Alimi, A. M.: An evolutionary neuro-fuzzy approach to recognize online Arabic handwriting, in: Document Analysis and Recognition, 1997., Proceedings of the Fourth International Conference on, IEEE. pp. 382–386, (1997)

  4. Al-Helali, B.M.: and Mahmoud. A survey. Submitted for publication, S. A., Arabic Online Text Recognition (AOTR) (2016)

    Google Scholar 

  5. Al-Helali, B.M., Mahmoud, S.A.: A statistical framework for online arabic character recognition. Cybern. Syst. 47(6), 478–498 (2016)

    Article  Google Scholar 

  6. Anh, D.. Le.., Indurkhya, B., Nakagawa, M.: Pattern generation strategies for improving recognition of handwritten mathematical expression. Pattern Recognit. Lett. 128, 255–262 (2019)

  7. Antreas A., Amos S., Harrison E.: Data augmentation generative adversarial networks. arXiv preprint arXiv: 1711.04340, (2017)

  8. Azeem, Sherif, A., Hany A.: “Combining online and offline systems for Arabic handwriting recognition.”Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). IEEE, (2012)

  9. Bahlmann, C., Burkhardt, H.:“The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 3, pp. 299–310, Mar. (2004)

  10. Berio, D., Akten, M., Fol Leymarie, F., Grierson, M., Plamondon, R.: Sequence generation with a physiologically plausible model of handwriting and Recurrent Mixture Density Networks , ICLR, (2017)

  11. Bezine, H., Kefi, M., Alimi, A.M.: On the beta-elliptic model for the control of the human arm movement. Int. J. Pattern Recognit Artif Intell. 21, 5–19 (2007)

    Article  Google Scholar 

  12. Boubaker, H., Elbaati, A., Tagougui, N., ElAbed, H., Kherallah, M., Alimi, A.M.: Online Arabic databases and applications gide to OCR for arabic scripts. Springer, New York (2012)

    Google Scholar 

  13. Boubaker, H., Chaabouni, A., Tagougui, N., Kherallah, M., Alimi, A.M.: “Handwriting and Hand drawing Velocity Modeling by Superposing Beta Impulses and Continuous Training Component,” International Journal of Computer Science Issues (UCS),pp S7-63, September (2013)

  14. Boubaker, H., Chaabouni, A., Haikal El-Abed, Alimi, A.M.:“GLoBD: Geometric and Learned Logic Algorithm for Straight or Curved Handwriting Baseline Detection” The International Arab Journal of Information Technology, Vol. 15, No. 1, January (2018)

  15. Brandão J.A., Almeida C.M.: The Italic Style: Getting It Straight. In: Rebelo F., Soares M. (eds) Advances in Ergonomics in Design. AHFE 2018. Advances in Intelligent Systems and Computing, vol 777. Springer, (2019)

  16. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: Delving deep into convolutional nets. Proc, BMVC (2014)

    Google Scholar 

  17. Derrode, S., Ghorbel, F.: Robust and efficient fourier-mellin transform approximations for gray-level image reconstruction and complete invariant description. Comput. Vis. Image Underst. 83(1), 57–78 (2001)

    Article  Google Scholar 

  18. Elleuch, M., Maalej, R., Kherallah, M.: A new design based-SVM of the CNN classifier architecture with dropout for offline arabic handwritten recognition. Procedia - Procedia Comput. Sci. 80, 1712–1723 (2016)

    Article  Google Scholar 

  19. Ghosh, R., Chirumavila, V., Prabhat, K.: RNN based online handwritten word recognition in Devanagari and Bengali scripts using horizontal zoning. Pattern Recogn. 92, 203–218 (2019)

    Article  Google Scholar 

  20. Graves, A., Liwicki, M., Bunke, H., Schmidhuber, J., Fernández, S.: Unconstrained on-line handwriting recognition with recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 577–584, (2008)

  21. Graves, A., Schmidhuber, J.: Offline handwriting recognition with multidimensional recurrent neural networks. Advances in Neural Information Processing Systems 21,(2008)

  22. Graves, A., Mohamed, A., Hinton. G.: Speech recognition with deep recurrent neural networks. In Proc. ICASSP, (2013)

  23. Graves, A.: Generating Sequences With Recurrent Neural Networks, arXiv:1308.0850, 5 Jun (2014)

  24. Guyon, I., Schomaker, L.R.B., Plamondon, R., Liberman, M. Janet, S.: “UNIPEN Project of On-Line Data Exchange and Recognizer Benchmarks,”Proc. 12th Int’l Conf

  25. Hamdi., Y., Boubaker H., Dhieb., T., Elbaati., A, Alimi, M.: Adel. Hybrid DBLSTM-SVM based Beta-elliptic-CNN Models for Online Arabic Characters Recognition. In International Conference on Document Analysis and Recognition (ICDAR), pages 803-808, (2019)

  26. Hamdi Y., Chaabouni A., Boubaker H., Alimi A.M.: Hybrid Neural Network and Genetic Algorithm for off-Lexicon Online Arabic Handwriting Recognition. In: Proceedings of the 16th International Conference on Hybrid Intelligent Systems (HIS 2016). HIS 2016. Advances in Intelligent Systems and Computing, vol 552. (2017)

  27. Keysers, D., Deselaers, T., Rowley, H., Wang, L.L.: Fast multi-language LSTM-based online handwriting recognition. International Journal on Document Analysis and Recognition (IJDAR) 89–102,(2020)

  28. Kherallah, M.,Najiba, T., Alimi,A.M., El Abed, H., Margner, M.: “Online Arabic Handwriting Recognition Competition,” International Conference on Document Analysis and Recognition, (2011)

  29. Krishnan, P., Jawahar, C. V.: “Matching handwritten document images,” The 14th European Conference on Computer Vision (ECCV), (2016)

  30. Ltaief, M., Njah, S., Bezine, H., Alimi, M.A.: Genetic algorithms for perceptual codes extraction. J. Intell. Learn. Syst. Appl. JILSA 4, 256–265 (2012)

    Google Scholar 

  31. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., Zitnick, C. L.:“Microsoft COCO: Common objects in context,” in Proc. Eur. Conf. Comp. Vis., (2014)

  32. Mahmoud, S., Luqman, H., Al-Helali, B., BinMakhashen, G., Tanvir Parvez, M.: ”Online-KHATT: An Open-Vocabulary Database for Arabic OnlineText Processing”. The Open Cybernetics & Systemics Journal, vol. 12, pp. 42-59, March. (2018)

  33. Patrice, J. P., Simard, Y., Dave S.: “Best practices for convolutional neural networks applied to visual document analysis.” Institute of Electrical and Electronics Engineers, Inc., August (2003)

  34. Plamondon, R.: A delta lognormal model for handwriting generation, in: Proceedings of the Biennal Conference of the International Graphonomics Society, pp. 126–127, (1995)

  35. Plamondon, R., Srihari, S. N.: “On-line and off-line handwriting recognition: A comprehensive survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 63–84, Jan. (2000)

  36. Plamondon, R., Pirlo, G., Anquetil, E., Rémi, C., Teulings, H., Nakagawa, M.: Personal digital bodyguards for e-Security, e-Learning and e-Health: a prospective surve. Pattern Recogn. 81, 633–659 (2018)

    Article  Google Scholar 

  37. Poznanski, A., Wolf, L.:“Cnn-n-gram for handwriting word recognition,” Proc. CVPR, (2016)

  38. Sen, S., Shaoo, D., Paul, S., Sarkar, R., Roy, K.: Online Handwritten Bangla Character Recognition Using CNN: A Deep Learning. Intelligent Engineering Informatics 695,(2018)

  39. Shen, X., Messina, R.: “A method of synthesizing handwritten chinese images for data augmentation,” The 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 114–119, (2016)

  40. Smach, F., Lemaître, C., Gauthier, J.-P., Mitéran, J., Atri, M.: Generalized fourier descriptors with applications to objects recognition in svm context. J. Math. Imaging Vision 30(1), 43–71 (2008)

    Article  MathSciNet  Google Scholar 

  41. Sennrich, R., Haddow, H.,Birch, A.: Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725. Association for Computational Linguistics, (2016)

  42. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6, 60 (2019)

    Article  Google Scholar 

  43. T. Lin, M. Maire, S. Belongie, et al., CoCo: Microsoft COCO: common objects in context, in: Proceedings of the ECCV, (2014)

  44. Wigington, C., Stewart, S., Davis, B., Barrett, B., Price, B, Cohen, S.:“Data Augmentation for Recognition of Handwritten Words and Lines Using a CNN-LSTM Network,” 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, pp. 639-645, (2017)

  45. Wilson-Nunn, D., Lyons, T., Papavasiliou, A., Ni, H.: “A Path Signature Approach to Online Arabic Handwriting Recognition”, International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR): 135-139, (2018)

  46. Wu, Y.C., Yin, F.: Liu, CL: & #x201C;Improving handwritten Chinese text recognition using neural network language models and convolutional neural network shape models. Pattern Recogn. 65, 251–264 (2017)

    Article  Google Scholar 

  47. Xiwen, Qu, Wang, W., Lu, K., Zhou, J.: Data augmentation and directional feature maps extraction for in-air handwritten Chinese character recognition based on convolutional neural network. Pattern Recogn, Lett (2018)

    Google Scholar 

  48. Yuan, A., Bai, G., Yang, P., Guo, Y., Zhao, X.:“ Handwritten English word recognition based on Convolutional Neural Networks” Proceedings of the 13th International Conference on Frontiers in Handwriting Recognition2, Bari, Italy, pp. 207–(2012)

  49. Online and offline handwritten chinese character recognition: Zhang, X.-Y., Bengio, Y. C.-L. Liu. A comprehensive study and new benchmark, Pattern Recognition 61, 348–360 (2017)

  50. Zitouni1, R., Bezine, H., Arous, N.:“Online Handwritten Arabic Scripts Recognition Using Stroke-Based Class Labeling Scheme”, International Journal of Computational Intelligence Systems (IJCIS), (2020)

Download references

Acknowledgements

The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48.

Author information

Authors and Affiliations

  1. REGIMLab.: Research Groups in Intelligent Machines, University of Sfax, National Engineering School of Sfax, LR11ES48, 3038, Sfax, Tunisia

    Yahia Hamdi, Houcine Boubaker & Adel M. Alimi

  2. Department of Electrical and Electronic Engineering Science, Faculty of Engineering and the Built Environment, University of Johannesburg, Johannesburg, South Africa

    Adel M. Alimi

Authors
  1. Yahia Hamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Houcine Boubaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yahia Hamdi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamdi, Y., Boubaker, H. & Alimi, A.M. Data Augmentation using Geometric, Frequency, and Beta Modeling approaches for Improving Multi-lingual Online Handwriting Recognition. IJDAR 24, 283–298 (2021). https://doi.org/10.1007/s10032-021-00376-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-021-00376-2

Keywords

Search

Navigation