Skip to main content
SpringerLink
Log in

An End-to-End Recognition System for Unconstrained Vietnamese Handwriting

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Inspired by recent successes in neural machine translation and image caption generation, we present an attention-based encoder–decoder model (AED) to recognize handwritten Vietnamese text. The model consists of two parts: a Convolutional Neural Network-based (CNN) encoder and a Long Short-Term Memory-based (LSTM) decoder. The encoder is based on DenseNet for extracting invariant features. The LSTM-based decoder with an attention model incorporated generates output text. The input of the encoder is a handwritten text image and the target of the decoder is the corresponding text of the input image. Our model is trained end-to-end to predict the text from a given input image since all the parts are differential components. In the experiment section, we evaluate our proposed AED model on the VNOnDB-Word and VNOnDB-Line datasets to verify its efficiency. The experiential results show that our model achieves 4.10% of character error eate (CER) and 10.24% of word error rate (WER) on the testing set of VNOnDB-Word and 4.67% of CER and 13.33% of WER on the testing set of VNOnDB-Line without using any language model. This result is higher than our previous system, Nguyen et al.’s system, and the system provided by Google in the Vietnamese Online Handwritten Text Recognition competition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Plamondon R, Srihari SN. Online and off-line handwriting recognition: a comprehensive survey. IEEE Trans Pattern Anal Mach Intell. 2000;22:63–84. https://doi.org/10.1109/34.824821.

    Article  Google Scholar 

  2. Nakagawa M, Matsumoto K. Collection of on-line handwritten Japanese character pattern databases and their analyses. Int J Doc Anal Recognit. 2004;7:69–81. https://doi.org/10.1007/s10032-004-0125-4.

    Article  Google Scholar 

  3. Liu CL, Jaeger S, Nakagawa M. Online recognition of Chinese characters: the state-of-the-art. IEEE Trans Pattern Anal Mach Intell. 2004;26:198–213. https://doi.org/10.1109/TPAMI.2004.1262182.

    Article  Google Scholar 

  4. Frinken V, Fischer A, Bunke H. A novel word spotting algorithm using bidirectional long short-term memory neural networks. In: Proceedings of the 4th workshop on artificial neural networks in pattern recognition. 2010. pp. 185–196. https://doi.org/10.1007/978-3-642-12159-3_17.

  5. Al Hamad HA, Abu Zitar R. Development of an efficient neural-based segmentation technique for Arabic handwriting recognition. Pattern Recognit. 2010;43:2773–98. https://doi.org/10.1016/j.patcog.2010.03.005.

    Article  MATH  Google Scholar 

  6. Graves A, Fernández S, Liwicki M, Bunke H, Schmidhuber U, Schmidhuber J, Fernández S. Unconstrained online handwriting recognition with recurrent neural networks. Adv Neural Inf Process Syst. 2008;20:577–84. https://doi.org/10.1057/9780230226203.3287.

    Article  Google Scholar 

  7. Zhu B, Nakagawa M. Building a compact online MRF recognizer for large character set by structured dictionary representation and vector quantization technique. Pattern Recognit. 2014;47:982–93. https://doi.org/10.1016/j.patcog.2013.09.031.

    Article  Google Scholar 

  8. Sun L, Su T, Liu C, Wang R. Deep LSTM networks for online chinese handwriting recognition. In: Proceedings of the 15th international conference on frontiers in handwriting recognition. 2016. pp. 271–276. https://doi.org/10.1109/ICFHR.2016.0059.

  9. Zhang X-Y, Bengio Y, Liu C-L. Online and offline handwritten Chinese character recognition: a comprehensive study and new benchmark. Pattern Recognit. 2016;61:348–60. https://doi.org/10.1016/j.patcog.2016.08.005.

    Article  Google Scholar 

  10. Biadsy F, Saabni R, El-Sana J. Segmentation-free online arabic handwriting recognition. Int J Pattern Recognit Artif Intell. 2011;25:1009–33. https://doi.org/10.1142/S0218001411008956.

    Article  Google Scholar 

  11. Álvaro F, Sánchez JA, Benedí JM. Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Pattern Recognit Lett. 2014;35:58–67. https://doi.org/10.1016/j.patrec.2012.09.023.

    Article  Google Scholar 

  12. Le AD, Nakagawa M. A system for recognizing online handwritten mathematical expressions by using improved structural analysis. Int J Doc Anal Recognit. 2016;19:305–19. https://doi.org/10.1007/s10032-016-0272-4.

    Article  Google Scholar 

  13. Nguyen DK, Bui TD. On the problem of classifying vietnamese online handwritten characters. In: Proceedings of the 10th international conference on control, automation, robotics and vision. 2008. pp. 803–808. https://doi.org/10.1109/ICARCV.2008.4795620.

  14. Nguyen DK, Bui TD. Recognizing Vietnamese online handwritten separated characters. In: Proceedings of the 7th international conference on advanced language processing and web information technology. 2008. pp. 279–284. https://doi.org/10.1109/ALPIT.2008.58.

  15. Phuong PA, Tao NQ, Mai LC: An efficient model for isolated Vietnamese handwritten recognition. In: Proceedings of the international conference on intelligent information hiding and multimedia signal processing. 2008. pp. 358–361. https://doi.org/10.1109/IIH-MSP.2008.67.

  16. Tran DC. An efficient method for on-line Vietnamese handwritten character recognition. In: Proceedings of the 3rd symposium on information and communication technology. 2012. pp. 135–141. https://doi.org/10.1145/2350716.2350737.

  17. Nguyen HT, Nguyen CT, Bao PT, Nakagawa M. A database of unconstrained Vietnamese online handwriting and recognition experiments by recurrent neural networks. Pattern Recognit. 2018;78:291–306. https://doi.org/10.1016/j.patcog.2018.01.013.

    Article  Google Scholar 

  18. Guyon I, Schomaker L. UNIPEN project of on-line data exchange and recognizer benchmarks. In: Proceedings of the 12th IAPR international conference on pattern recognition. 1994. pp. 29–33. https://doi.org/10.1109/ICPR.1994.576870.

  19. Viard-Gaudin C, Lallican PM, Knerr S, Binter P. The IRESTE on/off (IRONOFF) dual handwriting database. In: Proceedings of the 5th international conference on document analysis and recognition. 1999. pp. 455–458. https://doi.org/10.1109/ICDAR.1999.791823.

  20. Nguyen HT, Nguyen CT, Nakagawa M. ICFHR 2018—competition on Vietnamese online handwritten text recognition using HANDS-VNOnDB (VOHTR2018). In: Proceedings of the 18th international conference on frontiers in handwriting recognition. 2018. pp. 494–499. https://doi.org/10.1109/ICFHR-2018.2018.00092.

  21. Le AD, Nguyen HT, Nakagawa M. Recognizing unconstrained Vietnamese handwriting by attention based encoder decoder model. In: Proceedings of the 12th international conference on advanced computing and applications. 2018. pp. 83–87. https://doi.org/10.1109/ACOMP.2018.00021.

  22. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the 30th IEEE conference on computer vision and pattern recognition. 2017. pp. 2261–2269. https://doi.org/10.1109/CVPR.2017.243.

  23. Le AD, Nakagawa M. Training an end-to-end system for handwritten mathematical expression recognition by generated patterns. In: Proceedings of the 14th IAPR international conference on document analysis and recognition. 2017. pp. 1056–1061. https://doi.org/10.1109/ICDAR.2017.175.

  24. Deng Y, Kanervisto A, Ling J, Rush AM. Image-to-markup generation with coarse-to-fine attention. In: Proceedings of the 34th international conference on machine learning. 2016. pp. 980–989. https://doi.org/10.2340/16501977-0511.

  25. Zhang J, Du J, Dai L. Multi-scale attention with dense encoder for handwritten mathematical expression recognition. In: Proceedings of the 24th international conference on pattern recognition. 2018. pp. 2245–2250. https://doi.org/10.1109/ICPR.2018.8546031.

  26. Thi LN, My LH, Viet NH, Minh NTH, Hong LP. Building a treebank for Vietnamese dependency parsing. In: Proceedings of the international conference on computing and communication technologies: research, innovation, and vision for future. 2013. pp. 147–151. https://doi.org/10.1109/RIVF.2013.6719884.

Download references

Author information

Authors and Affiliations

  1. Center for Open Data in the Humanities, Tokyo, Japan

    Anh Duc Le

  2. Department of Computer and Information Sciences, Tokyo University of Agriculture and Technology, Tokyo, Japan

    Hung Tuan Nguyen & Masaki Nakagawa

Authors
  1. Anh Duc Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hung Tuan Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masaki Nakagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anh Duc Le.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Future Data and Security Engineering” guest edited by Tran Khanh Dang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Le, A.D., Nguyen, H.T. & Nakagawa, M. An End-to-End Recognition System for Unconstrained Vietnamese Handwriting. SN COMPUT. SCI. 1, 7 (2020). https://doi.org/10.1007/s42979-019-0001-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-019-0001-4

Keywords

Search

Navigation