Skip to main content
SpringerLink
Log in

Recurrent Neural Network Transducer for Japanese and Chinese Offline Handwritten Text Recognition

  • Conference paper
  • First Online:
Document Analysis and Recognition – ICDAR 2021 Workshops (ICDAR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12917))

Included in the following conference series:

Abstract

In this paper, we propose an RNN-Transducer model for recognizing Japanese and Chinese offline handwritten text line images. As far as we know, it is the first approach that adopts the RNN-Transducer model for offline handwritten text recognition. The proposed model consists of three main components: a visual feature encoder that extracts visual features from an input image by CNN and then encodes the visual features by BLSTM; a linguistic context encoder that extracts and encodes linguistic features from the input image by embedded layers and LSTM; and a joint decoder that combines and then decodes the visual features and the linguistic features into the final label sequence by fully connected and softmax layers. The proposed model takes advantage of both visual and linguistic information from the input image. In the experiments, we evaluated the performance of the proposed model on the two datasets: Kuzushiji and SCUT-EPT. Experimental results show that the proposed model achieves state-of-the-art performance on all datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mouchère, H., Zanibbi, R., Garain, U., Viard-Gaudin, C.: Advancing the state of the art for handwritten math recognition: the CROHME competitions, 2011–2014. International Journal on Document Analysis and Recognition (IJDAR) 19(2), 173–189 (2016). https://doi.org/10.1007/s10032-016-0263-5

    Article  Google Scholar 

  2. Zhu, Y., Xie, Z., Jin, L., Chen, X., Huang, Y., Zhang, M.: SCUT-EPT: new dataset and benchmark for offline Chinese text recognition in examination paper. IEEE Access 7, 370–382 (2019)

    Article  Google Scholar 

  3. Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., Ha, D.: Deep learning for classical Japanese literature. In: The Neural Information Processing Systems - Workshop on Machine Learning for Creativity and Design (2018)

    Google Scholar 

  4. Xu, Y., Yin, F., Wang, D.H., Zhang, X.Y., Zhang, Z., Liu, C.L.: CASIA-AHCDB: a large-scale chinese ancient handwritten characters database. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, pp. 793–798 (2019)

    Google Scholar 

  5. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 855–868 (2009)

    Article  Google Scholar 

  6. Bluche, T., Louradour, J., Knibbe, M., Moysset, B., Benzeghiba, M.F., Kermorvant, C.: The a2ia Arabic handwritten text recognition system at the openhart2013 evaluation. In: International Workshop on Document Analysis Systems, pp. 161–165 (2014)

    Google Scholar 

  7. Nguyen, H.T., Ly, N.T., Nguyen, K.C., Nguyen, C.T., Nakagawa, M.: Attempts to recognize anomalously deformed Kana in Japanese historical documents. In: Proceedings of the 4th International Workshop on Historical Document Imaging and Processing - HIP 2017, pp. 31–36 (2017)

    Google Scholar 

  8. Du, J., et al.: Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition. Pattern Recognit. 71, 196–206 (2017)

    Article  Google Scholar 

  9. Ly, N.T., Nguyen, C.T., Nguyen, K.C., Nakagawa, M.: Deep convolutional recurrent network for segmentation-free offline handwritten Japanese text recognition. In: Proceedings of the 14th International Conference on Document Analysis and Recognition, ICDAR, pp. 5–9 (2018)

    Google Scholar 

  10. Yin, F., Wang, Q.F., Zhang, X.Y., Liu, C.L.: ICDAR 2013 Chinese handwriting recognition competition. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, pp. 1464–1470 (2013)

    Google Scholar 

  11. Ly, N.T., Nguyen, C.T., Nakagawa, M.: An attention-based end-to-end model for multiple text lines recognition in Japanese historical documents. In: Proceedings of the 15th International Conference on Document Analysis and Recognition, pp. 629–634 (2019)

    Google Scholar 

  12. Hamdani, M., Doetsch, P., Kozielski, M., Mousa, A.E.D., Ney, H.: The RWTH large vocabulary arabic handwriting recognition system. In: International Workshop on Document Analysis Systems (DAS), DAS 2014, pp. 111–115 (2014)

    Google Scholar 

  13. Koerich, A.L., Sabourin, R., Suen, C.Y.: Large vocabulary off-line handwriting recognition: a survey (2003)

    Google Scholar 

  14. Nguyen, C.T., Nakagawa, M.: Finite state machine based decoding of handwritten text using recurrent neural networks. In: Proceedings of International Conference on Frontiers in Handwriting Recognition, ICFHR, pp. 246–251 (2016)

    Google Scholar 

  15. Ingle, R., Fujii, Y., Deselaers, T., Baccash, J.M., Popat, A.: A scalable handwritten text recognition system. In: Proceedings of the 15th International Conference on Document Analysis and Recognition, pp. 17–24 (2019)

    Google Scholar 

  16. Bluche, T., Louradour, J., Messina, R.: Scan, attend and read: end-to-end handwritten paragraph recognition with MDLSTM attention. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, pp. 1050–1055 (2017)

    Google Scholar 

  17. Puigcerver, J.: Are multidimensional recurrent layers really necessary for handwritten text recognition? In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, pp. 67–72 (2017)

    Google Scholar 

  18. Ly, N.T., Nguyen, C.T., Nakagawa, M.: Training an end-to-end model for offline handwritten Japanese text recognition by generated synthetic patterns. In: Proceedings of International Conference on Frontiers in Handwriting Recognition, ICFHR, pp. 74–79 (2018)

    Google Scholar 

  19. Wang, Q.F., Yin, F., Liu, C.L.: Handwritten Chinese text recognition by integrating multiple contexts. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1469–1481 (2012)

    Article  Google Scholar 

  20. Srihari, S.N., Yang, X., Ball, G.R.: Offline Chinese handwriting recognition: an assessment of current technology (2007). https://link.springer.com/article/10.1007/s11704-007-0015-2

  21. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning - ICML 2006, pp. 369–376 (2006)

    Google Scholar 

  22. Bahdanau, D., Cho, K.H., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of the 3rd International Conference on Learning Representations (2015)

    Google Scholar 

  23. Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., Bengio, Y.: End-to-end attention-based large vocabulary speech recognition. In: the 41thinternational Conference on Acoustics, Speech and Signal Processing, pp. 4945–4949 (2016)

    Google Scholar 

  24. Ly, N.T., Nguyen, C.T., Nakagawa, M.: Attention augmented convolutional recurrent network for handwritten Japanese text recognition. In: Proceedings of the 17th International Conference on Frontiers in Handwriting Recognition, ICFHR, pp. 163–168 (2020)

    Google Scholar 

  25. Kim, S., Hori, T., Watanabe, S.: Joint CTC-attention based end-to-end speech recognition using multi-task learning. In: the 42th International Conference on Acoustics, Speech and Signal Processing, pp. 4835–4839 (2017)

    Google Scholar 

  26. Hoang, H.-T., Peng, C.-J., Tran, H.V., Le, H., Nguyen, H.H.: lodenet: a holistic approach to offline handwritten Chinese and Japanese text line recognition. In: Proceedings of the 25th International Conference on Pattern Recognition (ICPR), pp. 4813–4820 (2021)

    Google Scholar 

  27. Wigington, C., Price, B., Cohen, S.: Multi-label connectionist temporal classification. In: Proceedings of the 15th International Conference on Document Analysis and Recognition, ICDAR, pp. 979–986 (2019)

    Google Scholar 

  28. Graves, A.: Sequence transduction with recurrent neural networks (2012)

    Google Scholar 

Download references

Acknowledgement

The authors would like to thank Dr. Cuong Tuan Nguyen for his valuable comments. This research is being partially supported by A-STEP JPMJTM20ML, the grant-in-aid for scientific research (S) 18H05221 and (A) 18H03597.

Author information

Authors and Affiliations

  1. Tokyo University of Agriculture and Technology, Tokyo, Japan

    Trung Tan Ngo, Hung Tuan Nguyen, Nam Tuan Ly & Masaki Nakagawa

Authors
  1. Trung Tan Ngo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hung Tuan Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nam Tuan Ly
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masaki Nakagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hung Tuan Nguyen .

Editor information

Editors and Affiliations

  1. Boise State University, Boise, ID, USA

    Elisa H. Barney Smith

  2. Indian Statistical Institute, Kolkata, India

    Umapada Pal

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ngo, T.T., Nguyen, H.T., Ly, N.T., Nakagawa, M. (2021). Recurrent Neural Network Transducer for Japanese and Chinese Offline Handwritten Text Recognition. In: Barney Smith, E.H., Pal, U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science(), vol 12917. Springer, Cham. https://doi.org/10.1007/978-3-030-86159-9_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86159-9_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86158-2

  • Online ISBN: 978-3-030-86159-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation