Skip to main content
Log in

A unified method for augmented incremental recognition of online handwritten Japanese and English text

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

We present a unified method to augmented incremental recognition for online handwritten Japanese and English text, which is used for busy or on-the-fly recognition while writing, and lazy or delayed recognition after writing, without incurring long waiting times. It extends the local context for segmentation and recognition to a range of recent strokes called “segmentation scope” and “recognition scope,” respectively. The recognition scope is inside of the segmentation scope. The augmented incremental recognition triggers recognition at every several recent strokes, updates the segmentation and recognition candidate lattice, and searches over the lattice for the best result incrementally. It also incorporates three techniques. The first is to reuse the segmentation and recognition candidate lattice in the previous recognition scope for the current recognition scope. The second is to fix undecided segmentation points if they are stable between character/word patterns. The third is to skip recognition of partial candidate character/word patterns. The augmented incremental method includes the case of triggering recognition at every new stroke with the above-mentioned techniques. Experiments conducted on TUAT-Kondate and IAM online database show its superiority to batch recognition (recognizing text at one time) and pure incremental recognition (recognizing text at every input stroke) in processing time, waiting time, and recognition accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Gao, J., Zhu, B., Nakagawa, M.: Building compact recognizer with recognition rate maintained for on-line handwritten Japanese text recognition. Pattern Recognit. Lett. 35, 169–177 (2014). https://doi.org/10.1016/j.patrec.2013.08.014

    Article  Google Scholar 

  2. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 855–868 (2009). https://doi.org/10.1109/TPAMI.2008.137

    Article  Google Scholar 

  3. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18, 602–610 (2005). https://doi.org/10.1016/j.neunet.2005.06.042

    Article  Google Scholar 

  4. Jaeger, S., Manke, S., Reichert, J., Waibel, A.: Online handwriting recognition: the NPen++ recognizer. Int. J. Doc. Anal. Recognit. 3, 169–180 (2001). https://doi.org/10.1007/PL00013559

    Article  Google Scholar 

  5. Johansson, S., Leech, G.N., Goodluck, H.: Manual of Information to accompany the Lancaster-Oslo/Bergen Corpus of British English, for use with digital Computers (1978)

  6. Keysers, D., Deselaers, T., Rowley, H.A., Wang, L.-L., Carbune, V.: Multi-language online handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1180–1194 (2017). https://doi.org/10.1109/TPAMI.2016.2572693

    Article  Google Scholar 

  7. Liu, C.-L., Jaeger, S., Nakagawa, M.: Online recognition of Chinese characters: the state-of-the-art. IEEE Trans. Pattern Anal. Mach. Intell. 26, 198–213 (2004). https://doi.org/10.1109/TPAMI.2004.1262182

    Article  Google Scholar 

  8. Liwicki, M., Bunke, H.: IAM-OnDB: an on-line English sentence database acquired from handwritten text on a whiteboard. In: 8th International Conference on Document Analysis and Recognition, vol. 2, pp. 956–961. IEEE (2005)

  9. Liwicki, M., Bunke, H.: HMM-based on-line recognition of handwritten whiteboard notes. In: 10th International Workshop on Frontiers in Handwriting Recognition. La Baule (France) (2006)

  10. Liwicki, M., Bunke, H., Pittman, J.A., Knerr, S.: Combining diverse systems for handwritten text line recognition. Mach. Vis. Appl. 22, 39–51 (2011). https://doi.org/10.1007/s00138-009-0208-9

    Article  Google Scholar 

  11. Matic, N.P., Platt, J.C., Wang, T.: QuickStroke: an incremental on-line Chinese handwriting recognition system, vol. 3, pp. 435–439 (2002)

  12. Matsushita, T., Nakagawa, M.: A database of on-line handwritten mixed objects named Kondate. In: 14th International Conference on Frontiers in Handwriting Recognition, pp. 369–374 (2014)

  13. McDermott, E., Katagiri, S.: Minimum classification error for large scale speech recognition tasks using weighted finite state transducers. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 113–116. IEEE (2005)

  14. Nakagawa, M., Machii, K., Kato, N., Souya, T.: Lazy recognition as a principle of pen interfaces. In: INTERACT’93 and CHI’93 Conference Companion on Human Factors in Computing Systems, pp. 89–90. ACM Press, New York (1993)

  15. Nakagawa, M., Matsumoto, K.: Collection of on-line handwritten Japanese character pattern databases and their analyses. Doc. Anal. Recognit. (2004). https://doi.org/10.1007/s10032-004-0125-4

    Article  Google Scholar 

  16. Nakagawa, M., Zhu, B., Onuma, M.: A model of on-line handwritten Japanese text recognition free from line direction and writing format constraints. IEICE Trans. Inf. Syst. 88, 1815–1822 (2005)

    Article  Google Scholar 

  17. Nguyen, C.T., Nakagawa, M.: An improved segmentation of online English handwritten text using recurrent neural networks. In: 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 176–180. IEEE (2015)

  18. Nguyen, C.T., Zhu, B., Nakagawa, M.: A Semi-incremental recognition method for on-line handwritten Japanese text. In: 12th International Conference on Document Analysis and Recognition, pp. 84–88. IEEE (2013)

  19. Nguyen, C.T., Zhu, B., Nakagawa, M.: A Semi-incremental recognition method for on-line handwritten English text. In: 14th International Conference on Frontiers in Handwriting Recognition, pp. 234–239 (2014)

  20. Nguyen, C.T., Zhu, B., Nakagawa, M.: Semi-incremental recognition of on-line handwritten Japanese text. IEICE Trans. Inf. Syst. 99, 2619–2628 (2016). https://doi.org/10.1587/transinf.2016edp7051

    Article  Google Scholar 

  21. Okamoto, M., Yamamoto, K.: On-line handwriting character recognition using direction-change features that consider imaginary strokes. Pattern Recognit. 32, 1115–1128 (1999). https://doi.org/10.1016/S0031-3203(98)00153-8

    Article  Google Scholar 

  22. Plamondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22, 63–84 (2000). https://doi.org/10.1109/34.824821

    Article  Google Scholar 

  23. Shivram, A., Zhu, B., Setlur, S., Nakagawa, M., Govindaraju, V.: Segmentation based online word recognition: a conditional random field driven beam search strategy. In: 12th International Conference on Document Analysis and Recognition, pp. 852–856. IEEE (2013)

  24. Tanaka, H., Akiyama, K., Ishigaki, K.: Realtime box-free on-line handwriting string recognition using layer-delayed segmentation method. IEICE Technical Report, vol. 101, pp. 155–162. Institute of Electronics, Information and Communication Engineers (2002)

  25. Wang, D.H., Liu, C.L., Zhou, X.D.: An approach for real-time recognition of online Chinese handwritten sentences. Pattern Recognit. 45, 3661–3675 (2012). https://doi.org/10.1016/j.patcog.2012.04.020

    Article  Google Scholar 

  26. Wang, Q., Yin, F., Liu, C.-L., Member, S.: Handwritten Chinese text recognition by integrating multiple contexts. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1469–1481 (2012). https://doi.org/10.1109/TPAMI.2011.264

    Article  Google Scholar 

  27. Zhou, Xiang-Dong, Wang, Da-Han, Tian, Feng, Liu, Cheng-Lin, Nakagawa, M.: Handwritten Chinese/Japanese text recognition using semi-Markov conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2413–2426 (2013). https://doi.org/10.1109/TPAMI.2013.49

    Article  Google Scholar 

  28. Zhou, X.-D., Wang, D.-H., Liu, C.-L.: Grouping text lines in online handwritten Japanese documents by combining temporal and spatial information. In: 8th IAPR International Workshop on Document Analysis Systems, pp. 61–68. https://doi.org/10.1109/das.2008.15 (2008)

  29. Zhu, B., Gao, J., Nakagawa, M.: Objective function design for MCE-based combination of on-line and off-line character recognizers for on-line handwritten Japanese text recognition. In: 11th International Conference on Document Analysis and Recognition, pp. 594–598. IEEE

  30. Zhu, B., Shivram, A., Setlur, S., Govindaraju, V., Nakagawa, M.: Online handwritten cursive word recognition using segmentation-free MRF in combination with P2DBMN-MQDF. In: 12th International Conference on Document Analysis and Recognition, pp. 349–353. IEEE (2013)

  31. Zhu, B., Zhou, X.-D., Liu, C.-L., Nakagawa, M.: A robust model for on-line handwritten japanese text recognition. Int. J. Doc. Anal. Recognit. 13, 121–131 (2010). https://doi.org/10.1007/s10032-009-0111-y

    Article  Google Scholar 

Download references

Acknowledgements

This research has been partially supported by NEDO under the contract number 27J1103, JSPS KAKENHI Grant Number JP 18K18068.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Cuong Tuan Nguyen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, C.T., Indurkhya, B. & Nakagawa, M. A unified method for augmented incremental recognition of online handwritten Japanese and English text. IJDAR 23, 53–72 (2020). https://doi.org/10.1007/s10032-019-00343-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-019-00343-y

Keywords

Navigation