Advertisement

A comparative study of delayed stroke handling approaches in online handwriting

  • Esma F. Bilgin TasdemirEmail author
  • Berrin Yanikoglu
Original Paper
  • 122 Downloads

Abstract

Delayed strokes, such as i-dots and t-crosses, cause a challenge in online handwriting recognition by introducing an extra source of variation in the sequence order of the handwritten input. The problem is especially relevant for languages where delayed strokes are abundant and training data are limited. Studies for handling delayed strokes have mainly focused on Arabic and Farsi scripts where the problem is most severe, with less attention devoted for scripts based on the Latin alphabet. This study aims to investigate the effectiveness of the delayed stroke handling methods proposed in the literature. Evaluated methods include the removal of delayed strokes and embedding delayed strokes in the correct writing order, together with their variations. Starting with new definitions of a delayed stroke, we tested each method using both hidden Markov model classifiers separately for English and Turkish and bidirectional long short-term memory networks for English. For both the UNIPEN and Turkish datasets, the best results are obtained with hidden Markov model recognizers by removing all delayed strokes, with up to 2.13% and 2.03% points accuracy increases over the respective baselines. In case of the bidirectional long short-term memory networks, stroke order correction of the delayed strokes by embedding performs the best, with 1.81% (raw) and 1.72% (post-processed) points improvements above the baseline.

Keywords

Online handwriting Delayed strokes Accented characters 

Notes

Acknowledgements

This work was supported by The Scientific and Technological Research Council of Turkey (TÜBITAK), under Grant Number 113E062.

References

  1. 1.
    Abdelazeem, S., Eraqi, H.M.: On-line Arabic handwritten personal names recognition system based on HMM. In: 2011 International Conference on Document Analysis and Recognition, ICDAR 2011, Beijing, China, September 18–21, 2011, pp. 1304–1308 (2011)Google Scholar
  2. 2.
    Abdelaziz, I., Abdou, S., Al-Barhamtoshy, H.: Large vocabulary Arabic online handwriting recognition system. CoRR. arxiv:abs/1410.4688 (2014)
  3. 3.
    Alimi, A.M.: An evolutionary neuro-fuzzy approach to recognize on-line Arabic handwriting. In: 4th International Conference Document Analysis and Recognition, ICDAR 1997, 2-Volume Set, August 18–20, 1997, Ulm, Germany, Proceedings, pp. 382–386 (1997)Google Scholar
  4. 4.
    Biadsy, F., El-Sana, J., Habash, J.: Online Arabic handwriting recognition using Hidden Markov Models. In: Tenth International Workshop on Frontiers in Handwriting Recognition, IWFHR 2007, IAPR. Newyork,USA, 2006 (2006)Google Scholar
  5. 5.
    Flann, N.S.: Recognition-based segmentation of on-line cursive handwriting. In: Advances in Neural Information Processing Systems 6, 7th NIPS Conference, Denver, Colorado, USA, 1993, pp. 777–784 (1993)Google Scholar
  6. 6.
    Gauthier, N., Artières, T., Gallinari, P., Dorizzi, B.: Strategies for combining on-line and off-line information in an on-line handwriting recognition system. In: 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10–13 September 2001, pp. 412–416. Seattle, WA, USA (2001)Google Scholar
  7. 7.
    Ghods, V., Kabir, E., Razzazi, F.: Effect of delayed strokes on the recognition of online Farsi handwriting. Pattern Recognit. Lett. 34(5), 486–491 (2013)CrossRefGoogle Scholar
  8. 8.
    Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning, ICML 2014, Beijing, China, 21–26 June 2014, pp. 1764–1772 (2014)Google Scholar
  9. 9.
    Graves, A., Fernández, S., Gomez, F.J., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Machine Learning, Proceedings of the Twenty-Third International Conference (ICML 2006), Pittsburgh, Pennsylvania, USA, June 25–29, 2006, pp. 369–376 (2006)Google Scholar
  10. 10.
    Graves, A., Fernández, S., Liwicki, M., Bunke, H., Schmidhuber, J.: Unconstrained on-line handwriting recognition with recurrent neural networks. In: Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, NIPS, Vancouver, British Columbia, Canada, December 3–6, 2007, pp. 577–584 (2007)Google Scholar
  11. 11.
    Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009)CrossRefGoogle Scholar
  12. 12.
    Günter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer. In: 7th International Conference on Document Analysis and Recognition, (ICDAR 2003), 2-Volume Set, 3–6 August 2003, pp. 472–476. Edinburgh, Scotland, UK (2003)Google Scholar
  13. 13.
    Ha, J., Oh, S., J.K., Kwon, Y.: Unconstrained handwritten word recognition with interconnected Hidden Markov Models. In: Third International Workshop on Frontiers in Handwriting Recognition, IWFHR 1993, IAPR. Buffalo, USA, May 1993, pp 455–560 (1993)Google Scholar
  14. 14.
    HTK The Hidden Markov Model toolkit htk. http://htk.eng.cam.ac.uk/. Accessed 29 Oct 2016
  15. 15.
    Hu, J., Brown, M.K., Turin, W: Handwriting recognition with Hidden Markov Models and grammatical constraints. In: Fourth International Workshop on Frontiers in Handwriting Recognition, IWFHR 1994, IAPR.Taipei, Dec 1994 (1994)Google Scholar
  16. 16.
    Hu, J., Lim, S.G., Brown, M.K.: Writer independent on-line handwriting recognition using an HMM approach. Pattern Recognit. 33(1), 133–147 (2000)CrossRefGoogle Scholar
  17. 17.
    Jäger, S., Manke, S., Reichert, J., Waibel, A.: Online handwriting recognition: the NPen++ recognizer. IJDAR 3(3), 169–180 (2001)CrossRefGoogle Scholar
  18. 18.
    Koerich, A.L., Sabourin, R., Suen, C.Y.: Large vocabulary off-line handwriting recognition: a survey. Pattern Anal. Appl. 6(2), 97–121 (2003)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Liwicki, M., Bunke, H.: IAM-OnDB—an on-line English sentence database acquired from handwritten text on a whiteboard. In: Eighth International Conference on Document Analysis and Recognition, ICDAR 2005, 29 August–1 September 2005, pp. 956–961. Seoul, Korea (2005)Google Scholar
  20. 20.
    Liwicki, M., Bunke, H.: Hmm-based on-line recognition of handwritten whiteboard notes. In: Tenth International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006 (2006)Google Scholar
  21. 21.
    Liwicki, M., Bunke, H.: Handwriting recognition of whiteboard notes—studying the influence of training set size and type. IJPRAI 21(1), 83–98 (2007)Google Scholar
  22. 22.
    Liwicki, M., Graves, A., Bunke, H., Schmidhuber, J.: A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In: Proceedings of the 9th International Conference on Document Analysis and Recognition, ICDAR 2007 (2007)Google Scholar
  23. 23.
    Marukatat, S., Artières, T., Gallinari, P., Dorizzi, B.: Sentence recognition through hybrid neuro-Markovian modeling. In: 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10–13 September 2001, pp. 731–737. Seattle, WA, USA (2001)Google Scholar
  24. 24.
    Plamondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)CrossRefGoogle Scholar
  25. 25.
    Plötz, T., Fink, G.A.: Markov models for offline handwriting recognition: a survey. IJDAR 12(4), 269–298 (2009)CrossRefGoogle Scholar
  26. 26.
    Rabiner, L.R.: A tutorial on Hidden Markov Models and selected applications in speech recognition. In: Proceedings of the IEEE, pp. 257–286 (1989)Google Scholar
  27. 27.
    Ratcliff, J.W., Metzener, D.E.: Pattern matching: the gestalt approach. Dr Dobbs J. 13(7), 46–72 (1988)Google Scholar
  28. 28.
    Schenk, J., Rigoll, G.: Novel hybrid nn/hmm modelling techniques for on-line handwriting recognition. In: 10th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006, IAPR. , La Baule, France, Oct 2006, pp. 619–6230 (2006)Google Scholar
  29. 29.
    Schenkel, M., Guyon, I., Henderson, D.: On-line cursive script recognition using time-delay neural networks and Hidden Markov Models. Mach. Vis. Appl. 8(4), 215–223 (1995)CrossRefGoogle Scholar
  30. 30.
    Sutskever, I.: (2013) Training Recurrent Neural Networks. Ph.D. thesis, Toronto, Ont., CanadaGoogle Scholar
  31. 31.
    The UNIPEN Consortium The UNIPEN project. http://www.unipen.org/products.html. Accessed 20 Oct 2016
  32. 32.
    Viard-Gaudin, C., Lallican, P.M., Binter, P., Knerr, S.: The IRESTE on/off (IRONOFF) dual handwriting database. In: Fifth International Conference on Document Analysis and Recognition, ICDAR 1999, 20–22 September, 1999, pp. 455–458. Bangalore, India (1999)Google Scholar
  33. 33.
    Vural, E., Erdogan, H., Oflazer, K., Yanikoglu, B.A.: An online handwriting recognition system for Turkish. In: Document Recognition and Retrieval XII,DRR 2005,San Jose, California, USA, January 16–20, 2005, Proceedings, pp. 56–65 (2005)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Engineering and Natural SciencesSabancı UniversityIstanbulTurkey

Personalised recommendations