Abstract
It is very common for human beings to include both text and non-text data in a handwritten document. Text portion contains alphabets, digits, and mathematical symbols, whereas non-text portion includes various graphical entities like flow chart, transition diagram, etc. This paper proposes a novel method for online handwritten text and non-text stroke classification with text written in Devanagari script, the most popular script in India, using two different architectures of artificial Recurrent Neural Network (RNN)—long-short term memory (LSTM) and bidirectional long-short term memory (BLSTM). In the present work, the classifier classifies an ink as text or non-text stroke when a sequence of strokes of any online handwritten document of both text and non-text data is presented to it. Various structural and directional features related to online handwriting have been extracted from the basic strokes of text and non-text data. The system has been trained in both LSTM and BLSTM architecture based classification platforms. Experiment has also been performed using Convolutional Neural Network (CNN) to make a comparative performance analysis with RNN classifier based results. The classification performance of the present work has been evaluated using a self-generated dataset and it outperforms the CNN based results as well as the existing studies available in the literature in this regard.
Similar content being viewed by others
References
Awal AM, Mouchre H, Viard-Gaudin C (2014) A global learning approach for an on-line handwritten mathematical expression recognition system. Pattern Recogn Lett 35:68–77
Blanchard J, Artieres T (2004) On-line handwritten documents segmentation. In: Proceedings of the 9th international workshop on frontiers in handwriting recognition. IEEE Press, Tokyo, pp 148–153
Bresler M, VanPhan T, Prusa D, Nakagawa M, Hlavc V (2014) Recognition system for online sketched diagrams. In: Proceedings of the 14th international conference on frontiers in handwriting recognition. IEEE Press, Crete, pp 563–568
Delaye A, Lee K (2015) A flexible framework for online document segmentation by pairwise stroke distance learning. Pattern Recogn 48:1197–1210
Delaye A, Liu CL (2013) Graphics extraction from heterogeneous online documents with hierarchical random fields. In: Proceedings of the 12th international conference on document analysis and recognition. IEEE Press, Washington DC, pp 1007–1011
Delaye A, Liu CL (2014) Contextual text/non-text stroke classification in online handwritten notes with conditional random fields. Pattern Recogn 47 (3):959–968
Feng G, Viard-Gaudin C, Sun Z (2009) Online Hand-drawn electric circuit diagram recognition using 2d dynamic programming. Pattern Recogn 42 (12):3215–3223
Ghosh R, Keshri P, Kumar P (2018) RNN based online handwritten word recognition in Devanagari 431 script. In: Proceedings of the 16th international conference on frontiers in handwriting recognition. IEEE 432 Press, Niagra Falls, pp 517–522
Ghosh R, Kumar S, Kumar P (2018). Online Handwritten Text and Non-Text Classification in Devanagari Script using Elliptical Region-wise Features. In: Proceedings of the first international conference on secure cyber computing and communications. IEEE Press, Jalandhar, pp 163–167
Ghosh R, Shanu S, Ranjan S, Kumari K (2019) An approach based on classifier combination for online handwritten text and non-text classification in Devanagari script. SADHANA 44:8
Ghosh R, Vamsi C, Kumar P (2019) RNN based online handwritten word recognition in Devanagari and bengali scripts using horizontal zoning. Pattern Recogn 92:203–218
Graves A, Liwicki M, Fernandez S, Bertolami R, Bunke H, Schmidhuber J (2009) A novel connectionist system for unconstrained handwriting recognition. IEEE Trans Pattern Anal Mach Intell 31(5):855–868
Jaeger S, Manke S, Reichert J, Waibel A (2001) Online handwriting recognition: the NPen++ recognizer. Int J Doc Anal Recogn 3(3):169–180
Liwicki M, Indermuhle E, Bunke H (2007) Online hand written text line detection using dynamic programming. In: Proceedings of the 9th international conference on document analysis and recognition. IEEE Press, Curitiba, pp 447–451
Mochida K, Nakagawa M (2004) Separating figures, mathematical formulas and Japanese text from free handwriting in mixed on-line documents. Int J Pattern Recogn Artif Intell 18(7):1173–1187
Phan TV, Nakagawa M (2014) Text/non-text classification in online handwritten documents with recurrent neural networks. In: Proceedings of the 14th international conference on frontiers in handwriting recognition. IEEE Press, Crete, pp 23–28
Phan TV, Nakagawa M (2016) Combination of global and local contexts for text/non-text classification in heterogeneous online handwritten documents. Pattern Recogn 51:112–124
Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681
Zhou XD, Liu CL (2007) Text/non-text ink stroke classification in Japanese handwriting based on Markov random fields. In: Proceedings of the 9th international conference on document analysis and recognition. IEEE Press, Curitiba, pp 377–381
Zhou XD, Wang DH, Liu C (2009) A robust approach to text line grouping in online handwritten Japanese documents. Pattern Recogn 42(9):2077–2088
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The author has no conflict of interest/competing interest to declare that are relevant to the content of this article.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ghosh, R. A recurrent neural network based deep learning model for text and non-text stroke classification in online handwritten Devanagari document. Multimed Tools Appl 81, 24245–24263 (2022). https://doi.org/10.1007/s11042-022-12767-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-12767-6