Abstract
Stroke order and velocity are helpful features in the fields of signature verification, handwriting recognition, and handwriting synthesis. Recovering these features from offline handwritten text is a challenging and well-studied problem. We propose a new model called TRACE (Trajectory Recovery by an Adaptively-trained Convolutional Encoder). TRACE is a differentiable approach that uses a convolutional recurrent neural network (CRNN) to infer temporal stroke information from long lines of offline handwritten text with many characters and dynamic time warping (DTW) to align predictions and ground truth points. TRACE is perhaps the first system to be trained end-to-end on entire lines of text of arbitrary width and does not require the use of dynamic exemplars. Moreover, the system does not require images to undergo any pre-processing, nor do the predictions require any post-processing. Consequently, the recovered trajectory is differentiable and can be used as a loss function for other tasks, including synthesizing offline handwritten text.
We demonstrate that temporal stroke information recovered by TRACE from offline data can be used for handwriting synthesis and establish the first benchmarks for a stroke trajectory recovery system trained on the IAM online handwriting dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ardizzone, L., Kruse, J., Rother, C., Köthe, U.: Analyzing inverse problems with invertible neural networks. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=rJed6j0cKX
Breuel, T.M.: Tutorial on OCR and layout analysis. Technical report (2018)
Carbune, V., et al.: Fast multi-language LSTM-based online handwriting recognition. Int. J. Doc. Anal. Recognit. (IJDAR) 23(2), 89–102 (2020). https://doi.org/10.1007/s10032-020-00350-4
Davis, B., Tensmeyer, C., Price, B., Wigington, C., Morse, B., Jain, R.: Text and style conditioned GAN for generation of offline handwriting lines. In: 31st British Machine Vision Conference, BMVC (2020)
Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using Real NVP. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, May 2016. http://arxiv.org/abs/1605.08803
Fogel, S., Averbuch-Elor, H., Cohen, S., Mazor, S., Litman, R.: Scrabblegan: semi-supervised varying length handwritten text generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4324–4333 (2020)
Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
Graves, A.: Stochastic Backpropagation through Mixture Density Distributions, July 2016. http://arxiv.org/abs/1607.05690
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR, December 2015. https://arxiv.org/abs/1412.6980v9
Kumarbhunia, A., et al.: Handwriting trajectory recovery using end-to-end deep encoder-decoder network. In: Proceedings - International Conference on Pattern Recognition, vol. 2018-August, pp. 3639–3644. Institute of Electrical and Electronics Engineers Inc. (2018). https://doi.org/10.1109/ICPR.2018.8546093
Liwicki, M., Bunke, H.: IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, vol. 2005, pp. 956–961 (2005). https://doi.org/10.1109/ICDAR.2005.132
Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recogn. 5(1), 39–46 (2002)
Nguyen, H.T., Nakamura, T., Nguyen, C.T., Nakawaga, M.: Online trajectory recovery from offline handwritten Japanese kanji characters of multiple strokes. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 8320–8327. IEEE (2021)
Nguyen, V., Blumenstein, M.: Techniques for static handwriting trajectory recovery. In: Proceedings of the 8th IAPR International Workshop on Document Analysis Systems - DAS 2010, pp. 463–470. ACM Press, New York (2010). https://doi.org/10.1145/1815330.1815390. http://portal.acm.org/citation.cfm?doid=1815330.1815390
Plamondon, R., Privitera, C.M.: The segmentation of cursive handwriting: an approach based on off-line recovery of the motor-temporal information. IEEE Trans. Image Process. 8(1), 80–91 (1999). https://doi.org/10.1109/83.736691
Plamondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)
Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978)
Sumi, T., Iwana, B.K., Hayashi, H., Uchida, S.: Modality conversion of handwritten patterns by cross variational autoencoders. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 407–412. IEEE (2019)
Wigington, C., Stewart, S., Davis, B., Barrett, B., Price, B., Cohen, S.: Data augmentation for recognition of handwritten words and lines using a CNN-LSTM network. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, vol. 1, pp. 639–645. IEEE Computer Society (2017). https://doi.org/10.1109/ICDAR.2017.110
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)
Zhao, B., Yang, M., Tao, J.: Pen tip motion prediction for handwriting drawing order recovery using deep neural network. In: Proceedings - International Conference on Pattern Recognition, vol. 2018-August, pp. 704–709. Institute of Electrical and Electronics Engineers Inc. (2018). https://doi.org/10.1109/ICPR.2018.8546086
Acknowledgements
We would like to thank Chris Tensmeyer for his suggestions and feedback.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Archibald, T., Poggemann, M., Chan, A., Martinez, T. (2021). TRACE: A Differentiable Approach to Line-Level Stroke Recovery for Offline Handwritten Text. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12823. Springer, Cham. https://doi.org/10.1007/978-3-030-86334-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-86334-0_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86333-3
Online ISBN: 978-3-030-86334-0
eBook Packages: Computer ScienceComputer Science (R0)
