Abstract
Finding key terms in scanned historical manuscripts is invaluable for accessing our written cultural heritage. While keyword spotting (KWS) approaches based on machine learning achieve the best spotting results in the current state of the art, they are limited by the fact that annotated learning samples are needed to infer the writing style of a particular manuscript collection. In this paper, we propose an annotation-free KWS method that does not require any labeled handwriting sample but learns from a printed font instead. First, we train a deep convolutional character detection system on synthetic pages using printed characters. Afterwards, the structure of the detected characters is modeled by means of graphs and is compared with search terms using graph matching. We evaluate our method for spotting logographic Chu Nom characters on the newly introduced Kieu database, which is a historical Vietnamese manuscripts containing 719 scanned pages of the famous Tale of Kieu. Our results show that search terms can be found with promising precision both when providing handwritten samples (query by example) as well as printed characters (query by string).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Available here: https://github.com/asciusb/Kieu-database.
- 3.
github.com/ultralytics/yolov5, commit cc03c1d5727e178438e9f0ce0450fa6bdbbe1ea7.
References
Ameri, M.R., Stauffer, M., Riesen, K., Bui, T.D., Fischer, A.: Graph-based keyword spotting in historical manuscripts using Hausdorff edit distance. Pattern Recogn. Lett. 121, 61–67 (2019)
Conte, D., Foggia, P., Sansone, C., Vento, M.: Thirty years of graph matching in pattern recognition. Int. J. Pattern Recogn. Artif. Intell. 18(3), 265–298 (2004)
Fischer, A., Riesen, K., Bunke, H.: Graph similarity features for HMM-based handwriting recognition in historical documents. In: Proceedings of the International Conference on Frontiers in Handwriting Recognition, pp. 253–258 (2010)
Fischer, A., Suen, C.Y., Frinken, V., Riesen, K., Bunke, H.: Approximation of graph edit distance based on Hausdorff matching. Pattern Recogn. 48(2), 331–343 (2015)
Fischer, A., Liwicki, M., Ingold, R. (eds.): Handwritten Historical Document Analysis, Recognition, and Retrieval - State of the Art and Future Trends. World Scientific, Singapore (2020)
Giotis, A.P., Sfikas, G., Gatos, B., Nikou, C.: A survey of document image word spotting techniques. Pattern Recogn. 68, 310–332 (2017)
Jocher, G., et al.: ultralytics/yolov5: v4.0 - nn.SILU() activations, weights & biases logging, PyTorch hub integration (2021). https://doi.org/10.5281/ZENODO.4418161
Maergner, P., et al.: Combining graph edit distance and triplet networks for offline signature verification. Pattern Recogn. Lett. 125, 527–533 (2019)
Nguyen, K.C., Nguyen, C.T., Nakagawa, M.: Nom document digitalization by deep convolution neural networks. Pattern Recogn. Lett. 133, 8–16 (2020)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016)
Scius-Bertrand, A., Jungo, M., Wolf, B., Fischer, A., Bui, M.: Annotation-free character detection in historical Vietnamese stele images. In: Proceedings of the 16th International Conference on Document Analysis and Recognition (ICDAR), pp. 432–447 (2021)
Scius-Bertrand, A., Jungo, M., Wolf, B., Fischer, A., Bui, M.: Transcription alignment of historical Vietnamese manuscripts without human-annotated learning samples. Appl. Sci. 11(11), 4894 (2021)
Sudholt, S., Fink, G.A.: PHOCNet: a deep convolutional neural network for word spotting in handwritten documents. In: Proceedings of the 15th International Conference on Frontiers in Handwriting Recognition, pp. 277–282 (2016)
Vento, M.: A one hour trip in the world of graphs, looking at the papers of the last ten years. In: Proceedings of the International Workshop on Graph-Based Representation, pp. 1–10 (2013)
Vidal, E., Toselli, A.H., Puigcerver, J.: A probabilistic framework for lexicon-based keyword spotting in handwritten text images. CoRR abs/2104.04556 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Scius-Bertrand, A., Studer, L., Fischer, A., Bui, M. (2022). Annotation-Free Keyword Spotting in Historical Vietnamese Manuscripts Using Graph Matching. In: Krzyzak, A., Suen, C.Y., Torsello, A., Nobile, N. (eds) Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2022. Lecture Notes in Computer Science, vol 13813. Springer, Cham. https://doi.org/10.1007/978-3-031-23028-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-23028-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23027-1
Online ISBN: 978-3-031-23028-8
eBook Packages: Computer ScienceComputer Science (R0)