Abstract
This paper presents an end-to-end attention-based neural network for identifying writers in historical documents. The proposed network does not require any preprocessing stages such as binarization or segmentation. It has three main parts: a feature extractor using Convolutional Neural Network (CNN) to extract features from an input image; an attention filter to select key points; and a generalized deep neural VLAD model to form a representative vector by aggregating the extracted key points. The whole network is trained end-to-end by a combination of cross-entropy and triplet losses. In the experiments, we evaluate the performance of our model on the HisFragIR20 dataset that consists of about 120,000 historical fragments from many writers. The experiments demonstrate better mean average precision and accuracy at top-1 in comparison with the state-of-the-art results on the HisFragIR20 dataset. This model is rather new for dealing with various sizes of historical document fragments in the writer identification and image retrieval.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recognit. 5, 39–46 (2003)
Bulacu, M., Schomaker, L., Vuurpijl, L.: Writer identification using edge-based directional features. In: Proceedings of the 7th International Conference on Document Analysis and Recognition, pp. 937–941 (2003)
Louloudis, G., Gatos, B., Stamatopoulos, N., Papandreou, A.: ICDAR 2013 competition on writer identification. In: Proceedings of the 12th International Conference on Document Analysis and Recognition, pp. 1397–1401 (2013)
Fiel, S., et al.: ICDAR2017 competition on historical document writer identification (Historical-WI). In: Proceedings of the 14th International Conference on Document Analysis and Recognition, pp. 1377–1382 (2018)
Christlein, V., Nicolaou, A., Seuret, M., Stutzmann, D., Maier, A.: ICDAR 2019 competition on image retrieval for historical handwritten documents. In: Proceedings of the 15th International Conference on Document Analysis and Recognition, pp. 1505–1509 (2019)
Seuret, M., Nicolaou, A., Stutzmann, D., Maier, A., Christlein, V.: ICFHR 2020 competition on image retrieval for historical handwritten fragments. In: Proceedings of 17th International Conference on Frontiers in Handwriting Recognition, pp. 216–221 (2020)
Jaakkola, T., Diekhans, M., Haussler, D.: Using the Fisher kernel method to detect remote protein homologies. In: Proceedings of the 7th International Conference on Intelligent Systems for Molecular Biology, pp. 149–158 (1999)
Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: Proceedings of the 23th IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3304–3311 (2010)
Abdeljalil, G., Djeddi, C., Siddiqi, I., Al-Maadeed, S.: Writer identification on historical documents using oriented basic image features. In: Proceedings of 16th International Conference on Frontiers in Handwriting Recognition, pp. 369–373 (2018)
He, Z., You, X., Zhou, L., Cheung, Y., Du, J.: Writer identification using fractal dimension of wavelet subbands in gabor domain. Integr. Comput. Aided Eng. 17, 157–165 (2010)
Nicolaou, A., Bagdanov, A.D., Liwicki, M., Karatzas, D.: Sparse radial sampling LBP for writer identification. In: Proceedings of the 13th International Conference on Document Analysis and Recognition, pp. 716–720 (2015)
Lai, S., Zhu, Y., Jin, L.: Encoding pathlet and SIFT features with bagged VLAD for historical writer identification. IEEE Trans. Inf. Forensics Secur. 15, 3553–3566 (2020)
Fiel, S., Sablatnig, R.: Writer identification and writer retrieval using the fisher vector on visual vocabularies. In: Proceedings of the 12th International Conference on Document Analysis and Recognition, pp. 545–549 (2013)
Khan, F.A., Khelifi, F., Tahir, M.A., Bouridane, A.: Dissimilarity Gaussian mixture models for efficient offline handwritten text-independent identification using SIFT and RootSIFT descriptors. IEEE Trans. Inf. Forensics Secur. 14, 289–303 (2018)
He, S., Schomaker, L.: FragNet: writer identification using deep fragment networks. IEEE Trans. Inf. Forensics Secur. 15, 3013–3022 (2020)
Xing, L., Qiao, Y.: DeepWriter: a multi-stream deep CNN for text-independent writer identification. In: Proceedings of the 15th International Conference on Frontiers in Handwriting Recognition, pp. 584–589 (2016)
Nguyen, H.T., Nguyen, C.T., Ino, T., Indurkhya, B., Nakagawa, M.: Text-independent writer identification using convolutional neural network. Pattern Recognit. Lett. 121, 104–112 (2019)
Tang, Y., Wu, X.: Text-independent writer identification via CNN features and joint Bayesian. In: Proceedings of the 15th International Conference on Frontiers in Handwriting Recognition, pp. 566–571 (2016)
Christlein, V., Gropp, M., Fiel, S., Maier, A.: Unsupervised feature learning for writer identification and writer retrieval. In: Proceedings of the 14th International Conference on Document Analysis and Recognition, pp. 991–997 (2017)
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1437–1451 (2018)
Bui, T., Ribeiro, L., Ponti, M., Collomosse, J.: Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network. Comput. Vis. Image Underst. 164, 27–37 (2017)
Aggarwal, C.C., Hinneburg, A., Keim, D.A.: On the surprising behavior of distance metrics in high dimensional space. In: Van den Bussche, J., Vianu, V. (eds.) ICDT 2001. LNCS, vol. 1973, pp. 420–434. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44503-X_27
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the 28th IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Acknowledgement
The authors would like to thank Dr. Cuong Tuan Nguyen for his valuable comments. This research is being partially supported by the grant-in-aid for scientific research (S) 18H05221 and (A) 18H03597.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ngo, T.T., Nguyen, H.T., Nakagawa, M. (2021). A-VLAD: An End-to-End Attention-Based Neural Network for Writer Identification in Historical Documents. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12822. Springer, Cham. https://doi.org/10.1007/978-3-030-86331-9_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-86331-9_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86330-2
Online ISBN: 978-3-030-86331-9
eBook Packages: Computer ScienceComputer Science (R0)
