Skip to main content

Scalable Handwritten Text Recognition System for Lexicographic Sources of Under-Resourced Languages and Alphabets

  • 945 Accesses

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12742)

Abstract

The paper discusses an approach to decipher large collections of handwritten index cards of historical dictionaries. Our study provides a working solution that reads the cards, and links their lemmas to a searchable list of dictionary entries, for a large historical dictionary entitled the Dictionary of the 17\(^{th}\)- and 18\(^{th}\)-century Polish, which comprizes 2.8 million index cards. We apply a tailored handwritten text recognition (HTR) solution that involves (1) an optimized detection model; (2) a recognition model to decipher the handwritten content, designed as a spatial transformer network (STN) followed by convolutional neural network (RCNN) with a connectionist temporal classification layer (CTC), trained using a synthetic set of 500,000 generated Polish words of different length; (3) a post-processing step using constrained Word Beam Search (WBC): the predictions were matched against a list of dictionary entries known in advance. Our model achieved the accuracy of 0.881 on the word level, which outperforms the base RCNN model. Within this study we produced a set of 20,000 manually annotated index cards that can be used for future benchmarks and transfer learning HTR applications.

Keywords

  • Handwritten text recognition
  • Index cards archives
  • Lexicography
  • Neural network
  • Convolutional neural network
  • Recurrent neural network
  • Connectionist temporal classification
  • Keras OCR
  • ResNet
  • Spatial transformer networks
  • Synthetic dataset

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-77961-0_13
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-77961-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Baek, J., et al.: What is wrong with scene text recognition model comparisons? dataset and model analysis. In: International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  2. Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9365–9374 (2019)

    Google Scholar 

  3. Bilińska-Brynk, J., Rodek, E.: Paper quotation slips to the Electronic Dictionary of the 17th-and 18th-Century Polish - digital index and its integration with the Dictionary. In: EURALEX XIX Proceedings, pp. 465–470 (2020)

    Google Scholar 

  4. Bronikowska, R., Majdak, M., Wieczorek, A., Żółtak, M.: The Electronic Dictionary of the 17th-and 18th-century Polish - towards the open formula asset of the historical vocabulary. In: EURALEX XIX Proceedings pp. 471–475 (2020)

    Google Scholar 

  5. Chu, W.: Text renderer (2021). https://github.com/Sanster/text_renderer

  6. Doetsch, P., Kozielski, M., Ney, H.: Fast and robust training of Recurrent Neural Networks for offline handwriting recognition. In: 2014 14th International Conference on Frontiers in Handwriting Recognition, pp. 279–284 (2014). https://doi.org/10.1109/ICFHR.2014.54

  7. Farra, N., Tomeh, N., Rozovskaya, A., Habash, N.: Generalized character-level spelling error correction. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. vol. 2, pp. 161–167. Association for Computational Linguistics, Baltimore, Maryland (2014). https://doi.org/10.3115/v1/P14-2027. http://aclweb.org/anthology/P14-2027

  8. Franzini, G., et al.: Attributing authorship in the noisy digitized correspondence of Jacob and Wilhelm Grimm. Frontiers Digital Humanities 5 (2018). https://doi.org/10.3389/fdigh.2018.00004

  9. Graves, A., Fernández, S., Schmidhuber, J.: Multi-dimensional recurrent neural networks. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds.) ICANN 2007. LNCS, vol. 4668, pp. 549–558. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74690-4_56

    CrossRef  Google Scholar 

  10. Graves, A., Schmidhuber, J.: Offline handwriting recognition with multidimensional Recurrent Neural Networks. Adv. Neural Inf. Process. Syst. 21, 545–552 (2008). https://proceedings.neurips.cc/paper/2008/hash/66368270ffd51418ec58bd793f2d9b1b-Abstract.html

  11. Grzelak, D., Podlaski, K., Wiatrowski, G.: Analyze the effectiveness of an algorithm for identifying Polish characters in handwriting based on neural machine learning technologies. J. King Saud University - Comput. Inf. Sci. (2019). https://doi.org/10.1016/j.jksuci.2019.08.001

    CrossRef  Google Scholar 

  12. Gupta, A., Vedaldi, A., Zisserman, A.: Synthetic data for text localisation in natural images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2315–2324 (2016) https://openaccess.thecvf.com/content_cvpr_2016/html/Gupta_Synthetic_Data_for_CVPR_2016_paper.html

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015) http://arxiv.org/abs/1512.03385

  14. Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Synthetic data and artificial neural networks for natural scene text recognition (2014). http://arxiv.org/abs/1406.2227

  15. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial Transformer Networks (2016), http://arxiv.org/abs/1506.02025

  16. Kahle, P., Colutto, S., Hackl, G., Mühlberger, G.: Transkribus: A service platform for transcription, recognition and retrieval of historical documents. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 04, pp. 19–24 (2007). https://doi.org/10.1109/ICDAR.2017.307

  17. Krishnan, P., Jawahar, C.V.: Matching handwritten document images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 766–782. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_46

    CrossRef  Google Scholar 

  18. Landau, S.I.: Dictionaries: The art and craft of lexicography. Cambridge University Press, 2 edn. (2001)

    Google Scholar 

  19. Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recogn. 5(1), 39–46 (2002). https://doi.org/10.1007/s100320200071

    CrossRef  MATH  Google Scholar 

  20. Muehlberger, G., et al.: Transforming scholarship in the archives through handwritten text recognition: Transkribus as a case study. J. Documentation 75(5), 954–976 (2019). https://doi.org/10.1108/JD-07-2018-0114

    CrossRef  Google Scholar 

  21. Ogrodniczuk, M., Gruszczyński, W.: Connecting Data for Digital Libraries: The Library, the Dictionary and the Corpus. In: Jatowt, A., Maeda, A., Syn, S.Y. (eds.) ICADL 2019. LNCS, vol. 11853, pp. 125–138. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34058-2_13

    CrossRef  Google Scholar 

  22. Pal, A., Singh, D.: Handwritten English character recognition using neural network. Int. J. Comput. Sci. Commun. 1(2), 141–144 (2010)

    Google Scholar 

  23. Rigaud, C., Doucet, A., Coustaty, M., Moreux, J.P.: ICDAR 2019 competition on Post-OCR text correction. 15th International Conference on Document Analysis and Recognition, pp. 1588–1593 (2019). https://hal.archives-ouvertes.fr/hal-02304334/document

  24. Scheidl, H., Fiel, S., Sablatnig, R.: Word beam search: a connectionist temporal classification decoding algorithm. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 253–258 (2018). https://doi.org/10.1109/ICFHR-2018.2018.00052

  25. Sánchez, J.A., Romero, V., Toselli, A.H., Vidal, E.: Icfhr 2014 competition on handwritten text recognition on Transcriptorium datasets (HTRtS). In: 2014 14th International Conference on Frontiers in Handwriting Recognition, pp. 785–790 (2014). https://doi.org/10.1109/ICFHR.2014.137

  26. Sánchez, J.A., Romero, V., Toselli, A.H., Vidal, E.: Icfhr 2016 competition on handwritten text recognition on the READ dataset. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR) pp. 630–635 (2016). https://doi.org/10.1109/ICFHR.2016.0120

  27. Voigtlaender, P., Doetsch, P., Ney, H.: Handwriting recognition with large multidimensional long short-term memory recurrent neural networks. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 228–233 (2016). https://doi.org/10.1109/ICFHR.2016.0052

  28. Xiao, S., Peng, L., Yan, R., Wang, S.: Deep network with pixel-level rectification and robust training for handwriting recognition. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 9–16 (2019). https://doi.org/10.1109/ICDAR.2019.00012

  29. Yin, X., Yin, X., Huang, K., Hao, H.: Robust text detection in natural scene images. IEEE Trans. Pattern Anal. Mach. Intell. 36(5), 970–983 (2014). https://doi.org/10.1109/TPAMI.2013.182

    CrossRef  Google Scholar 

Download references

Acknowledgements

This research was partly conducted as a result of a project supported by Poland’s National Science Centre (project number UMO-2013/11/B/HS2/02795). The authors are grateful to Bartłomiej Borek for his IT support, which included setting up the server and the environment for conducting our experiments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maciej Eder .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Idziak, J., Šeļa, A., Woźniak, M., Leśniak, A., Byszuk, J., Eder, M. (2021). Scalable Handwritten Text Recognition System for Lexicographic Sources of Under-Resourced Languages and Alphabets. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12742. Springer, Cham. https://doi.org/10.1007/978-3-030-77961-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77961-0_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77960-3

  • Online ISBN: 978-3-030-77961-0

  • eBook Packages: Computer ScienceComputer Science (R0)