Skip to main content

End-to-End Approach for Recognition of Historical Digit Strings

  • Conference paper
  • First Online:
Document Analysis and Recognition – ICDAR 2021 (ICDAR 2021)

Abstract

The plethora of digitalised historical document datasets released in recent years has rekindled interest in advancing the field of handwriting pattern recognition. In the same vein, a recently published data set, known as ARDIS, presents handwritten digits manually cropped from 15.000 scanned documents of Swedish churches’ books that exhibit various handwriting styles. To this end, we propose an end-to-end segmentation- free deep learning approach to handle this challenging ancient handwriting style of dates present in the ARDIS dataset (4-digits long strings). We show that with slight modifications in the VGG-16 deep model, the framework can achieve a recognition rate of 93.2%, resulting in a feasible solution free of heuristic methods, segmentation, and fusion methods. Moreover, the proposed approach outperforms the well-known CRNN method (a model widely applied in handwriting recognition tasks).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://ardisdataset.github.io/ARDIS/.

References

  1. Almazan, J., Gordo, A., Fornés, A., Valveny, E.: Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2552–2566 (2014). https://doi.org/10.1109/TPAMI.2014.2339814

    Article  Google Scholar 

  2. Aly, S., Mohamed, A.: Unknown-length handwritten numeral string recognition using cascade of PCA-SVMNET classifiers. IEEE Access 7, 52024–52034 (2019). https://doi.org/10.1109/ACCESS.2019.2911851

    Article  Google Scholar 

  3. Cecotti, H.: Active graph based semi-supervised learning using image matching: application to handwritten digit recognition. Pattern Recogn. Lett. 73, 76–82 (2016)

    Article  Google Scholar 

  4. Cheddad, A.: Towards query by text example for pattern spotting in historical documents. In: 2016 7th International Conference on Computer Science and Information Technology (CSIT), pp. 1–6 (2016). https://doi.org/10.1109/CSIT.2016.7549479

  5. Cheng, S., Shang, G., Zhang, L.: Handwritten digit recognition based on improved vgg16 network. In: Tenth International Conference on Graphics and Image Processing (ICGIP 2018), vol. 11069, p. 110693B. International Society for Optics and Photonics (2019)

    Google Scholar 

  6. Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Conference on Empirical Methods in Natural Language Processing (EMNLP 2014) (2014)

    Google Scholar 

  7. Choi, S.M., Oh, I.S.: A segmentation-free recognition of handwritten touching numeral pairs using modular neural network. Int. J. Pattern Recogn. Artif. Intell. 15(06), 949–966 (2001)

    Article  Google Scholar 

  8. Cilia, N., De Stefano, C., Fontanella, F., Marrocco, C., Molinara, M., Di Freca, A.S.: An end-to-end deep learning system for medieval writer identification. Pattern Recogn. Lett. 129, 137–143 (2020). https://doi.org/10.1016/j.patrec.2019.11.025, https://www.sciencedirect.com/science/article/pii/S0167865519303460

  9. Cimpoi, M., Maji, S., Vedaldi, A.: Deep filter banks for texture recognition and segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3828–3836 (2015). https://doi.org/10.1109/CVPR.2015.7299007

  10. Ciresan, D.: Avoiding segmentation in multi-digit numeral string recognition by combining single and two-digit classifiers trained without negative examples. In: 2008 10th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, pp. 225–230. IEEE (2008)

    Google Scholar 

  11. Diem, M., et al.: ICFHR 2014 competition on handwritten digit string recognition in challenging datasets (HDSRC 2014). In: 2014 14th International Conference on Frontiers in Handwriting Recognition, pp. 779–784. IEEE (2014)

    Google Scholar 

  12. Dutta, K., Krishnan, P., Mathew, M., Jawahar, C.V.: Improving CNN-RNN hybrid networks for handwriting recognition. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 80–85, August 2018. https://doi.org/10.1109/ICFHR-2018.2018.00023

  13. Gao, B.B., Wei, X.S., Wu, J., Lin, W.: Deep spatial pyramid: the devil is once again in the details. arXiv preprint arXiv:1504.05277 (2015)

  14. Gao, Y., Mishchenko, Y., Shah, A., Matsoukas, S., Vitaladevuni, S.: Towards data-efficient modeling for wake word spotting. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7479–7483 (2020). https://doi.org/10.1109/ICASSP40776.2020.9053313

  15. Ghodrati, A., Diba, A., Pedersoli, M., Tuytelaars, T., Gool, L.V.: DeepProposals: hunting objects and actions by cascading deep convolutional layers. Int. J. Comput. Vis. 124(2), 115–131 (2017). https://doi.org/10.1007/s11263-017-1006-x

    Article  Google Scholar 

  16. Gilloux, M.: Document analysis in postal applications and check processing. In: Doermann, D., Tombre, K. (eds.) Handbook of Document Image Processing and Recognition, pp. 705–747. Springer, London (2014). https://doi.org/10.1007/978-0-85729-859-1_22

    Chapter  Google Scholar 

  17. Hochuli, A.G., Britto, A.S., Barddal, J.P., Sabourin, R., Oliveira, L.E.S.: An end-to-end approach for recognition of modern and historical handwritten numeral strings. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2020). https://doi.org/10.1109/IJCNN48605.2020.9207468

  18. Hochuli, A.G., Britto, A.S., Jr., Saji, D.A., Saavedra, J.M., Sabourin, R., Oliveira, L.S.: A comprehensive comparison of end-to-end approaches for handwritten digit string recognition. Expert Syst. Appl. 165, 114196 (2021)

    Google Scholar 

  19. Hochuli, A.G., Oliveira, L.S., Britto, A., Jr., Sabourin, R.: Handwritten digit segmentation: Is it still necessary? Pattern Recogn. 78, 1–11 (2018)

    Article  Google Scholar 

  20. Kusetogullari, H., Yavariabdi, A., Cheddad, A., Grahn, H., Hall, J.: ARDIS: a Swedish historical handwritten digit dataset. Neural Comput. Appl. 32, 16505–16518 (2019)

    Article  Google Scholar 

  21. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  22. Matan, O., Burges, C.J., LeCun, Y., Denker, J.S.: Multi-digit recognition using a space displacement neural network. In: Advances in Neural Information Processing Systems, pp. 488–495 (1992)

    Google Scholar 

  23. Neto, A.F.D.S., Bezerra, B.L.D., Lima, E.B., Toselli, A.H.: HDSR-Flor: a robust end-to-end system to solve the handwritten digit string recognition problem in real complex scenarios. IEEE Access 8, 208543–208553 (2020)

    Article  Google Scholar 

  24. Neudecker, C., et al.: OCR-D: an end-to-end open source OCR framework for historical printed documents. In: Proceedings of the 3rd International Conference on Digital Access to Textual Cultural Heritage, pp. 53–58. DATeCH2019, Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3322905.3322917

  25. Palm, R.B., Laws, F., Winther, O.: Attend, copy, parse end-to-end information extraction from documents. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 329–336 (2019). https://doi.org/10.1109/ICDAR.2019.00060

  26. Ribas, F.C., Oliveira, L., Britto, A., Sabourin, R.: Handwritten digit segmentation: a comparative study. Int. J. Doc. Anal. Recogn. (IJDAR) 16(2), 127–137 (2013)

    Article  Google Scholar 

  27. Roy, K., Vajda, S., Pal, U., Chaudhuri, B.B., Belaid, A.: A system for Indian postal automation. In: Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), vol. 2, pp. 1060–1064 (2005).https://doi.org/10.1109/ICDAR.2005.259

  28. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sig. Process. 45(11), 2673–2681 (1997). https://doi.org/10.1109/78.650093

    Article  Google Scholar 

  29. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2017). https://doi.org/10.1109/TPAMI.2016.2646371

  30. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2016)

    Article  Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  32. Vellasques, E., Oliveira, L.S., Britto, A., Jr., Koerich, A.L., Sabourin, R.: Filtering segmentation cuts for digit string recognition. Pattern Recogn. 41(10), 3044–3053 (2008)

    Article  Google Scholar 

  33. Voigtlaender, P., Doetsch, P., Ney, H.: Handwriting recognition with large multidimensional long short-term memory recurrent neural networks. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 228–233, October 2016. https://doi.org/10.1109/ICFHR.2016.0052

  34. Wei, X.S., Luo, J.H., Wu, J., Zhou, Z.H.: Selective convolutional descriptor aggregation for fine-grained image retrieval. Trans. Img. Proc. 26(6), 2868–2881 (2017)

    Article  MathSciNet  Google Scholar 

  35. Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, pp. 2082–2090. Curran Associates Inc., Red Hook (2016)

    Google Scholar 

Download references

Acknowledgments

This project is supported by the research project “DocPRESERV: Preserving and Processing Historical Document Images with Artificial Intelligence”, STINT, the Swedish Foundation for International Cooperation in Research and Higher Education (Grant: AF2020-8892).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Abbas Cheddad .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, M., Hochuli, A.G., Cheddad, A. (2021). End-to-End Approach for Recognition of Historical Digit Strings. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12823. Springer, Cham. https://doi.org/10.1007/978-3-030-86334-0_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86334-0_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86333-3

  • Online ISBN: 978-3-030-86334-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics