Skip to main content

A Transcription Is All You Need: Learning to Align Through Attention

  • 1353 Accesses

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12916)

Abstract

Historical ciphered manuscripts are a type of document where graphical symbols are used to encrypt their content instead of regular text. Nowadays, expert transcriptions can be found in libraries alongside the corresponding manuscript images. However, those transcriptions are not aligned, so these are barely usable for training deep learning-based recognition methods. To solve this issue, we propose a method to align each symbol in the transcript of an image with its visual representation by using an attention-based Sequence to Sequence (Seq2Seq) model. The core idea is that, by learning to recognise symbols sequence within a cipher line image, the model also identifies their position implicitly through an attention mechanism. Thus, the resulting symbol segmentation can be later used for training algorithms. The experimental evaluation shows that this method is promising, especially taking into account the small size of the cipher dataset.

Keywords

  • Handwritten symbol alignment
  • Hand-drawn symbol recognition
  • Sequence to Sequence
  • Attention models

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-86198-8_11
  • Chapter length: 6 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-86198-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

Notes

  1. 1.

    https://cl.lingfil.uu.se/~bea/copiale/.

References

  1. Baró, A., Chen, J., Fornés, A., Megyesi, B.: Towards a generic unsupervised method for transcription of encoded manuscripts. In: DATeCH, pp. 73–78 (2019)

    Google Scholar 

  2. Ezra, D.S.B., Brown-DeVost, B., Dershowitz, N., Pechorin, A., Kiessling, B.: Transcription alignment for highly fragmentary historical manuscripts: the dead sea scrolls. In: ICFHR, pp. 361–366 (2020)

    Google Scholar 

  3. Fischer, A., Frinken, V., Fornés, A., Bunke, H.: Transcription alignment of Latin manuscripts using hidden Markov models. In: HIP, pp. 29–36 (2011)

    Google Scholar 

  4. Kang, L., Toledo, J.I., Riba, P., Villegas, M., Fornés, A., Rusiñol, M.: Convolve, attend and spell: an attention-based sequence-to-sequence model for handwritten word recognition. In: Brox, T., Bruhn, A., Fritz, M. (eds.) GCPR 2018. LNCS, vol. 11269, pp. 459–472. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-12939-2_32

    CrossRef  Google Scholar 

  5. Kassis, M., Nassour, J., El-Sana, J.: Alignment of historical handwritten manuscripts using Siamese neural network. In: ICDAR, vol. 1, pp. 293–298 (2017)

    Google Scholar 

  6. Megyesi, B., et al.: Decryption of historical manuscripts: the decrypt project. Cryptologia 44(6), 545–559 (2020)

    CrossRef  Google Scholar 

  7. Riba, P., Fornés, A., Lladós, J.: Towards the alignment of handwritten music scores. In: GREC, pp. 103–116 (2015)

    Google Scholar 

  8. Romero-Gómez, V., Toselli, A.H., Bosch, V., Sánchez, J.A., Vidal, E.: Automatic alignment of handwritten images and transcripts for training handwritten text recognition systems. In: DAS, pp. 328–333 (2018)

    Google Scholar 

  9. Souibgui, M.A., Fornés, A., Kessentini, Y., Tudor, C.: A few-shot learning approach for historical ciphered manuscript recognition. In: ICPR, pp. 5413–5420 (2021)

    Google Scholar 

Download references

Acknowledgement

This work has been supported by the Swedish Research Council, grant 2018-06074, DECRYPT – Decryption of Historical Manuscripts, the Spanish project RTI2018-095645-B-C21 and the CERCA Program / Generalitat de Catalunya.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Pau Torras .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Torras, P., Souibgui, M.A., Chen, J., Fornés, A. (2021). A Transcription Is All You Need: Learning to Align Through Attention. In: Barney Smith, E.H., Pal, U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science(), vol 12916. Springer, Cham. https://doi.org/10.1007/978-3-030-86198-8_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86198-8_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86197-1

  • Online ISBN: 978-3-030-86198-8

  • eBook Packages: Computer ScienceComputer Science (R0)