Skip to main content

Strikethrough Removal from Handwritten Words Using CycleGANs

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12824)

Abstract

Obtaining the original, clean forms of struck-through handwritten words can be of interest to literary scholars, focusing on tasks such as genetic criticism. In addition to this, replacing struck-through words can also have a positive impact on text recognition tasks. This work presents a novel unsupervised approach for strikethrough removal from handwritten words, employing cycle-consistent generative adversarial networks (CycleGANs). The removal performance is improved upon by extending the network with an attribute-guided approach. Furthermore, two new datasets, a synthetic multi-writer set, based on the IAM database, and a genuine single-writer dataset, are introduced for the training and evaluation of the models. The experimental results demonstrate the efficacy of the proposed method, where the examined attribute-guided models achieve \(F_1\) scores above 0.8 on the synthetic test set, improving upon the performance of the regular CycleGAN. Despite being trained exclusively on the synthetic dataset, the examined models even produce convincing cleaned images for genuine struck-through words.

Keywords

  • Strikethrough removal
  • CycleGAN
  • Handwritten words
  • Document image processing

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-86337-1_38
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-86337-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

References

  1. Adak, C., Chaudhuri, B.B., Blumenstein, M.: Impact of struck-out text on writer identification. In: IJCNN, pp. 1465–1471 (2017)

    Google Scholar 

  2. Almahairi, A., Rajeshwar, S., Sordoni, A., Bachman, P., Courville, A.: Augmented CycleGAN: learning many-to-many mappings from unpaired data. In: ICML, pp. 195–204 (2018). http://proceedings.mlr.press/v80/almahairi18a.html

  3. Brink, A., van der Klauw, H., Schomaker, L.: Automatic removal of crossed-out handwritten text and the effect on writer verification and identification. In: Document Recognition and Retrieval XV, vol. 6815, pp. 79–88. SPIE (2008). https://doi.org/10.1117/12.766466

  4. Chang, B., Zhang, Q., Pan, S., Meng, L.: Generating Handwritten Chinese Characters Using CycleGAN. In: WACV, pp. 199–207 (2018)

    Google Scholar 

  5. Chang, H., Lu, J., Yu, F., Finkelstein, A.: PairedCycleGAN: asymmetric style transfer for applying and removing makeup. In: CVPR, June 2018

    Google Scholar 

  6. Chaudhuri, B.B., Adak, C.: An approach for detecting and cleaning of struck-out handwritten text. Pattern Recogn. 61, 282–294 (2017). https://doi.org/10.1016/j.patcog.2016.07.032

    CrossRef  Google Scholar 

  7. Dheemanth Urs, R., Chethan, H.K.: A study on identification and cleaning of struck-out words in handwritten documents. In: Jeena Jacob, I., Kolandapalayam Shanmugam, S., Piramuthu, S., Falkowski-Gilski, P. (eds.) Data Intelligence and Cognitive Informatics. AIS, pp. 87–95. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-8530-2_6

    CrossRef  Google Scholar 

  8. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR, July 2017

    Google Scholar 

  9. Hulle, D.V.: The stuff of fiction: digital editing, multiple drafts and the extended mind. Text. Cult. 8(1), 23–37 (2013). http://www.jstor.org/stable/10.2979/textcult.8.1.23

  10. Jung, A.B., et al.: imgaug (2020). https://github.com/aleju/imgaug. Accessed 15 May 2021

  11. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  12. Likforman-Sulem, L., Vinciarelli, A.: Hmm-based offline recognition of handwritten words crossed out with different kinds of strokes (2008). http://eprints.gla.ac.uk/59027/

  13. Lu, Y., Tai, Y.-W., Tang, C.-K.: Attribute-guided face generation using conditional CycleGAN. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11216, pp. 293–308. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01258-8_18

    CrossRef  Google Scholar 

  14. Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. IJDAR 5(1), 39–46 (2002). https://doi.org/10.1007/s100320200071

    CrossRef  MATH  Google Scholar 

  15. Nisa, H., Thom, J.A., Ciesielski, V., Tennakoon, R.: A deep learning approach to handwritten text recognition in the presence of struck-out text. In: IVCNZ, pp. 1–6 (2019)

    Google Scholar 

  16. Papandreou, A., Gatos, B.: Slant estimation and core-region detection for handwritten Latin words. Pattern Recogn. Lett. 35, 16–22 (2014). https://doi.org/10.1016/j.patrec.2012.08.005

    CrossRef  Google Scholar 

  17. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS, pp. 8024–8035. Curran Associates, Inc. (2019)

    Google Scholar 

  18. Sharma, M., Verma, A., Vig, L.: Learning to clean: A GAN perspective. In: Carneiro, G., You, S. (eds.) ACCV 2018. LNCS, vol. 11367, pp. 174–185. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21074-8_14

    CrossRef  Google Scholar 

  19. Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR (2003). https://doi.org/10.1109/ICDAR.2003.1227801

  20. Vats, E., Hast, A., Singh, P.: Automatic document image binarization using Bayesian optimization. In: HIP, p. 89–94 (2017). https://doi.org/10.1145/3151509.3151520

  21. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV, October 2017

    Google Scholar 

  22. Öfverstedt, J., Lindblad, J., Sladoje, N.: Fast and robust symmetric image registration based on distances combining intensity and spatial information. EEE Trans. Image Process. 28(7), 3584–3597 (2019). https://doi.org/10.1109/TIP.2019.2899947

    MathSciNet  CrossRef  MATH  Google Scholar 

Download references

Acknowledgements

R.Heil would like to thank Nicolas Pielawski, Håkan Wieslander, Johan Öfverstedt and Anders Brun for their helpful comments and fruitful discussions. The computations were enabled by resources provided by the Swedish National Infrastructure for Computing (SNIC) at the High Performance Computing Center North (HPC2N) partially funded by the Swedish Research Council through grant agreement no. 2018-05973. This work is partially supported by the Riksbankens Jubileumsfond (RJ) (Dnr P19-0103:1).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Raphaela Heil .

Editor information

Editors and Affiliations

A Dataset and Code Availability

A Dataset and Code Availability

Synthetic strikethrough dataset: https://doi.org/10.5281/zenodo.4767094. Strikethrough generation code: https://doi.org/10.5281/zenodo.4767063. Genuine strikethrough dataset: https://doi.org/10.5281/zenodo.4765062. Deep learning code and checkpoints: https://doi.org/10.5281/zenodo.4767168.

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Heil, R., Vats, E., Hast, A. (2021). Strikethrough Removal from Handwritten Words Using CycleGANs. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12824. Springer, Cham. https://doi.org/10.1007/978-3-030-86337-1_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86337-1_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86336-4

  • Online ISBN: 978-3-030-86337-1

  • eBook Packages: Computer ScienceComputer Science (R0)