Skip to main content
SpringerLink
Log in

R-GRU: Regularized gated recurrent unit for handwritten mathematical expression recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Handwritten mathematical expression recognition(HMER) plays a wide variety of roles in different domains like online teaching, scientific research, etc. Due to its two-dimensional non-linear structure, it is a challenging problem. In this work, we proposed a novel architecture using regularization(dropout), attention & gated recurrent unit(R-GRU). R-GRU is used as the central decoding unit that takes the intermediate representation produced by the encoder as an input and generates corresponding LaTeX sequences. Simulation on CROHME 2014 and 2016 datasets achieve results comparable to the latest state-of-the-art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Anderson R. H. (1968) Syntax-Directed Recognition of Hand-Printed Two-Dimensional mathematics. In:

  2. Awal AM, Mouchére H, Viard-Gaudin C (2012) A global learning approach for an online handwritten mathematical expression recognition system. In: Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2012.10.024

  3. Álvaro F, Sánchez J-A, Benedí J-M (2014) Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. In: Pattern Recognition Letters, vol 35. https://doi.org/10.1016/j.patrec.2012.09.023

  4. Álvaro F, Sánchez J-A, Benedí J-M (2015) An Integrated Grammar-based Approach for Mathematical Expression Recognition Recognition. In: Pattern Recognition, vol 51. https://doi.org/10.1016/j.patcog.2015.09.013

  5. Deng Y, Kanervisto A, Rush AM (2016) What You Get Is What You See: A Visual Markup Decompiler. In: CoRR, . arXiv:1609.04938

  6. Duc AL, Nakagawa M (2017) Training an End-to-End System for Handwritten Mathematical Expression Recognition by Generated Patterns. In: https://doi.org/10.1109/ICDAR.2017.175

  7. Garain U, Chaudhuri B (2005) Recognition of Online Handwritten Mathematical Expressions. In: IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publi- cation of the IEEE Systems, Man, and Cyber- netics Society, vol 34, pp 2366–76. https://doi.org/10.1109/TSMCB.2004.836817

  8. Huang G et al (2017) Densely Connected Convolutional Networks. In: https://doi.org/10.1109/CVPR.2017.243

  9. Le AD (2020) Recognizing handwritten mathematical expressions via paired dual loss attention network and printed mathematical expressions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

  10. Moon T et al (2015) RnnDrop: a novel dropout for RNNs in ASR. In: pp 65–70. https://doi.org/10.1109/ASRU.2015.7404775

  11. Mouchére H et al (2014) ICFHR 2014 Competition on Recognition of On-line Handwritten Mathematical Expressions (CROHME 2014). In: Proceedings of International Conference on Frontiers in Handwriting Recognition, ICFHR 2014. https://doi.org/10.1109/ICFHR.2014.42

  12. Mouchére H et al (2014) ICFHR 2014 Competition on Recognition of On-line Handwritten Mathematical Expressions (CROHME 2014). In: Proceedings of International Conference on Frontiers in Handwriting Recognition, ICFHR 2014. https://doi.org/10.1109/ICFHR.2014.42

  13. Mouchére H et al (2016) ICFHR2016 CROHME: Competition on recognition of online handwritten mathematical expressions. In: pp 607–612. https://doi.org/10.1109/ICFHR.2016.0116

  14. Vaswani A et al (2017) Attention Is All You Need. In: CoRR arXiv:1706.03762

  15. Wang H, Shan G (2020) Recognizing handwritten mathematical expressions as latex sequences using a multiscale robust neural network. In:

  16. Wang J et al (2019) Multi-modal Attention Network for Handwritten Mathematical Expression Recognition. In: pp 1181–1186. https://doi.org/10.1109/ICDAR.2019.00191

  17. Wu J-W et al (2019) Image-to-Markup Generation via Paired Adversarial Learning: Recognizing Outstanding Ph.D. Research. In: pp 18–34. https://doi.org/10.1007/978-3-030-10925-72

  18. Wu J-W et al (2020) Handwritten Mathematical Expression Recognition via Paired Adversarial Learning. In: International Journal of Computer Vision. https://doi.org/10.1007/s11263-020-01291-5

  19. Yamamoto R et al (2006) On-Line Recognition of handwritten mathematical expressions based on Stroke-Based stochastic Context-Free grammar. In:

  20. Zanibbi R, Blostein D, Cordy J (2002) Recognizing mathematical expressions using tree transformation. IEEE Trans Pattern Anal Mach Intell. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 24, pp 1455–1467. https://doi.org/10.1109/TPAMI.2002.1046157

  21. Zaremba W, Sutskever I, Vinyals O (2014) Recurrent neural network regularization. In:

  22. Zhang J, Du J, Dai L (2018) Multiscale attention with dense encoder for handwritten mathematical expression recognition. In: In- ternational Conference on Pattern Recognition, pp 2245–2250

  23. Zhang J et al (2017) Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition. In: Pattern Recognition, vol 71, pp 196–206

Download references

Acknowledgements

We would like to thank anonymous reviewers for their insightful comments and suggestions. Also, we would like to thank MoE, Government of India and, IIIT Allahabad for providing research funds and support.

Author information

Authors and Affiliations

  1. Machine Learning & Optimization Lab, Department of Information Technology, Indian Institute of Information Technology, Allahabad, Prayagraj, India

    Aniket Pal & Krishna Pratap Singh

Authors
  1. Aniket Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krishna Pratap Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aniket Pal.

Ethics declarations

Conflict of Interests

There is no conflict of interest from any of these authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pal, A., Singh, K.P. R-GRU: Regularized gated recurrent unit for handwritten mathematical expression recognition. Multimed Tools Appl 81, 31405–31419 (2022). https://doi.org/10.1007/s11042-022-12889-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12889-x

Keywords

Search

Navigation