Skip to main content
SpringerLink
Log in

Neural machine translation for Indian language pair using hybrid attention mechanism

  • S.I. : Multifaceted Intelligent Computing Systems (MICS)
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

Machine translation is an effective way of achieving suitable translations for a given language pair without human intervention. Machine translation can be a very tedious task when the resources for the concerned language pair are low, and the English–Assamese language pair is one such. Neural machine translation is among the different machine translation techniques that use artificial neural networks to learn the nonlinear relationship between bilingual sentence pairs. In this paper, a neural machine translation model is implemented, which uses our proposed hybrid attention mechanism. This hybrid attention mechanism consists of additive and multiplicative attention mechanisms. The proposed attention mechanism is not only capable of selecting an optimal attention mechanism, but also gives the average loss value per epoch to be minimal. The model with our proposed hybrid attention mechanism outperforms the other two models with additive and multiplicative attention by achieving a BLEU score of 27.20 on English–Assamese and a BLEU score of 25.88 on the Assamese–English language pair.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Allen RB (1987) Several studies on natural language and back-propagation. In: Proceedings of the IEEE first international conference on neural networks, IEEE 2, pp 335–341

  2. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. In: Proceedings of the 2014 international conference on learning representations

  3. Bengio Y, Ducharme R, Vincent P, Jauvin C (2003) A neural probabilistic language model. J Mach Learn Res 3:1137–1155

    MATH  Google Scholar 

  4. Brown PF, Cocke J, Pietra SAD, Pietra VJD, Jelinek F, Mercer RL, Lafferty J, Roossin PS (1990) A statistical approach to machine translation. Comput Linguist 16(2):79–85

    Google Scholar 

  5. Cho K, Merrienboer BV, Bahdanau D, Bengio Y (2014) Learning phase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). https://doi.org/10.3115/v1/d14-1179

  6. Cho K, Merrienboer BV, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of SSST-8, eighth workshop on syntax, semantics and structure in statistical translation. https://doi.org/10.3115/v1/w14-4012

  7. Choudhary H, Rao S, Rohilla R (2018) Neural machine translation for English-Tamil. In: Proceedings of the third conference on machine translation: shared task papers. https://doi.org/10.1145/3140107.3140111

  8. Choudhary H, Rao S, Rohilla R (2020) Neural machine translation for low resourced languages. In: Proceedings of The 12th Language resources and evaluation conference, LREC 2020, Marseille, France

  9. Christopher MTLHP, Manning D (2015) Effective approaches to attention based neural machine translation. In: Proceedings of the 2015 conference on empirical methods in natural language processing. https://doi.org/10.18653/v1/d15-1166

  10. Das A, Yerra P, Kumar K, Sarkar S (2016) A study of attention-based neural machine translation models on Indian languages. In: Proceedings of the 6th workshop on south and southeast asian natural language processing, pp 163–172

  11. Firat O, Cho K, Bengio Y (2016) Multi-way, multilingual neural machine. Translation with a shared attention mechanism. In: Proceedings of the 2016 conference of the North American Chapter of the Association for 17 computational linguistic: human language technologies. https://doi.org/10.1016/j.csl.2016.10.006

  12. Johnson M, Schuster M, Le QV, Krikun M, Wu Y, Chen Z, Thorat N, Viégas F, Wattenberg M, Corrado G, Hughes M, Dean J (2017) Google’ s multilingual neural machine translation system: enabling zero-shot translation. Trans Assoc Comput Linguist 5:339–351

    Article  Google Scholar 

  13. Narayan R, Chakraverty S, Singh VP (2016) Quantum neural network based machine translator for English to Hindi. Appl Soft Comput 38(1060):1075. https://doi.org/10.1016/j.asoc.2015.08.031

    Article  Google Scholar 

  14. Papneni K, Roukos S, Ward T, Zhu, W (2002) BLEU: a method for automatic evaluation for machine translation. In: Proceedings of the 40th annual meeting of the association for computational linguistics (ACL). https://doi.org/10.3115/1073083.1073135

  15. Sennrich R, Haddow B, Birch A (2016) Neural machine translation of rare words with subword units. In: Proceedings of the 54th annual meeting of the association for computational linguistics, 1. https://doi.org/10.18653/v1/p16-1162

  16. Shah P, Bakarola V (2019) Neural machine translation system of Indic language-attention based approach. In: Second international conference on advanced computational and communication paradigms. https://doi.org/10.1109/ICACCP.2019.8882969

  17. Singh S, Anand Kumar M, Soman KP (2018) Attention based English to Punjabi neural machine translation. J Intell Fuzzy Syst 34(3):1551–1559. https://doi.org/10.3233/jifs-169450

    Article  Google Scholar 

  18. Singh M, Kumar R, Chana I (2020) Corpus based machine translation system with deep neural network for Sanskrit to Hindi Translation. Procedia Comput Sci 167:2534–2544. https://doi.org/10.1016/j.procs.2020.03.306

  19. Sutskever I, Vinyals O, Le Q (2014) Sequence to sequence learning with neural networks. Adv Neural Inf Process Syst 27:3104–3112

    Google Scholar 

  20. Verma C, Singh A, Seal S, Singh VMI (2019) Hindi-English neural machine translation using attention model. Int J Sci Technol Res 8:11–11

  21. Weaver W (1949) The mathematics of communication. Sci Am 181(1):11–15

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Assam University, Silchar, India

    Basab Nath, Sunita Sarkar, Surajeet Das & Somnath Mukhopadhyay

Authors
  1. Basab Nath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sunita Sarkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Surajeet Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Somnath Mukhopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nath, B., Sarkar, S., Das, S. et al. Neural machine translation for Indian language pair using hybrid attention mechanism. Innovations Syst Softw Eng (2022). https://doi.org/10.1007/s11334-021-00429-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11334-021-00429-z

Keywords

Search

Navigation