Skip to main content
SpringerLink
Log in

Bidirectional Encoder–Decoder Model for Arabic Named Entity Recognition

  • Research Article - Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Sequence labeling models with recurrent neural network variants, such as long short-term memory (LSTM) and gated recurrent units, show promising performance in several natural language processing tasks, such as named entity recognition (NER). Most current models use unidirectional decoders, which reason only about the past and remain limited to retaining future contexts while generating predictions. Therefore, these models suffer from their generation of unbalanced outputs. Moreover, most existing NER models utilize word embeddings for capturing similarities between words but sustain when handling previously unobserved or infrequently used words. We propose a bidirectional encoder–decoder model for addressing the problem of Arabic NER on the basis of recent work in deep learning, in which the encoder and decoder are bidirectional LSTMs. In addition to word-level embeddings, character-level embeddings are adopted, and they are combined via an embedding-level attention mechanism. Our model can dynamically determine the information that must be utilized from a word- or character-level component through this attention mechanism. Experimental results on the ANERCorp and AQMAR datasets show that the model with a bi-encoder–decoder network and embedding attention layer achieves a high F-score measure of approximately 92%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://users.dsic.upv.es/~ybenajiba/downloads.html.

  2. http://curtis.ml.cmu.edu/w/courses/index.php/ANERcorp.

  3. http://www.ark.cs.cmu.edu/AQMAR/.

References

  1. Shaalan, K.: A survey of Arabic named entity recognition and classification. Comput. Linguist. 40(2), 469–510 (2014)

    Article  Google Scholar 

  2. Nydell, M.K.: Understanding Arabs: A Guide for Modern Times. Hodder & Stoughton, London (2018)

    Google Scholar 

  3. Oudah, M.; Shaalan, K.: NERA 2.0: improving coverage and performance of rule-based named entity recognition for Arabic. Nat. Lang. Eng. 23(3), 441–472 (2017)

    Article  Google Scholar 

  4. Zirikly, A.; Diab, M.: Named entity recognition for Arabic social media. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pp. 176–185 (2015)

  5. Zaghouani, W.: RENAR: a rule-based Arabic named entity recognition system. ACM Trans. Asian Lang. Inf. Process. 11(1), 2 (2012)

    Article  MathSciNet  Google Scholar 

  6. Kim, Y.; Jernite, Y.; Sontag, D.; Rush, A.M.: Character-aware neural language models. In: AAAI, pp. 2741–2749 (2016)

  7. Dahou, A.; Xiong, S.; Zhou, J.; Haddoud, M.H.; Duan, P.: Word embeddings and convolutional neural network for Arabic sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2418–2427 (2016)

  8. Ali, M.; Tan, G.; Hussain, A.: Bidirectional recurrent neural network approach for Arabic named entity recognition. Future Internet 10(12), 123 (2018)

    Article  Google Scholar 

  9. Sun, Q.; Lee, S.; Batra, D.: Bidirectional beam search: forward–backward inference in neural sequence models for fill-in-the-blank image captioning. arXiv preprint, arXiv:1705.08759 (2017).

  10. Doetsch, P.; Zeyer, A.; Ney, H.: Bidirectional decoder networks for attention-based end-to-end offline handwriting recognition. In 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 361–366 (2016)

  11. Goyal, A.; Gupta, V.; Kumar, M.: Recent named entity recognition and classification techniques: a systematic review. Comput. Sci. Rev. 29, 21–43 (2018)

    Article  Google Scholar 

  12. Etaiwi, W.; Awajan, A.; Suleiman, D.: Statistical Arabic name entity recognition approaches: a survey. Procedia Comput. Sci. 113, 57–64 (2017)

    Article  Google Scholar 

  13. Al-Ayyoub, M.; Nuseir, A.; Alsmearat, K.; Jararweh, Y.; Gupta, B.: Deep learning for Arabic NLP: a survey. J. Comput. Sci. 26, 522–531 (2018)

    Article  Google Scholar 

  14. Shaalan, K.: Rule-based approach in Arabic natural language processing. Int. J. Inf. Commun. Technol. 3(3), 11–19 (2010)

    Google Scholar 

  15. Benajiba, Y.; Rosso, P.: Arabic named entity recognition using conditional random fields. In: Proceedings of Workshop HLT NLP Within Arabic World, LREC, pp. 143–153 (2008)

  16. Meselhi, M.A.; Bakr, H.M.A.; Ziedan, I.; Shaalan, K.: Hybrid named entity recognition-application to Arabic language. In: 2014 9th International Conference on Computer Engineering & Systems (ICCES), pp. 80–85 (2014)

  17. Ali, M.N.A.; Tan, G.; Hussain, A.: Boosting Arabic named-entity recognition with multi-attention layer. IEEE Access 7, 46575–46582 (2019)

    Article  Google Scholar 

  18. Mohammed, N.F.; Omar, N.: Arabic named entity recognition using artificial neural network. J. Comput. Sci. 8(8), 1285–1293 (2012)

    Article  Google Scholar 

  19. Gridach, M.; Haddad, H.: Arabic named entity recognition: a bidirectional GRU-CRF approach. In: International Conference on Intelligent Text Processing and Computational Linguistics, pp. 264–275 (2017)

  20. Awad, D.; Sabty, C.; Elmahdy, M.; Abdennadher, S.: Arabic name entity recognition using deep learning. In: International Conference on Statistical Language and Speech Processing, pp. 105–116 (2018)

  21. Cho, K., et al.: Learning phrase representations using RNN encoder–decoder for statistical machine translation. arXiv preprint, arXiv:1406.1078 (2014)

  22. Xu, K., et al.: Show, attend and tell: neural image caption generation with visual attention. In: International Conference on Machine Learning, pp. 2048–2057 (2015)

  23. Mikolov, T.; Chen, K.; Corrado, G.; Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint, arXiv:1301.3781 (2013)

  24. Li, F.; Zhang, M.; Fu, G.; Ji, D.: A neural joint model for entity and relation extraction from biomedical text. BMC Bioinformatics 18(1), 198 (2017)

    Article  Google Scholar 

  25. Ling, W., et al.: Finding function in form: Compositional character models for open vocabulary word representation. arXiv preprint, arXiv:1508.02096 (2015)

  26. Luong, M.-T.; Manning, C.D.: Achieving open vocabulary neural machine translation with hybrid word-character models. arXiv preprint, arXiv:1604.00788 (2016)

  27. Cocos, A.; Fiks, A.G.; Masino, A.J.: Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts. J. Am. Med. Inform. Assoc. 24(4), 813–821 (2017)

    Article  Google Scholar 

  28. Soliman, A.B.; Eissa, K.; El-Beltagy, S.R.: AraVec: a set of arabic word embedding models for use in Arabic NLP. Procedia Comput. Sci. 117, 256–265 (2017)

    Article  Google Scholar 

  29. Rei, M.; Crichton, G.K.O.; Pyysalo, S.: Attending to characters in neural sequence labeling models. arXiv preprint, arXiv:1611.04361 (2016)

  30. Benajiba, Y.; Rosso, P.; Miguel, J.; Ruiz, B.: ANERsys: an Arabic named entity recognition system based on maximum entropy. In: International Conference on Intelligent Text Processing and Computational Linguistics, pp. 143–153 (2007)

  31. Mohit, B.; Schneider, N.; Bhowmick, R.; Oflazer, K.; Smith, N.A.: Recall-oriented learning of named entities in Arabic Wikipedia. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 162–173 (2012)

  32. Duchi, J.; Hazan, E.; Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

  33. Azroumahli, C.; El Younoussi, Y.; Achbal, F.: An overview of a distributional word representation for an Arabic named entity recognition system. In: International Conference on Soft Computing and Pattern Recognition, pp. 130–140 (2017)

  34. Devarakonda, A.; Naumov, M.; Garland, M.: AdaBatch: adaptive batch sizes for training deep neural networks. arXiv preprint arXiv:1712.02029 (2017)

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Central South University, Changsha, 410083, China

    Mohammed N. A. Ali & Guanzheng Tan

Authors
  1. Mohammed N. A. Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanzheng Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanzheng Tan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ali, M.N.A., Tan, G. Bidirectional Encoder–Decoder Model for Arabic Named Entity Recognition. Arab J Sci Eng 44, 9693–9701 (2019). https://doi.org/10.1007/s13369-019-04068-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-019-04068-2

Keywords

Search

Navigation