Advertisement

Arabic named entity recognition via deep co-learning

  • Chadi HelweEmail author
  • Shady Elbassuoni
Article

Abstract

Named entity recognition (NER) is an important natural language processing (NLP) task with many applications. We tackle the problem of Arabic NER using deep learning based on Arabic word embeddings that capture syntactic and semantic relationships between words. Deep learning has been shown to perform significantly better than other approaches for various NLP tasks including NER. However, deep-learning models also require a significantly large amount of training data, which is highly lacking in the case of the Arabic language. To remedy this, we adopt the semi-supervised co-training approach to the realm of deep learning, which we refer to as deep co-learning. Our deep co-learning approach makes use of a small amount of labeled data, which is augmented with partially labeled data that is automatically generated from Wikipedia. Our approach relies only on word embeddings as features and does not involve any additional feature engineering. Nonetheless, when tested on three different Arabic NER benchmarks, our approach consistently outperforms state-of-the-art Arabic NER approaches, including ones that employ carefully-crafted NLP features. It also consistently outperforms various baselines including purely-supervised deep-learning approaches as well as semi-supervised ones that make use of only unlabeled data such as self-learning and the traditional co-training approach.

Keywords

NER Arabic NLP Co-training Deep Learning 

Notes

Acknowledgements

The authors would like to thank the American University of Beirut Research Board (URB) for funding this project. This work is supported by the American University of Beirut Research Board (URB), award number 103367.

References

  1. Abdallah S, Shaalan K, Shoaib M (2012) Integrating rule-based system with classification for Arabic named entity recognition. In: International conference on intelligent text processing and computational Linguistics, Springer, pp 311–322Google Scholar
  2. Abdelali A, Darwish K, Durrani N, Mubarak H (2016) Farasa: a fast and furious segmenter for arabic. In: Proceedings of the 2016 conference of the North American chapter of the association for computational Linguistics: demonstrations, pp 11–16. Association for computational Linguistics. San Diego, CaliforniaGoogle Scholar
  3. AbdelRahman S, Elarnaoty M, Magdy M, Fahmy A (2010) Integrated machine learning techniques for arabic named entity recognition. IJCSI 7:27–36Google Scholar
  4. Abdul-Hamid A, Darwish K (2010) Simplified feature set for arabic named entity recognition. In: Proceedings of the 2010 named entities workshop, pp 110–115. Association for computational LinguisticsGoogle Scholar
  5. Abuleil S (2004) Extracting names from arabic text for question-answering systems. In coupling approaches, coupling media and coupling languages for information retrieval, pp 638–647. LE CENTRE DE HAUTES ETUDES INTERNATIONALES D’INFORMATIQUE DOCUMENTAIREGoogle Scholar
  6. Al-Ahmari S, Al-Johar B (2016) Cross domains arabic named entity recognition system. In: First international workshop on pattern recognition, pp 100111I–100111I. International society for optics and photonicsGoogle Scholar
  7. Al-Rfou R, Perozzi B, Skiena S (2013) Polyglot: distributed word representations for multilingual nlp. arXiv preprint arXiv:1307.1662
  8. Al-Shalabi R, Kanaan G, Al-Sarayreh B, Khanfar K, Al-Ghonmein A, Talhouni H, Al-Azazmeh S (2009) Proper noun extracting algorithm for arabic language. In International conference on IT, ThailandGoogle Scholar
  9. Alkharashi I (2009) Person named entity generation and recognition for arabic language. In: Proceedings of the second international conference on Arabic language resources and tools, pp 205–208. CiteseerGoogle Scholar
  10. Alotaibi F, Lee MG (2014) A hybrid approach to features representation for fine-grained Arabic named entity recognition. In COLING, pp 984–995Google Scholar
  11. Althobaiti M, Kruschwitz U, Massimo P (2014) Aranlp: a java-based library for the processing of arabic text, pp 4134–4138Google Scholar
  12. Althobaiti M, Kruschwitz U, Poesio M (2014) Automatic creation of arabic named entity annotated corpus using wikipedia, pp 106–115Google Scholar
  13. Benajiba Y, Diab M, Rosso P (2008) Arabic named entity recognition using optimized feature sets. In: Proceedings of the conference on empirical methods in natural language processing, pp 284–293. Association for computational LinguisticsGoogle Scholar
  14. Benajiba Y, Diab M, Rosso P (2009) Arabic named entity recognition: a feature-driven study. IEEE Trans Audio Speech Lang Process 17(5):926–934CrossRefGoogle Scholar
  15. Benajiba Y, Diab M, Rosso P, et al. (2008) Arabic named entity recognition: an svm-based approach. In: Proceedings of 2008 Arab international conference on information technology (ACIT), pp 16–18Google Scholar
  16. Benajiba Y, Rosso P (2007) Anersys 2.0: conquering the ner task for the Arabic language by combining the maximum entropy with pos-tag information. In IICAI, pp 1814–1823Google Scholar
  17. Benajiba Y, Rosso P (2008) Arabic named entity recognition using conditional random fields. In: Proceedings of Workshop on HLT & NLP within the Arabic World, LREC, volume 8, pp 143–153. CiteseerGoogle Scholar
  18. Benajiba Y, Rosso P, Benedíruiz JM (2007) Anersys: an arabic named entity recognition system based on maximum entropy. In: international conference on intelligent text processing and computational Linguistics, Springer, pp 143–153Google Scholar
  19. Benajiba Y, Zitouni I, Diab M, Rosso P (2010) Arabic named entity recognition: using features extracted from noisy data. In: Proceedings of the ACL 2010 conference short papers, pp 281–285. Association for computational LinguisticsGoogle Scholar
  20. Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on computational learning theory, pp 92–100. ACMGoogle Scholar
  21. Buckwalter T (2002) Buckwalter Arabic morphological analyzer version 1.0Google Scholar
  22. Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th international conference on machine learning, pp 160–167. ACMGoogle Scholar
  23. Darwish K (2013) Named entity recognition using cross-lingual resources: Arabic as an example. ACL 1:1558–1567Google Scholar
  24. Doddington GR, Mitchell A, Przybocki MA , Ramshaw LA , Strassel S, Weischedel RM (2004) The automatic content extraction (ace) program-tasks, data, and evaluation. In LREC, vol. 2, p. 1Google Scholar
  25. El-Haj M, Koulali R (2013) Kalimat a multipurpose arabic corpus. In Second Workshop on Arabic corpus Linguistics (WACL-2), pp 22–25Google Scholar
  26. Elrazzaz M, Elbassuoni S, Shaban K, Helwe C (2017) Methodical evaluation of Arabic word embeddings. In: Proceedings of the 55th annual meeting of the association for computational Linguistics (Vol. 2: Short Papers), pp 454–458, Vancouver, Canada, Association for computational LinguisticsGoogle Scholar
  27. Elsebai A, Meziane F, Belkredim FZ (2009) A rule based persons names arabic extraction system. Communications of the IBIMA, 11(6):53–59Google Scholar
  28. Finkel JR, Grenager T, Manning C (2005) Incorporating non-local information into information extraction systems by gibbs sampling. In: Proceedings of the 43rd annual meeting on association for computational linguistics, pp 363–370. Association for computational LinguisticsGoogle Scholar
  29. Gao B, Bian J, Liu TY (2014) Wordrep: a benchmark for research on learning word representations. arXiv preprint arXiv:1407.1640
  30. Gers FA, Schmidhuber J (2000) Recurrent nets that time and count. In: Neural networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS international joint conference on, vol. 3, pp 189–194Google Scholar
  31. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. Book in preparation for MIT PressGoogle Scholar
  32. Gopee N (2016) Applying recurrent neural network for arabic named entity recognitionGoogle Scholar
  33. Gridach M (2016) Character-aware neural networks for Arabic named entity recognition for social media. In: Proceedings of the 6th workshop on South and Southeast Asian natural language processing (WSSANLP2016), pp 23–32Google Scholar
  34. Habash Nizar Y (2010) Introduction to Arabic natural language processing. Synth Lect Hum Lang Technol 3(1):1–187CrossRefGoogle Scholar
  35. Halpern J et al. (2009) Lexicon-driven approach to the recognition of Arabic named entities. In: Second international conference on Arabic language resources and toolsGoogle Scholar
  36. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural comput 9(8):1735–1780CrossRefGoogle Scholar
  37. Koulali R, Meziane A (2012) A contribution to Arabic named entity recognition. In: ICT and knowledge Engineering (ICT & knowledge Engineering), 2012 10th international conference on IEEE, pp 46–52Google Scholar
  38. Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. ArXiv preprint arXiv:1603.01360
  39. Maamouri M, Bies A, Jin H, Buckwalter T (2010) The penn arabic tree bank. Current implementations in Arabic NLP. CSLI NLP Series, Computational approaches to Arabic script-based languagesGoogle Scholar
  40. Maloney J, Niv M (1998) Tagarab: a fast, accurate arabic name recognizer using high-precision morphological analysis. In: Proceedings of the workshop on computational approaches to semitic languages, pp 8–15. Association for computational LinguisticsGoogle Scholar
  41. Mayhew S, Tsai CT, Roth D (2017) Cheap translation for cross-lingual named entity recognition. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 2536–2545Google Scholar
  42. Mesfar S (2007) Named entity recognition for Arabic using syntactic grammars. In natural language processing and information systems, pp 305–316. SpringerGoogle Scholar
  43. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781
  44. Mohit B, Schneider N, Bhowmick R, Oflazer K, Smith NA (2012) Recall-oriented learning of named entities in Arabic Wikipedia. In: Proceedings of the 13th conference of the European chapter of the association for computational Linguistics, pp 162–173. Association for Computational LinguisticsGoogle Scholar
  45. Nothman J, Ringland N, Radford W, Murphy T, Curran James R (2013) Learning multilingual named entity recognition from Wikipedia. Artif Intell 194:151–175MathSciNetCrossRefzbMATHGoogle Scholar
  46. Oudah M, Shaalan KF (2012) A pipeline Arabic named entity recognition using a hybrid approach. In COLING, pp 2159–2176Google Scholar
  47. Pascanu R, Gulcehre C, Cho K, Bengio Y (2013) How to construct deep recurrent neural networks? ArXiv preprint arXiv:1312.6026
  48. Pasha A, Al-Badrashiny M, Diab MT , El Kholy A, Eskander R, Habash N, Pooleery M, Rambow O, Roth R (2014) Madamira: a fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In LREC, vol. 14, pp 1094–1101Google Scholar
  49. Pennington J, Socher R, Manning C (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543, Doha, Qatar, Association for Computational LinguisticsGoogle Scholar
  50. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), pp 1532–1543Google Scholar
  51. Richman AE, Schone P (2008) Mining Wiki resources for multilingual named entity recognition. In: Proceedings of ACL-08: HLT, pp 1–9Google Scholar
  52. Rosenberg C, Hebert M, Schneiderman H (2005) Semi-supervised self-training of object detection modelsGoogle Scholar
  53. Samy D, Moreno A, Guirao JM (2005) A proposal for an Arabic named entity tagger leveraging a parallel corpus. In: International conference RANLP, Borovets, Bulgaria, pp 459–465Google Scholar
  54. Shaalan K (2014) A survey of Arabic named entity recognition and classification. Comput Linguist 40(2):469–510CrossRefGoogle Scholar
  55. Shaalan K, Raza H (2007) Person name entity recognition for Arabic. In: Proceedings of the 2007 workshop on computational approaches to semitic languages: common issues and resources, pp 17–24. Association for computational LlinguisticsGoogle Scholar
  56. Shaalan K, Raza H (2008) Arabic named entity recognition from diverse text types. In advances in natural language processing, Springer, pp 440–451Google Scholar
  57. Shaalan K, Raza H (2009) Nera: named entity recognition for Arabic. J Am Soc Inf Sci Technol 60(8):1652–1663CrossRefGoogle Scholar
  58. Tieleman T, Hinton G (2012) Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: neural networks for machine learning, 4(2):26–31Google Scholar
  59. Tjong EF, Sang K, De Meulder F (2003) Introduction to the conll-2003 shared task: language-independent named entity recognition. In: Proceedings of the seventh conference on natural language learning at HLT-NAACL 2003, Vol. 4, pp 142–147. Association for computational LinguisticsGoogle Scholar
  60. Zaghouani W, Pouliquen B, Ebrahim M, Steinberger R (2010) Adapting a resource-light highly multilingual named entity recognition system to arabic. In LRECGoogle Scholar
  61. Zahran MA, Magooda A, Mahgoub AY , Raafat H, Rashwan M, Atyia A (2015) Word representations in vector space and their applications for Arabic. In: International conference on intelligent text processing and computational Linguistics, Springer, pp 430–443Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Computer Science DepartmentAmerican University of BeirutBeirutLebanon

Personalised recommendations