Arabic Language Analyzer with Lemma Extraction and Rich Tagset
Arabic language analyzers have been studied constructed and studied by many researchers. Most of research projects and commercial applications in the area of Arabic Natural Language Processing (ANLP) build their own analyzer. Typically they are intended for just one project or application and can not be generalized to work in other areas of ANLP. All of these analyzers didn’t cover our requirements from the analyzer which make us to build a new one. Our analyzer is also a part of a complete Arabic tagging system. It receives the output of a tokenizer in the form of inflected words, and produces for each of them a set of several possible analyzes, consisting of: a POS tag, features and a lemma. It differs from most of the existing analyzers because it produces a lemma rather than stem or root, which is a significantly harder task in Arabic, and because POS and features are described by a new very rich tagset, described in a previous publication. The test dataset was a small corpus of 16 k words, manually annotated by a single analysis for each word, correct for this particular use of that word. In the test, for 99.67% of words, the correct analysis was among those produced by the analyzer. On the other hand, in a manual verification of the output of the analyzer, only 0.1% of all analyses were grammatically incorrect.
KeywordsArabic language analyzer lemma generation analyzer and POS tagging
Unable to display preview. Download preview PDF.
- 1.Tarek, E., Fatma, E.: An Accurate Arabic Root-Based Lemmatizer for Information Retrieval Purposes. IJCSI International Journal of Computer Science Issues 9(1,3) (January 2012)Google Scholar
- 2.Khoja, S.: Stemming Arabic Text. Computing Department, Lancaster University, Lancaster (1999)Google Scholar
- 3.Nizar, H., Owen, R., George, K.: Morphological Analysis and Generation for Arabic Dialects. In: Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, Michigan, USA (2005)Google Scholar
- 4.Kareem, D.: Building a Shallow Arabic Morphological Analyzer in One Day. In: Proceedings of the ACL 2002 Workshop on Computational Approaches to Semitic Languages, PA, USA (2002)Google Scholar
- 5.Mourad, G., Noureddine, C.: Developing a New System for Arabic Morphological Analysis and Generation. In: Proceedings of the 2nd Workshop on South and Southeast Asian Natural Language Processing (WSSANLP), IJCNLP 2011, Chiang Mai, Thailand, pp. 52–57 (2011)Google Scholar
- 6.Otakar, S.: Functional Arabic Morphology, Formal System and Implementation. Doctoral Thesis, Institute of Formal and Applied Linguistics, Faculty of Mathematics and Physics, Charles University, Prague (2007)Google Scholar
- 7.Buckwalter, T.: Isues in Arabic Orthography and morphology Analysis. In: Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, COLING, Geneva (2004)Google Scholar
- 8.El-Dahdah, A.: An Intermediate Dictionary of Verb Conjugation, 1st edn. Libaririe Du Liban, Beirut (1994) (in Aabic)Google Scholar