Chapter

AI 2003: Advances in Artificial Intelligence

Volume 2903 of the series Lecture Notes in Computer Science pp 910-921

Evaluating Corpora for Named Entity Recognition Using Character-Level Features

  • Casey WhitelawAffiliated withSydney Language Technology Research Group Capital Markets Co-operative Research Centre, University of Sydney
  • , Jon PatrickAffiliated withSydney Language Technology Research Group Capital Markets Co-operative Research Centre, University of Sydney

Abstract

We present a new collection of training corpora for evaluation of language-independent named entity recognition systems. For the five languages included in this initial release, Basque, Dutch, English, Korean, and Spanish, we provide an analysis of the relative difficulty of the NER task for both the language in general, and as a supervised task using these corpora. We construct three strongly language-independent systems, each using only orthographic features, and compare their performance on both seen and unseen data. We achieve improved results through combining these classifiers, showing that ensemble approaches are suitable when dealing with language-independent problems.

Keywords

computational linguistics machine learning