Language Resources and Evaluation

, Volume 44, Issue 3, pp 263–280

SpatialML: annotation scheme, resources, and evaluation

  • Inderjeet Mani
  • Christy Doran
  • Dave Harris
  • Janet Hitzeman
  • Rob Quimby
  • Justin Richer
  • Ben Wellner
  • Scott Mardis
  • Seamus Clancy
Article

DOI: 10.1007/s10579-010-9121-0

Cite this article as:
Mani, I., Doran, C., Harris, D. et al. Lang Resources & Evaluation (2010) 44: 263. doi:10.1007/s10579-010-9121-0

Abstract

SpatialML is an annotation scheme for marking up references to places in natural language. It covers both named and nominal references to places, grounding them where possible with geo-coordinates, and characterizes relationships among places in terms of a region calculus. A freely available annotation editor has been developed for SpatialML, along with several annotated corpora. Inter-annotator agreement on SpatialML extents is 91.3 F-measure on a corpus of SpatialML-annotated ACE documents released by the Linguistic Data Consortium. Disambiguation agreement on geo-coordinates on ACE is 87.93 F-measure. An automatic tagger for SpatialML extents scores 86.9 F on ACE, while a disambiguator scores 93.0 F on it. Results are also presented for two other corpora. In adapting the extent tagger to new domains, merging the training data from the ACE corpus with annotated data in the new domain provides the best performance.

Keywords

AnnotationGuidelinesSpatial languageGeographyInformation extractionEvaluationAdaptation

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Inderjeet Mani
    • 1
  • Christy Doran
    • 1
  • Dave Harris
    • 1
  • Janet Hitzeman
    • 1
  • Rob Quimby
    • 1
  • Justin Richer
    • 1
  • Ben Wellner
    • 1
  • Scott Mardis
    • 1
  • Seamus Clancy
    • 1
  1. 1.The MITRE CorporationBedfordUSA