Using Parallel Corpora for Multilingual (Multi-document) Summarisation Evaluation

  • Marco Turchi
  • Josef Steinberger
  • Mijail Kabadjov
  • Ralf Steinberger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6360)

Abstract

We are presenting a method for the evaluation of multilingual multi-document summarisation that allows saving precious annotation time and that makes the evaluation results across languages directly comparable. The approach is based on the manual selection of the most important sentences in a cluster of documents from a sentence-aligned parallel corpus, and by projecting the sentence selection to various target languages. We also present two ways of exploiting inter-annotator agreement levels, apply them both to a baseline sentence extraction summariser in seven languages, and discuss the result differences between the two evaluation versions, as well as a preliminary analysis between languages. The same method can in principle be used to evaluate single-document summarisers or information extraction tools.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Marco Turchi
    • 1
  • Josef Steinberger
    • 1
  • Mijail Kabadjov
    • 1
  • Ralf Steinberger
    • 1
  1. 1.European Commission - Joint Research Centre (JRC), IPSC - GlobSecIspra (VA)Italy

Personalised recommendations