Skip to main content
View expanded cover

European Conference on Information Retrieval

ECIR 2010: Advances in Information Retrieval pp 662–666Cite as

Enhancing N-Gram-Based Summary Evaluation Using Information Content and a Taxonomy

  • Conference paper

  • 1923 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 5993)

Abstract

In this paper we propose a novel information-theoretic metric for automatic summary evaluation when model summaries are available as in the setting of the AESOP task of the Update Summarization track of the Text Analysis Conference (TAC). The metric is based on the concept of information content operationalized by using a taxonomy. Hereby, we present and discuss the results obtained at TAC 2009.

Keywords

  • Information Content
  • Semantic Similarity
  • Machine Translation
  • Automatic Evaluation
  • Model Summary

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-12275-0_71
  • Chapter length: 5 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-12275-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Lin, C.-Y., Hovy, E.: Automatic evaluation of summaries using n-gram co-occurrence statistics. In: Proceedings of HLT-NAACL, Edmonton, Canada (2003)

    Google Scholar 

  • Nenkova, A., Passonneau, R.: Evaluating content selection in summarization: The pyramid method. In: Proceedings of the Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL (2004)

    Google Scholar 

  • Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. Technical Report RC22176 (W0109-022), IBM Corporation (2001)

    Google Scholar 

  • Resnik, P.: Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the National Conference on Artificial Intelligence, AAAI (November 1995)

    Google Scholar 

  • Steinberger, J., Kabadjov, M., Pouliquen, B., Steinberger, R., Poesio, M.: WB-JRC-UT’s participation in TAC 2009: Update summarization and AESOP tasks. In: National Institute of Standards and Technology (eds.) Proceedings of the Text Analysis Conference, Gaithersburg, MD (November 2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Joint Research Centre, European Commission, Via E. Fermi 2749, Ispra, (VA), Italy

    Mijail Kabadjov, Josef Steinberger, Ralf Steinberger & Bruno Pouliquen

  2. University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, United Kingdom

    Massimo Poesio

  3. Universitá di Trento, Povo, TN, 38100, Italy

    Massimo Poesio

Authors
  1. Mijail Kabadjov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josef Steinberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ralf Steinberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Massimo Poesio
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bruno Pouliquen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Adaptive Information Cluster, Dublin City University, Dublin, 9, Ireland

    Cathal Gurrin

  2. The Open University, Walton Hall, MK7 6HF, Milton Keynes, UK

    Yulan He

  3. Microsoft Research Ltd, 7 JJ Thomson Avenue, CB3 0FB, Cambridge, UK

    Gabriella Kazai

  4. Department of Computer Science, University of Essex, Wivenhoe Park, CO4 3SQ, Colchester, UK

    Udo Kruschwitz

  5. The Open University, Walton Hall, Milton Keynes, UK

    Suzanne Little

  6. University of London, London, UK

    Thomas Roelleke

  7. Knowledge Media Institute, The Open University, MK7 6AA, Milton Keynes, UK

    Stefan Rüger

  8. Department of Computing Science, University of Glasgow, 17 Lilybank Gardens, G12 8QQ, Glasgow, UK

    Keith van Rijsbergen

Rights and permissions

Reprints and Permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kabadjov, M., Steinberger, J., Steinberger, R., Poesio, M., Pouliquen, B. (2010). Enhancing N-Gram-Based Summary Evaluation Using Information Content and a Taxonomy. In: , et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12275-0_71

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12275-0_71

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12274-3

  • Online ISBN: 978-3-642-12275-0

  • eBook Packages: Computer ScienceComputer Science (R0)