Skip to main content

Enhancing N-Gram-Based Summary Evaluation Using Information Content and a Taxonomy

  • Conference paper
  • 2112 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 5993)


In this paper we propose a novel information-theoretic metric for automatic summary evaluation when model summaries are available as in the setting of the AESOP task of the Update Summarization track of the Text Analysis Conference (TAC). The metric is based on the concept of information content operationalized by using a taxonomy. Hereby, we present and discuss the results obtained at TAC 2009.


  • Information Content
  • Semantic Similarity
  • Machine Translation
  • Automatic Evaluation
  • Model Summary

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  • Lin, C.-Y., Hovy, E.: Automatic evaluation of summaries using n-gram co-occurrence statistics. In: Proceedings of HLT-NAACL, Edmonton, Canada (2003)

    Google Scholar 

  • Nenkova, A., Passonneau, R.: Evaluating content selection in summarization: The pyramid method. In: Proceedings of the Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL (2004)

    Google Scholar 

  • Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. Technical Report RC22176 (W0109-022), IBM Corporation (2001)

    Google Scholar 

  • Resnik, P.: Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the National Conference on Artificial Intelligence, AAAI (November 1995)

    Google Scholar 

  • Steinberger, J., Kabadjov, M., Pouliquen, B., Steinberger, R., Poesio, M.: WB-JRC-UT’s participation in TAC 2009: Update summarization and AESOP tasks. In: National Institute of Standards and Technology (eds.) Proceedings of the Text Analysis Conference, Gaithersburg, MD (November 2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kabadjov, M., Steinberger, J., Steinberger, R., Poesio, M., Pouliquen, B. (2010). Enhancing N-Gram-Based Summary Evaluation Using Information Content and a Taxonomy. In: Gurrin, C., et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12274-3

  • Online ISBN: 978-3-642-12275-0

  • eBook Packages: Computer ScienceComputer Science (R0)