Skip to main content

Enhancing N-Gram-Based Summary Evaluation Using Information Content and a Taxonomy

  • Conference paper
  • 1923 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 5993)

Abstract

In this paper we propose a novel information-theoretic metric for automatic summary evaluation when model summaries are available as in the setting of the AESOP task of the Update Summarization track of the Text Analysis Conference (TAC). The metric is based on the concept of information content operationalized by using a taxonomy. Hereby, we present and discuss the results obtained at TAC 2009.

Keywords

  • Information Content
  • Semantic Similarity
  • Machine Translation
  • Automatic Evaluation
  • Model Summary

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-12275-0_71
  • Chapter length: 5 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-12275-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Lin, C.-Y., Hovy, E.: Automatic evaluation of summaries using n-gram co-occurrence statistics. In: Proceedings of HLT-NAACL, Edmonton, Canada (2003)

    Google Scholar 

  • Nenkova, A., Passonneau, R.: Evaluating content selection in summarization: The pyramid method. In: Proceedings of the Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL (2004)

    Google Scholar 

  • Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. Technical Report RC22176 (W0109-022), IBM Corporation (2001)

    Google Scholar 

  • Resnik, P.: Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the National Conference on Artificial Intelligence, AAAI (November 1995)

    Google Scholar 

  • Steinberger, J., Kabadjov, M., Pouliquen, B., Steinberger, R., Poesio, M.: WB-JRC-UT’s participation in TAC 2009: Update summarization and AESOP tasks. In: National Institute of Standards and Technology (eds.) Proceedings of the Text Analysis Conference, Gaithersburg, MD (November 2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kabadjov, M., Steinberger, J., Steinberger, R., Poesio, M., Pouliquen, B. (2010). Enhancing N-Gram-Based Summary Evaluation Using Information Content and a Taxonomy. In: , et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12275-0_71

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12275-0_71

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12274-3

  • Online ISBN: 978-3-642-12275-0

  • eBook Packages: Computer ScienceComputer Science (R0)