Abstract
The use of predefined phrase patterns like: N-grams (N>=2), longest common sub sequences or pre defined linguistic patterns etc do not give any credit to non-matching/smaller-size useful patterns and thus, may result in loss of information. Next, the use of 1-gram based model results in several noisy matches. Additionally, due to presence of more than one topic with different levels of importance in summary, we consider summarization evaluation task as topic based evaluation of information content. Means at first stage, we identify the topics covered in given model/reference summary and calculate their importance. At the next stage, we calculate the information coverage in test / machine generated summary, w.r.t. every identified topic. We introduce a graph based mapping scheme and the concept of closeness centrality measure to calculate the information depth and sense of the co-occurring words in every identified topic. Our experimental results show that devised system is better than/comparable with best results of TAC 2011 AESOP dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Nenkova, A., Passonneau, R., McKeown, K.: The pyramid method: Incorporating human content selection variation in summarization evaluation. ACM Trans. Speech Lang. Process. 4(2), 4 (2007)
Lin, C.-Y., Hovy, E.: Automatic evaluation of summaries using n-gram co-occurance statistics. In: Proceedings of HLT-NAACL 2003 (2003)
Teufel, S., van Halteren, H.: Evaluating Information Content by Factoid Analysis: Human Annotation and Stability. In: Proceedings of the NLP 2004 Conference, Barcelona, Spain (2004)
Nenkova, A., Passonneau, R.: Evaluating Content Selection in Summarization: The Pyramid Method. In: Proceedings of the HLT-NAACL 2004 Conference (2004)
Hovy, E.H., Lin, C.Y., Zhou, L.: Evaluating DUC 2005 using Basic Elements. In: Proceedings of DUC-2005 Workshop (2005)
Hovy, E.H., Lin, C.Y., Zhou, L., Fukumoto, J.: Automated Summarization Evaluation with Basic Elements. In: Full paper. Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006), Genoa, Italy (2006)
Porter Stemming Algorithm for suffix stripping, web –link, http://telemat.det.unifi.it/book/2001/wchange/download/stem_porter.html
Kumar, N., Srinathan, K., Varma, V.: Evaluating Information Coverage in Machine Generated Summary and Variable Length Documents. In: COMAD-2010, Nagpur, India (2010)
Kumar, N., Srinathan, K., Varma, V.: An Effective Approach for AESOP and Guided Summarization Task. In: TAC 2010 Workshop (2010), www.nist.gov/tac
Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: bringing order to the web. Technical report, Stanford digital library technologies project (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kumar, N., Srinathan, K., Varma, V. (2012). Using Graph Based Mapping of Co-occurring Words and Closeness Centrality Score for Summarization Evaluation. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2012. Lecture Notes in Computer Science, vol 7182. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28601-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-28601-8_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28600-1
Online ISBN: 978-3-642-28601-8
eBook Packages: Computer ScienceComputer Science (R0)