Abstract
A plethora of extractive summarisation techniques have been developed in the past decade, but very few enquiries have been made as to how these differ from each other or what factors affect these systems. Such meaningful comparison if available can be used to create a robust ensemble of these approaches, which has the possibility to consistently outperform each individual summarisation system. In this chapter we examine the roles of three principle components of an extractive summarisation technique: sentence ranking algorithm, sentence similarity metric and text representation scheme. We show that using a combination of several different sentence similarity measures, rather than choosing any particular measure, significantly improves performance of the resultant meta-system. Even simple ensemble techniques, when used in an informed manner, prove to be very effective in improving the overall performance and consistency of summarisation systems. While aggregating multiple ranking algorithms or text similarity measures, though the improvement in ROUGE score is not always significant, the resultant meta-systems are more robust than candidate systems. The results suggest that, when proposing a sentence extraction technique, defining better sentence similarity metrics would be more impactful than a new ranking algorithm. Also using multiple sentence similarity scores and ranking algorithms in favour of a particular combination always results in an improved and robust performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
ROUGE-1.5.5.pl -n 4 -m -a -x -l 100 -c 95 -r 1000 -f A -p 0.5 -t 0.
- 2.
- 3.
References
Campos, R., Mangaravite, V., Pasquali, A., Jorge, A.M., Nunes, C., Jatowt, A.: A text feature based automatic keyword extraction method for single documents. In: European Conference on Information Retrieval, pp. 684–691. Springer (2018)
Cohn, T.A., Lapata, M.: Sentence compression as tree transduction. J. Artif. Intell. Res. 34, 637–674 (2009)
Dang, H.T.: Overview of duc 2005. Proc. Doc. Underst. Conf. 2005, 1–12 (2005)
Dumais, S., Furnas, G., Landauer, T., Deerwester, S., Deerwester, S., et al.: Latent semantic indexing. In: Proceedings of the Text Retrieval Conference (1995)
Erkan, G., Radev, D.R.: Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 457–479, (2004)
Haghighi, A., Vanderwende, L.: Exploring content models for multi-document summarization. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 362–370. Association for Computational Linguistics (2009)
Hong, K., Conroy, J.M., Favre, B., Kulesza, A., Lin, H., Nenkova, A.: A repository of state of the art and competitive baseline summaries for generic news summarization. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC 2014, Reykjavik, Iceland, 26–31 May 2014, pp. 1608–1616 (2014). http://www.lrec-conf.org/proceedings/lrec2014/summaries/1093.html
Hong, K., Marcus, M., Nenkova, A.: System combination for multi-document summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 107–117. Association for Computational Linguistics, Lisbon, Portugal (2015)
Kulesza, A., Taskar, B., et al.: Determinantal point processes for machine learning. Found. Trends® Mach. Learn. 5(2–3), 123–286 (2012)
Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pp. 74–81 (2004)
Lin, C.Y., Hovy, E.: The automated acquisition of topic signatures for text summarization. In: Proceedings of the 18th conference on Computational linguistics, vol. 1, pp. 495–501. Association for Computational Linguistics (2000)
Lin, H., Bilmes, J.: Learning mixtures of submodular shells with application to document summarization. In: Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, pp. 479–490. AUAI Press (2012)
Mandal, A., Ghosh, K., Pal, A., Ghosh, S.: Automatic catchphrase identification from legal court case documents. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 2187–2190. ACM (2017)
Mehta, P., Majumder, P.: Effective aggregation of various summarization techniques. Inf. Process. Manag. 54(2), 145–158 (2018)
Mihalcea, R., Tarau, P.: Textrank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2004)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
Mogren, O., Kågebäck, M., Dubhashi, D.: Extractive summarization by aggregating multiple similarities. In: Proceedings of Recent Advances In Natural Language Processing, pp. 451–457 (2015)
Nenkova, A., Vanderwende, L., McKeown, K.: A compositional context sensitive multi-document summarizer: exploring the factors that influence summarization. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 573–580. ACM (2006)
Owczarzak, K., Conroy, J.M., Dang, H.T., Nenkova, A.: An assessment of the accuracy of automatic evaluation in summarization. In: Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization, pp. 1–9. Association for Computational Linguistics (2012)
Owczarzak, K., Dang, H.T.: Overview of the tac 2011 summarization track: Guided task and aesop task. In: Proceedings of the Text Analysis Conference (TAC 2011), Gaithersburg, Maryland, USA (2011)
Page, L., Brin, S., Motwani, R., Winograd, T.: The Pagerank Citation Ranking: Bringing Order to the Web. Technical report, Stanford InfoLab (1999)
Pei, Y., Yin, W., Fan, Q., Huang, L.: A supervised aggregation framework for multi-document summarization. In: Proceedings of 24th International Conference on Computational Linguistics: Technical Papers, pp. 2225–2242 (2012)
Radev, D.R., Jing, H., Styś, M., Tam, D.: Centroid-based summarization of multiple documents. Inf. Process. Manag. 40(6), 919–938 (2004)
Steinberger, J.: Using latent semantic analysis in text summarization and summary evaluation. In: Proceedings of ISIM04, pp. 93–100 (2004)
Voorhees, E.M.: The trec robust retrieval track. ACM SIGIR Forum 39(1), 11–20 (2005)
Wang, D., Li, T.: Weighted consensus multi-document summarization. Inf. Process. Manag. 48(3), 513–523 (2012)
Acknowledgements
Adapted/Translated by permission from Elsevier: Elsevier, Information processing and management, vol 54/2, pages no. 145–158, Effective aggregation of various summarisation techniques, Parth Mehta and Prasenjit Majumder, Copyright (2018).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Mehta, P., Majumder, P. (2019). Improving Sentence Extraction Through Rank Aggregation. In: From Extractive to Abstractive Summarization: A Journey. Springer, Singapore. https://doi.org/10.1007/978-981-13-8934-4_5
Download citation
DOI: https://doi.org/10.1007/978-981-13-8934-4_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-8933-7
Online ISBN: 978-981-13-8934-4
eBook Packages: Computer ScienceComputer Science (R0)