Abstract
In this paper, we present a novel approach that makes use of topic models based on Latent Dirichlet allocation(LDA) for generating single document summaries. Our approach is distinguished from other LDA based approaches in that we identify the summary topics which best describe a given document and only extract sentences from those paragraphs within the document which are highly correlated given the summary topics. This ensures that our summaries always highlight the crux of the document without paying any attention to the grammar and the structure of the documents. Finally, we evaluate our summaries on the DUC 2002 Single document summarization data corpus using ROUGE measures. Our summaries had higher ROUGE values and better semantic similarity with the documents than the DUC summaries.
Keywords
- Single Document Summaries
- Latent Dirichlet Allocation
- SVM
- Naïve Bayes Classifier
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Blei, D.: Probabilistic Topic Models. Commun. ACM 55(4), 77–84 (2012)
Changm, Y., Chien, J.: Latent Dirichlet Learning for Document Summarization. In: ICASSP, pp. 1689–1692 (2009)
Edmundson, H.P.: New Methods in Automatic Extracting. J. ACM 16(2), 264–285 (1969)
Luhn, H.P.: The Automatic Creation of Literature Abstracts. Presented at IRE National Convention (1958)
Conroy, J.M., O’leary, D.P.: Text Summarization via Hidden Markov Models. In: SIGIR, pp. 406–407 (2001)
Arora, R., Ravindran, B.: Latent Dirichlet Allocation Based Multi-Document Summarization. In: AND, pp. 91–97 (2008)
Hanna, W., David, M., Andrew, M.: Rethinking LDA: Why Priors Matter. In: NIPS, pp. 1973–1981 (2009)
Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval, 1st edn. Cambridge University Press (2008)
Lin, C.Y.: Rouge: A package for Automatic Evaluation of Summaries. In: Proceedings of the Workshop on Text Summarization Branches Out, WAS 2004 (2004)
Simpson, J., Weiner, E.: Oxford English Dictionary, 2nd edn. Oxford University Press (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nagesh, K., Murty, M.N. (2012). Obtaining Single Document Summaries Using Latent Dirichlet Allocation. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7666. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34478-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-34478-7_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34477-0
Online ISBN: 978-3-642-34478-7
eBook Packages: Computer ScienceComputer Science (R0)
