Skip to main content

Obtaining Single Document Summaries Using Latent Dirichlet Allocation

  • Conference paper
  • 4091 Accesses

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7666)

Abstract

In this paper, we present a novel approach that makes use of topic models based on Latent Dirichlet allocation(LDA) for generating single document summaries. Our approach is distinguished from other LDA based approaches in that we identify the summary topics which best describe a given document and only extract sentences from those paragraphs within the document which are highly correlated given the summary topics. This ensures that our summaries always highlight the crux of the document without paying any attention to the grammar and the structure of the documents. Finally, we evaluate our summaries on the DUC 2002 Single document summarization data corpus using ROUGE measures. Our summaries had higher ROUGE values and better semantic similarity with the documents than the DUC summaries.

Keywords

  • Single Document Summaries
  • Latent Dirichlet Allocation
  • SVM
  • Naïve Bayes Classifier

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Blei, D.: Probabilistic Topic Models. Commun. ACM 55(4), 77–84 (2012)

    CrossRef  Google Scholar 

  2. Changm, Y., Chien, J.: Latent Dirichlet Learning for Document Summarization. In: ICASSP, pp. 1689–1692 (2009)

    Google Scholar 

  3. Edmundson, H.P.: New Methods in Automatic Extracting. J. ACM 16(2), 264–285 (1969)

    CrossRef  MATH  Google Scholar 

  4. Luhn, H.P.: The Automatic Creation of Literature Abstracts. Presented at IRE National Convention (1958)

    Google Scholar 

  5. Conroy, J.M., O’leary, D.P.: Text Summarization via Hidden Markov Models. In: SIGIR, pp. 406–407 (2001)

    Google Scholar 

  6. Arora, R., Ravindran, B.: Latent Dirichlet Allocation Based Multi-Document Summarization. In: AND, pp. 91–97 (2008)

    Google Scholar 

  7. Hanna, W., David, M., Andrew, M.: Rethinking LDA: Why Priors Matter. In: NIPS, pp. 1973–1981 (2009)

    Google Scholar 

  8. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval, 1st edn. Cambridge University Press (2008)

    Google Scholar 

  9. Lin, C.Y.: Rouge: A package for Automatic Evaluation of Summaries. In: Proceedings of the Workshop on Text Summarization Branches Out, WAS 2004 (2004)

    Google Scholar 

  10. Simpson, J., Weiner, E.: Oxford English Dictionary, 2nd edn. Oxford University Press (1989)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nagesh, K., Murty, M.N. (2012). Obtaining Single Document Summaries Using Latent Dirichlet Allocation. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7666. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34478-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34478-7_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34477-0

  • Online ISBN: 978-3-642-34478-7

  • eBook Packages: Computer ScienceComputer Science (R0)