Long-Span Language Models for Query-Focused Unsupervised Extractive Text Summarization

  • Mittul Singh
  • Arunav Mishra
  • Youssef Oualil
  • Klaus Berberich
  • Dietrich Klakow
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10772)


Effective unsupervised query-focused extractive summarization systems use query-specific features along with short-range language models (LMs) in sentence ranking and selection summarization subtasks. We hypothesize that applying long-span n-gram-based and neural LMs that better capture larger context can help improve these subtasks. Hence, we outline the first attempt to apply long-span models to a query-focused summarization task in an unsupervised setting. We also propose the A cross S entence B oundary LSTM-based LMs, ASB LSTM and bi ASB LSTM, that is geared towards the query-focused summarization subtasks. Intrinsic and extrinsic experiments on a real word corpus with 100 Wikipedia event descriptions as queries show that using the long-span models applied in an integer linear programming (ILP) formulation of MMR criterion are the most effective against several state-of-the-art baseline methods from the literature.


  1. 1.
    Cao, Z., et al.: AttSum: joint learning of focusing and summarization with neural attention. arXiv preprint arXiv:1604.00125 (2016)
  2. 2.
    Nallapati, R., et al.: Classify or select: neural architectures for extractive document summarization. arXiv preprint arXiv:1611.04244 (2016)
  3. 3.
    Mishra, A., et al.: Event digest: a holistic view on past events. In: SIGIR (2016)Google Scholar
  4. 4.
    Zhong, S., et al.: Query-oriented unsupervised multi-document summarization via deep learning model. Expert Syst. Appl. 42(21) (2015)Google Scholar
  5. 5.
    Yousefi-Azar, M., et al.: Text summarization using unsupervised deep learning. Expert Syst. Appl. 68, 93–105 (2017)CrossRefGoogle Scholar
  6. 6.
    Cao, Z., et al.: Ranking with recursive neural networks and its application to multi-document summarization. In: AAAI (2015)Google Scholar
  7. 7.
    Palangi, H., et al.: Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval. In: TASLP (2016)Google Scholar
  8. 8.
    Momtazi, S., et al.: Trained trigger language model for sentence retrieval in QA: bridging the vocabulary gap. In: CIKM (2011)Google Scholar
  9. 9.
    McDonald, R.: A study of global inference algorithms in multi-document summarization. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 557–564. Springer, Heidelberg (2007). CrossRefGoogle Scholar
  10. 10.
    Gillick, D., et al.: A scalable global model for summarization. In: ILP-NAACL-HLT (2009)Google Scholar
  11. 11.
    Riedhammer, K., et al.: Long story short - global unsupervised models for keyphrase based meeting summarization. Speech Commun. 52(10), 801–815 (2010)CrossRefGoogle Scholar
  12. 12.
    Carbonell, J., et al.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR (1998)Google Scholar
  13. 13.
    Oualil, Y., et al.: Long-short range context neural networks for language modeling. arXiv preprint arXiv:1708.06555 (2017)
  14. 14.
  15. 15.
    Zhai, C.X., et al.: Statistical language models for information retrieval. Synth. Lect. Hum. Lang. Technol. 1(1), 1–141 (2008)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Radev, D.R., et al.: MEAD-a platform for multidocument multilingual text summarization. In: LREC (2004)Google Scholar
  17. 17.
    Riedhammer, K., et al.: Packing the meeting summarization knapsack. In: INTERSPEECH (2008)Google Scholar
  18. 18.
    Guo, J., et al.: A deep relevance matching model for ad-hoc retrieval. In: CIKM (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Spoken Language Systems (LSV)Saarland Informatics CampusSaarbrückenGermany
  2. 2.Max Planck Institute for InformaticsSaarland Informatics CampusSaarbrückenGermany

Personalised recommendations