Skip to main content
SpringerLink
Log in

Parallel Relationship Graph to Improve Multi-Document Summarization

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2022 (ICANN 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13530))

Included in the following conference series:

  • 2244 Accesses

Abstract

Multi-document summarization (MDS) is an important branch of information aggregation. Compared with the single-document summary (SDS), MDS faces the problem of large search space, redundancy and complex cross-document relation. In this paper, we propose an abstractive MDS model based on Transformer, which considers the parallel information of documents with the graph attention network. During decoding, our model can utilize graph information to guide the summary generation. In addition, combined with the pre-trained language model, our model can further improve the summarization performance. Empirical results on the Multi-News and WikiSum datasets show that our model brings substantial improvements over several strong baselines, and ablation studies verify the effectiveness of our key mechanisms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Liu, Y., Lapata, M.: Hierarchical transformers for multi-document summarization. arXiv preprint arXiv:1905.13164 (2019)

  2. Li, W., Xiao, X., Liu, J., Wu, H., Wang, H., Du, J.: Leveraging graph to improve abstractive multi-document summarization. arXiv preprint arXiv:2005.10043 (2020)

  3. Fabbri, A.R., Li, I., She, T., Li, S., Radev, D.R.: Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. arXiv preprint arXiv:1906.01749 (2019)

  4. Radev, D.: A common theory of information fusion from multiple text sources step one: cross-document structure. In: 1st SIGdial Workshop on Discourse and Dialogue, pp. 74–83 (2000)

    Google Scholar 

  5. Zhou, H., Ren, W., Liu, G., Su, B., Lu, W.: Entity-aware abstractive multi-document summarization. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 351–362 (2021)

    Google Scholar 

  6. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  7. Banerjee, S., Mitra, P., Sugiyama, K.: Multi-document abstractive summarization using ILP based multi-sentence compression. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)

    Google Scholar 

  8. Li, W., Zhuge, H.: Abstractive multi-document summarization based on semantic link network. In: IEEE Transactions on Knowledge and Data Engineering (2019)

    Google Scholar 

  9. Bing, L., Li, P., Liao, Y., Lam, W., Guo, W., Passonneau, R.J.: Abstractive multi-document summarization via phrase selection and merging. arXiv preprint arXiv:1506.01597 (2015)

  10. Chu, E., Liu, P.: Meansum: a neural model for unsupervised multi-document abstractive summarization. In: International Conference on Machine Learning, pp. 1223–1232. PMLR (2019)

    Google Scholar 

  11. Zhang, J., Tan, J., Wan, X.: Adapting neural single-document summarization model for abstractive multi-document summarization: a pilot study. In: Proceedings of the 11th International Conference on Natural Language Generation, pp. 381–390 (2018)

    Google Scholar 

  12. Lebanoff, L., Song, K., Liu, F.: Adapting the neural encoder-decoder framework from single to multi-document summarization. arXiv preprint arXiv:1808.06218 (2018)

  13. Pasunuru, R., et al.: Data augmentation for abstractive query-focused multi-document summarization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 13666–13674 (2021)

    Google Scholar 

  14. Lebanoff, L., Wang, B., Feng, Z., Liu, F.: Modeling endorsement for multi-document abstractive summarization. In: Proceedings of the Third Workshop on New Frontiers in Summarization, pp. 119–130 (2021)

    Google Scholar 

  15. Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)

    Article  Google Scholar 

  16. Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 404–411 (2004)

    Google Scholar 

  17. Wan, X.: An exploration of document impact on graph-based multi-document summarization. In: Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 755–762 (2008)

    Google Scholar 

  18. Tan, J., Wan, X., Xiao, J.: Abstractive document summarization with a graph-based attentional neural model. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1171–1181 (2017)

    Google Scholar 

  19. Huang, L., Wu, L., Wang, L.: Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. arXiv preprint arXiv:2005.01159 (2020)

  20. Christensen, J., Soderland, S., Etzioni, O., et al.: Towards coherent multi-document summarization. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1163–1173 (2013)

    Google Scholar 

  21. Fan, A., Gardent, C., Braud, C., Bordes, A.: Using local knowledge graph construction to scale seq2seq models to multi-document inputs. arXiv preprint arXiv:1910.08435 (2019)

  22. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

  23. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  24. Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)

  25. Wang, D., Liu, P., Zheng, Y., Qiu, X., Huang, X.J.: Heterogeneous graph neural networks for extractive document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6209–6219 (2020)

    Google Scholar 

  26. See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)

  27. Liu, P.J., et al.: Generating wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198 (2018)

  28. Gehrmann, S., Deng, Y., Rush, A.M.: Bottom-up abstractive summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4098–4109 (2018)

    Google Scholar 

Download references

Acknowledgments

This research work has been funded by the National Natural Science Foundation of China (Grant No. U21B2020).

Author information

Authors and Affiliations

  1. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Menghua Lu, Lijia Liang & Gongshen Liu

Authors
  1. Menghua Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lijia Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gongshen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gongshen Liu .

Editor information

Editors and Affiliations

  1. University of the West of England, Bristol, UK

    Elias Pimenidis

  2. Lancaster University, Lancaster, UK

    Plamen Angelov

  3. Digital Innovation, Teeside University, Middlesbrough, UK

    Chrisina Jayne

  4. Democritus University of Thrace, Xanthi, Greece

    Antonios Papaleonidas

  5. The University of the West of England, Bristol, UK

    Mehmet Aydin

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lu, M., Liang, L., Liu, G. (2022). Parallel Relationship Graph to Improve Multi-Document Summarization. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13530. Springer, Cham. https://doi.org/10.1007/978-3-031-15931-2_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15931-2_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15930-5

  • Online ISBN: 978-3-031-15931-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation