Abstract
With the massive growth of blogs, news stories, and reports, extracting useful information from such a large quantity of textual documents has become a difficult task. Automatic text summarization is an excellent approach for summarising these documents. Text summarization aims to condense large documents into concise summaries while preserving essential information and meaning. A variety of fascinating summarising models have been developed to achieve state-of-the-art performance in terms of fluency, human readability, and semantically meaningful summaries. In this paper, we have investigated the OpenNMT tool for task text summarization. The OpenNMT is the encoder-decoder-based neural machine translation model which has been fine-tuned for the task of abstractive text summarization. The proposed OpenNMT based text summarization approach has been tested on freely available dataset such as CNNDM & MSMO dataset and depicts their proficiency in terms of ROUGE and BLEU score.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Banerjee, S., Lavie, A.: METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72 (2005)
Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 335–336 (1998)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Du, Z., et al.: All NLP tasks are generation tasks: a general pretraining framework. arXiv preprint arXiv:2103.10360 (2021)
Edmundson, H.P.: New methods in automatic extracting. J. ACM (JACM) 16(2), 264–285 (1969)
Edmundson, H.P., Wyllys, R.E.: Automatic abstracting and indexing-survey and recommendations. Commun. ACM 4(5), 226–234 (1961)
Fattah, M.A., Ren, F.: Automatic text summarization. World Acad. Sci. Eng. Technol. 37(2), 192 (2008)
Gehrmann, S., Deng, Y., Rush, A.M.: Bottom-up abstractive summarization. arXiv preprint arXiv:1808.10792 (2018)
Klein, G., Kim, Y., Deng, Y., Nguyen, V., Senellart, J., Rush, A.M.: OpenNMT: neural machine translation toolkit. arXiv preprint arXiv:1805.11462 (2018)
Klein, G., Kim, Y., Deng, Y., Senellart, J., Rush, A.M.: OpenNmt: open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810 (2017)
Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)
Luhn, H.P.: The automatic creation of literature abstracts. IBM J. Res. Dev. 2(2), 159–165 (1958)
Nallapati, R., Zhou, B., Gulcehre, C., Xiang, B., et al.: Abstractive text summarization using sequence-to-sequence RNNs and beyond. arXiv preprint arXiv:1602.06023 (2016)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Qi, W., et al.: ProphetNet: predicting future n-gram for sequence-to-sequence pre-training. arXiv preprint arXiv:2001.04063 (2020)
Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685 (2015)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Xue, L., et al.: mT5: a massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934 (2020)
Zaheer, M., et al.: Big bird: transformers for longer sequences. In: NeurIPS (2020)
Zhang, J., Zhao, Y., Saleh, M., Liu, P.: PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In: International Conference on Machine Learning, pp. 11328–11339. PMLR (2020)
Acknowledgement
The work presented here falls under the Research Project Grant No. IFC/4130/DST-CNRS/2018-19/IT25 (DST-CNRS targeted program). The authors would like to express gratitude to the Centre for Natural Language Processing and Artificial Intelligence Lab, Department of Computer Science and Engineering, National Institute of Technology Silchar, India for providing infrastructural facilities and support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tawmo, Adhikary, P.K., Dadure, P., Pakray, P. (2022). An Empirical Analysis on Abstractive Text Summarization. In: Mukhopadhyay, S., Sarkar, S., Dutta, P., Mandal, J.K., Roy, S. (eds) Computational Intelligence in Communications and Business Analytics. CICBA 2022. Communications in Computer and Information Science, vol 1579. Springer, Cham. https://doi.org/10.1007/978-3-031-10766-5_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-10766-5_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10765-8
Online ISBN: 978-3-031-10766-5
eBook Packages: Computer ScienceComputer Science (R0)