Abstract
The Prevalence of information and its magnitude mandates a short description of the core of a document, an article, or legal documents. Abstractive summarization helps to concur with this problem utilizing the evolutions in machine learning and deep neural network. Attention-mechanism has extensively applied in the challenging issue of abstraction a text, in shorter length yet informative. We noticed in [13] after removing the attention layer from their proposed model, the performance only experience soft drawback, even can be ignored. Thus, motivates us to survey the latest models using attention-mechanism and its achievements, and the second objective is to run an experiment to compare standard stacked 3- Long Short-Term Memory (LSTM) layers incorporated with attention layer only (without any other hand-crafted algorithm) to explore how efficient this technique can generate better summarization, then a stand-alone model. The standard proposed model incorporated with attention-mechanism suffered from drawback performance and scored less than a stand-alone model by at least 6 point scores on ROUGE-1&2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mohd, M., Jan, R., Shah, M.: Text document summarization using word embedding. Expert Syst. Appl. 143, 112958 (2020)
Gupta, S., Gupta, S.K.: Abstractive summarization: an overview of the state of the art. Expert Syst. Appl. 121, 49–65 (2019)
Lin, H., Ng, V.: Abstractive summarization: a survey of the state of the art. Proc. AAAI Conf. Artif. Intell. 33(01), 9815–9822 (2019)
Liu, Y., Titov, I., Lapata, M.: Single document summarization as tree induction. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long and Short Papers), vol. 1, pp. 1745–1755 (2019)
Liu, K., Koyuncu, A., Kim, D., Bissyandé, T.F.: Tbar: revisiting template-based automated program repair. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 31–42 (2019)
MacAvaney, S., Sotudeh, S., Cohan, A., Goharian, N., Talati, I., Filice, R.W.: Ontology-aware clinical abstractive summarization. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1013–1016 (2019)
Mallick, C., Das, A.K., Dutta, M., Das, A.K., Sarkar, A.: Graph-based text summarization using modified TextRank. In: Nayak, J., Abraham, A., Krishna, B.M., Chandra Sekhar, G.T., Das, A.K. (eds.) Soft Computing in Data Analytics. AISC, vol. 758, pp. 137–146. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-0514-6_14
Ullah, S., Al Islam, A.B.M.A.: A framework for extractive text summarization using semantic graph based approach. In: Proceedings of the 6th International Conference on Networking, Systems and Security, pp. 48–56 (2019)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv1409.0473 (2014)
Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. arXiv1509.00685 (2015)
Yuan, C., Bao, Z., Sanderson, M., Tang, Y.: Incorporating word attention with convolutional neural networks for abstractive summarization. World Wide Web 23(1), 267–287 (2019)
Sheela, J., Janet, B.: An abstractive summary generation system for customer reviews and news article using deep learning. J. Ambient Intell. Humaniz. Comput. 12(7), 7363–7373 (2020)
Chen, Y., Ma, Y., Mao, X., Li, Q.: Multi-task learning for abstractive and extractive summarization. Data Sci. Eng. 4(1), 14–23 (2019)
Gao, Y., Wang, Y., Liu, L., Guo, Y., Huang, H.: Neural abstractive summarization fusing by global generative topics. Neural Comput. Appl. 32(9), 5049–5058 (2019)
Luong, M.-T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv1508.04025 (2015)
Lin, J., Sun, X., Ma, S., Su, Q.: Global encoding for abstractive summarization. arXiv1805.03989 (2018)
Acknowledgment
This work is a part of a project submitted to The British University in Dubai.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
AlMazrouei, R.Z., Nelci, J., Salloum, S.A., Shaalan, K. (2022). Feasibility of Using Attention Mechanism in Abstractive Summarization. In: Al-Emran, M., Al-Sharafi, M.A., Al-Kabi, M.N., Shaalan, K. (eds) Proceedings of International Conference on Emerging Technologies and Intelligent Systems. ICETIS 2021. Lecture Notes in Networks and Systems, vol 299. Springer, Cham. https://doi.org/10.1007/978-3-030-82616-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-82616-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82615-4
Online ISBN: 978-3-030-82616-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)