Skip to main content

Feasibility of Using Attention Mechanism in Abstractive Summarization

  • Conference paper
  • First Online:
Proceedings of International Conference on Emerging Technologies and Intelligent Systems (ICETIS 2021)

Abstract

The Prevalence of information and its magnitude mandates a short description of the core of a document, an article, or legal documents. Abstractive summarization helps to concur with this problem utilizing the evolutions in machine learning and deep neural network. Attention-mechanism has extensively applied in the challenging issue of abstraction a text, in shorter length yet informative. We noticed in [13] after removing the attention layer from their proposed model, the performance only experience soft drawback, even can be ignored. Thus, motivates us to survey the latest models using attention-mechanism and its achievements, and the second objective is to run an experiment to compare standard stacked 3- Long Short-Term Memory (LSTM) layers incorporated with attention layer only (without any other hand-crafted algorithm) to explore how efficient this technique can generate better summarization, then a stand-alone model. The standard proposed model incorporated with attention-mechanism suffered from drawback performance and scored less than a stand-alone model by at least 6 point scores on ROUGE-1&2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mohd, M., Jan, R., Shah, M.: Text document summarization using word embedding. Expert Syst. Appl. 143, 112958 (2020)

    Article  Google Scholar 

  2. Gupta, S., Gupta, S.K.: Abstractive summarization: an overview of the state of the art. Expert Syst. Appl. 121, 49–65 (2019)

    Article  Google Scholar 

  3. Lin, H., Ng, V.: Abstractive summarization: a survey of the state of the art. Proc. AAAI Conf. Artif. Intell. 33(01), 9815–9822 (2019)

    Google Scholar 

  4. Liu, Y., Titov, I., Lapata, M.: Single document summarization as tree induction. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long and Short Papers), vol. 1, pp. 1745–1755 (2019)

    Google Scholar 

  5. Liu, K., Koyuncu, A., Kim, D., Bissyandé, T.F.: Tbar: revisiting template-based automated program repair. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 31–42 (2019)

    Google Scholar 

  6. MacAvaney, S., Sotudeh, S., Cohan, A., Goharian, N., Talati, I., Filice, R.W.: Ontology-aware clinical abstractive summarization. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1013–1016 (2019)

    Google Scholar 

  7. Mallick, C., Das, A.K., Dutta, M., Das, A.K., Sarkar, A.: Graph-based text summarization using modified TextRank. In: Nayak, J., Abraham, A., Krishna, B.M., Chandra Sekhar, G.T., Das, A.K. (eds.) Soft Computing in Data Analytics. AISC, vol. 758, pp. 137–146. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-0514-6_14

    Chapter  Google Scholar 

  8. Ullah, S., Al Islam, A.B.M.A.: A framework for extractive text summarization using semantic graph based approach. In: Proceedings of the 6th International Conference on Networking, Systems and Security, pp. 48–56 (2019)

    Google Scholar 

  9. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv1409.0473 (2014)

    Google Scholar 

  10. Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. arXiv1509.00685 (2015)

    Google Scholar 

  11. Yuan, C., Bao, Z., Sanderson, M., Tang, Y.: Incorporating word attention with convolutional neural networks for abstractive summarization. World Wide Web 23(1), 267–287 (2019)

    Article  Google Scholar 

  12. Sheela, J., Janet, B.: An abstractive summary generation system for customer reviews and news article using deep learning. J. Ambient Intell. Humaniz. Comput. 12(7), 7363–7373 (2020)

    Article  Google Scholar 

  13. Chen, Y., Ma, Y., Mao, X., Li, Q.: Multi-task learning for abstractive and extractive summarization. Data Sci. Eng. 4(1), 14–23 (2019)

    Article  Google Scholar 

  14. Gao, Y., Wang, Y., Liu, L., Guo, Y., Huang, H.: Neural abstractive summarization fusing by global generative topics. Neural Comput. Appl. 32(9), 5049–5058 (2019)

    Article  Google Scholar 

  15. Luong, M.-T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv1508.04025 (2015)

    Google Scholar 

  16. Lin, J., Sun, X., Ma, S., Su, Q.: Global encoding for abstractive summarization. arXiv1805.03989 (2018)

    Google Scholar 

Download references

Acknowledgment

This work is a part of a project submitted to The British University in Dubai.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Said A. Salloum .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

AlMazrouei, R.Z., Nelci, J., Salloum, S.A., Shaalan, K. (2022). Feasibility of Using Attention Mechanism in Abstractive Summarization. In: Al-Emran, M., Al-Sharafi, M.A., Al-Kabi, M.N., Shaalan, K. (eds) Proceedings of International Conference on Emerging Technologies and Intelligent Systems. ICETIS 2021. Lecture Notes in Networks and Systems, vol 299. Springer, Cham. https://doi.org/10.1007/978-3-030-82616-1_2

Download citation

Publish with us

Policies and ethics