Skip to main content
SpringerLink
Log in

A hierarchical framework based on transformer technology to achieve factual consistent and non-redundant abstractive text summarization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Abstractive summarization is one of the popular topics that has been the researchers’ attention for several years. This is because of the widespread application frameworks included in this field. Most of the existing summarization frameworks cannot provide effective abstracts as the contextual information of the input is not given importance. To deal with the problem, this work introduces a hierarchical framework using transformer technology to produce effective abstracts. The proposed framework includes preprocessing, extractive summarization, and abstractive summarization as the basic steps of the work. Initially, the input contents are preprocessed to obtain a clean document, and then the contents are provided to the extractive summarization unit. This unit consists of a fine-tuned BERTSum model (FTBS), which is a pre-trained model to produce the required extractive summary. The output is then provided to the proposed convolutional bidirectional gated recurrent unit transformer (CBi-GRUT) model, where an additional encoder model is introduced with the traditional transformer technology to obtain the output. The outcomes of the model are then assessed with the existing models to prove its efficacy, and the evaluations are carried out using the CNN/Daily Mail dataset. The proposed method achieved an average ROUGE-1 score of 0.78, average ROUGE-2 score of 0.68 and an average ROUGE-L score of 0.77.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Alomari A, Idris N, Sabri AQ, Alsmadi I (2022) Deep reinforcement and transfer learning for abstractive text summarization: A review. Comput Speech Lang 71:101276

    Article  Google Scholar 

  2. Saiyyad MM, Patil NN (2022) The State of the Art Text Summarization Techniques. Appl Comput Technol: Proc ICCET 2022:434–447

    Article  Google Scholar 

  3. Huang Y, Feng X, Feng X, Qin B (2021) The factual inconsistency problem in abstractive text summarization: A survey. arXiv preprint arXiv:2104.14839

  4. El-Kassas WS, Salama CR, Rafea AA, Mohamed HK (2021) Automatic text summarization: A comprehensive survey. Expert Syst Appl 165:113679

    Article  Google Scholar 

  5. Syed AA, Gaol FL, Matsuo T (2021) A survey of the state-of-the-art models in neural abstractive text summarization. IEEE Access 9:13248–13265

    Article  Google Scholar 

  6. Magdum PG, Rathi S (2021) A survey on deep learning-based automatic text summarization models. In Advances in Artificial Intelligence and Data Engineering: Select Proceedings of AIDE 2019, Springer Singapore 377–392

  7. Moradi M, Dorffner G, Samwald M (2020) Deep contextualized embeddings for quantifying the informative content in biomedical text summarization. Comput Methods Programs Biomed 184:105117

    Article  Google Scholar 

  8. Ma C, Zhang WE, Guo M, Wang H, Sheng QZ (2022) Multi-document summarization via deep learning techniques: A survey. ACM Comput Surv 55(5):1–37

    Article  Google Scholar 

  9. Abu Nada AM, Alajrami E, Al-Saqqa AA, Abu-Naser SS. Arabic text summarization using arabert model using extractive text summarization approach

  10. Abualigah L, Bashabsheh MQ, Alabool H, Shehab M (2020) Text summarization: a brief review. Recent Advances in NLP: the case of Arabic language. 1–5

  11. Belwal RC, Rai S, Gupta A (2021) Text summarization using topic-based vector space model and semantic measure. Inf Process Manage 58(3):102536

    Article  Google Scholar 

  12. Jia R, Cao Y, Tang H, Fang F, Cao C, Wang S (2020) Neural extractive summarization with hierarchical attentive heterogeneous graph network. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) 3622–3631

  13. Nambiar SK, Peter SD, Idicula SM (2021) Abstractive summarization of Malayalam document using sequence to sequence model. In 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS). IEEE 1: 347–352

  14. Awasthi I, Gupta K, Bhogal PS, Anand SS, Soni PK (2021) Natural language processing (NLP) based text summarization-a survey. In 2021 6th International Conference on Inventive Computation Technologies (ICICT). IEEE 1310–1317

  15. Afzal M, Alam F, Malik KM, Malik GM (2020) Clinical context–aware biomedical text summarization using deep neural network: model development and validation. J Med Internet Res 22(10):e19810

    Article  Google Scholar 

  16. Haider MM, Hossin MA, Mahi HR, Arif H (2020) Automatic text summarization using gensim word2vec and k-means clustering algorithm. In 2020 IEEE Region 10 Symposium (TENSYMP). IEEE 283–286

  17. Yang M, Wang X, Lu Y, Lv J, Shen Y, Li C (2020) Plausibility-promoting generative adversarial network for abstractive text summarization with multi-task constraint. Inf Sci 521:46–61

    Article  Google Scholar 

  18. Lamsiyah S, El Mahdaouy A, Espinasse B, Ouatik SE (2021) An unsupervised method for extractive multi-document summarization based on centroid approach and sentence embeddings. Expert Syst Appl 167:114152

    Article  Google Scholar 

  19. Suleiman D, Awajan A (2020) Deep learning based abstractive text summarization: approaches, datasets, evaluation measures, and challenges. Math Probl Eng 2020:1–29

    Article  Google Scholar 

  20. Zad S, Heidari M, Hajibabaee P, Malekzadeh M (2021) A survey of deep learning methods on semantic similarity and sentence modeling. In2021 IEEE 12th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) 0466–0472

  21. Aliakbarpour H, Manzuri MT, Rahmani AM (2022) Improving the readability and saliency of abstractive text summarization using combination of deep neural networks equipped with auxiliary attention mechanism. The Journal of Supercomputing. 1–28

  22. Poornima M, Pulipati VR, Sunil Kumar T (2022) Abstractive multi-document summarization using deep learning approaches. In Proceedings of Second International Conference on Advances in Computer Engineering and Communication Systems: ICACECS 2021, Singapore: Springer Nature Singapore 57–68

  23. Alahmadi D, Wali A, Alzahrani S (2022) TAAM: Topic-aware abstractive arabic text summarisation using deep recurrent neural networks. J King Saud Univ-Comput Inform Sci 34(6):2651–2665

    Google Scholar 

  24. Abdi A, Hasan S, Shamsuddin SM, Idris N, Piran J (2021) A hybrid deep learning architecture for opinion-oriented multi-document summarization based on multi-feature fusion. Knowl-Based Syst 213:106658

    Article  Google Scholar 

  25. Liao W, Ma Y, Yin Y, Ye G, Zuo D (2021) Improving abstractive summarization based on dynamic residual network with reinforce dependency. Neurocomputing 448:228–237

    Article  Google Scholar 

  26. Mohsin M, Latif S, Haneef M, Tariq U, Khan MA, Kadry S, Yong HS, Choi JI (2021) Improved Text Summarization of News Articles Using GA-HC and PSO-HC. Appl Sci 11(22):10511

    Article  Google Scholar 

  27. Xu W, Nong G (2022) A study for extracting keywords from data with deep learning and suffix array. Multimed Tools App 81(5):7419–7437

    Article  Google Scholar 

  28. Muneera NM, Sriramya P (2022) Abstractive text summarization employing ontology-based knowledge-aware multi-focus conditional generative adversarial network (OKAM-CGAN) with hybrid pre-processing methodology. Multimed Tools App 82:23331

    Google Scholar 

  29. Moratanch N, Chitrakala S (2023) Anaphora resolved abstractive text summarization (AR-ATS) system. Multimed Tools App 82(3):4569–4597

    Article  Google Scholar 

  30. Song S, Huang H, Ruan T (2019) Abstractive text summarization using LSTM-CNN based deep learning. Multimed Tools App 78:857–875

    Article  Google Scholar 

  31. Li Y, Huang Y, Huang W and Wang W (2023) A global and local information extraction model incorporating selection mechanism for abstractive text summarization. Multimed Tools App 1–28

  32. Abualigah L, Diabat A, Mirjalili S, Abd Elaziz M, Gandomi AH (2021) The arithmetic optimization algorithm. Comput Methods Appl Mech Eng 376:113609

    Article  MathSciNet  Google Scholar 

Download references

Funding

No funding is provided for the preparation of manuscript.

Author information

Authors and Affiliations

  1. Research Scholar in CSE, GITAM (Deemed To Be University), Hyderabad, Telangana, 502329, India

    G. Swetha

  2. Department of Computer Science and Engineering, GITAM (Deemed To Be University), Hyderabad, Telangana, 502329, India

    S. Phani Kumar

Authors
  1. G. Swetha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Phani Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have equal contributions in this work.

Corresponding author

Correspondence to S. Phani Kumar.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Consent to participate

All the authors involved have agreed to participate in this submitted article.

Consent to publish

All the authors involved in this manuscript give full consent for publication of this submitted article.

Conflict of interest

Authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Swetha, G., Kumar, S.P. A hierarchical framework based on transformer technology to achieve factual consistent and non-redundant abstractive text summarization. Multimed Tools Appl 83, 47587–47608 (2024). https://doi.org/10.1007/s11042-023-17426-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17426-y

Keywords

Search

Navigation