Skip to main content
SpringerLink
Log in

General fine-grained event detection based on fusion of multi-information representation and attention mechanism

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Event extraction is an important field in information extraction, which aims to extract key information from unstructured text automatically. Event extraction is mainly divided into trigger identification and classification. The existing models are deficient in sentence representation in the initial word embeddings training process, which makes it difficult to capture the deep bidirectional representation and can’t handle the semantic information of the context well, thus affecting the performance of event detection. In this paper, a model BMRMC (BERT + Mean pooling layer + Relative position in multi-head attention + CRF) based on multi-information representation and attention mechanism is proposed. Firstly, the BERT pre-training model based on a bidirectional training transformer is used to embed words and extract word-level features. Then the sentence-level semantic representation is fused by mean pooling layer. In addition, relative position is combined with multi-head attention, which can strengthen the connection of contents. Finally, the sequences are labeled by CRF based on the BIO-labeling mechanism. The experimental results show that the proposed model BMRMC improves the performance of event detection, and the F value on the MAVEN dataset is 67.74%, which achieves state-of-the-art performance in the general fine-grained event detection task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Data availability

The dataset can be obtained from https://cloud.tsinghua.edu.cn/d/874e0ad810f34272a03b/.

References

  1. Wei X, Bang W (2019) A survey of event extraction from text. IEEE Access 99:1–1

    Google Scholar 

  2. Wei H, Ai Z et al (2021) Biomedical event trigger extraction based on multi-layer residual BiLSTM and contextualized word representations. Int J Mach Learn Cybern 12(18):1–13

    Google Scholar 

  3. Wan QZ, Wan CX, Hu R et al (2021) Chinese financial event extraction based on syntactic and semantic dependency parsing. Chin J Comput 44(3):508–530

    Google Scholar 

  4. Ribeiro S, Olivier F, and Xavier T (2017) Unsupervised event clustering and aggregation from newswire and web articles. In: Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, pp 62–67

  5. Yu SQ, Bin W (2018) Exploiting structured news information to improve event detection via dual-level clustering. In: IEEE Third International Conference on Data Science in Cyberspace, pp 873–880

  6. Björne J (2014) Biomedical event extraction with machine learning. University of Turku, Turku

    Google Scholar 

  7. Yu H, Wen Z et al. (2018) Self-regulation: employing a generative adversarial network to improve event detection. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, volume 1: Long Papers, Association for Computational Linguistics, pp 515–526

  8. Nie YF, Rong WG et al (2015) Embedding assisted prediction architecture for event trigger identification. J Bioinform Comput Biol 13(3):1540001

    Article  Google Scholar 

  9. Liu SL, Chen YB et al. (2017) Exploiting argument information to improve event detection via supervised attention mechanisms. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, volume 1: Long Papers, Association for Computational Linguistics, pp 1789–1798

  10. Liu SL, Chen YB et al. (2016) Leveraging FrameNet to improve automatic event detection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, volume 1: Long Papers, The Association for Computer Linguistics, pp 98–108

  11. Zhan LY, Jiang XP, Liu Q (2021) Research on Chinese event extraction method based on HMM and multi-stage method. J Phys: Conf Ser 1732(1):1–4

    Google Scholar 

  12. Collins M, Singer Y (1999) Unsupervised models for named entity classification. In: Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp 100–110

  13. Kim JT, Moldovan DI (1995) Acquisition of linguistic patterns for knowledge-based information extraction. IEEE Trans Knowl Data Eng 7(5):713–724

    Article  Google Scholar 

  14. Li Q, Ji H, Huang L (2013) Joint event extraction via structured prediction with global features. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, volume 1: Long Papers, pp 73–82

  15. Ahn D (2006) The stages of event extraction. In: Proceedings of the Workshop on Annotating and Reasoning about Time and Events. MIT Press, Cambridge, pp 1–8

  16. Saha S, Majumder A et al. (2011) Bio-molecular event extraction using support vector machine. In: Proceedings of the 3rd International Conference on Advanced Computing, Piscataway: IEEE, pp 298–303

  17. Nguyen TH, Grishman R (2015) Event detection and domain adaptation with convolutional neural network. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Stroudsburg, ACL, pp 365–371

  18. Liu J, Chen Y et al. (2018) Event detection via gated multilingual attention mechanism. In: AAAI Conference on Artificial Intelligence; Innovative Applications of Artificial Intelligence Conference; Symposium on Educational Advances in Artificial Intelligence in Proc. 32nd AAAI Conf, pp 4865–4872

  19. Tong M H, Bin X et al. (2020) Improving event detection via open-domain trigger knowledge. In: Association for Computational Linguistics in ACL, pp 5887–5897

  20. Xiang LA, Mc B et al (2021) A hybrid medical text classification framework: integrating attentive rule construction and neural network. Neurocomputing 443:345–355

    Article  Google Scholar 

  21. Xu R X, Liu T Y, Li L, et al. (2021) Document-level event extraction via heterogeneous graph-based interaction model with a tracker. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp 3533–3546

  22. Cai MD, Shen GH, Huang ZQ (2022) Semi-supervised learning key-phrase extraction method without manual annotation. J Chin Comput Syst. https://doi.org/10.20009/j.cnki.21-1106/TP

    Article  Google Scholar 

  23. Sheng J W, Guo S, Yu B W, et al. (2021) CasEE: a joint learning framework with cascade decoding for overlapping event extraction. In: The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp 164–174

  24. Vaswani A, Shazeer N et al. (2017) Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp 6000–6010

  25. Peters M E, Neumann M et al. (2018) Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. New Orleans, pp 2227

  26. Radford A, Karthik N et al. (2018) Improving language understanding by generative pre-training. Technical report, OpenAI, CA

  27. Devlin J, Chang M W et al. (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Conference on the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

  28. Chen Y B, Yang H, Liu K, et al. (2018) Collective event detection via a hierarchical and bias tagging networks with gated multi-level attention mechanisms. In: Proceedings of EMNLP: Conference on Empirical Methods in Natural Language Processing, pp d18–1158

  29. Lou D F, Liao Z L, Deng S M et al. (2021) MLBiNet: a cross-sentence collective event detection network. In: Proceedings of ACL: Annual Meeting of the Association for Computational Linguistics, acl-long.373

  30. Ren YG, Yan G, He XY (2023) general fine-grained event detection based on multi-scale CNN and CRF. J Chin Comput Systms: 1-8 [2023-03-09]. http://kns.cnki.net/kcms/detail/21.1106.tp.20230216.1653.003.html

  31. Wang X Z, Zi Q W et al. (2020) MAVEN: a massive general domain event detection dataset. In: EMNLP: Online, Association for Computational Linguistics, pp 1652–1671

  32. Pyysalo S, Ohta T, Miwa M et al (2012) Event extraction across multiple levels of biological organization. Bioinformatics 28(18):i575–i581

    Article  Google Scholar 

  33. Chen Y B, Li H X et al. (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: ACL-IJCNLP, as sociation for Computational Linguistics, pp 167–176

  34. Hochreiter S, Jürgen S (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  35. John L, McCallum A et al. (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. Department of Computer & Information Science, pp 6–28

  36. Yan H R, Xiao L J et al. (2019) Event detection with multi-order graph convolution and aggregated attention. In: EMNLP, Association for Computational Linguistics, pp 5765–5769

  37. Wang X Z, Xu H et al. (2019) Adversarial training for weakly supervised event detection. In: NAACL, Minneapolis, Minnesota. Association for Computational Linguistics, pp 998–718 1008

  38. Zhou D, Zhong D (2015) A semi-supervised learning framework for biomedical event extraction based on hidden topics. Artif Intell Med 64(1):51–58

    Article  Google Scholar 

  39. Wang J, Zhang J, Yuan A, et al. (2015) Biomedical event trigger detection by dependency-based word embedding. In: Proceeding of IEEE International Conference on Bioinformatics and Biomedicine. IEEE, pp 429–432

  40. Hai LT, Thy TT, Duong K et al (2020) DeepEventMine: end-to-end neural nested event extraction from biomedical texts. Bioinformatics 36(19):4910–4917

    Article  Google Scholar 

  41. Chen Y (2019) Multiple-level biomedical event trigger recognition with transfer learning. BMC Bioinform 20:459

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 62006108, 61976109), Postdoctoral Research Foundation of China (No. 2022M710593), Liaoning Provincial Science and Technology Fund project (No. 2021-BS-201), Scientific Research Project of Liaoning Province (No. LJKZ0963), Key Research and Development Project of Science and Technology Department of Liaoning Province (No. 2022JH2/101300271), Liaoning Revitalization Talents Program (No. XLYC2006005); Liaoning Provincial Key Laboratory Special Fund; Dalian Key Laboratory Special Fund.

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Liaoning Normal University, Dalian, Liaoning, China

    Xinyu He, Ge Yan & Yonggong Ren

  2. Information and Communication Engineering Postdoctoral Research Station, Dalian University of Technology, Dalian, Liaoning, China

    Xinyu He

  3. Postdoctoral Workstation of Dalian Yongjia Electronic Technology Co., Ltd, Dalian, Liaoning, China

    Xinyu He

  4. College of Computer Science and Technology, Shenyang University of Chemical Technology, Shenyang, Liaoning, China

    Changfu Si

Authors
  1. Xinyu He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ge Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changfu Si
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yonggong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XH participated in the general fine-grained event detection and drafted the manuscript. GY proposed a model based on the fusion of multi-information representation and attention mechanism, designed the experiments, and drafted the manuscript. CS reviewed and edited the manuscript. YR reviewed and edited the manuscript. All the authors read and approved the final manuscript.

Corresponding authors

Correspondence to Changfu Si or Yonggong Ren.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, X., Yan, G., Si, C. et al. General fine-grained event detection based on fusion of multi-information representation and attention mechanism. Int. J. Mach. Learn. & Cyber. 14, 4393–4403 (2023). https://doi.org/10.1007/s13042-023-01900-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01900-y

Keywords

Search

Navigation