Abstract
Automatic extraction of biomedical relation from text becomes critical because manual relation extraction requires significant time and resources. The extracted medical relations can be used in clinical diagnosis, medical knowledge discovery, and so on. The benefits for pharmaceutical companies, health care providers, and public health are enormous. Previous studies have shown that both semantic information and dependent information in the corpus are helpful to relation extraction. In this paper, we propose a novel neural network, named RD-MAGCN, for biomedical relation extraction. We use Multi-head Attention model to extract semantic features, syntactic dependency tree, and Graph Convolution Network to extract structural features from the text, and finally R-Drop regularization method to enhance network performance. Extensive results on a medical corpus extracted from PubMed show that our model achieves better performance than existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Islamaj, R., Murray, C., Névéol, A.: Understanding PubMed user search behavior through log analysis. Database 6(1), 1–18 (2009)
Yu, M., Yin, W., Hasan, S., Santos, C., Xiang, B., Zhou, B.: Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 571–581 (2017)
Quirk, C., Poon, H.: Distant supervision for relation extraction beyond the sentence boundary. In: Proceedings of the European Chapter of the Association for Computational Linguistics, pp. 1171–1182 (2017)
Ekbal, A., Saha, S., Bhattacharyya, P.: A deep learning architecture for protein-protein interaction article identification. In: Proceedings of the 23rd International Conference on Pattern Recognition (ICPR), pp. 3128–3133 (2016)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Conference on Neural Information Processing Systems (NIPS), pp. 5998–6008 (2017)
Zhou, P., Shi, W., Tian, J., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 207–212 (2016)
Guo, X., Zhang, H., Yang, H., et al.: A single attention-based combination of CNN and RNN for relation classification. IEEE Access 7(3), 12467–12475 (2019)
Zhang, Y., Qi, P., Manning, D.: Graph convolution over pruned dependency trees improves relation extraction. In: Conference on Empirical Methods in Natural Language Processing, pp. 2205–2215 (2018)
Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1105–1116 (2016)
Peng, N., Poon, H., Quirk, C., et al.: Cross-sentence N-ary relation extraction with graph LSTMs. Trans. Assoc. Comput. Linguist. 5(2), 101–115 (2017)
Song, L., Zhang, Y., Wang, Z., et al.: N-ary relation extraction using graph-state LSTM. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 2226–2235 (2018)
Jin, L., Song, L., Zhang, Y., et al.: Relation extraction exploiting full dependency forests. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (2020)
Guo, Z., Zhang, Y., Lu, W.: Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 241–251 (2019)
Jin, Y., Li, J., Lian, Z., et al.: Supporting medical relation extraction via causality-pruned semantic dependency forest. In: Proceedings of the COLING, pp. 2450–2460 (2022)
Qian, M., et al.: Auto-learning convolution-based graph convolutional network for medical relation extraction. In: Lin, H., Zhang, M., Pang, L. (eds.) CCIR 2021. LNCS, vol. 13026, pp. 195–207. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88189-4_15
Tang, W., Wang, J., Lin, H., et al.: A syntactic information-based classification model for medical literature: algorithm development and validation study. JMIR Med. Inform. 10(8), 378–387 (2022)
Chen, X., Zhang, M., Xiong, S., et al.: On the form of parsed sentences for relation extraction. Knowl.-Based Syst. 25(1), 109–184 (2022)
Zhang, D., Liu, Z., Jia, W., et al.: Dual attention graph convolutional network for relation extraction. IEEE Trans. Knowl. Data Eng. 10(11), 1–14 (2023)
Martin, J., Gompel, R.: Handbook of Psycholinguistics. 2nd edn. (2006)
Lee, J., Yoon, W., Kim, S., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2019)
Peters, M., Neumann, M., Iyyer, M., et al.: Deep contextualized word representations. In: Proceedings of the North American Chapter of the Association for Computational Linguistics, pp. 2227–2237 (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Li, P., Mao, K., Yang, X., et al.: Improving relation extraction with knowledge-attention. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 229–239 (2019)
Bordes, A., Usunier, N., Weston, J., et al.: Translating embeddings for modeling multi-relational data. In: Proceedings of the International Conference on Neural Information Processing Systems, pp. 87–95 (2013)
Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the International Conference on Learning Representations (2017)
Lee, K., He, L., Lewis, M., et al.: End-to-end neural coreference resolution. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2017)
Hinton, G., Srivastava, N., Krizhevsky, A., et al.: Improving neural networks by preventing co-adaptation of feature detectors. Comput. Sci. 3(4), 212–223 (2012)
Liang, X., Wu, L., Li, J., et al.: R-Drop: regularized dropout for neural networks. In: Advances in Neural Information Processing Systems, vol. 34, no. 1, pp. 10890–10905 (2021)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference for Learning Representations, San Diego (2015)
Guo, Z., Nan, G., Lu, W., et al.: Learning latent forests for medical relation extraction. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pp. 3651–3675 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Huang, M., Wang, J., Lin, H., Yang, Z. (2024). Multi-head Attention and Graph Convolutional Networks with Regularized Dropout for Biomedical Relation Extraction. In: Xu, H., et al. Health Information Processing. CHIP 2023. Communications in Computer and Information Science, vol 1993. Springer, Singapore. https://doi.org/10.1007/978-981-99-9864-7_7
Download citation
DOI: https://doi.org/10.1007/978-981-99-9864-7_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9863-0
Online ISBN: 978-981-99-9864-7
eBook Packages: Computer ScienceComputer Science (R0)