Applying Self-interaction Attention for Extracting Drug-Drug Interactions

  • Luca PutelliEmail author
  • Alfonso E. GereviniEmail author
  • Alberto LavelliEmail author
  • Ivan SerinaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11946)


Discovering the effect of the simultaneous assumption of drugs is a very important field in medical research that could improve the effectiveness of healthcare and avoid adverse drug reactions which can cause health problems to patients. Although there are several pharmacological databases containing information on drugs, this type of information is often expressed in the form of free text. Analyzing sentences in order to extract drug-drug interactions was the objective of the DDIExtraction-2013 task. Despite the fact that the challenge took place six years ago, the interest on this task is still active and several new methods based on Recurrent Neural Networks and Attention Mechanisms have been designed. In this paper, we propose a model that combines bidirectional Long Short Term Memory (LSTM) networks with the Self-Interaction Attention Mechanism. Experimental analysis shows how this model improves the classification accuracy reducing the tendency to predict the majority class resulting in false negatives, over several input configurations.


  1. 1.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate (2014). cite arxiv:1409.0473Comment. Accepted at ICLR 2015 as oral presentation
  2. 2.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012).
  3. 3.
    Björne, J., Kaewphan, S., Salakoski, T.: UTurku: drug named entity recognition and drug-drug interaction extraction using SVM classification and domain knowledge. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation, SemEval 2013, pp. 651–659. Association for Computational Linguistics, Atlanta, June 2013.
  4. 4.
    Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734. Association for Computational Linguistics, Doha, October 2014.,
  5. 5.
    Chowdhury, M.F.M., Lavelli, A.: FBK-irst: a multi-phase kernel based approach for drug-drug interaction detection and classification that exploits linguistic information. In: Proceedings of the 7th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2013, Atlanta, Georgia, USA, 14–15 June 2013, pp. 351–355 (2013).
  6. 6.
    Du, J., Han, J., Way, A., Wan, D.: Multi-level structured self-attentions for distantly supervised relation extraction. CoRR abs/1809.00699 (2018).
  7. 7.
    Gers, F.A., Schmidhuber, J., Cummins, F.A.: Learning to forget: continual prediction with LSTM. Neural Comput. 12, 2451–2471 (2000)CrossRefGoogle Scholar
  8. 8.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.orgzbMATHGoogle Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–80 (1997). Scholar
  10. 10.
    Kadlec, R., Schmid, M., Bajgar, O., Kleindienst, J.: Text understanding with the attention sum reader network. CoRR abs/1603.01547 (2016)Google Scholar
  11. 11.
    Kumar, S., Anand, A.: Drug-drug interaction extraction from biomedical text using long short term memory network. CoRR abs/1701.08303 (2017)Google Scholar
  12. 12.
    Lee, J., et al.: BioBERT: pre-trained biomedical language representation model for biomedical text mining. arXiv preprint arXiv:1901.08746 (2019)
  13. 13.
    Li, L., Guo, Y., Qian, S., Zhou, A.: An end-to-end entity and relation extraction network with multi-head attention. In: Sun, M., Liu, T., Wang, X., Liu, Z., Liu, Y. (eds.) CCL/NLP-NABD -2018. LNCS (LNAI), vol. 11221, pp. 136–146. Springer, Cham (2018). Scholar
  14. 14.
    Li, Y., Yang, T.: Word embedding for understanding natural language: a survey. In: Srinivasan, S. (ed.) Guide to Big Data Applications. SBD, vol. 26, pp. 83–104. Springer, Cham (2018). Scholar
  15. 15.
    Liu, S., Tang, B., Chen, Q., Wang, X.: Drug-drug interaction extraction via convolutional neural networks. Comput. Math. Methods Med. 2016, 8 (2016)zbMATHGoogle Scholar
  16. 16.
    Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)
  17. 17.
    McDonald, R., Brokos, G., Androutsopoulos, I.: Deep relevance ranking using enhanced document-query interactions. CoRR abs/1809.01682 (2018)Google Scholar
  18. 18.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 26, pp. 3111–3119. Curran Associates, Inc. (2013).
  19. 19.
    Tarwani, K.M., Edem, S.: Survey on recurrent neural network in natural language processing. Int. J. Eng. Trends Technol. 48, 301–304 (2017). Scholar
  20. 20.
    Nagi, J., et al.: Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pp. 342–347, November 2011.
  21. 21.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  22. 22.
    Quan, C., Hua, L., Sun, X., Bai, W.: Multichannel convolutional neural network for biological relation extraction. BioMed. Res. Int. 2016, 10 (2016)Google Scholar
  23. 23.
    Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems. CoRR abs/1512.08756 (2015)Google Scholar
  24. 24.
    Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sig. Process. 45(11), 2673–2681 (1997)CrossRefGoogle Scholar
  25. 25.
    Segura-Bedmar, I., Martínez, P., Herrero-Zazo, M.: Lessons learnt from the DDIExtraction-2013 shared task. J. Biomed. Inform. 51, 152–164 (2014)CrossRefGoogle Scholar
  26. 26.
    Suárez-Paniagua, V., Segura-Bedmar, I.: Evaluation of pooling operations in convolutional architectures for drug-drug interaction extraction. BMC Bioinform. 19, 209 (2018). Scholar
  27. 27.
    Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)Google Scholar
  28. 28.
    Weiss, G., Provost, F.: The effect of class distribution on classifier learning: an empirical study. Technical report, Department of Computer Science, Rutgers University (2001)Google Scholar
  29. 29.
    Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). Scholar
  30. 30.
    Yi, Z., et al.: Drug-drug interaction extraction via recurrent neural network with multiple attention layers. In: Cong, G., Peng, W.-C., Zhang, W.E., Li, C., Sun, A. (eds.) ADMA 2017. LNCS (LNAI), vol. 10604, pp. 554–566. Springer, Cham (2017). Scholar
  31. 31.
    Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 2335–2344. Dublin City University and Association for Computational Linguistics, Dublin, August 2014.
  32. 32.
    Zhang, Y., Zheng, W., Lin, H., Wang, J., Yang, Z., Dumontier, M.: Drug-drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths. Bioinformatics 34(5), 828–835 (2018)CrossRefGoogle Scholar
  33. 33.
    Zhang, Y., Qi, P., Manning, C.D.: Graph convolution over pruned dependency trees improves relation extraction. CoRR abs/1809.10185 (2018)Google Scholar
  34. 34.
    Zheng, J., Cai, F., Shao, T., Chen, H.: Self-interaction attention mechanism-based text representation for document classification. Appl. Sci. 8(4), 613 (2018). Scholar
  35. 35.
    Zheng, W., et al.: An attention-based effective neural model for drug-drug interactions extraction. BMC Bioinform. 18, 445 (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Universitá degli Studi di BresciaBresciaItaly
  2. 2.Fondazione Bruno KesslerTrentoItaly

Personalised recommendations