Trigger Words Detection by Integrating Attention Mechanism into Bi-LSTM Neural Network—A Case Study in PubMED-Wide Trigger Words Detection for Pancreatic Cancer

  • Kaiyin Zhou
  • Xinzhi Yao
  • Shuguang Wang
  • Jin-Dong Kim
  • Kevin Bretonnel Cohen
  • Ruiying Chen
  • Yuxing Wang
  • Jingbo XiaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11221)


A Bi-LSTM based encode/decode mechanism for named entity recognition was studied in this research. In the proposed mechanism, Bi-LSTM was used for encoding, an Attention method was used in the intermediate layers, and an unidirectional LSTM was used as decoder layer. By using element wise product to modify the conventional decoder layers, the proposed model achieved better F-score, compared with other three baseline LSTM-based models. For the purpose of algorithm application, a case study of causal gene discovery in terms of disease pathway enrichment was designed. In addition, the causal gene discovery rate of our proposed method was compared with another baseline methods. The result showed that trigger genes detection effectively increase the performance of a text mining system for causal gene discovery.


Natural language processing LSTM Encoder/decoder model Trigger words 



This work is funded by the Fundamental Research Funds for the Central Universities of China (Project No. 2662018PY096).


  1. 1.
    Nadeau, D., Sekine, S.: A survey of named entity recognition and classification. Lingvist. Investig. 30(1), 326 (2007)Google Scholar
  2. 2.
    Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need (2017)Google Scholar
  3. 3.
    Sintchenko, V., Anthony, S., Phan, X.H., Lin, F., Coiera, E.W.: A PubMed-wide associational study of infectious diseases. PLoS One 5(3), e9535 (2010)CrossRefGoogle Scholar
  4. 4.
    Allot, A., Peng, Y., Wei, C.H., Lee, K., Phan, L., Lu, Z.: LitVar: a semantic search engine for linking genomic variant data in PubMed and PMC. Nucl. Acids Res. 46(W1), W530–W536 (2018)CrossRefGoogle Scholar
  5. 5.
    Cohen, K.B., et al.: High-precision biological event extraction with a concept recognizer. In: Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task, 5 June 2009, pp. 50–58. Association for Computational Linguistics (2009)Google Scholar
  6. 6.
    Song, M., Kim, M., Kang, K., Kim, Y.H., Jeon, S.: Application of public knowledge discovery tool (PKDE4J) to represent biomedical scientific knowledge. Front. Res. Metr. Anal. 3, 7 (2018)CrossRefGoogle Scholar
  7. 7.
    Zhou, H., Yang, Y., Ning, S., Liu, Z., Lang, C., Lin, Y., Huang, D.: Combining context and knowledge representations for chemical-disease relation extraction. IEEE/ACM Trans. Comput. Biol. Bioinform. (2018).
  8. 8.
    Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
  9. 9.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
  10. 10.
    Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)
  11. 11.
    Zheng, S., Hao, Y., Lu, D., et al.: Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257, 1–8 (2017)CrossRefGoogle Scholar
  12. 12.
    Liu, B., Lane, I.: Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint arXiv:1609.01454, 6 September 2016
  13. 13.
    Wang, Y., et al.: Guideline design of an active gene annotation corpus for the purpose of drug repurposing. In: OHDSI 2018 Workshop, July, Guangzhou (2018, submitted)Google Scholar
  14. 14.
    Kim, J.D., Wang, Y.: PubAnnotation: a persistent and sharable corpus and annotation repository. In: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing, pp. 202–205. Association for Computational Linguistics (2012)Google Scholar
  15. 15.
    Wang, Z.Y., Zhang, H.Y.: Rational drug repositioning by medical genetics. Nat. Biotechnol. 31(12), 1080–1082 (2013)CrossRefGoogle Scholar
  16. 16.
    Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF (2016)Google Scholar
  17. 17.
    Huang, E.H., Socher, R., Manning, C.D., Ng, A.Y.: Improving word representations via global context and multiple word prototypes. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, pp. 873–882. Association for Computational Linguistics (2012)Google Scholar
  18. 18.
    Pavlopoulos, I., Kosmopoulos, A., Androutsopoulos, I.: Continuous space word vectors obtained by applying Word2Vec to abstracts of biomedical articles (2014)Google Scholar
  19. 19.
    Luong, M.-T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)
  20. 20.
    Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation. arXiv preprint arXiv:1412.2007 (2014)
  21. 21.
    Wei, C.H., Kao, H.Y., Lu, Z.: PubTator: a web-based text mining tool for assisting biocuration. Nucl. Acids Res. 41(W1), W518–W522 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Kaiyin Zhou
    • 1
    • 2
  • Xinzhi Yao
    • 1
  • Shuguang Wang
    • 1
  • Jin-Dong Kim
    • 3
  • Kevin Bretonnel Cohen
    • 4
  • Ruiying Chen
    • 1
  • Yuxing Wang
    • 1
    • 2
  • Jingbo Xia
    • 1
    • 2
    Email author
  1. 1.College of InformaticsHuazhong Agricultural UniversityWuhanChina
  2. 2.Hubei Key Laboratory of Agricultural BioinformaticsWuhanChina
  3. 3.Database Center for Life Science (DBCLS)Research Organization of Information and Systems (ROIS)TokyoJapan
  4. 4.School of MedicineUniversity of Colorado DenverAuroraUSA

Personalised recommendations