A Multi-domain Named Entity Recognition Method Based on Part-of-Speech Attention Mechanism

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1042)


Named entity recognition is an important and basic work in text mining. To overcome the shortcomings of existing multi-domain named entity recognition methods, a multi-domain named entity recognition method based on the part-of-speech attention mechanism, called BiLSTM-ATTENTION-CRF, was proposed in this paper. The domain dictionary was constructed to represent multi-domain semantic information and the BiLSTM network was used to capture the grammatical and syntactic features, as well as multi-domain semantic features in context information. A part-of-speech attention mechanism was designed to obtain the contribution weight of part-of-speech for entity recognition. Finally, a group of experiments were performed on the multi-domain dataset to compare various fusion strategies of multi-level entity information. The experimental results show that BiLSTM-ATTENTION-CRF has a high precision and recall rate, and can effectively recognizes the multi-domain named entities.


Multi-domain entity recognition Attention mechanism BiLSTM CRF 



The work received support from Science and Technology Project of Beijing Municipal Commission of Education (No. KM201710005026), National Basic Research Program of China (No. 2014CB744600), Open Foundation of Beijing Key Laboratory of MRI and Brain Informatics, Open Foundation of Beijing Key Laboratory of Multimedia and Intelligent Software (Beijing University of Technology).


  1. 1.
    Mikheev, A., Moens, M., Grover, C.: Named entity recognition without gazetteers. In: Ninth Conference on European Chapter of the Association for Computational Linguistics (EACL 1999), pp. 1–8. ACM Press (1999)Google Scholar
  2. 2.
    Chandel, A., Nagesh, P.C., Sarawagi, S.: Efficient batch top-k search for dictionary-based entity recognition. In: 22nd International Conference on Data Engineering (ICDE 2006), p. 28. ACM Press (2006)Google Scholar
  3. 3.
    Abacha, A.B., Zweigenbaum, P.: Medical entity recognition: a comparison of semantic and statistical methods. In: BioNLP 2011 Workshop (BioNLP 2011), pp. 56–64. ACM Press (2011)Google Scholar
  4. 4.
    Eftimov, T., Seljak, B.K., Korošec, P.: A rule-based named-entity recognition method for knowledge extraction of evidence-based dietary recommendations. PLoS One 12(6), e0179488 (2017)CrossRefGoogle Scholar
  5. 5.
    Gandhe, A., Rastrow, A., Hoffmeister, B.: Scalable language model adaptation for spoken dialogue systems. In: 2018 IEEE Spoken Language Technology Workshop (SLT 2018) (2018)Google Scholar
  6. 6.
    Teixeira, J., Sarmento, L., Oliveira, E.: A bootstrapping approach for training a NER with conditional random fields. In: Antunes, L., Pinto, H.S. (eds.) EPIA 2011. LNCS (LNAI), vol. 7026, pp. 664–678. Springer, Heidelberg (2011). Scholar
  7. 7.
    Ju, Z.F., Wang, J., Zhu, F.: Named entity recognition from biomedical text using SVM. In: 5th International Conference on Bioinformatics and Biomedical Engineering. IEEE Press (2011).
  8. 8.
    Morwal, S., Jahan, N., Chopra, D.: Named entity recognition using hidden markov model (HMM). Int. J. Nat. Lang. Comput. (IJNLC) 1(4), 15–23 (2012)CrossRefGoogle Scholar
  9. 9.
    Ding, P., Zhou, X.B., Zhang X.J., Wang, J., Lei, Z.F.: An attentive neural sequence labeling model for adverse drug reactions mentions extraction. IEEE Access 6 (2018).
  10. 10.
    Lipenkova, J.: A system for fine-grained aspect-based sentiment analysis of Chinese. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2015), pp. 55–60 (2015)Google Scholar
  11. 11.
    Liu, P.F., Joty, S., Meng, H.: Fine-grained opinion mining with recurrent neural networks and word embeddings. In: 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 1433–1443. ACL (2015)Google Scholar
  12. 12.
    Liu, Q., Liu, B., Zhang, Y., Kim, D.S., Gao, Z.: Improving opinion aspect extraction using semantic similarity and aspect associations. ACM SIGARCH Comput. Archit. News 44(3), 506–518 (2016)CrossRefGoogle Scholar
  13. 13.
    Silver, D.L., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: AAAI 2013 Spring Symposium on Lifelong Machine Learning (2013)Google Scholar
  14. 14.
    Liu, P.F., Qiu, X.P., Chen, X.C, Wu, S.Y.: Multi-timescale long short-term memory neural network for modelling sentences and documents. In: 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 2326–2335 (2015)Google Scholar
  15. 15.
    Jakob, N., Gurevych, I.: Extracting opinion targets in a single- and cross-domain setting with conditional random fields. In: 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), pp. 1035–1045 (2010)Google Scholar
  16. 16.
    Zhao, Y.Y., Che, W.X., Guo, H.L., Qin, B, Su, Z., Liu, T.: Sentence compression for target-polarity word collocation extraction. In: 25th International Conference on Computational Linguistics (COLING 2014), pp. 1360–1369 (2014)Google Scholar
  17. 17.
    Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  18. 18.
    Mikolov, T., Chen, K., Corrado, G.: Efficient estimation of word representations in vector space. Comput. Sci. (2013)Google Scholar
  19. 19.
    Beck, D., Cohn, T., Hardmeier, C., Specia, L.: Learning structural kernels for natural language processing. Trans. Assoc. Comput. Linguist. 3, 461–473 (2015)CrossRefGoogle Scholar
  20. 20.
    Shalaby, W., Zadrozny, W.: Mined semantic analysis: a new concept space model for semantic representation of textual data. In: 2017 IEEE International Conference on Big Data (Big Data 2017). IEEE Press (2017)Google Scholar
  21. 21.
    Ustun, V., Rosenbloom, P.S., Sagae, K., Demski, A.: Distributed vector representations of words in the sigma cognitive architecture. In: Goertzel, B., Orseau, L., Snaider, J. (eds.) AGI 2014. LNCS (LNAI), vol. 8598, pp. 196–207. Springer, Cham (2014). Scholar
  22. 22.
    Zilly, J.G., Srivastava, R.K., Koutník, J., Schmidhuber, J.: Recurrent highway networks. In: 34th International Conference on Machine Learning, pp. 4189–4198 (2017)Google Scholar
  23. 23.
    Strobelt, H., Gehrmann, S., Huber, B., Pfister, H.: LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans. Visual. Comput. Graphics (2016)Google Scholar
  24. 24.
    Yu, Z., et al.: Using bidirectional LSTM recurrent neural networks to learn high-level abstractions of sequential features for automated scoring of non-native spontaneous speech. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 338–345. IEEE Press (2015)Google Scholar
  25. 25.
    Long, D., Zhang, R., Mao, Y.Y.: Prototypical recurrent unit. Neurocomputing 311, 146–154 (2018)CrossRefGoogle Scholar
  26. 26.
    Graves, A., Fernández, S., Schmidhuber, J.: Bidirectional LSTM networks for improved phoneme classification and recognition. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 799–804. Springer, Heidelberg (2005). Scholar
  27. 27.
    Chen, Y., et al.: Named entity recognition from Chinese adverse drug event reports with lexical feature based BiLSTM-CRF and tri-training. J. Biomed. Inform. 96 (2019).
  28. 28.
    Sasaki, Y., et al.: Local and global attention are mapped retinotopically in human occipital cortex. Natl. Acad. Sci. U.S.A. 98(4), 2077–2082 (2001)CrossRefGoogle Scholar
  29. 29.
    Quan, C.Q., Ren, F.J.: Target based review classification for fine-grained sentiment analysis. Int. J. Innov. Comput. Inf. Control 10(1), 257–268 (2016)Google Scholar
  30. 30.
    Huang, H.J., Li, Z.-C.: A multiclass, multicriteria logit-based traffic equilibrium assignment model under ATIS. Eur. J. Oper. Res. 176(3), 1464–1477 (2007)CrossRefGoogle Scholar
  31. 31.
  32. 32.
    Zhang, S., Lin, S.F., Gao, J.F., Chen, J.H.: Recognizing small-sample biomedical named entity based on contextual domain relevance. In: 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC). IEEE Press (2019)Google Scholar
  33. 33.
    Dong, G.C., Chen, J.H., Wang, H.Y., Zhong, N.: A narrow-domain entity recognition method based on domain relevance measurement and context information. In: 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2017), pp. 623–628. ACM Press (2017)Google Scholar
  34. 34.
    Jagannatha, A.N., Yu, H.: Structured prediction models for RNN based sequence labeling in clinical text. In: 2016 Conference on Empirical Methods in Natural Language Processing, pp. 856–865 (2016)Google Scholar
  35. 35.
    Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: 54th Annual Meeting of the Association for Computational Linguistics, pp. 1105–1116 (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Faculty of Information TechnologyBeijing University of TechnologyBeijingChina
  2. 2.Beijing Institute of Smart CityBeijing University of TechnologyBeijingChina
  3. 3.Beijing Key Laboratory of MRI and Brain InformaticsBeijingChina

Personalised recommendations