Abstract
Named Entity Recognition (NER) on Clinical Electronic Medical Records (CEMR) is a fundamental step in extracting disease knowledge by identifying specific entity terms such as diseases, symptoms, etc. However, the state-of-the-art NER methods based on Long Short-Term Memory (LSTM) fail to fully exploit GPU parallelism under the massive medical records. Although a novel NER method based on Iterated Dilated CNNs (ID-CNNs) can accelerate network computing, it tends to ignore the word-order feature and semantic information of the current word. In order to enhance the performance of ID-CNNs-based models on NER tasks, an attention-based ID-CNNs-CRF model which combines word-order feature and local context is proposed. Firstly, Position Embedding is utilized to fuse word-order information. Secondly, ID-CNNs architecture is used to rapidly extract global semantic information. Simultaneously, the attention mechanism is employed to pay attention to the local context. Finally, we apply the CRF to obtain the optimal tag sequence. Experiments conducted on two CEMR datasets show that our model outperforms traditional ones. The F1-scores of 94.55% and 91.17% are obtained respectively on these two datasets and both are better than LSTMs-based models.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016)
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354 (2016). https://doi.org/10.18653/v1/p16-1101
Rondeau, M.-A., Su, Y.: LSTM-based NeuroCRFs for named entity recognition. In: INTERSPEECH, pp. 665–669 (2016). https://doi.org/10.21437/interspeech.2016-288
Rei, M., Crichton, G.K., Pyysalo, S.: Attending to characters in neural sequence labeling models. arXiv preprint arXiv:1611.04361 (2016)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011). https://doi.org/10.1016/j.chemolab.2011.03.009
Wang, C., Wei, C., Bo, X.: Named entity recognition with gated convolutional neural networks (2017). https://doi.org/10.1007/978-3-319-69005-6_10
Strubell, E., Verga, P., Belanger, D., McCallum, A.: Fast and accurate sequence labeling with iterated dilated convolutions. arXiv preprint arXiv:1702.02098 138 (2017)
Hirschman, L., Morgan, A.A., Yeh, A.S.: Rutabaga by any other name: extracting biological names. J. Biomed. Inform. 35(4), 247–259 (2002). https://doi.org/10.1016/s1532-0464(03)00014-5
Han, X., Ruonan, R.: The method of medical named entity recognition based on semantic model and improved SVM-KNN algorithm. In: 2011 Seventh International Conference on Semantics, Knowledge and Grids, pp. 21–27. IEEE (2011). https://doi.org/10.1109/skg.2011.24
Collier, N., Nobata, C., Tsujii, J.-I.: Extracting the names of genes and gene products with a hidden Markov model. In: Proceedings of the 18th Conference on Computational linguistics-Volume 1, pp. 201–207. Association for Computational Linguistics (2000). https://doi.org/10.3115/990820.990850
GuoDong, Z., Jian, S.: Exploring deep knowledge resources in biomedical name recognition. In: JNLPBA Workshop, pp. 96–99. Association for Computational Linguistics (2004). https://doi.org/10.3115/1567594.1567616
Chieu, H.L., Ng, H.T.: Named entity recognition with a maximum entropy approach. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4, pp. 160–163. Association for Computational Linguistics (2003). https://doi.org/10.3115/1119176.1119199
Leaman, R., Islamaj Doğan, R., Lu, Z.: DNorm: disease name normalization with pairwise learning to rank. Bioinformatics 29(22), 2909–2917 (2013). https://doi.org/10.1093/bioinformatics/btt474
Kaewphan, S., Van Landeghem, S., Ohta, T., Van de Peer, Y., Ginter, F., Pyysalo, S.: Cell line name recognition in support of the identification of synthetic lethality in cancer from text. Bioinformatics 32(2), 276–282 (2015). https://doi.org/10.1093/bioinformatics/btv570
Zhu, Q., Li, X., Conesa, A., Pereira, C.: GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics 34(9), 1547–1554 (2017). https://doi.org/10.1093/bioinformatics/btx815
Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016). https://doi.org/10.18653/v1/e17-2068
Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. arXiv preprint arXiv:1405.4053 (2014)
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)
Luo, L., et al.: An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition. Bioinformatics 34(8), 1381–1388 (2017). https://doi.org/10.1093/bioinformatics/btx761
Bharadwaj, A., Mortensen, D., Dyer, C., Carbonell, J.: Phonologically aware neural model for named entity recognition in low resource transfer settings. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1462–1472 (2016). https://doi.org/10.18653/v1/d16-1153
Li, J., Zhou, M., Qi, G., Lao, N., Ruan, T., Du, J. (eds.): CCKS 2017. CCIS, vol. 784. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-7359-5
Zhao, J., Harmelen, F., Tang, J., Han, X., Wang, Q., Li, X. (eds.): CCKS 2018. CCIS, vol. 957. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3146-6
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, M., Xiao, Q., Wu, S., Deng, K. (2019). An Attention-Based ID-CNNs-CRF Model for Named Entity Recognition on Clinical Electronic Medical Records. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions. ICANN 2019. Lecture Notes in Computer Science(), vol 11731. Springer, Cham. https://doi.org/10.1007/978-3-030-30493-5_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-30493-5_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30492-8
Online ISBN: 978-3-030-30493-5
eBook Packages: Computer ScienceComputer Science (R0)