Skip to main content
SpringerLink
Log in

Span Classification Based Model for Clinical Concept Extraction

  • Conference paper
  • First Online:
Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD 2020)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 88))

  • 159 Accesses

Abstract

Recently, how to structuralize electronic medical records (EMRs) has attracted considerable attention from researchers. Extracting clinical concepts from EMRs is a critical part of EMR structuralization. The performance of clinical concept extraction will directly affect the performance of the downstream tasks related to EMR structuralization. However, the mainstream method, sequence labeling model has some shortcomings. The clinical concept extraction method based on sequence labeling does not conform to the human cognitive model of language. At the same time, the extraction results produced by this method are difficult to couple with downstream tasks, which will cause error propagation and affect the performance of downstream tasks. To deal with these problems, we propose a span classification based method to improves the performance of clinical concept extraction tasks by considering the overall semantics of the token sequence instead of the semantics of each token. We call this model as span classification model.

Experiments show that the span classification model achieves the best micro-average F1 score (81.22%) on the corpora of the 2012 i2b2 NLP challenges, and obtained an F1 score (89.25%) comparable to SOTA in the 2010 i2b2 NLP challenges. Furthermore, the performance of our approach is always better than the sequence labeling model such as BiLSTM-CRF model and softmax classifier.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alsentzer, E., Murphy, J.R., Boag, W., weiHung Weng, Jin, D., Naumann, T., McDermott, M.: Publicly available clinical bert embeddings. arxiv preprint. In: arXiv preprint, arXiv:1904.03323 (2019)

  2. Bruijin, B.D., Cherry, C., Kirichenko, S., Martin, J., Xhu, X.: Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010. J. Am. Med. Inform. Assoc. 18(5), 557–562 (2011)

    Article  Google Scholar 

  3. Chalapathy, R., Borzeshi, E.Z., Picardi, M.: Bidirectional lstm-crf for clinical concept extraction. In: arXiv preprint, arXiv:1611.08373 (2016)

  4. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT 2019 (2018)

    Google Scholar 

  5. Dixit, K., Al-Onaizan, Y.: Span-level model for relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5308–5314. Association for Computational Linguistics, Florence, Italy (2019). https://doi.org/10.18653/v1/P19-1525. https://www.aclweb.org/anthology/P19-1525

  6. Finkel, J.R., Manning, C.D.: Nested named entity recognition. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pp. 141–150. Association for Computational Linguistics, Singapore (2009). https://www.aclweb.org/anthology/D09-1015

  7. Florez, E., Precioso, F., Riveill, M., Pighetti, R.: Named entity recognition using neural networks for clinical notes. In: International Workshop on Medication and adverse Drug Event Detection, pp. 7–15 (2018)

    Google Scholar 

  8. Habibi, M., Weber, L., Neves, M., Wiegandt, D.L., Leser, U.: Deep learn with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14), i37–i48 (2017)

    Article  Google Scholar 

  9. Huang, K., Altosaar, J., Ranganath, R.: Clinicalbert: modeling clinical notes and predicting hospital readmission. In: arXiv preprint, arXiv:1904.05342 (2019)

  10. Jiang, Z., Xu, W., Araki, J., Neubig, G.: Generalizing natural language analysis through span-relation representations. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2120–2133. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.192. https://www.aclweb.org/anthology/2020.acl-main.192

  11. Johnson, A.E., Pollard, T.J., Shen, L., Li-wei, H.L., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Celi, L.A., Mark, R.G.: Mimic-iii, a freely accessible critical care database. Sci. Data 3, 160,035 (2016)

    Google Scholar 

  12. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: arXiv preprint, arXiv:1603.01360 (2016)

  13. Lee, H., Peirsman, Y., Nathanael Chamberts, A.C., Surdeanu, M., Jurafsky, D.: Stanford’s multi-pass sieve coreference resolution system at the conll-2011 shared task. In: In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pp. 28–34 (2011)

    Google Scholar 

  14. Lee, J., Yoon, W., Kim, S.K.D., Kim, S., So, C.H., Kang, J.: Biobert:pre-trained biomedical language representation model for biomedical text mining. In: arXiv preprint, arXiv:1901.08746 (2019)

  15. Lee, K., He, L., Lewis, M., Zettlemoyer, L.: End-to-end neural coreference resolution. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 188–197. Association for Computational Linguistics, Copenhagen, Denmark (2017). https://doi.org/10.18653/v1/D17-1018. https://www.aclweb.org/anthology/D17-1018

  16. Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 318–327 (2020)

    Article  Google Scholar 

  17. Liu, Z., Yang, M., Wang, X., Chen, Q., Tang, B., Wang, Z., Xu, H.: Entity recognition from clinical texts via recurrent neural network. BMC Med. Inf. Decision Making 17(supple), 67 (2017)

    Google Scholar 

  18. Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1105–1116. Association for Computational Linguistics, Berlin, Germany (2016). https://doi.org/10.18653/v1/P16-1105. https://www.aclweb.org/anthology/P16-1105

  19. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. In: NAACL 2018 (2018)

    Google Scholar 

  20. Rink, B., Harabagiu, S., Roberts, K.: Automatic extraction of relations between medical concepts in clinical texts. J. Am. Med. Inform. Assoc. 18(5), 594–600 (2011)

    Article  Google Scholar 

  21. dos Santos, C., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 626–634. Association for Computational Linguistics, Beijing, China (2015). https://doi.org/10.3115/v1/P15-1061. https://www.aclweb.org/anthology/P15-1061

  22. Si, Y., Wang, J., Xu, H., Roberts, K.: Enhancing clinical concept extraction with contextual embeddings. In: arXiv preprint, arXiv:1902.08691 (2019)

  23. Sun, W., Rumshisky, A., Uzuner, O.: Evaluating temporal relations in clinical text: 2012 i2b2 challenge. J. Am. Med. Inform. Assoc. 20(5), 806–813 (2013)

    Article  Google Scholar 

  24. Uzuner, O., South, B.R., Shen, S., Piccardi, M.: 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. J. Am. Med. Inform. Assoc. 18(5), 552–556 (2011)

    Article  Google Scholar 

  25. Wang, Y., Wang, L., Moon, M.R.M.S., Shen, F., Affza, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S.: Clinical information extraction applications: a literature review. J. Biomed. Inf. 77, 34–49 (2018)

    Article  Google Scholar 

  26. Zhu, H., Paschalidis, I.C., Tahmasebi, A.: Clinical concept extraction with contextual word embedding. In: arXiv preprint, arXiv:1810.10566 (2018)

Download references

Author information

Authors and Affiliations

  1. College of Computer, National University of Defense Technology, Changsha, China

    Yongtao Tang, Jie Yu, Shasha Li, Bin Ji, Yusong Tan & Qingbo Wu

Authors
  1. Yongtao Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shasha Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yusong Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingbo Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge, UK

    Hongying Meng

  2. School of Electronical Information and Artificial Engineering, Shaanxi University of Science and Technology, Xi’an, China

    Tao Lei

  3. College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge, UK

    Maozhen Li

  4. College of Electrical and Information, Hunan University, Changsha, China

    Kenli Li

  5. Division of Intelligent Future Technologies, Mälardalen University, Västerås, Västmanlands Län, Sweden

    Ning Xiong

  6. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    Lipo Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tang, Y., Yu, J., Li, S., Ji, B., Tan, Y., Wu, Q. (2021). Span Classification Based Model for Clinical Concept Extraction. In: Meng, H., Lei, T., Li, M., Li, K., Xiong, N., Wang, L. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2020. Lecture Notes on Data Engineering and Communications Technologies, vol 88. Springer, Cham. https://doi.org/10.1007/978-3-030-70665-4_203

Download citation

Publish with us

Policies and ethics

Search

Navigation