MedNLU: Natural Language Understander for Medical Texts

  • H. B. Barathi GaneshEmail author
  • U. Reshma
  • K. P. Soman
  • M. Anand Kumar
Part of the Studies in Big Data book series (SBD, volume 68)


Natural Language Understanding is one of the essential tasks for building clinical text-based applications. Understanding of these clinical texts can be achieved through Vector Space Models and Sequential Modelling tasks. This paper is focused on sequential modelling i.e. Named Entity Recognition and Part of Speech Tagging by attaining a state of the art performance of 93.8% as F1 score for i2b2 clinical corpus and achieves 97.29% as F1 score for GENIA corpus. This paper also states the performance of feature fusion by integrating word embedding, feature embedding and character embedding for sequential modelling tasks. We also propose a framework based on a sequential modelling architecture, named MedNLU, which has the capability of performing Part of Speech Tagging, Chunking, and Entity Recognition on clinical texts. The sequence modeler in MedNLU is an integrated framework of Convolutional Neural Network, Conditional Random Fields and Bi-directional Long-Short Term Memory network.


  1. 1.
    Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S. et al.: Clinical information extraction applications: a literature review. J. Biomed. Inform, 2017Google Scholar
  2. 2.
    Yogatama, D., Liu, F., Smith, N.A.: Extractive summarization by maximizing semantic volume. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1961–1966, (2015)Google Scholar
  3. 3.
    Pestian, J.P., Itert, L., Duch, W.: Development of a pediatric text-corpus for part-of-speech tagging. In: Proceedings of the International IIS: IIPWM‘04 Conference held in Zakopane, Poland. Springer, pp. 219–26 (2004)Google Scholar
  4. 4.
    Pakhomov, S.V., Coden, A., Chute, C.G.: Developing a corpus of clinical notes manually annotated for part-of-speech. Int J Med Inform. 75(6), 418–429 (2006)CrossRefGoogle Scholar
  5. 5.
    Hirschman, L., Morgan, A.A., Yeh, A.S.: The MITRE Corporation. Rutabaga by any other name: extracting biological names. J. Biomed. Inform. 35(4), 247–259 (2002)CrossRefGoogle Scholar
  6. 6.
    Savova, G.K., Masanz, J.J., Ogren, P.V., Zheng, J., Sohn, S., Kipper-Schuler, K.C., Chute, C.G.: Mayo clinical text analysis and knowledge extraction system (ctakes): architecture, component evaluation and applications. J. Am. Med. Inform. Assoc. 17(5), 507–513 (2010)CrossRefGoogle Scholar
  7. 7.
    Boag, W., Wacome, K, Naumann, T., Rumshisky, A.: Cliner: a lightweight tool for clinical named entity recognition. AMIA Joint Summits on Clinical Research Informatics (poster) (2015)Google Scholar
  8. 8.
    Fu, X., Ananiadou, S.: Improving the extraction of clinical concepts from clinical records. In: Proceedings of BioTxtM14 (2014)Google Scholar
  9. 9.
    Lv, X., Guan, Y., Yang, J., Wu, J.: Clinical relation extraction with deep learning. International Journal of Hybrid Information Technology, pp. 237–248 (2016)CrossRefGoogle Scholar
  10. 10.
    Wu, Y., Jiang, M,, Lei, J., Xu, H.: Named entity recognition in Chinese clinical text using deep neural networks. Studies in Health Technology and Informatics, pp. 624 (2015)Google Scholar
  11. 11.
    Dong, X., Qian, L., Guan, Y., Huang, L., Yu, Q., Yang, J.: A multiclass classification method based on deep learning for named entity recognition in electronic medical records. In: Scientific Data Summit (NYSDS), IEEE, pp. 1–10 (2016)Google Scholar
  12. 12.
    Pakhomov, S.V., Finley, G., McEwan, R., Wang, Y., Melton, G.B.: Corpus domain effects on distributional semantic modeling of medical terms. Bioinformatics 32(23), 3635–3644 (2016)Google Scholar
  13. 13.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  14. 14.
    Ganguly, D., Roy, D., Mitra, M., Jones, G.J.: Word embedding based generalized language model for information retrieval. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp. 795–798 (2015)Google Scholar
  15. 15.
    Ganesh, H.B., Kumar, M.A., Soman, K.P.: From vector space models to vector space models of semantics. In: Forum for Information Retrieval Evaluation, Springer, Cham, pp. 50–60 (2018)Google Scholar
  16. 16.
    Tang, B., Cao, H., Wang, X., Chen, Q., Xu, H.: Evaluating word representation features in biomedical named entity recognition tasks. BioMed research International, 2014 (2014)Google Scholar
  17. 17.
    Jagannatha, A., Chen, J., Yu, H.: Mining and ranking biomedical synonym candidates from wikipedia. In: Proceedings of the Sixth International Workshop on Health Text Mining and Information Analysis, pp. 142–151 (2015)Google Scholar
  18. 18.
    Gurulingappa, H., Toldo, L., Schepers, C., Bauer, A., Megaro, G.: Semi-supervised information retrieval system for clinical decision support. In TREC (2016)Google Scholar
  19. 19.
    Peter, D.T.: A uniform approach to analogies, synonyms, antonyms, and associations. In: Proceedings of the 22nd International Conference on Computational Linguistics, Vol. 1. Association for Computational Linguistics, pp. 905–912 (2008)Google Scholar
  20. 20.
    Landauer, T.K., Dumais, S.T.: A solution to plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104(2), 211 (1997)CrossRefGoogle Scholar
  21. 21.
    Liu, K., Chapman, W., Hwa, R., Crowley, R.S.: Heuristic sample selection to minimize reference standard training set for a part-of-speech tagger. J. Am. Med. Inform. Assoc. 14(5), 641–650 (2007)CrossRefGoogle Scholar
  22. 22.
    Fan, J.W., Prasad, R., Yabut, R.M., Loomis, R.M., Zisook, D.S., Mattison, J.E., Huang, Y.: Part-of-speech tagging for clinical text: wall or bridge between institutions?” In: AMIA Annual Symposium Proceedings, vol. 2011. American Medical Informatics Association, p. 382–391 (2011)Google Scholar
  23. 23.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. ICML. pp. 282–289 (2001)Google Scholar
  24. 24.
    de Bruijn, Berry, Cherry, Colin, Kiritchenko, Svetlana, Martin, Joel, Zhu, Xiaodan: Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010. J. Am. Med. Inform. Assoc. 18(5), 557–562 (2011)CrossRefGoogle Scholar
  25. 25.
    Jonnalagadda, S., Cohen, T., Wu, S., Gonzalez, G.: Enhancing clinical concept extraction with distributional semantics. J. Biomed. Inform. 45(1), 129–140 (2012)CrossRefGoogle Scholar
  26. 26.
    Wu, Y., Xu, J., Jiang, M., Zhang, Y., Xu, H.: A study of neural word embeddings for named entity recognition in clinical text. In: AMIA Annual Symposium Proceedings, vol. 2015, p. 1326. American Medical Informatics Association (2015)Google Scholar
  27. 27.
    Chiu, J.P.C., Nichols, E.: Named entity recognition with bidirectional lstm-cnns. arXiv preprint arXiv:1511.08308 (2015)
  28. 28.
    Ganesh, H.B., Kumar, M.A., Soman, K.P.: Distributional semantic representation in health care text classification. In: International Conference on Forum of Information Retrieval and Evaluation, pages 201–204, 2016Google Scholar
  29. 29.
    Dyer, C., Ballesteros, M., Ling, W., Matthews, A., Smith, N.A..: Transition based dependency parsing with stack long short-term memory. In: Proceedings of ACL-2015 (Volume1: Long Papers), pages 334–343 (2015)Google Scholar
  30. 30.
    Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Networks 18(5–6), 602–610 (2005)CrossRefGoogle Scholar
  31. 31.
    Settles, B.: Biomedical named entity recognition using conditional random fields and rich feature sets. In: Proceedings of the COLING 2004 NLPBA,. 2004, pp 104–108 (2004)Google Scholar
  32. 32.
    Verspoor, K., Cohen, K.B., Lanfranchi, A., Warner, C., Johnson, H.L., Roeder, C., Choi, J.D., Funk, C., Malenkiy, Y., Eckert, M., et al.: A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools. BMC Bioinformatics 13(1), 207 (2012)CrossRefGoogle Scholar
  33. 33.
    Uzuner, O., South, B.R., Shen, S., DuVall, S.L.: 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J Am Med Inform Assoc. Sep-Oct 18(5), 552–556 (2011)CrossRefGoogle Scholar
  34. 34.
    Ghannay, S., Favre, B., Esteve, Y., Camelin, N.: Word embedding evaluation and combination. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pp. 300–305 (2016)Google Scholar
  35. 35.
    Baroni, M., Dinu, G., Kruszewski, G.: Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 238–247 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • H. B. Barathi Ganesh
    • 1
    Email author
  • U. Reshma
    • 2
  • K. P. Soman
    • 1
  • M. Anand Kumar
    • 3
  1. 1.Amrita School of EngineeringCenter for Computational Engineering and Networking (CEN), Amrita Vishwa VidyapeethamCoimbatoreIndia
  2. 2.Arnekt Solutions Pvt. Ltd.Magarpatta City, PuneIndia
  3. 3.Department of Information TechnologyNational Institute of Technology KarnatakaSurathkalIndia

Personalised recommendations