Abstract
In this paper we study the problem of predicting clinical diagnoses from textual Electronic Health Records (EHR) data. We show the importance of this problem in medical community and present comprehensive historical review of the problem and proposed methods. As the main scientific contributions we present a modification of Bidirectional Encoder Representations from Transformers (BERT) model for sequence classification that implements a novel way of Fully-Connected (FC) layer composition and a BERT model pretrained only on domain data. To empirically validate our model, we use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits. This is the largest such study for the Russian language and one of the largest globally. We performed a number of comparative experiments with other text representation models on the task of multiclass classification for 265 disease subset of ICD-10. The experiments demonstrate improved performance of our models compared to other baselines, including a fine-tuned Russian BERT (RuBERT) variant. We also show comparable performance of our model with a panel of experienced medical experts. This allows us to hope that implementation of this system will reduce misdiagnosis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Although the data does not contain any personal information, we cannot publicly release it due to certain legal restrictions.
References
Atasoy, H., Greenwood, B.N., McCullough, J.S.: The digitization of patient care: a review of the effects of electronic health records on health care quality and utilization. Ann. Rev. Public Health 40(5), 487–500 (2019)
Fast Healthcare Interoperability Resources (FHIR). https://www.hl7.org/fhir/. Accessed 20 Apr 2020
Liberman, A.L., Newman-Toker, D.E.: Symptom-disease pair analysis of diagnostic error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data. BMJ Qual. Saf. 27(7), 557–566 (2018)
Vardanyan, G.J., et al.: Medical errors: modern condition of the problem. Med. Sci. Armenia 59(4), 105–120 (2019)
World Health Organization: International Statistical Classification of Diseases and Related Health Problems. 10th revision, Fifth edition (2016)
Kuratov, Y., Arkhipov, M.: Adaptation of deep bidirectional multilingual transformers for Russian language. arXiv preprint arXiv:1905.07213 (2019)
Vasiljeva, I., Arandjelovic, O.: Diagnosis prediction from electronic health records (EHR) using the binary diagnosis history vector representation. J. Comput. Biol. 24(8), 767–786 (2017)
Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T., Gao, J.: Dipole: diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1903–1911 (2017). https://doi.org/10.1145/3097983.3098088
Shi, J., Fan, X., Wu, J., et al.: DeepDiagnosis: DNN-based diagnosis prediction from pediatric big healthcare data. In: Sixth International Conference on Advanced Cloud and Big Data (CBD), pp. 287–292. IEEE (2018)
Qiao, Z., Wu, X., Ge, S., Fan, W.: MNN: multimodal attentional neural networks for diagnosis prediction. In: Proceedings of the 28th International Joint Conference on Artifcial Intelligence, pp. 5937–5943. AAAI Press (2019)
Devlin, J., Chang, M.-W., Lee, K., et al.: Bert: pretraining of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Amin, S., Neumann, G., Dunfield, K., Vechkaeva, A., Chapman, K., Wixted, M.: MLT-DFKI at CLEF eHealth 2019: multi-label classification of ICD-10 codes with BERT. In: Conference and Labs of the Evaluation Forum (Working Notes) (2019)
Li, F., Jin, Y., Liu, W., et al.: Fine-tuning bidirectional encoder representations from transformers (BERT)-based models on large-scale electronic health record notes: an empirical study. JMIR Med. Inform. 7(3), e14830 (2019). https://doi.org/10.2196/14830
Sakhibgareeva, M.V., Zaozersky, A.Y.: Developing an artificial intelligence-based system for medical prediction. Bull. Russ. State Med. Univ. 6, 42–46 (2017). https://doi.org/10.24075/brsmu.2017-06-07
Johnson, A.E.W., et al.: MIMIC-III, a freely accessible critical care database. Sci. Data 3, 160035 (2016). https://doi.org/10.1038/sdata.2016.35
Malakouti, S., Hauskrecht, M.: Predicting patient’s diagnoses and diagnostic categories from clinical-events in EHR data. In: Riaño, D., Wilk, S., ten Teije, A. (eds.) AIME 2019. LNCS (LNAI), vol. 11526, pp. 125–130. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21642-9_17
Wu, Y., Schuster, M., Chen, Z., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)
Cer, D., Yang, Y., et al.: Universal sentence encoder, CoRR. arXiv preprint arXiv:1803.11175 (2018)
Manning, C., Raghavan, P., Schutze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)
Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Blinov, P., Avetisian, M., Kokh, V., Umerenkov, D., Tuzhilin, A. (2020). Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-Based Neural Networks. In: Michalowski, M., Moskovitch, R. (eds) Artificial Intelligence in Medicine. AIME 2020. Lecture Notes in Computer Science(), vol 12299. Springer, Cham. https://doi.org/10.1007/978-3-030-59137-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-59137-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59136-6
Online ISBN: 978-3-030-59137-3
eBook Packages: Computer ScienceComputer Science (R0)