Augmenting Embedding with Domain Knowledge for Oral Disease Diagnosis Prediction

  • Guangkai Li
  • Songmao ZhangEmail author
  • Jie Liang
  • Zhanqiang Cao
  • Chuanbin Guo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11344)


In this paper, we propose to add domain knowledge from the most comprehensive biomedical ontology SNOMED CT to facilitate the embedding of EMR symptoms and diagnoses for oral disease prediction. We first learn embeddings of SNOMED CT concepts by applying the TransE algorithm prevalent for representation learning of knowledge base. Secondly, the mapping from symptoms/diagnoses to biomedical concepts and the corresponding semantic relations defined in SNOMED CT are modeled mathematically. We design a neural network to train embeddings of EMR symptoms and diagnoses and ontological concepts in a coherent way, for the latter the TransE-learned vectors being used as initial values. The evaluation on real-world EMR datasets from Peking University School and Hospital Stomatology demonstrates the prediction performance improvement over embeddings solely based on EMRs. This study contributes as a first attempt to learn distributed representations of EMR symptoms and diagnoses under the constraint of embeddings of biomedical concepts from comprehensive clinical ontology. Incorporating domain knowledge can augment embedding as it reveals intrinsic correlation among symptoms and diagnoses that cannot be discovered by EMR data alone.


Biomedical ontology Embedding Diagnosis prediction EMR data 



This work has been supported by the National Key Research and Development Program of China under grant 2016YFB1000902, Projects of Beijing Municipal Science & Technology Commission, and the Natural Science Foundation of China grant 61621003.


  1. 1.
    Jonnalagadda, S.R., Adupa, A.K., Garg, R.P.: Text mining of the electronic health record: an information extraction approach for automated identification and subphenotyping of HFPEF patients for clinical trials. J. Cardiovasc. transl. Res. 10(3), 313–321 (2017)CrossRefGoogle Scholar
  2. 2.
    Sesen, M.B., Kadir, T., Alcantara, R.B., Fox, J., Brady, M.: Survival prediction and treatment recommendation with Bayesian techniques in lung cancer. In: AMIA Annual Symposium Proceedings, pp. 838 (2012)Google Scholar
  3. 3.
    Mani, S., Chen, Y., Elasy, T., Clayton, W., Denny, J.: Type 2 diabetes risk forecasting from EMR data using machine learning. In: AMIA Annual Symposium Proceedings, pp. 606–615 (2012)Google Scholar
  4. 4.
    Mani, S., Chen, Y., Arlinghaus, L.R., Li, X., Chakravarthy, B., Bhave, R.: Early prediction of the response of breast tumors to neoadjuvant chemotherapy using quantitative MRI and machine learning. In: AMIA Annual Symposium Proceedings, pp. 868–877 (2011)Google Scholar
  5. 5.
    Kawaler, E., Cobian, A., Peissig, P., Cross, D., Yale, S., Craven, M.: Learning to predict post-hospitalization VTE risk from EHR data. In: AMIA Annual Symposium Proceedings, pp. 436–445 (2012)Google Scholar
  6. 6.
    Kim, Y.J., Lee, Y.G., Kim, J.W., Park, J.J., Ryu, B., Ha, J.W.: Highrisk Prediction from electronic medical records via deep attention networks. arXiv preprint arXiv:1712.00010 (2017)
  7. 7.
    Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3(Feb), 1137–1155 (2003)zbMATHGoogle Scholar
  8. 8.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  9. 9.
    Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., Qin, B.: Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 1555–1565 (2014)Google Scholar
  10. 10.
    Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 142–150 (2011)Google Scholar
  11. 11.
    Ganguly, D., Roy, D., Mitra, M., Jones, G. J.: Word embedding based generalized language model for information retrieval. In: 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 795–798 (2015)Google Scholar
  12. 12.
    Li, G., Zhang, S., Liang, J., Cao, Z., Guo, C.: An embedding-based approach for oral disease diagnosis prediction from electronic medical records. In: 2nd International Conference on Medical and Health Informatics, pp. 125–133 (2018)Google Scholar
  13. 13.
    Rosse, C., Mejino Jr., J.L.: A reference ontology for biomedical informatics: the foundational model of anatomy. J. Biomed. Inform. 36(6), 478–500 (2003)CrossRefGoogle Scholar
  14. 14.
    Hayamizu, T.F., Mangan, M., Corradi, J.P., Kadin, J.A., Ringwald, M.: The adult mouse anatomical dictionary: a tool for annotating and integrating data. Genome Biol. 6(3), R29 (2005)CrossRefGoogle Scholar
  15. 15.
    Golbeck, J., Fragoso, G., Hartel, F., Hendler, J., Oberthaler, J., Parsia, B.: The national cancer institute’s thesaurus and ontology. J. Web Semant. 1(1) (2003)Google Scholar
  16. 16.
    Donnelly, K.: SNOMED-CT: the advanced terminology and coding system for eHealth. Stud. Health Technol. Inf. 121, 279 (2006)Google Scholar
  17. 17.
    Guo, S., Wang, Q., Wang, B., Wang, L., Guo, L.: Semantically smooth knowledge graph embedding. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 84–94 (2015)Google Scholar
  18. 18.
    Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)CrossRefGoogle Scholar
  19. 19.
    Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: AAAI, p. 6 (2011)Google Scholar
  20. 20.
    Bordes, A., Glorot, X., Weston, J., Bengio, Y.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2014)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Socher, R., Chen, D., Manning, C. D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp. 926–934 (2013)Google Scholar
  22. 22.
    Jenatton, R., Roux, L., Bordes, A., Obozinski, R.: A latent factor model for highly multi-relational data. In: Advances in Neural Information Processing Systems, pp. 3167–3175 (2012)Google Scholar
  23. 23.
    Nickel, M., Tresp, V., Kriegel, H. P.: A three-way model for collective learning on multi-relational data. In: ICML, pp. 809–816 (2011)Google Scholar
  24. 24.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)Google Scholar
  25. 25.
    Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)zbMATHGoogle Scholar
  26. 26.
    Minarro-Giménez, J.A., Marin-Alonso, O., Samwald, M.: Exploring the application of deep learning techniques on medical text corpora. Stud. Health technol. Inf. 205, 584–588 (2014)Google Scholar
  27. 27.
    Ye, C., Fabbri, D.: Extracting similar terms from multiple EMR-based semantic embeddings to support chart reviews. J. Biomed. Inf. 83, 63–72 (2018)CrossRefGoogle Scholar
  28. 28.
    Choi, E., Schuetz, A., Stewart, W. F., Sun, J.: Medical concept representation learning from electronic health records and its application on heart failure prediction. arXiv preprint arXiv:1602.03686 (2016)
  29. 29.
    Choi, Y., Chiu, C.Y.I., Sontag, D.: Learning low-dimensional representations of medical concepts. AMIA Summits on Transl. Sci. Proc. 2016, 41 (2016)Google Scholar
  30. 30.
    Henriksson, A.: Representing clinical notes for adverse drug event detection. In: 6th International Workshop on Health Text Mining and Information Analysis, pp. 152–158 (2015)Google Scholar
  31. 31.
    Wang, S., Chang, X., Li, X., Long, G., Yao, L., Sheng, Q.: Diagnosis code assignment using sparsity-based disease correlation embedding. IEEE Trans. Knowl. Data Eng. 1, 1 (2016)Google Scholar
  32. 32.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Knowledge graph embedding with iterative guidance from soft rules. arXiv preprint arXiv:1711.11231 (2017)
  33. 33.
    Dasigi, P., Ammar, W., Dyer, C., Hovy, E. Ontology-aware token embeddings for prepositional phrase attachment. arXiv preprint arXiv:1705.02925 (2017)
  34. 34.
    Ruder, S., Vulić, I., Søgaard, A.: A survey of cross-lingual word embedding models. arXiv preprint arXiv:1706.04902 (2017)
  35. 35.
    Kolyvakis, P., Kalousis, A., Smith, B., Kiritsis, D.: Biomedical ontology alignment: an approach based on representation learning. J. Biomed. Semant. 9(1), 21 (2018)CrossRefGoogle Scholar
  36. 36.
    Diaz, G., Fokoue, A., Sadoghi, M.: EmbedS: scalable, ontology-aware graph embeddings. In: 21st International Conference on Extending Database Technology, pp. 433–436 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Guangkai Li
    • 1
  • Songmao Zhang
    • 1
    Email author
  • Jie Liang
    • 2
  • Zhanqiang Cao
    • 3
  • Chuanbin Guo
    • 2
  1. 1.MADIS, Academy of Mathematics and Systems Science, Chinese Academy of SciencesUniversity of Chinese Academy of SciencesBeijingChina
  2. 2.Department of Oral and Maxillofacial SurgeryPeking University School and Hospital of StomatologyBeijingChina
  3. 3.Information CenterPeking University School and Hospital of StomatologyBeijingChina

Personalised recommendations