Drug Safety

, Volume 42, Issue 5, pp 657–670 | Cite as

Development of a Controlled Vocabulary-Based Adverse Drug Reaction Signal Dictionary for Multicenter Electronic Health Record-Based Pharmacovigilance

  • Suehyun Lee
  • Jongsoo Han
  • Rae Woong Park
  • Grace Juyun Kim
  • John Hoon Rim
  • Jooyoung Cho
  • Kye Hwa Lee
  • Jisan Lee
  • Sujeong Kim
  • Ju Han KimEmail author
Original Research Article



Integration of controlled vocabulary-based electronic health record (EHR) observational data is essential for real-time large-scale pharmacovigilance studies.


To provide a semantically enriched adverse drug reaction (ADR) dictionary for post-market drug safety research and enable multicenter EHR-based extensive ADR signal detection and evaluation, we developed a comprehensive controlled vocabulary-based ADR signal dictionary (CVAD) for pharmacovigilance.


A CVAD consists of (1) administrative disease classifications of the International Classification of Diseases (ICD) codes mapped to the Medical Dictionary for Regulatory Activities Preferred Terms (MedDRA® PTs); (2) two teaching hospitals’ codes for laboratory test results mapped to the Logical Observation Identifiers Names and Codes (LOINC) terms and MedDRA® PTs; and (3) clinical narratives and ADRs encoded by standard nursing statements (encoded by the International Classification for Nursing Practice [ICNP]) mapped to the World Health Organization–Adverse Reaction Terminology (WHO-ART) terms and MedDRA® PTs.


Of the standard 4514 MedDRA® PTs from Side Effect Resources (SIDER) 4.1, 1130 (25.03%), 942 (20.86%), and 83 (1.83%) terms were systematically mapped to clinical narratives, laboratory test results, and disease classifications, respectively. For the evaluation, we loaded multi-source EHR data. We first performed a clinical expert review of the CVAD clinical relevance and a three-drug ADR case analyses consisting of linezolid-induced thrombocytopenia, warfarin-induced bleeding tendency, and vancomycin-induced acute kidney injury.


CVAD had a high coverage of ADRs and integrated standard controlled vocabularies to the EHR data sources, and researchers can take advantage of these features for EHR observational data-based extensive pharmacovigilance studies to improve sensitivity and specificity.



The authors would like to acknowledge the National Research Foundation of Korea (NRF) and Korean Health Technology R&D Project, Ministry of Health and Welfare. The authors thank the anonymous reviewers for their helpful feedback.

Compliance with Ethical Standards


This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Science ICT and Future Planning (MSIP) (2018R1D1A1B07049155) and by a grant from the Korean Health Technology R&D Project, Ministry of Health and Welfare (HI13C2164, HI16C11280000). This research was supported by a grant (16183MFDS541) from Ministry of Food and Drug Safety in 2018.

Conflict of interest

Ju Han Kim, Suehyun Lee, Jongsoo Han, Rae Woong Park, Grace Juyun Kim, John Hoon Rim, Jooyoung Cho, Kye Hwa Lee, Jisan Lee, and Sujeong Kim have no conflicts of interest directly relevant to the content of this study. The results of this study do not reflect the views of the National Research Foundation of Korea (NRF) or Ministry of Health and Welfare.

Supplementary material

40264_2018_767_MOESM1_ESM.xlsx (98 kb)
Supplementary material 1 (XLSX 98 kb)
40264_2018_767_MOESM2_ESM.xlsx (5.1 mb)
Supplementary material 2 (XLSX 5220 kb)


  1. 1.
    Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000;356(9237):1255–9.CrossRefGoogle Scholar
  2. 2.
  3. 3.
    Koutkias VG, Jaulent MC. Computational approaches for pharmacovigilance signal detection: toward integrated and semantically-enriched frameworks. Drug Saf. 2015;38(3):219–32. Scholar
  4. 4.
    Hauben M, Madigan D, Gerrits CM, Walsh L, Van Puijenbroek EP. The role of data mining in pharmacovigilance. Expert Opin Drug Saf. 2005;4(5):929–48.CrossRefGoogle Scholar
  5. 5.
    Harpaz R, DuMouchel W, Shah NH, Madigan D, Ryan P, Friedman C. Novel data-mining methodologies for adverse drug event discovery and analysis. Clin Pharmacol Ther. 2012;91(6):1010–21. Scholar
  6. 6.
    Koutkias V, Jaulent M-C. Leveraging post-marketing drug safety research through semantic technologies. In: The PharmacoVigilance signal detectors ontology, SWAT4LS workshop, 10 Dec 2014, Berlin; 2014.Google Scholar
  7. 7.
    Declerck G, Hussain S, Daniel C, Yuksel M, Laleci GB, Twagirumukiza M, et al. Bridging data models and terminologies to support adverse drug event reporting using EHR data. Methods Inf Med. 2015;54(1):24–31. Scholar
  8. 8.
    Lee S, Choi J, Kim HS, Kim GJ, Lee KH, Park CH, et al. Standard-based comprehensive detection of adverse drug reaction signals from nursing statements and laboratory results in electronic health records. J Am Med Inform Assoc. 2017;24(4):697–708. Scholar
  9. 9.
    Backstrom M, Mjorndal T, Dahlqvist R. Spontaneous reporting of adverse drug reactions by nurses. Pharmacoepidemiol Drug Saf. 2002;118:647–50.CrossRefGoogle Scholar
  10. 10.
    Ranganathan SS, Houghton JE, Davies DP, Routledge PA. The involvement of nurses in reporting suspected adverse drug reactions: experience with the meningococcal vaccination scheme. Br J Clin Pharmacol. 2003;566:658–63.CrossRefGoogle Scholar
  11. 11.
    Ahn HJ, Park HA. Adverse-drug-event surveillance using narrative nursing records in electronic nursing records. Comput Inform Nurs. 2013;311:45–51.CrossRefGoogle Scholar
  12. 12.
    Conforti A, Opri S, D’Incau P, et al. Adverse drug reaction reporting by nurses: analysis of Italian pharmacovigilance database. Pharmacoepidemiol Drug Saf. 2012;216:597–602.CrossRefGoogle Scholar
  13. 13.
    Hripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ, et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. 2015;216:574–8.Google Scholar
  14. 14.
    WHO. ICD-10: international statistical classification of diseases and health related problems: tenth revision. 2nd ed. Geneva: World Health Organization; 2004.Google Scholar
  15. 15.
    Park MY, Yoon D, Lee K, Kang SY, Park I, Lee SH, et al. A novel algorithm for detection of adverse drug reaction signals using a hospital electronic medical record database. Pharmacoepidemiol Drug Saf. 2011;20(6):598–607. Scholar
  16. 16.
    Liu M, McPeek Hinz ER, Matheny ME, Denny JC, Schildcrout JS, Miller RA, et al. Comparative analysis of pharmacovigilance methods in the detection of adverse drug reactions using electronic medical records. J Am Med Inform Assoc. 2013;20(3):420–6. Scholar
  17. 17.
    Ji Y, Ying H, Dews P, Mansour A, Tran J, Miller RE, et al. A potential causal association mining algorithm for screening adverse drug reactions in postmarketing surveillance. IEEE Trans Inf Technol Biomed. 2011;15(3):428–37. Scholar
  18. 18.
    Yoon D, Park MY, Choi NK, Park BJ, Kim JH, Park RW. Detection of adverse drug reaction signals using an electronic health records database: comparison of the Laboratory Extreme Abnormality Ratio (CLEAR) algorithm. Clin Pharmacol Ther. 2012;91(3):467–74. Scholar
  19. 19.
    LePendu P, Iyer SV, Bauer-Mehren A, Harpaz R, Mortensen JM, Podchiyska T, et al. Pharmacovigilance using clinical notes. Clin Pharmacol Ther. 2013;93(6):547–55. Scholar
  20. 20.
    Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54–60. Scholar
  21. 21.
    Coloma PM, Schuemie MJ, Trifirò G, Gini R, Herings R, Hippisley-Cox J, et al. EU-ADR Consortium. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR Project. Pharmacoepidemiol Drug Saf. 2011;20(1):1–11. Scholar
  22. 22.
    Eriksson R, Jensen PB, Frankild S, Jensen LJ, Brunak S. Dictionary construction and identification of possible adverse drug events in Danish clinical narrative text. J Am Med Inform Assoc. 2013;20(5):947–53. Scholar
  23. 23.
    Stausberg J. International prevalence of adverse drug events in hospitals: an analysis of routine data from England, Germany, and the USA. BMC Health Serv Res. 2014;13(14):125. Scholar
  24. 24.
    Neubert A, Dormann H, Prokosch HU, Bürkle T, Rascher W, Sojer R, et al. E-pharmacovigilance: development and implementation of a computable knowledge base to identify adverse drug reactions. Br J Clin Pharmacol. 2013;76(Suppl 1):69–77. Scholar
  25. 25.
    Patel VN, Kaelber DC. Using aggregated, de-identified electronic health record data for multivariate pharmacosurveillance: a case study of azathioprine. J Biomed Inform. 2014;52:36–42. Scholar
  26. 26.
    Haerian K, Varn D, Vaidya S, Ena L, Chase HS, Friedman C. Detection of pharmacovigilance-related adverse events using electronic health records and automated methods. Clin Pharmacol Ther. 2012;92(2):228–34. Scholar
  27. 27.
    Li Y, Ryan PB, Wei Y, Friedman C. A method to combine signals from spontaneous reporting systems and observational healthcare data to detect adverse drug reactions. Drug Saf. 2015;38(10):895–908. Scholar
  28. 28.
    Li Y, Salmasian H, Vilar S, Chase H, Friedman C, Wei Y. A method for controlling complex confounding effects in the detection of adverse drug reactions using electronic health records. J Am Med Inform Assoc. 2014;21(2):308–14. Scholar
  29. 29.
    Reich C, Ryan PB, Stang PE, Rocca M. Evaluation of alternative standardized terminologies for medical conditions within a network of observational healthcare databases. J Biomed Inform. 2012;45(4):689–96. Scholar
  30. 30.
    Reisinger SJ, Ryan PB, O’Hara DJ, Powell GE, Painter JL, Pattishall EN, et al. Development and evaluation of a common data model enabling active drug safety surveillance using disparate healthcare databases. J Am Med Inform Assoc. 2010;17(6):652–62. Scholar
  31. 31.
    Ryan PB, Madigan D, Stang PE, Overhage JM, Racoosin JA, Hartzema AG. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31(30):4401–15. Scholar
  32. 32.
    Wang L, Rastegar-Mojarad M, Ji Z, Liu S, Liu K, Moon S, et al. Detecting pharmacovigilance signals combining electronic medical records with spontaneous reports: a case study of conventional disease-modifying antirheumatic drugs for rheumatoid arthritis. Front Pharmacol. 2018;7(9):875. Scholar
  33. 33.
    Kuhn M, Campillos M, Letunic I, Jensen LJ, Bork P. A side effect resource to capture phenotypic effects of drugs. Mol Syst Biol. 2010;6:343. Scholar
  34. 34.
    Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc. 2014;21(3):547–57. Scholar
  35. 35.
    Classification of Disease (ICD). Accessed 15 May 2016.
  36. 36.
    Korean Standard Classification of Diseases (KCD). Accessed 12 Dec 2018.
  37. 37.
    Yu OS, Park IS, Joo YH, Woo KS, Shin HJ, Ahn TS, et al. Classification of nursing statements based on the ICNP, the HHCC, and the nursing process for use in electronic nursing records. Stud Health Technol Inform. 2006;122:718–21.Google Scholar
  38. 38.
    Park IS, Shin HJ, Kim EM, Park HA, Kim YA, Jo EM. Mapping nursing statements with the ICNP and its practical use in electronic nursing records. Stud Health Technol Inform. 2006;122:989–90.Google Scholar
  39. 39.
    Tajima M, Kato Y, Matsumoto J, Hirosawa I, Suzuki M, Takashio Y, et al. Linezolid-induced thrombocytopenia is caused by suppression of platelet production via phosphorylation of myosin light chain 2. Biol Pharm Bull. 2016;39(11):1846–51.CrossRefGoogle Scholar
  40. 40.
    Shoeb M, Fang MC. Assessing bleeding risk in patients taking anticoagulants. J Thromb Thrombolysis. 2013;35(3):312–9. Scholar
  41. 41.
    Fitzmaurice DA, Blann AD, Lip GY. Bleeding risks of antithrombotic therapy. BMJ. 2002;325(7368):828–31.CrossRefGoogle Scholar
  42. 42.
    van Hal SJ, Paterson DL, Lodise TP. Systematic review and meta-analysis of vancomycin-induced nephrotoxicity associated with dosing schedules that maintain troughs between 15 and 20 milligrams per liter. Antimicrob Agents Chemother. 2013;57(2):734–44. Scholar
  43. 43.
    Ramírez E, Jiménez C, Borobia AM, Tong HY, Medrano N, Krauel-Bidwell L, et al. Vancomycin-induced acute kidney injury detected by a prospective pharmacovigilance program from laboratory signals. Ther Drug Monit. 2013;35(3):360–6. Scholar
  44. 44.
    Lobo MG, Pinheiro SM, Castro JG, Momenté VG, Pranchevicius MC. Adverse drug reaction monitoring: support for pharmacovigilance at a tertiary care hospital in Northern Brazil. BMC Pharmacol Toxicol. 2013;14:5. Scholar
  45. 45.
    Härmark L, van Grootheest AC. Pharmacovigilance: methods, recent developments and future perspectives. Eur J Clin Pharmacol. 2008;64(8):743–52. Scholar
  46. 46.
    Xu R, Wang Q. Automatic construction of a large-scale and accurate drug-side-effect association knowledge base from biomedical literature. J Biomed Inform. 2014;51:191–9. Scholar
  47. 47.
    Gurulingappa H, Mateen-Rajput A, Toldo L. Extraction of potential adverse drug events from medical case reports. J Biomed Semant. 2012;3(1):15. Scholar
  48. 48.
    Cai MC, Xu Q, Pan YJ, Pan W, Ji N, Li YB, et al. ADReCS: an ontology database for aiding standardization and hierarchical classification of adverse drug reaction terms. Nucleic Acids Res. 2015;43(Database issue):D907–13. Scholar
  49. 49.
    Juan-Blanco T, Duran-Frigola M, Aloy P. IntSide: a web server for the chemical and biological examination of drug side effects. Bioinformatics. 2015;31(4):612–3. Scholar
  50. 50.
    Khan LM, Al-Harthi SE, Alkreathy HM, Osman A-MM, Ali AS. Detection of adverse drug reactions by medication antidote signals and comparison of their sensitivity with common methods of ADR detection. Saudi Pharm J. 2015;23(5):515–22. Scholar
  51. 51.
    Hui C, Vaillancourt R, Bair L, Wong E, King JW. Accuracy of adverse drug reaction documentation upon implementation of an ambulatory electronic health record system. Drugs Real World Outcomes. 2016;3(2):231–8. Scholar
  52. 52.
    Belenkaya R, Natarajan K, Velez M, Voss E. OMOP common data model (CDM) & extract-transform-load (ETL) tutorial. 24 Sep 2016. Accessed 4 Dec 2018.
  53. 53.
    Santoro A, Genov G, Spooner A, Raine J, Arlett P. Promoting and protecting public health: how the European Union pharmacovigilance system works. Drug Saf. 2017;40(10):855–69. Scholar
  54. 54.
    Wise L, Parkinson J, Raine J, Breckenridge A. New approaches to drug safety: a pharmacovigilance tool kit. Nat Rev Drug Discov. 2009;8(10):779–82. Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Division of Biomedical InformaticsSeoul National University Biomedical Informatics (SNUBI), Seoul National University College of MedicineSeoulKorea
  2. 2.Department of Biomedical Informatics, College of MedicineKonyang UniversityDaejeonKorea
  3. 3.Department of Biomedical InformaticsAjou University School of MedicineSuwonKorea
  4. 4.Department of Laboratory Medicine, Severance HospitalYonsei University College of MedicineSeoulKorea
  5. 5.Physician-Scientist Program, Department of MedicineYonsei University Graduate School of MedicineSeoulKorea
  6. 6.Department of PharmacologyYonsei University College of MedicineSeoulKorea
  7. 7.Department of Laboratory Medicine, Wonju Severance Christian HospitalYonsei University Wonju College of MedicineWonjuKorea
  8. 8.Precision Medicine CenterSeoul National University HospitalSeoulKorea
  9. 9.College of NursingCatholic University of PusanBusanKorea
  10. 10.College of NursingSeattle UniversitySeattleUSA

Personalised recommendations