Drug Safety

, Volume 40, Issue 8, pp 715–727 | Cite as

Preliminary Results of a Novel Algorithmic Method Aiming to Support Initial Causality Assessment of Routine Pharmacovigilance Case Reports for Medication-Induced Liver Injury: The PV-RUCAM

  • Erik ScalfaroEmail author
  • Henk Johan Streefkerk
  • Michael Merz
  • Christoph Meier
  • David Lewis
Original Research Article



Data incompleteness in pharmacovigilance (PV) health records limits the use of current causality assessment methods for drug-induced liver injury (DILI). In addition to the inherent complexity of this adverse event, identifying cases of high causal probability is difficult.


The aim was to evaluate the performance of an improved, algorithmic and standardised method called the Pharmacovigilance-Roussel Uclaf Causality Assessment Method (PV-RUCAM), to support assessment of suspected DILI. Performance was compared in different settings with regard to applicability and differentiation capacity.


A PV-RUCAM score was developed based on the seven sections contained in the original RUCAM. The score provides cut-off values for or against DILI causality, and was applied on two datasets of bona fide individual case safety reports (ICSRs) extracted randomly from clinical trial reports and a third dataset of electronic health records from a global PV database. The performance of PV-RUCAM adjudication was compared against two standards: a validated causality assessment method (original RUCAM) and global introspection.


The findings showed moderate agreement against standards. The overall error margin of no false negatives was satisfactory, with 100% sensitivity, 91% specificity, a 25% positive predictive value and a 100% negative predictive value. The Spearman’s rank correlation coefficient illustrated a statistically significant monotonic association between expert adjudication and PV-RUCAM outputs (R = 0.93). Finally, there was high inter-rater agreement (K w = 0.79) between two PV-RUCAM assessors.


Within the PV setting of a pharmaceutical company, the PV-RUCAM has the potential to facilitate and improve the assessment done by non-expert PV professionals compared with other methods when incomplete reports must be evaluated for suspected DILI. Prospective validation of the algorithmic tool is necessary prior to implementation for routine use.



The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement n° 115003, the resources of which are composed of financial contributions from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies, in kind contribution.

Compliance with Ethical Standards


The first author was funded as a Novartis associate during the execution of this work. No other sources of funding were used to assist in the preparation of this study.

Conflicts of interest

David Lewis has no conflicts of interest that are directly relevant to the content of this study. Dr Lewis is a full-time employee of Novartis Pharmaceuticals AG, Basel Switzerland. Dr Lewis holds shares in Novartis and GlaxoSmithKline, and has options on Novartis stock. Statement regarding conflicts of interest: ‘This publication has not been influenced or amended as a result of my employment or my shareholdings or stock options. I affirm that the content of the draft is scientific, evidenced-based, current and is not subject to bias.’ Christoph Meier has no conflicts of interest that are directly relevant to the content of this study. Henk Johan Streefkerk has no conflicts of interest that are directly relevant to the content of this study. Dr Streefkerk is a full-time employee of Novartis Pharmaceuticals AG, Basel Switzerland. Dr Streefkerk has options on Novartis stock. Michael Merz has no conflicts of interest that are directly relevant to the content of this study. Dr Merz is a full-time employee of Novartis Institutes for BioMedical Research, Basel, Switzerland. Erik Scalfaro has no conflicts of interest that are directly relevant to the content of this study. Mr Scalfaro was an associate of Novartis Pharmaceuticals AG, Basel Switzerland.

Supplementary material

40264_2017_541_MOESM1_ESM.pdf (188 kb)
Supportive information may be found via online access of this article. ESM: (1) specific hepatotoxicity-related PTs; (2) list of concomitant medicinal products; (3i) and (3ii) list of comorbidities and their relevance to the domain ‘exclusion of other causes of injury’; (4) list of PTs to define ‘alcohol use’ and ‘alcoholism’; and (5) dataset of clinical characteristics. (PDF 188 kb)
40264_2017_541_MOESM2_ESM.pdf (244 kb)
Supplementary material 2 (PDF 243 kb)
40264_2017_541_MOESM3_ESM.pdf (203 kb)
Supplementary material 3 (PDF 203 kb)
40264_2017_541_MOESM4_ESM.pdf (203 kb)
Supplementary material 4 (PDF 202 kb)
40264_2017_541_MOESM5_ESM.pdf (188 kb)
Supplementary material 5 (PDF 187 kb)
40264_2017_541_MOESM6_ESM.pdf (193 kb)
Supplementary material 6 (PDF 193 kb)


  1. 1.
    Goldkind L, Laine L. A systematic review of NSAIDs withdrawn from the market due to hepatotoxicity: lessons learned from the bromfenac experience. Pharmacoepidemiol Drug Saf. 2006;15(4):213–20. doi: 10.1002/pds.1207.CrossRefPubMedGoogle Scholar
  2. 2.
    Sarges P, Steinberg JM, Lewis JH. Drug-induced liver injury: highlights from a review of the 2015 literature. Drug Saf. 2016;38:801. doi: 10.1007/s40264-016-0427-8.CrossRefGoogle Scholar
  3. 3.
    Watkins PB, Merz M, Avigan MI, Kaplowitz N, Regev A, Senior JR. The Clinical Liver Safety Assessment Best Practices Workshop: rationale, goals, accomplishments and the future. Drug Saf. 2014;37(1):1–7. doi: 10.1007/s40264-014-0181-8.CrossRefPubMedCentralGoogle Scholar
  4. 4.
    Teschke R, Danan G. Diagnosis and management of drug-induced liver injury (DILI) in patients with pre-existing liver disease. Drug Saf. 2016;39(8):729–44. doi: 10.1007/s40264-016-0423-z.CrossRefPubMedGoogle Scholar
  5. 5.
    Teschke R, Wolff A, Frenzel C, Schwarzenboeck A, Schulze J, Eickhoff A. Drug and herb induced liver injury: Council for International Organizations of Medical Sciences scale for causality assessment. World J Hepatol. 2014;6(1):17–32. doi: 10.4254/wjh.v6.i1.17.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Regev A, Seeff LB, Merz M, Ormarsdottir S, Aithal GP, Gallivan J, et al. Causality assessment for suspected DILI during clinical phases of drug development. Drug Saf. 2014;37(1):47–56. doi: 10.1007/s40264-014-0185-4.CrossRefPubMedCentralGoogle Scholar
  7. 7.
    Senior JR. Evolution of the Food and Drug Administration approach to liver safety assessment for new drugs: current status and challenges. Drug Saf. 2014;37(1):9–17. doi: 10.1007/s40264-014-0182-7.CrossRefPubMedCentralGoogle Scholar
  8. 8.
    Kaplowitz N. Idiosyncratic drug hepatotoxicity. Nat Rev Drug Discov. 2005;4(6):489–99. doi: 10.1038/nrd1750.CrossRefPubMedGoogle Scholar
  9. 9.
    Danan G, Teschke R. RUCAM in drug and herb induced liver injury: the update. Int J Mol Sci. 2015;17(1):1–33. doi: 10.3390/ijms17010014.CrossRefGoogle Scholar
  10. 10.
    Close P, Collins W, Pellet P, Warner BA. Hepatotoxicity clinical safety standard guideline. Novartis Drug Saf Epidemiol. 2013;1:22–37.Google Scholar
  11. 11.
    FDA. Guidance for Industry: Drug-induced liver injury: premarketing clinical evaluation. 2009;…/Guidances/UCM174090.pdf. Accessed 06 Jul 2016.
  12. 12.
    Avigan MI, Bjornsson ES, Pasanen M, Cooper C, Andrade RJ, Watkins PB, et al. Liver safety assessment: required data elements and best practices for data collection and standardization in clinical trials. Drug Saf. 2014;37(1):19–31. doi: 10.1007/s40264-014-0183-6.CrossRefPubMedCentralGoogle Scholar
  13. 13.
    Agarwal VK, McHutchison JG, Hoofnagle JH. Important elements for the diagnosis of drug-induced liver injury. Clin Gastroenterol Hepatol. 2010;8(5):463–70. doi: 10.1016/j.cgh.2010.02.008.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Miljkovic M, Dobric S, Dragojevic-Simic V. Accuracy and reproducibility of two scales in causality assessment of unexpected hepatotoxicity. J Clin Pharm Ther. 2012;37(2):196–203. doi: 10.1111/j.1365-2710.2011.01282.x.CrossRefPubMedGoogle Scholar
  15. 15.
    Garcia-Cortés M, Stephens C, Lucena MI, Fernandez-Castañer A, Andrade RJ. Causality assessment methods in drug induced liver injury: strengths and weaknesses. J Hepatol. 2011;55(3):683–91. doi: 10.1016/j.jhep.2011.02.007.CrossRefPubMedGoogle Scholar
  16. 16.
    Raschi E, de Ponti F. Drug- and herb-induced liver injury: progress, current challenges and emerging signals of post-marketing risk. World J Hepatol. 2015;7(13):1761–71. doi: 10.4254/wjh.v7.i13.1761.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Danan G, Benichou C. Causality assessment of adverse reactions to drugs-I. A novel method based on the conclusions of international consensus meetings: application to drug-induced liver injuries. J Clin Epidemiol. 1993;46(11):1323–30. doi: 10.1016/0895-4356(93)90101-6.CrossRefPubMedGoogle Scholar
  18. 18.
    Benichou C, Danan G, Flahault A. Causality assessment of adverse reactions to drugs-II. An original model for validation of drug causality assessment methods: case reports with positive rechallenge. J Clin Epidemiol. 1993;46(II):1331–6. doi: 10.1016/0895-4356(93)90102-7.CrossRefPubMedGoogle Scholar
  19. 19.
    Björnsson E, Olsson R. Outcome and prognostic markers in severe drug-induced liver disease. Hepatology. 2005;42(2):481–9. doi: 10.1002/hep.20800.CrossRefPubMedGoogle Scholar
  20. 20.
    Marrero J, Ahn J, Rajender Reddy K. ACG clinical guideline: the diagnosis and management of focal liver lesions. Am J Gastroenterol. 2014;109:1–20. doi: 10.1038/ajg.2014.213.CrossRefGoogle Scholar
  21. 21.
    United States National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) -. LiverTox, clinical and research information on drug-induced liver injury. Accessed 26 Jun 2016.
  22. 22.
    Rockey DC, Seeff LB, Rochon J, Freston J, Chalasani N, Bonacini M, et al. Causality assessment in drug-induced liver injury using a structured expert opinion process: comparison to the Roussel-Uclaf causality assessment method. Hepatology. 2010;51(6):2117–26. doi: 10.1002/hep.23577.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Abraham M, Berhkin M, Weitbruch M, Romano M, Zhang A. Sacubitril/valsartan 50,100, 200 mg film-coated tablets Core Data Sheet. In: Novartis Pharma AG, Drug Regulatory Affairs. 2015. p. 107–23.Google Scholar
  24. 24.
    Hoofnagle JH. Drug-Induced Liver Injury Network (DILIN). Hepatology. 2004;40:773.CrossRefPubMedGoogle Scholar
  25. 25.
    Fontana RJ, Watkins PB, Bonkovsky HL, Chalasani N, Davern T, Serrano J, et al. Drug-induced liver injury network (DILIN) prospective study: rationale, design and conduct. Drug Saf. 2009;32(1):55–68. doi: 10.2165/00002018-200932010-00005.CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85(3):257–68.Google Scholar
  27. 27.
    Altman D. Practical statistics for medical research. Chapman and Hall; 1991. doi:ISBN:9780412276309.Google Scholar
  28. 28.
    Ludbrook J. Statistical techniques for comparing measurers and methods of measurement: a critical review. Clin Exp Pharmacol Physiol. 2002;29(7):527–36. doi: 10.1046/j.1440-1681.2002.03686.x.CrossRefPubMedGoogle Scholar
  29. 29.
    Cohen J. A coefficient of agreement for nominal scales. In: Educational and Psychological Measurement. 1960. p. 37–46. doi: 10.1177/001316446002000104.CrossRefGoogle Scholar
  30. 30.
    Walsh JF, Reznikoff M. Bootstrapping: a tool for clinical research. J Clin Psychol. 1990;46:928– 930.CrossRefPubMedGoogle Scholar
  31. 31.
    Attia J. Moving beyond sensitivity and specificity: using likelihood ratios to help interpret diagnostic tests. Aust Prescr. 2003;26(5):111–3. doi: 10.18773/austprescr.2003.082.CrossRefGoogle Scholar
  32. 32.
    International Conference on Harmonisation. ICH harmonised tripartite guideline post-approval safety data management: definitions and standards for expedited reporting E2D. 2003; Accessed 18 Jan 2016.
  33. 33.
    Moore TJ, Furberg CD. Electronic health data for postmarket surveillance: a vision not realized. Drug Saf. 2015;38(7):601–10. doi: 10.1007/s40264-015-0305-9.CrossRefPubMedGoogle Scholar
  34. 34.
    Maria VA, Victorino RM. Development and validation of a clinical scale for the diagnosis of drug-induced hepatitis. Hepatology. 1997;26(3):664–9. doi: 10.1002/hep.510260319.CrossRefPubMedGoogle Scholar
  35. 35.
    Théophile H, André M, Miremont-Salamé G, Arimone Y, Bégaud B. Comparison of three methods (an updated logistic probabilistic method, the Naranjo and Liverpool algorithms) for the evaluation of routine pharmacovigilance case reports using consensual expert judgement as reference. Drug Saf. 2013;36(10):1033–44. doi: 10.1007/s40264-013-0083-1.CrossRefPubMedGoogle Scholar
  36. 36.
    Agbabiaka TB, Savović J, Ernst E. Methods for causality assessment of adverse drug reactions: a systematic review. Drug Saf. 2008;31(1):21–37. doi: 10.2165/00002018-200831010-00003.CrossRefPubMedGoogle Scholar
  37. 37.
    Arimone Y, Bégaud B, Miremont-Salamé G, Fourrier-Réglat A, Molimard M, Moore N, et al. A new method for assessing drug causation provided agreement with experts’ judgment. J Clin Epidemiol. 2006;59(3):308–14. doi: 10.1016/j.jclinepi.2005.08.012.CrossRefPubMedGoogle Scholar
  38. 38.
    Théophile H, André M, Arimone Y, Haramburu F, Miremont-Salamé G, Bégaud B. An updated method improved the assessment of adverse drug reaction in routine pharmacovigilance. J Clin Epidemiol. 2012;65(10):1069–77. doi: 10.1016/j.jclinepi.2012.04.015.CrossRefPubMedGoogle Scholar
  39. 39.
    Garcia-Cortes M, Lucena MI, Pachkoria K, Borraz Y, Hidalgo R, Andrade RJ. Evaluation of Naranjo Adverse Drug Reactions Probability Scale in causality assessment of drug-induced liver injury. Aliment Pharmacol Ther. 2008;27(9):780–9. doi: 10.1111/j.1365-2036.2008.03655.x.CrossRefPubMedGoogle Scholar
  40. 40.
    Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts E a, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981;30:239–45. doi: 10.1038/clpt.1981.154.CrossRefPubMedGoogle Scholar
  41. 41.
    Kaplowitz N. Causality assessment versus guilt-by-association in drug hepatotoxicity. Hepatology. 2001;33(1):308–10. doi: 10.1053/jhep.2001.21083.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Patient SafetyNovartis Pharma AGBaselSwitzerland
  2. 2.Preclinical SafetyNovartis Institutes for BioMedical ResearchBaselSwitzerland
  3. 3.Division of Clinical Pharmacy and Epidemiology, Department of Pharmaceutical SciencesUniversity of BaselBaselSwitzerland
  4. 4.School of Life and Medical SciencesUniversity of HertfordshireHatfieldEngland, UK

Personalised recommendations