Skip to main content
SpringerLink
Log in

Comparison of Three Methods (An Updated Logistic Probabilistic Method, the Naranjo and Liverpool Algorithms) for the Evaluation of Routine Pharmacovigilance Case Reports Using Consensual Expert Judgement as Reference

  • Original Research Article
  • Published:
Drug Safety Aims and scope Submit manuscript

Abstract

Background

An updated probabilistic causality assessment method and the Liverpool algorithm presented as an improved version of the Naranjo algorithm, one of the most used and accepted causality assessment methods, have recently been proposed.

Objective

In order to test the validity of the probabilistic method in routine pharmacovigilance, results provided by the Naranjo and Liverpool algorithms, as well as the updated probabilistic method, were each compared with a consensual expert judgement taken as reference.

Methods

A sample of 59 drug–event pairs randomly sampled from spontaneous reports to the French pharmacovigilance system was assessed by expert judgement until reaching consensus and by members of a pharmacovigilance unit using the updated probabilistic method, the Naranjo and Liverpool algorithms. Probabilities given by the probabilistic method, and categories obtained by both the Naranjo and the Liverpool algorithms were compared as well as their sensitivity, specificity, positive and negative predictive values.

Results

The median probability for drug causation given by the consensual expert judgement was 0.70 (inter-quartile range, IQR 0.54–0.84) versus 0.77 (IQR 0.54–0.91) for the probabilistic method. For the Naranjo algorithm, the ‘possible’ causality category was predominant (61 %), followed by ‘probable’ (35 %), ‘doubtful’, and ‘almost certain’ categories (2 % each). Category distribution obtained with the Liverpool algorithm was similar to that obtained by the Naranjo algorithm with a majority of ‘possible’ (61 %) and ‘probable’ (30 %) followed by ‘definite’ (7 %) and ‘unlikely’ (2 %). For the probabilistic method, sensitivity, specificity, positive and negative predictive values were 0.96, 0.56, 0.92 and 0.71, respectively. For the Naranjo algorithm, depending on whether the ‘possible’ category was considered in favour or in disfavour of drug causation, sensitivity was, respectively, 1 or 0.42, specificity 0.11 or 0.89, negative predictive value 1 or 0.22 and positive predictive value 0.86 or 0.95; results were identical for the Liverpool algorithm.

Conclusion

The logistic probabilistic method gave results closer to the consensual expert judgment than either the Naranjo or Liverpool algorithms whose performance were strongly dependent on the meaning given to the ‘possible’ category. Owing to its good sensitivity and positive predictive value and by providing results as continuous probabilities, the probabilistic method seems worthy to use for a trustable assessment of adverse drug reactions in routine practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Agbabiaka TB, Savovic J, Ernst E. Methods for causality assessment of adverse drug reactions: a systematic review. Drug Saf. 2008;31(1):21–37.

    Article  PubMed  Google Scholar 

  2. Meyboom RH, Hekster YA, Egberts AC, et al. Causal or casual? The role of causality assessment in pharmacovigilance. Drug Saf. 1997;17:374–89.

    Article  PubMed  CAS  Google Scholar 

  3. Stephens MD. The diagnosis of adverse medical events associated with drug treatment. Adverse Drug React Acute Poisoning Rev. 1987;6:1–35.

    PubMed  CAS  Google Scholar 

  4. Blanc S, Leuenberger P, Berger JP, et al. Judgments of trained observers on adverse drug reactions. Clin Pharmacol Ther. 1979;25:493–8.

    PubMed  CAS  Google Scholar 

  5. Karch FE, Smith CL, Kerzner B, et al. Adverse drug reactions: a matter of opinion. Clin Pharmacol Ther. 1976;19:489–92.

    PubMed  CAS  Google Scholar 

  6. Koch-Weser J, Sellers EM, Zacest R. The ambiguity of adverse drug reactions. Eur J Clin Pharmacol. 1977;11:75–8.

    Article  PubMed  CAS  Google Scholar 

  7. Kramer MS. Difficulties in assessing the adverse effects of drugs. Br J Clin Pharmacol. 1981;11(Suppl. 1):105S–10S.

    PubMed  Google Scholar 

  8. Rowe G, Wright G. The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast. 1999;15:353–75.

    Article  Google Scholar 

  9. Karch FE, Lasagna L. Toward the operational identification of adverse drug reactions. Clin Pharmacol Ther. 1977;21:247–54.

    PubMed  CAS  Google Scholar 

  10. Macedo AF, Marques FB, Ribeiro CF. Can decisional algorithms replace global introspection in the individual causality assessment of spontaneously reported ADRs? Drug Saf. 2006;29:697–702.

    Article  PubMed  Google Scholar 

  11. Teschke R, Schulze J. Suspected herbal hepatotoxicity: requirements for appropriate causality assessment by the US Pharmacopeia. Drug Saf. 2012;35(12):1091–7.

    PubMed  Google Scholar 

  12. Auriche M. Bayesian approach to the imputability of undesirable phenomena to drugs. Therapie. 1985;40:301–6.

    PubMed  CAS  Google Scholar 

  13. Pere JC, Begaud B, Haramburu F, et al. Computerized comparison of six adverse drug reaction assessment procedures. Clin Pharmacol Ther. 1986;40(4):451–61.

    Article  PubMed  CAS  Google Scholar 

  14. Lanctot KL, Naranjo CA. Computer-assisted evaluation of adverse events using a Bayesian approach. J Clin Pharmacol. 1994;34(2):142–7.

    Article  PubMed  CAS  Google Scholar 

  15. Arimone Y, Bégaud B, Miremont-Salamé G, et al. A new method for assessing drug causation provided agreement with experts’ judgment. J Clin Epidemiol. 2006;59:308–14.

    Article  PubMed  Google Scholar 

  16. Théophile H, Arimone Y, André M, et al. Computerized version of the logistic method. http://www.arme.u-bordeaux2.fr/. Accessed 7 May 2013.

  17. Theophile H, Arimone Y, Miremont-Salame G, et al. Comparison of three methods (consensual expert judgement, algorithmic and probabilistic approaches) of causality assessment of adverse drug reactions: an assessment using reports made to a French pharmacovigilance centre. Drug Saf. 2010;33(11):1045–54.

    Article  PubMed  Google Scholar 

  18. Theophile H, André M, Arimone Y, et al. An updated method improved the assessment of adverse drug reaction in routine pharmacovigilance. J Clin Epidemiol. 2012;65(10):1069–77.

    Article  PubMed  Google Scholar 

  19. Naranjo CA, Busto U, Sellers EM, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981;30:239–45.

    Article  PubMed  CAS  Google Scholar 

  20. Garcia-Cortes M, Lucena MI, Pachkoria K, et al. Evaluation of Naranjo adverse drug reactions probability scale in causality assessment of drug-induced liver injury. Aliment Pharmacol Ther. 2008;27(9):780–9.

    Article  PubMed  CAS  Google Scholar 

  21. Kane-Gill SL, Kirisci L, Pathak DS. Are the Naranjo criteria reliable and valid for determination of adverse drug reactions in the intensive care unit? Ann Pharmacother. 2005;39(11):1823–7.

    Article  PubMed  Google Scholar 

  22. Koh Y, Li SC. A new algorithm to identify the causality of adverse drug reactions. Drug Saf. 2005;28(12):1159–61.

    Article  PubMed  Google Scholar 

  23. Koh Y, Yap CW, Li SC. A quantitative approach of using genetic algorithm in designing a probability scoring system of an adverse drug reaction assessment system. Int J Med Inform. 2008;77(6):421–30.

    Article  PubMed  Google Scholar 

  24. Teschke R, Schmidt-Taenzer W, Wolff A. Spontaneous reports of assumed herbal hepatotoxicity by black cohosh: is the liver-unspecific Naranjo scale precise enough to ascertain causality? Pharmacoepidemiol Drug Safety. 2011;20(6):567–82.

    Article  Google Scholar 

  25. FDA. Drugcite: drug side effects reported to the FDA. http://www.drugcitecom/. Accessed 7 May 2013.

  26. Instructions to Authors for submission case reports in Pharmacotherapy. www.pharmacotherapy.org/. Accessed 7 May 2013.

  27. Instructions to Authors for submission case reports in The Annals of Pharmacotherapy. www.theannals.com/. Accessed 7 May 2013.

  28. Davies EC, Green CF, Taylor S, et al. Adverse drug reactions in hospital in-patients: a prospective analysis of 3695 patient-episodes. PLoS One. 2009;4(2):e4439.

    Article  PubMed  Google Scholar 

  29. Reis AM, Cassiani SH. Adverse drug events in an intensive care unit of a university hospital. Eur J Clin Pharmacol. 2011;67(6):625–32.

    Article  PubMed  Google Scholar 

  30. Gallagher RM, Kirkham JJ, Mason JR, et al. Development and inter-rater reliability of the Liverpool adverse drug reaction causality assessment tool. PLoS One. 2011;6(12):e28096.

    Article  PubMed  CAS  Google Scholar 

  31. Karch FE, Lasagna L. Toward the operational identification of adverse drug reactions. Clin Pharmacol Ther. 1977;21(3):247–54.

    PubMed  CAS  Google Scholar 

  32. Hutchinson TA, Leventhal JM, Kramer MS, et al. An algorithm for the operational assessment of adverse drug reactions. II. Demonstration of reproducibility and validity. JAMA. 1979;242(7):633–8.

    Article  PubMed  CAS  Google Scholar 

  33. Leventhal JM, Hutchinson TA, Kramer MS, et al. An algorithm for the operational assessment of adverse drug reactions. III. Results of tests among clinicians. JAMA. 1979;242:1991–4.

    Article  PubMed  CAS  Google Scholar 

  34. Miljkovic MM, Dobric S, Dragojevic-Simic V. Consistency between causality assessments obtained with two scales and their agreement with clinical judgments in hepatotoxicity. Pharmacoepidemiol Drug Saf. 2011;20(3):272–85.

    Article  PubMed  Google Scholar 

  35. Theophile H, Schaeverbeke T, Miremont-Salame G, et al. Sources of information on lymphoma associated with anti-tumour necrosis factor agents: comparison of published case reports and cases reported to the French pharmacovigilance system. Drug Saf. 2011;34(7):577–85.

    Article  PubMed  Google Scholar 

  36. Thiessard F, Roux E, Miremont-Salamé G, et al. Trends in spontaneous adverse drug reaction reports to the French pharmacovigilance system (1986–2001). Drug Saf. 2005;28:731–40.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors wish to thank to Philip Robinson (Univ of Bordeaux) for his advice and help in writing this paper. No sources of funding were used to assist in the preparation of the manuscript. Hélène Théophile, Manon André, Ghada Miremont-Salamé, Yannick Arimone and Bernard Bégaud declare no conflicts of interest that could have directly or indirectly affected the content of this manuscript.

Author information

Authors and Affiliations

  1. Univ de Bordeaux, U657, 33000, Bordeaux, France

    Hélène Théophile, Ghada Miremont-Salamé, Yannick Arimone & Bernard Bégaud

  2. INSERM U657, 33000, Bordeaux, France

    Hélène Théophile, Manon André, Ghada Miremont-Salamé, Yannick Arimone & Bernard Bégaud

  3. Département de Pharmacologie, Centre Régional de Pharmacovigilance, CHU Bordeaux, 33000, Bordeaux, France

    Hélène Théophile, Ghada Miremont-Salamé & Bernard Bégaud

  4. Arme-P, 33000, Bordeaux, France

    Hélène Théophile & Yannick Arimone

  5. Centre de Pharmacovigilance, Hôpital Pellegrin, CHU, 33076, Bordeaux Cedex, France

    Hélène Théophile

Authors
  1. Hélène Théophile
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manon André
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ghada Miremont-Salamé
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yannick Arimone
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bernard Bégaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hélène Théophile.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Théophile, H., André, M., Miremont-Salamé, G. et al. Comparison of Three Methods (An Updated Logistic Probabilistic Method, the Naranjo and Liverpool Algorithms) for the Evaluation of Routine Pharmacovigilance Case Reports Using Consensual Expert Judgement as Reference. Drug Saf 36, 1033–1044 (2013). https://doi.org/10.1007/s40264-013-0083-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-013-0083-1

Keywords

Search

Navigation