Comparison of Three Methods (Consensual Expert Judgement), (Algorithmic and Probabilistic Approaches) of Causality Assessment of Adverse Drug Reactions

Abstract

Background: Different methods have been proposed for assessing a possible causal link between a drug treatment and an adverse event in individual patients. They approximately belong to three main categories: expert judgement, operational algorithms and probabilistic approaches.

Objective: To compare, in a set of actual drug adverse event reports, three different methods for assessing drug causality, each belonging to one of the three main categories: expert judgement, the algorithm used by the French pharmacovigilance centres since 1985, and a novel method based on the logistic function.

Methods: Fifty drug-event pairs were randomly sampled from the database of the Bordeaux pharmacovigilance centre, France. To serve as the gold standard, the probability for drug causation, from 0 to 1, was first determined for each drug-event pair by a panel of senior experts until consensus was reached. Causality was then assessed by members of the Bordeaux pharmacovigilance centre by using the French algorithm and the logistic method. Results expressed as a probability with the logistic method and as a score from 0 to 4 with the French algorithm were then compared with consensual expert judgement, as were the sensitivity, specificity and positive and negative predictive values.

Results: Probabilities ranged from 0.08 to 0.99 (median 0.58; mean 0.60) for experts versus 0.18–0.88 (median 0.73; mean 0.67) for the logistic method. Consensual expert judgement was not discriminant (p = 0.50) in ten cases. For the algorithm, only three of five causality scores were found, doubtful scores being clearly predominant (74%) followed by possible (16%) and probable (10%) scores. Sensitivity and specificity were 0.96 and 0.42, respectively, for the logistic method versus 0.42 and 0.92 for the algorithm. Positive and negative predictive values were 0.78 and 0.83, respectively, for the logistic method versus 0.92 and 0.42 for the algorithm.

Conclusions: Agreement between the three approaches was poor, and only satisfactory for drug events judged as drug-induced by consensual expert judgement. The logistic method showed high sensitivity at the expense of poor specificity. Conversely, the algorithm had poor sensitivity but good specificity. The comparatively good sensitivity and positive predictive values of the logistic method suggest that it may be more useful in the routine or automated assessment of case reports of suspected but still unknown adverse drug reactions. With a substantial rate of false positives relative to true negatives (low specificity), the logistic method does not replace, but can be complemented by, critical clinical assessment of individual cases in evaluating drug-related risk.