The bulk of this study was carried out in September 2016. Statistical interaction signal detection was combined with triage filters to direct signal assessment. Preliminary signal assessment was performed by a multidisciplinary group of pharmacovigilance professionals over eight dedicated days. Subsequently, in-depth signal assessment was performed by experienced pharmacovigilance assessors.
The basis for the study were reports of suspected ADRs in VigiBase, the WHO global database of individual case safety reports . Established in 1968, it is the world’s largest and broadest such collection, holding more than 21 million reports from 136 countries (as of November 2019). Reports received into the database until 31 August 2016 were included in the analysis. Suspected duplicate reports were excluded using the vigiMatch algorithm for probabilistic duplicate detection . For technical reasons, deduplication was performed for reports received up until 6 April 2016 only. In total, 13.6 million reports were included in the study.
Sources of Drug Information
The UK Summary of Product Characteristics (SmPC) in the Electronic Medicines Compendium (eMC) , and the US FDA’s drug label in either Drugs@FDA  or DailyMed  were the primary sources of approved drug information, to establish whether a certain drug–drug adverse event (DDA) interaction could be considered as labelled. The DrugDex  and Janusmed interactions Footnote 1 databases were used as complementary sources when relevant information was not found in the first two sources.
Natural Language Processing of Literature Sources
Automated screening of drug information sources was implemented, and findings on relevant ADRs and interactions were presented to the assessors for each combination. The sources for known ADRs were the PROTECT Database of ADRs listed in the SmPCs of centrally authorised medicinal products within the EU , DailyMed, and Martindale. Further sources for known drug interactions were Stockley’s Drug Interaction Alerts  and Janusmed. The natural language processing algorithm  preprocessed text by stop word removal, stemming and synonym replacement, and then matched to Medical Dictionary for Regulatory Activities (MedDRA®) terms accounting for word permutations and spelling variations. Drug names extracted from the source were mapped to the corresponding drug names in the WHODrug international reference for medicinal product information.
Statistical Signal Detection for Possible Adverse Drug Interactions
Statistical signal detection for possible adverse drug interactions in VigiBase was performed using a predictive model described by Strandell et al. Footnote 2 and summarized below. In this model, DDA combinations are ranked by probability scores (in the range of 0.04–1.0), where a higher value indicates a higher likelihood that the combinations will be classified as an interaction signal. Table 1 summarizes all predictors and their definitions.
This algorithm includes, as one of its predictors, disproportionality analysis based on the Omega (Ω) statistical interaction measure . Another predictor accounts for whether the two drugs of interest affect the same cytochrome P450 (CYP) enzyme in a way that might lead to an interaction based on the WHODrug Standardised Drug Groupings . Furthermore, it considers expressed suspicions of a drug interaction by the reporter, such as coding the two drugs as ‘interacting’, use of the MedDRA Preferred Term (PT) ‘drug interaction’, and/or inclusion of words starting with ‘interact’ or ‘interakt’ in the case narrative. Reports of altered therapeutic effect are also accounted for, if in combination with (1) specified dosages for both drugs; (2) positive dechallenge intervention (the adverse event abated upon withdrawal of one or both drugs); (3) positive rechallenge intervention (the adverse event reoccurred after one or both drugs were reintroduced); or (4) the two drugs of interest were the only ones listed on the report. Suggestions of unexpected therapeutic response as indicated by the MedDRA PTs ‘therapeutic response unexpected’ and ‘paradoxical drug reaction’ were included as a separate predictor. Finally, there is a predictor accounting for reports of positive dechallenge and overlapping treatment periods when the two drugs of interest were the only ones listed on the report.
We chose this predictive model as the basis for our statistical interaction detection since a retrospective evaluation showed that it performed better in this setting than the Ω interaction disproportionality measure when used on its own . As for the use of Ω, empirical evaluations have shown that statistical interaction models that, like Ω, assume additive effects of non-interacting drugs perform better than those that assume multiplicative effects, such as logistic regression [8, 9].
Disproportionality and CYP are evaluated at the level of the case series. The other predictors are evaluated per report, with scores added together for every report that fulfils a criterion, providing a total contribution score for the case series. With some exceptions specified below, all predictors are computed based only on the reports that listed the two drugs of interest as suspected or interacting, disregarding reports where at least one of the two drugs was listed as concomitant. This includes disproportionality analysis, as well as all but two of the predictors based on the case series. The exceptions are Unexpected therapeutic response and the predictor based on altered therapeutic effect in the presence of solely two drugs included in the report, which count all reports listing the two drugs with the adverse event of interest, including when one or more drug is noted as concomitant, in line with the method proposed in the original publication .
The original development and evaluation of the algorithm was based on the WHO Adverse Drug Reaction Terminology (WHO-ART). For the purpose of this study, we mapped WHO-ART PTs used by the algorithm to MedDRA (version 19.0) using the official WHO-ART to MedDRA bridge. In addition, the predictors MedDRA interaction, Unexpected therapeutic response and Altered effect were complemented with relevant PTs containing the words ‘drug’, ‘therapeutic’, ‘level’, ‘increased’, ‘decreased’, ‘clearance’, ‘enzyme’ and ‘treatment’. All included terms are listed in electronic supplementary material 1.
Automated filters were applied stepwise with the ambition to restrict the list of DDA combinations to those most likely to be signals for in-depth assessment. An earlier pilot study had indicated that a majority of highlighted DDA combinations reflected already known drug interactions or were due to data quality issues, such as report duplication (unpublished results). To address this, triage filters were targeted at automatically eliminating already known drug interactions and addressing some of the previously identified data quality problems.
The screening was restricted to DDA combinations with no more than 30 reports. This reflected our intent to primarily focus on early signals of drug interactions and to reduce the number of well-known DDA combinations to be assessed.
Beyond that we applied the filters specified in Tables 2 and 3, respectively.
The study design allowed for the implementation of additional (tertiary) filters at one specific time during the course of the 8-day period, with the aim to further focus our assessments. These filters were not prespecified but were designed based on the outcome of the assessments up until that point. Consequently, after about 2 days and having assessed 129 DDA combinations with no signals selected for in-depth assessment, the tertiary filters described in Table 4 were applied in an and/or fashion. In other words, all combinations with either three or more narratives and/or with supporting evidence in Janusmed interactions were selected for preliminary signal assessment.
Preliminary Signal Assessment
During the preliminary signal assessment step, a group of 21 pharmacovigilance professionals (pharmacists, medical doctors and data scientists) from Uppsala Monitoring Centre and collaborating organizations served as assessors and manually reviewed DDA combinations highlighted by our predictive algorithm and automated triage filters. Their aim was to identify signals that were suitable for in-depth signal assessment. Preliminary signal assessment was performed according to the flowchart shown in Fig. 1.
The first step in the preliminary signal assessment evaluated whether drugs or reported adverse events for each combination were specific enough to support further action. A DDA combination was, for example, deemed non-actionable if the drugs or adverse event were unspecific (e.g. ‘vitamins NOS’ and ‘therapeutic response decreased’), if any of the drugs had been withdrawn from the market, or for other motivations at the assessors’ discretion.
In assessing whether possible drug interactions were already known, a manual review of the information sources specified above was carried out. A drug interaction was considered known when the interaction was mentioned in at least one of the sources of the interacting drugs. In cases where there was information to suggest a drug interaction in DrugDex/Janusmed, but not in the SmPCs/product labels, consultation with one of three designated senior assessors participating in the study was required to judge whether a drug interaction should be considered known. For DDAs where there was no known interaction, the SmPCs/product labels were reviewed to establish whether the adverse event was a known ADR for any or both of the drugs individually. Literature findings and decision-making choices for each manually reviewed combination were documented.
After confirmation by one of the designated senior assessors, DDA combinations whose series of individual case report were assessed as supportive of an interaction were classified as meriting in-depth signal assessment.
Assessors specified the reason for closing signals, e.g. when the DDA combination was non-actionable, the evidence was non-suggestive, or there was a lack of data. An initial analysis of this data suggested heterogeneity in the application of these categories. Results are therefore presented below in aggregate form. Signals that could neither be classified as meriting in-depth assessment or be closed were placed on a monitoring list to keep under review awaiting additional and more informative reports.
In-Depth Signal Assessment
In-depth assessment was performed by experienced pharmacovigilance assessors, either among the original 21 assessors or within Uppsala Monitoring Centre’s signal review panel.Footnote 3 The details of each in-depth assessment relied on the domain expertise of our individual assessors and the specific circumstances of the signal being assessed; using a common, standardized approach was out of scope for this study. At the end of this process, remaining signals were written, disseminated to the national centres in the WHO Programme for International Drug Monitoring, relevant marketing authorisation holders, and ultimately published in the WHO Pharmaceuticals Newsletter. In case a signal could neither be closed nor support broader dissemination at the time, it was placed on the monitoring list to keep under review awaiting additional reports.