FormalPara Key Points

Utilizing routinely collected health data to support the validation and prioritization of pharmacovigilance signals is largely unexplored.

The results of this study indicate that descriptive analyses of routine health data from a distributed network can provide useful insights in a timely manner to support the validation and prioritization of signals, but the impact on decision making varies by signal.

The identified key user requirements highlight areas of further development to fully harness the potential of these data in signal management.

1 Introduction

The detection, analysis and communication of signals that indicate a possible causal relationship between a medicine and an adverse event are key pharmacovigilance priorities. Signal management relies extensively on adverse event reports submitted by health care professionals, patients and pharmaceutical manufacturers [1, 2]. Their analysis informs most regulatory decisions related to safety signals for marketed medicinal products [3, 4]. Signal detection is the first stage of signal management and can be based on case-by-case human review of incoming adverse event reports and/or rely on statistical signal detection of disproportional reporting patterns, to identify case series for expert review. After a signal has been detected, the case series is assessed by pharmacovigilance specialists. In the European Union (EU), this first stage of analysis is referred to as signal validation and its aim is to determine if there is sufficient evidence of a possible new causal association or a new aspect of a known causal association to justify further assessment [5]. Signals passing this stage are subjected to signal assessment where more extensive analyses of individual case reports are performed, and other data sources may be consulted. Throughout signal management, a prioritization of signals is continually made, accounting for strength of evidence and clinical relevance [5], which reflects both the potential public health impact [6] and impact on vulnerable groups and patients at risk [7] (see also Fig. 1 for an overview of the different stages of the signal management process).

Fig. 1
figure 1

Overview of the first stages of a signal management process as employed by the Uppsala Monitoring Centre. During signal detection, drug–event combinations captured by individual case reports are selected for expert review based on case-by-case review and/or statistical algorithms. Next, top-ranked signals undergo initial review or signal validation, during which regulatory documents, aggregate statistics of case series and the literature are screened to determine if further investigation is required. Signals passing this stage are subjected to signal assessment which involves a more comprehensive analysis of individual case reports, in-depth literature review and possible consultation of other data sources to determine whether communication is warranted

Full size image

Adverse event reports have well-known strengths and limitations [8, 9]. Among their strengths are that they cover all types of medicinal products and adverse events, and that they capture data with the specific aim of supporting individual case causality assessment. Among their limitations are the variable quality of information on individual reports, (selective) under-reporting of adverse events and absence of information on medicinal product usage in the population. Moreover, duplicate reports exist and reasons for reporting can differ by reporter qualification, setting and awareness. Altogether, these limitations can complicate the assessment of possible bias and confounding as well as the clinical relevance of a signal.

Alternative methods have been proposed to provide contextual information that individual case reports lack such as cohort event monitoring and other targeted efforts [10]. Whilst these methods can facilitate the identification and characterization of specific signals, their reliance on primary data collection make them less suited for identifying or assessing pharmacovigilance signals at scale. Routinely collected health data are observational data gathered over time from a population as part of routine healthcare delivery and/or administrative processes and are thus better suited for broader surveillance. Routine health data are traditionally used at later stages of signal management, to assess signals in pharmacoepidemiological studies focusing on characterization and causal effect estimation but can also be of value earlier in the process. For example, several international initiatives have explored the use of routine health data for signal detection including the Vaccine Safety Datalink [11], EU-ADR [12,13,14], IMI PROTECT [15], OMOP [16, 17] and the FDA’s Sentinel Initiative [18]. A recent review describes several other studies [19]. There have also been efforts to shorten the time required for follow-up pharmacoepidemiological studies, through so-called rapid-cycle assessments [11, 20,21,22]. In contrast, few studies [15, 23,24,25] have investigated the use of routine health data to validate and prioritize detected signals before assessment, all of which were based on single databases.

The European Health Data and Evidence Network (EHDEN) [26] is a federated network of data partners aimed at generating evidence from routinely collected health data at scale to advance medical research and patient care by harmonizing observational health databases across Europe and developing new research methodologies and analytical tools. Besides providing context for signal validation, a distributed data network like EHDEN could inform the feasibility of follow-up pharmacoepidemiological studies for further assessment.

The aim of this study was to examine the feasibility and utility of analysing routine health data within EHDEN to support signal validation and prioritization and to reflect on key requirements for these data to become an integral part of signal management.

2 Methods

A 5-day study-a-thon [27], an event aimed at generating knowledge and evidence for the specific aim of this study, was organized on September 5–9, 2022 and attended by 31 scientists from partners of the Innovative Medicines Initiative (IMI) EHDEN consortium. Statistical signal detection was performed in VigiBase [28].

As the consortium operates as a public–private partnership, the scope of the study was limited to suspected drugs with generic manufacturers (excluding vaccines and biologicals) to avoid potential conflicts of interest. Moreover, to prevent bias due to misclassification, only adverse events with pre-specified definitions or phenotypes validated in EHDEN were considered. In general, the  use of phenotypes instead of single diagnostic terms also improves the ability to recognize relevant adverse events and thereby increase statistical power of any analysis. A total of 16 adverse event phenotypes developed by the Observational Health Data Sciences and Informatics (OHDSI) network and validated by the consortium for its research on COVID-19 vaccines [29, 30] met this criterion at the time of the study.

Signal validation and prioritization considered information from VigiBase, regulatory documents and the scientific literature alongside descriptive analyses of routine health data from participating EHDEN data partners.

2.1 Study-a-thon Execution

Six pharmacovigilance specialists experienced in signal management (four pharmacists; two medical doctors) and four data scientists from the Uppsala Monitoring Centre (UMC) participated in the study. They worked in two signal validation teams, analysing reports in VigiBase and reviewing regulatory information and scientific literature according to UMC's routine signal validation and prioritization process.

Supporting these teams were four epidemiologists with expertise in performing analyses across large database networks using tools and packages developed by the OHDSI community including ATLAS [31]. They translated questions raised by the assessors during signal validation and prioritization into descriptive analyses with scripts to be executed across databases of participating data partners. All analyses were designed centrally in the ATLAS user interface, and JSON specifications were shared with the data partners for execution. Additionally, ad-hoc custom R/SQL scripts were developed on site.

During the study-a-thon, representatives of participating data partners were on call to run the analysis scripts in their respective databases to help answer the questions. They also provided interpretation and context for results based on their expert knowledge of the source data. Relevant findings were returned to the corresponding signal validation team (Fig. 2), and together with insights from VigiBase, regulatory documents and the scientific literature, they informed decisions about which signals to forward for assessment. These decisions were made by the signal validation teams through consensus.

Fig. 2
figure 2

Overall execution of the study-a-thon. Two signal validation teams (blue) examined the available information from VigiBase and other sources of information, resulting in questions that could potentially be answered with descriptive analyses of routine health data. Epidemiologists well versed in Observational Health Data Sciences and Informatics (OHDSI) analytical tools (red) translated those questions into scripts for execution across routine health databases of the participating European Health Data and Evidence Network (EHDEN) data partners (green)

Full size image

2.2 Data

2.2.1 VigiBase

Reports in VigiBase are shared by the 155 full member countries in the WHO Programme for International Drug Monitoring (February 2023) [32]. Medicinal products (drugs) are coded using the WHODrug Global dictionary [33] and adverse events are coded using the Medical Dictionary for Regulatory Activities (MedDRA®). The MedDRA® terminology is the international medical terminology developed under the auspices of the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). VigiBase held 32.0 million reports at the data lock point, 4 July, 2022 [29]. We excluded reports with only vaccines (ATC = J07) listed as suspected medicinal products and reports identified as suspected duplicates through the vigiMatch algorithm [34], resulting in a data set of 25.6 million reports (encompassing 3.7 million drug–event combinations) on which statistical signal detection was performed (see Fig. 3).

Fig. 3
figure 3

Flow chart presenting the flow of signals including those studied during the study-a-thon. Numbers of exclusions refer to the number of drug–event combinations, unless specified differently. MedDRA Medical Dictionary for Regulatory Activities, PTs preferred terms

Full size image

2.2.2 Routinely Collected Health Data from EHDEN

In EDHEN, individual-level data are maintained by data partners across Europe and mapped to a common standard, the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM). In this model, drugs are coded using RxNorm Extension [35] and outcomes using SNOMED CT, enabling execution of standardized analysis scripts across different databases. All 16 phenotypes used for the study (Supplementary Table 1, see electronic supplementary material [ESM]) were defined based on rule-based algorithms including relevant diagnostic codes [29, 30] (see also [36] for a detailed description of the definitions used). Mapped individual-level data are stored locally at the data partner site and only aggregate statistics are shared within the network. All data partners in EHDEN were invited to the study-a-thon. In total, 10 data partners accepted the invitation, a brief description of their databases is provided in Table 1.

Table 1 EHDEN data partners who participated in the study-a-thon
Full size table

2.3 Statistical Signal Detection

Statistical signal detection in VigiBase was carried out using vigiRank, a data-driven predictive model for emerging safety signals [37]. vigiRank combines disproportionality analyses (using the information component [IC] as measure of disproportionate reporting [38]) with predictors related to the completeness, recency, geographic spread and availability of case narratives in a logistic regression model. Since vaccine reports were excluded prior to statistical signal detection, masking of signals by these reports (which make up a large proportion of VigiBase) was prevented. For this study, we selected drug–event combinations with a maximum vigiRank score related to a generic drug and a MedDRA® Preferred Term (PT) mapped to one of the 16 selected phenotypes. Drugs with generic manufacturers were identified using publicly available data [39,40,41] and drug names were mapped from WHODrug Global to the RxNorm Extension vocabulary of the OMOP-CDM. Drugs with no verbatim match in RxNorm Extension were excluded. Mapping between each of the 16 phenotypes and MedDRA® PTs was based on expert medical knowledge. Phenotypes correspond to multiple MedDRA® PTs and the one-to-many mapping included less specific terms to increase the sensitivity of statistical signal detection (see Supplementary Table 2 in the ESM). During the signal validation process, signals were discarded if they contained a non-specific PT and were the sole term mapped to a phenotype. This resulted in a set of 1175 statistical signals related to 218 generic drugs and 72 MedDRA® PTs (see Fig. 3), which were listed in random order for analysis. During the study-a-thon, 95 statistical signals of this list (covering 65 drugs and 28 MedDRA® PTs) could be subjected to signal validation and prioritization.

2.4 Analysis of Case Series in VigiBase and Review of Regulatory Information and the Scientific Literature

The purpose of signal validation and prioritization is to determine which statistical signals merit assessment. Since we used a more inclusive approach for mapping phenotypes to MedDRA® PTs, we had to determine during signal validation whether there was support for a signal in VigiBase considering more specific PTs mapped to the phenotype (see Supplementary Table 2 in the ESM). Signals were closed upon initial inspection if:

  1. i)

    at least one of the PTs mapped to a phenotype or the phenotype itself was listed as an already known adverse drug reaction in the European Summaries of Products Characteristics or the US Food and Drug Administration product labels, or had been discussed by the Pharmacovigilance Risk Assessment Committee of the European Medicines Agency or included in Drug Safety Communications or Potential Signals of Serious Risk of the US Food and Drug Administration, and there was no information in the VigiBase case series to suggest new aspects of the association; or

  2. ii)

    the case series in question lacked clinical coherence or consistency (e.g., invalid case diagnosis, implausible time to onset or unclear clinical picture due to the PT being unspecific and the sole term mapped to the phenotype).

For the other signals, the following aspects were taken into consideration to determine whether a signal was eligible for assessment: the number of reported cases and whether this exceeded the number expected based on how commonly the drug and adverse event were reported overall; the time interval between drug administration and adverse event onset (time to onset); the course of the event when the drug was stopped (so-called dechallenge) and possibly re-administered (so-called rechallenge); the presence of a dose–response relationship; consistency of reporting across geographic regions; consistency of reporting for drugs belonging to the same substance class; existence of a plausible biological mechanism; existence of possible confounding by the underlying disease and/or concomitant treatments; coherence with other findings in published reports and the scientific literature. In addition, we examined descriptive characteristics of the case series and considered the possibility of comparing these against characteristics of other case series in VigiBase to identify relevant key features [42].

2.5 Analyses of Routine Health Data

Analyses of the routine health data to support signal validation and prioritization were descriptive in nature, i.e., focusing on so-called characterizations. The purpose of these analyses was not to assess causal associations directly, but rather to provide contextual information to assist signal assessors in their assessments of possible alternative explanations and potential public health and clinical implications. Analyses were also conducted to determine if follow-up pharmacoepidemiological investigations would be feasible. For signals that could not be assessed in VigiBase, these could impact the decision to forward a signal to assessment. For example, if a signal was difficult to assess because of suspected confounding by the underlying disease and/or concomitant medication(s), the decision whether to forward the signal for assessment was partly guided by the possibility of assessing it further in a pharmacoepidemiological study.

A cohort design was used for all analyses and the following cohorts were identified as key for supporting signal validation and prioritization:

  • Drug cohort: A cohort of new users of the drug, indexed on the drug start date. New use was defined as the first time a subject had a drug record in the database after at least 365 days of database observation.

  • Adverse event cohort: A cohort of subjects with a new diagnosis of the adverse event of interest, indexed on the diagnosis date. Event-free windows (see Supplementary Table 1 in the ESM) were used to distinguish new diagnoses from repeated reporting of the same diagnosis in the database.

  • Indication cohort: A cohort of subjects with the indication for drug use, indexed on the first diagnosis date of the indication.

Besides the cohorts defined above, we also considered the possibility of performing additional descriptive analyses including any subject with at least 365 days of database observation. All analyses were performed within each individual database and when interpreting the results, database-specific features were taken into consideration such as setting (hospital vs community-based), database capture of the drug, the adverse event, relevant covariates as well as sample size.

The design of all descriptive analyses was tailored to questions raised by assessors of the signal validation teams and where analyses of routine health data were considered of potential added value. Consequently, the analysis themes targeted by the routine health data (possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological investigations) could vary per signal.

2.5.1 Assessment of Possible Alternative Explanations

Possible bias and confounding were examined by characterizing new users of the drug (i.e., the drug cohort). To better understand characteristics of drug exposure (i.e., indications for treatment, concomitant treatments and other comorbid diseases) and their sequence leading up to drug initiation, descriptive summary statistics of relevant covariates in the drug cohort were obtained before and at the drug start date. Likewise, occurrence of the adverse event prior to or at drug start was assessed. The look-back window for examining the distribution of relevant covariates and the adverse event before the index date was set to 365 days and where possible further split into shorter time intervals. To evaluate potential confounding, associations between characteristics of drug exposure and the adverse event were explored. This was done by comparing incidence rates of the adverse event in a 365-day risk window (N per 10,000 person-years) between patients with certain characteristics of drug exposure (e.g., indications for treatment, concomitant treatments or other comorbid diseases) and at-risk subjects in the general population captured by the database. Results from this analysis were reviewed overall and in age and sex strata to account for imbalances in demographic characteristics in this comparison as appropriate.

2.5.2 Assessment of Potential Public Health and Clinical Impact

To better understand the potential public health and clinical impact of a signal, estimates of drug usage (i.e., number of new users of the drug (see also drug cohort definition in Sect. 2.5) as well as the proportion of patients with a specific indication receiving the drug) were obtained and the incidence rate of the adverse event among new users of the drug (N per 10,000 person-years) was estimated. To estimate the incidence, the allowable gap between successive drug records for defining continuous exposure during follow-up was tailored to the drug under investigation. To better understand the seriousness of the adverse event, hospitalization and death rates were computed in subjects experiencing the adverse event and those with prior drug exposure. The start date of follow-up for these analyses was defined as the diagnosis date (see also adverse event cohort definition in Sect. 2.5). Routine health data further enabled the identification of potential vulnerable subgroups through comparing descriptive characteristics of all new users of the drug and those who also experienced the adverse event. This comparison included descriptive summary statistics of relevant covariates before and at the drug start date.

2.5.3 Feasibility of Follow-up Pharmacoepidemiological Investigations

To assess the feasibility of pharmacoepidemiological follow-up investigations, the number of new users of the drug was obtained, either overall or for a specific indication. Similarly, the number of patients with the adverse event was computed, either overall or in a specific period after initiating treatment with the drug. If case counts for a drug–event combination were considered sufficient for further analysis, treatment pathway analyses (using so-called sunburst plots [43, 44]) were performed to display the sequence of common treatments for specific indications. These analyses were used to suggest relevant comparator drugs for follow-up pharmacoepidemiological studies, using active comparator designs. Sunburst plots are doughnut-shaped graphs with stacked layers, each representing different lines of treatment. The inner circle represents the first treatment and subsequent treatments are shown in the surrounding outer layers, with each drug represented by its own colour. Treatment pathway analyses were restricted to a set of drugs selected by the clinical experts. All analyses relied on cohorts as defined in Sect. 2.5, and data available within the participating databases. In addition, we also evaluated feasibility of follow-up pharmacoepidemiological investigations considering data availability in the entire EHDEN network.

3 Results

3.1 Overall Findings

Figure 3 presents the flow of signals during the study. Of the 95 statistical signals subjected to signal validation and prioritization, 66 (70%) were identified as already labelled and 21 (22%) were discarded due to lack of clinical coherence or consistency in the corresponding case series in VigiBase. For eight signals (8%), routine health data were considered during signal validation and prioritization. The eight signals covered seven generic drugs from different therapeutic areas including dementia, cancer and inflammatory bowel disease and adverse event PTs mapping to various phenotypes including appendicitis, acute myocardial infarction, deep vein thrombosis, pulmonary embolism, non-haemorrhagic stroke, myocarditis/pericarditis and anaphylaxis. Table 2 presents a summary of the insights gained from the different data sources for each of the eight signals and how these influenced the decision whether to forward the signal for assessment.

Table 2 Signal validation and prioritization results for the eight drug–event combinations studied
Full size table

Design, execution and interpretation of results from descriptive analyses of the routine health data took up to a few hours for each signal, of which 15–60 minutes were used for execution. Routine health data were interrogated for all signals where it was considered except for the diphenhydramine–anaphylaxis signal as the temporal resolution and capture of over-the-counter drugs in the data were deemed insufficient to answer the query for this specific signal. Descriptive analyses of routine health data informed decision making for five out of the seven signals (mesalazine–pericarditis/myocarditis, rivastigmine–non-hemorrhagic stroke, dexamethasone–acute myocardial infarction, dexamethasone–appendicitis and melphalan–deep vein thrombosis/pulmonary embolism) and the process resulted in three signals (mesalazine–pericarditis/myocarditis, rivastigmine–non-hemorrhagic stroke and sorafenib–deep vein thrombosis) being forwarded for assessment. Further elaborations on insights gained from the routine health data are provided per signal in Sects. 3.2–3.4 grouped by analysis theme (possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological investigations).

3.2 Possible Alternative Explanations

Routine health data were considered to explore potential alternative explanations in seven of the eight signals.

Descriptive analyses supported the validation of the dexamethasone–acute myocardial infarction signal. The small fraction of dexamethasone users with a history of multiple myeloma highlighted possible bias in the reported case series. Multiple myeloma (i.e., the most common indication in the case series in VigiBase) was also identified as a potential confounder and this was evaluated further in the routine health data by comparing the incidence rate of acute myocardial infarction in multiple myeloma patients with a cohort of at-risk subjects. The increased rate of acute myocardial infarction among multiple myeloma patients strengthened the suspicion of confounding by the underlying disease and/or its attributes (i.e., advanced age) and together with insights from VigiBase (i.e., reporting pattern suggesting intensified monitoring of selected patient groups) resulted in the signal being closed. The same approach was used for addressing potential confounding by indication for the rivastigmine–non-hemorrhagic stroke signal. Here, descriptive analyses strengthened the suspicion of confounding by dementia. However, other insights from VigiBase (higher reporting rate of the adverse event for rivastigmine compared with some other dementia drugs) as well as the possibility to assess the signal in more detail in a follow-up pharmacoepidemiological investigation resulted in it being forwarded for assessment. Descriptive analyses also reinforced the suspicion of confounding by indication (i.e., multiple myeloma) for melphalan–deep vein thrombosis/pulmonary embolism, but this insight was not instrumental in the decision to close this signal (see details in Table 2). Other insights from regulatory information (i.e., thromboembolic events listed in some labels) and anticipated confounding by other commonly co-administered drugs contributed to the signal not being prioritized for assessment. More contextual information regarding the temporal relation of drug exposure and event occurrence was requested for cyclophosphamide–appendicitis. Almost all cases of this signal lacked narratives and had limited time-to-onset information, and confounding by underlying disease was considered as a possible alternative explanation. None of the analysed databases, however, had sufficient capture of appendicitis cases around the initiation of cyclophosphamide treatment to inform signal validation. In all other signals (dexamethasone–appendicitis, diphenhydramine–anaphylaxis, sorafenib–deep vein thrombosis), attempts to explore possible alternative explanations were unsuccessful because of limited data or insufficient data capture.

3.3 Potential Public Health and Clinical Impact

Among the eight signals for which routine health data were considered, the potential public health and clinical impact was assessed in three. This involved assessment of number of new users of the drug, and the incidence of the adverse event among these subjects.

The mesalazine–myocarditis/pericarditis signal was selected for signal assessment as review of the case series, regulatory information and published case reports highlighted potential new aspects warranting further investigation. Mesalazine-induced myocarditis/pericarditis is listed in the Summaries of Products Characteristics (SmPCs) from the European Union [45] and labels from the United States [46] but there are no population-level estimates of the frequency and seriousness of these events in mesalazine users. SmPCs list the ADR as ‘rare’ (<1/1000) or ‘very rare’ (<1/10,000). In VigiBase, 597 myocarditis/pericarditis cases were observed and 82% of these cases were marked as serious, with narratives and published case reports showing evidence of life-threatening episodes and hospitalizations. In the routine health data, mesalazine was the most common drug recorded in patients with inflammatory bowel disease. The incidence rate of myocarditis/pericarditis among new users of mesalazine ranged between 1 and 4 per 10,000 per year, indicating that the ADR is rare. Data capture of hospitalization and death records, however, was insufficient to characterize the seriousness of myocarditis/pericarditis following mesalazine use in the routine health data. Given the relatively large number of patients exposed to the drug and the potential seriousness of the adverse event, it was selected for signal assessment.

Descriptive analyses of routine health data also supported the closure of two signals (dexamethasone–acute myocardial infarction and dexamethasone–appendicitis). Together with evidence suggesting intensified monitoring for dexamethasone in selected patient groups, these signals received lower priority because of the small group of patients considered to be at potential risk. In both combinations, multiple myeloma (i.e., the most frequent indication reported among these cases in VigiBase) represented only a small fraction of all indications for dexamethasone use and only a limited number of appendicitis cases was observed in the routine health data.

3.4 Feasibility of Follow-up Pharmacoepidemiological Investigations

For six of the eight signals, routine health data enabled rapid analyses of feasibility for follow-up epidemiological investigations.

The selection of mesalazine–pericarditis/myocarditis for assessment was supported by the observation that there would be a sufficiently large number of incident pericarditis/myocarditis cases among mesalazine users in a pharmacoepidemiological analysis. Similar observations supported the selection of rivastigmine–non-hemorrhagic stroke for in-depth assessment. For sorafenib–deep vein thrombosis, the small number of patients treated with sorafenib in the databases analysed indicated that it could not be evaluated further in these data. Whilst the decision to forward this signal for assessment was not directly informed by the routine health data that were part of the study-a-thon, a follow-up pharmacoepidemiological study was deemed feasible considering the availability of cancer-specific data in the larger data network. The very low numbers of multiple myeloma patients treated with dexamethasone across databases included in this study indicated that assessment of acute myocardial infarction and dexamethasone specific to multiple myeloma patients would not be feasible in the larger data network either. The same conclusion was drawn for appendicitis with dexamethasone in multiple myeloma patients, for which there was even less data—reflecting the lower population incidence of appendicitis compared with acute myocardial infarction. For the cyclophosphamide–appendicitis signal, the limited number of identified cases highlighted the difficulty of assessing this signal further in a pharmacoepidemiological study.

For rivastigmine–non-hemorrhagic stroke, treatment pathway analysis shown in Fig. 4 identified other first-line treatments (galantamine, donepezil and memantine) that could serve as active comparators for rivastigmine in a pharmacoepidemiological study. It also showed the heterogeneity of treatment pathways reflecting differences in care setting of databases and prescribing practices across countries; for example, in the IPCI database, galantamine was the second most common first-line treatment for dementia whereas in CPRD Aurum, NHR and IMASIS, other drugs were more commonly observed as first-line treatment. Treatment pathway analyses were also instrumental for identifying the candidate active comparator (i.e., corticosteroids) for follow-up pharmacoepidemiological analysis of the mesalazine–myocarditis/pericarditis signal.

Fig. 4
figure 4

Treatment pathway analyses for patients with dementia in different databases. From left to right: CPRD (Clinical Practice Research Datalink) Aurum, NHR (Norwegian health registries), IMASIS (Institut Municipal Assistència Sanitària Information System) and IPCI (Integrated Primary Care Information). In the inner circle, the first relevant medication that was taken by patients is shown, in the second circle, the second medication, etc.

Full size image

4 Discussion

Analyses of routine health data to support signal validation and prioritization are feasible and can inform decision making. The design and execution of descriptive analyses across ten European databases were completed in a time frame of hours for each drug–event combination. Of eight signals where routine health data were considered for signal validation, descriptive analyses of the data informed decisions in five. Possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological investigations each impacted these decisions. The signal validation process resulted in three signals being selected for signal assessment; two of these decisions were supported by insights from the routine health data.

The impact of insights from the routine health data varied between the signals. For example, the mesalazine–myocarditis/pericarditis signal would not have been selected for signal assessment without the ability to analyse routine health data at this stage, whereas the melphalan–deep vein thrombosis/pulmonary embolism signal would have probably been closed either way. Clearly, insights from routine health data will not resolve all uncertainties in causality assessments, and insights derived from the case series, routine health data and other sources of evidence must be combined in a holistic assessment. For example, signal assessors need to bear in mind that local safety concerns from other parts of the world may not be reflected in a European network of routine health data. Also, integrating routine health data in signal validation not only requires pharmacovigilance knowledge, but also clinical and epidemiological expertise, as well as a good understanding of the statistical methods and data used for analyses [15]. Lastly, the cost–benefit of integrating routine health data at this stage of signal management cannot be assessed until we know the outcome of the signal assessment process. If many signals supported by routine health data are closed after signal assessment, it may indicate that restricting analyses to descriptive analyses focusing on characterizations and feasibility assessments only as was done here is not effective. If so, it may be better to move more rapidly to signal assessment and apply a wider range of pharmacoepidemiological analyses right away, or if data governance allows, to perform some of these analyses during signal validation, with the aim to eliminate false positives, as has been done in some earlier studies [15, 23].

Access to harmonized routine health databases across Europe embedded within an infrastructure with real-time analytical tools facilitated and quality assured the execution of the descriptive analyses. However, a narrow selection of drugs and adverse events was assessed in the study, which may limit the generalisability of our findings. The focus on generic products may have increased the proportion of signals related to new aspects of known ADRs as opposed to new ADRs. Moreover, generic products tend to be used in many countries where adoption of new products may be slower, and because these drugs have been on the market for a long time, their exposure is more likely to be captured in routine health data. Also, the adverse events studied did not include very rare ones, which might have been more difficult to study in these data [14]. We further acknowledge that the eligibility criteria for the drug–event combinations in this study were driven by reasons to avoid potential bias and conflicts of interest. Whilst the selected adverse events are routinely used for safety monitoring of vaccines, most of these are also considered relevant for generic drugs due to their multifactorial etiology.

The EHDEN databases we had at our disposal varied in coverage both in terms of geography, setting and data capture. However, not all databases could contribute meaningfully to all analyses and sometimes data capture was insufficient to answer specific questions. Overall, this emphasizes the importance of having access to a large and diverse data network. Furthermore, the inferences that can be drawn from this study are determined by the analytical choices that were made. Future initiatives may need to consider flexibility in defining the input parameters including time at risk windows for descriptive analyses. For this study we used data visualization tools available in the OHDSI ATLAS interface. Future research could explore the adoption of other tools [47, 48] or develop novel ones tailored to the signal management process. Lastly, we were only able to consider analyses in the routine health data for a small set of signals (8 in total) and we did not systematically assess all possible analyses that could have informed decision making for each of these. The design of descriptive analyses was driven by questions from the signal validation team which limited the scope of analyses that were undertaken, and we envision that a more systematic data-driven exploration in parallel could have further enhanced the value of the data by providing more hypothesis-free contextual information. For instance, assessing the potential public health and clinical impact as well as characterizing all events occurring prior to drug initiation could provide relevant contextual information irrespective of the drug–event combination under investigation. On the other hand, identification of possible alternative explanations requires a more customized approach as it requires defining the main indications for treatment that are specific for each combination. A more systematic approach would also include pre-determined criteria for when a signal would be suited for further analysis in routine health data.

Signal validation and prioritization tend to be performed under significant time pressure, with timescales ranging from hours to days. The additional insights from analysing routine health data described above did come at a cost in added time and complexity of each signal reviewed at this stage of signal management. Future research may explore to what extent these analyses can be performed in a more formulaic way, either via large-scale pre-computed analytics or via standardized analyses allowing limited customizable input from signal assessors and epidemiologists. Examples of analyses lending themselves well to pre-computation may be those based on pre-specified drugs and established phenotypes that have been validated. Analyses better suited for execution of customizable scripts may be those that require relevant comparator cohorts, subgroups, or phenotypes to be defined after a signal has been identified. Generally, the development and evaluation of additional phenotype definitions is critical to enable broader use and greater impact of analyses of these data in support of signal management. There is also a need for more effective bridges between pharmacovigilance adverse event data coded in MedDRA® and phenotype definitions in the routine health data. Similarly, governance around the access and use of routine health data may need adaptation, to better support the pharmacovigilance use case. For this study, we were able to base our applications for data access approval to each data partner on a single protocol (adhering to the European PAS format) without requests for individual adaptation, and time to approval ranged from a few weeks to months. However, harmonized approval of the same protocol for an entire database network would help further streamline and speed up this process.

5 Conclusions

Analyses of routine health data within EHDEN can support signal validation and prioritization. The impact on decision making in this study varied between signals, and further research will be required to determine to what extent this can ultimately improve our ability to identify and communicate relevant safety signals. Broader use of routine health data in signal management will require a more formulaic approach including pre-computation and standardized analyses with limited customizable input, more extensive phenotyping of relevant adverse events, as well as more effective data access reviews.