Background

The thalidomide catastrophe around 1960 and additional experiences such as serious adverse drug reactions to high oestrogen oral contraceptives in the 1960s were probably the main reasons for the increasingly stringent requirements set to document development safety and the establishment of spontaneous reporting systems [1, 2]. Over the years, the repeated occurrence of unexpected, serious adverse drug reactions (ADRs) has attracted wide professional and public attention, with the result that doubt has been cast on the effectiveness and quality of drug safety surveillance systems. The COX-2 scandal resulting in worldwide withdrawal of Vioxx® (rofecoxib) from the market in 2004 is a recent example of an ADR case that emerged unexpectedly and took the world by surprise [3]. Several other ADR cases have been discovered after marketing; well known are fenfluramine and the risk of pulmonal hypertension, vigabatrine and visual field defects and tolcapone and the risk of liver toxicity [4–6]. The repeated occurrence of serious ADR cases after medicines have been released on the market questions the extent to which existing systems and methods for predicting ADRs are effective [7]. Information about the ADR profile of a new medicine appears from observations made during the clinical development process [8, 9]. The gold standard for the design of these clinical trials is the randomised controlled clinical trial (RCCT) [8, 9]. The RCCT was designed to measure efficacy rather than ADRs as outcome. The design of the RCCT as hypothesis testing in itself sets narrow limits for the detection of information about serious and unexpected ADRs due to the short treatment period, the relatively small number of carefully selected participants in the trial, fixed drug doses, and hospital settings that do not reflect the conditions under which the medicines are used after marketing [8, 9]. Data on well-recognised, easily detectable ADRs may potentially be observed in RCCTs, but unknown, rare or long-term adverse effects are seldom detected in these trials due to the limitations of the RCCT. Detection of unknown or rare ADRs may include other pharmacovigilance designs, e.g. the spontaneous reporting systems, cohort or case-control studies [1, 10–12]. This article aims to review ADRs reported by available ADR signal detection approaches and to explore which information about new and unexpected ADRs these approaches have detected.

Methods

We selected three different therapeutic groups of medicines for review. The groups were characterised by different:

a. Therapeutic groups

  • Antibiotics for systemic use

  • Non-steroidal anti-inflammatory drugs (NSAIDs)

  • Selective serotonin re-uptake inhibitors (SSRIs)

b. Market launch

Antibiotics were first marketed in the 1940s and NSAIDs in the 1960s, while SSRIs were not launched until the middle of the 1980s (internal documents, The Danish Medicines Agency).

c. ADR profiles

The therapeutic categories present different ADR profiles due to their specific pharmacological characteristics and functions.

Literature search

Studies were identified through Medline (from 1966) and Embase (from 1989) using the following MESH terms: serotonin re-uptake inhibitors, anti-inflammatory agents, non-steroidal, anti-bacterial agents, adverse drug reaction reporting systems, pharmacoepidemiology and the key words: adverse drug reactions and information in combination. The literature search was conducted in July 2007 without language restriction. Studies written in non-European languages were later excluded. To be considered relevant for this review, articles had to be empirical in origin and focus on signal detection. Titles and abstracts of the search results were screened and relevant articles identified. The reference lists of included publications were hand-searched for possible additional relevant studies. Non peer-reviewed articles or unpublished observations were not considered. A flow chart of the study selection process for the therapeutic cases is illustrated in figure 1.

Figure 1
figure 1

Flow chart of the study selection process for the cases.

Full size image

Characteristics of the included studies

We developed a taxonomy inspired by general guidelines for pharmacoepidemiological research to analyse the studies systematically [13]. The taxonomy covers the following characteristics: publication year, design, method, explored medicine and adverse drug reactions, geographic setting, sampling period, sample size, outcome measures and results. We extracted and compared the results of published empirical studies in which various signal detection methods were used. Extracted information was entered into data sheets, one for each article. Data were extracted and handled by the first author and checked by the second author.

Analyses

For each of the three selected therapeutic groups, we analysed the time of publication, the strengths of the evidence in the different approaches, reported ADRs and whether the studies had produced new information about ADRs compared to the information available at the time of marketing.

Classification of the tested/detected ADRs

For each included literature reference, the ADRs tested or detected via the various signal detection approaches were classified according to Rawlins' classification system [14]. An overview of the classification system is shown in table 1. The reported/detected ADRs were also classified according to System Organ Classes in keeping with MedDRA terminology [13].

Table 1 Rawlins' classification system of ADRs
Full size table

Classification of applied approaches

The explored approaches were classified into analytical or observational approaches according to Strom's definitions [13]. Case-control and cohort studies are classified as analytical methods, while spontaneous reporting, case series/case reports and PEM studies are observational [13].

Time of publication

For each therapeutic group, we analysed whether there was a connection between time of publication and the applied study design.

Strength of evidence

Evidence-based medicine operates with an evidence hierarchy for evaluating the quality of the various study designs used for therapeutic studies [13]. At the top of this hierarchy are the meta-analyses (level 1), followed by RCCTs at the second level and other controlled trials at the third level. Cohort studies are placed at the fourth level, followed by case-control studies at the fifth level. At the bottom of the evidence hierarchy are cross-sectional surveys (level 6) and anecdotal case reports (level 7) [13].

Results

The literature search identified 327 potentially relevant references for all three therapeutic groups, 149 of which were selected from the titles and abstracts and further screened for relevance. Eventually 79 references were included in this analysis. A flow chart of the selection and exclusion process is illustrated in figure 1. The included studies were distributed on the three therapeutic cases as follows: antibiotics: 23 studies; NSAID: 35 studies; SSRI: 20 studies. One reference was not accessible.

ADR detection approaches applied

Table 2 provides an overview of the categorisation of the designs used in the included studies and their rank in the evidence hierarchy [13]. As the table indicates, the majority of the included studies dealt with analyses of data reported in Prescription Event Monitoring (PEM) programs and ADRs reported to national ADR databases, approaches ranking at levels six and seven in the evidence hierarchy.

Table 2 The analysed studies categorised by study design
Full size table

Study characteristics

Tables 3, 4 and 5 display the characteristics and descriptions of the analysed studies for each therapeutic case [15–92]. The tables show that the studies primarily dealt with analyses of ADRs of the type A and B, and to a lesser extent C and D. The evidence level of ADRs varied widely; some of the ADRs were documented in both the analytical and observational studies, others in only one of the designs.

Table 3 Characteristics of studies of the occurrence of ADRs related to antibiotics use
Full size table
Table 4 Characteristics of studies of the occurrence of ADRs related to NSAID use
Full size table
Table 5 Studies of the occurrence of ADRs related to SSRI use
Full size table

Data sources

Case-control studies were carried out on data from various national registers and/or data from spontaneous ADR databases, physicians' databases such as the General Practitioners' database in the UK and Health Insurance Databases [15, 16, 37, 38, 72, 73]. The studies were reported in the literature from the mid-1980s to the end of the 1990s. Cohort studies analysed ADR data collected from the mid-1980s to the end of the 1990s. The cohort studies varied in size from less than 20,000 patients to between 20,000–50,000 and more than 100,000 patients [17, 19, 21, 40, 41]. The PEM studies were conducted in the UK at the Drug Safety Unit in Southampton, and were based on data collected from the mid-1980s to the end of the 1990s [22, 23, 58–66, 86–89]. Studies analysing spontaneously reported ADRs were conducted on large spontaneous reporting databases such as the French, American, British and the Uppsala Monitoring Centre WHO database [44–48, 51, 55–57, 62, 74–76, 79, 82, 83, 85].

Design and historical perspective

The antibiotic studies were published from 1990 and onwards, most of them from 1995. Cohort studies were published during 1990–1994, while the PEM studies, spontaneous reporting, case reports/case series primarily were published after 1995. The majority of the NSAID studies were published after year 2000. The SSRI studies were published from 1990 to present, most of them from 1995 to 2005. Table 6 shows the distribution of the analysed studies by type of approach, therapeutic case, and time of publication. For all therapeutic cases, data were collected and the studies published a long time after the drugs were first marketed. Despite the decades of difference in market launches for the therapeutic cases, the studies are mainly published from the end of the 1990s and on. Data were collected earlier.

Table 6 Number of studies categorised by number, design and time of publication
Full size table

Explored and detected ADRs

Antibiotics

ADRs from newer types of antibiotics, such as fluoroquinolones, have been reported much more frequently in the literature than ADRs from the older antibiotics, such as penicillins and macrolides [15–17, 19, 20, 22–24, 26, 29, 33, 34, 36]. The studies explore a possible risk between the use of antibiotics and the risk of liver, cardiovascular, CNS and dermatological ADRs [18–20, 22, 24, 27, 29, 30, 32, 33, 36]. Three cohort studies documented a correlation between cholestatic hepatitis and the use of flucloxacillin [18–20]. Increased risk of palpitation from the use of norfloxacin compared to ciprofloxacin/ofloxacin was demonstrated [22]. Cohort studies further demonstrated a risk of pemphigus related to penicillins, liver injury related to flucloxacillin and erythromycin [18–21]. CNS and dermatological ADRs from treatment with antibiotics have been reported rarely and on the case report level [30, 32, 33]. New information about ADRs was only produced by case reports: acute psychotic stress and glossitis/black tongue [34, 35].

NSAIDs

Studies explored the risk of gastrointestinal [37–44, 50] and dermatological ADRs as well as the development of liver and kidney toxicity which are well known ADRs associated with NSAIDs and their pharmacological characteristics[38, 39, 48, 51, 53, 54, 57, 65, 69, 70]. The studies were generated after the launch of COX-2 inhibitors in the mid-1990s. A case-control study documented increased risk of developing dermatological ADRs of the type Steven-Johnson Syndrome and toxic epidermal necrolysis as did spontaneously reported ADRs [38, 48]. A case-control study documented hepatic injury related to the use of NSAIDs, as did spontaneously reported ADRs, while renal injury and hypertension was documented in spontaneous reports and thromboembolic events in a PEM study [39, 45, 47, 51, 53, 54, 63, 70]. With the exception of case reports, the approaches used did not produce information about ADRs that had not been reported previously.

SSRIs

Studies explored the risk of extrapyramidal symptoms, withdrawal syndromes and serotonin syndrome with the use of SSRIs, other ADRs investigated were: changes in testosterone and natrium level, alopecia, liver injury and bleeding. ADRs reported only via spontaneous reports are first-trimester exposure on newborns and neonatal withdrawal syndrome, hepatic injury and pancreatitis, suicide, non-puerperal lactation and serotonin syndrome [72–81, 79, 83, 85, 86, 90–92]. With the exception of case reports, the approaches used did not detect new ADR signals that had not been reported previously [90–92].

Information about ADRs reported across approaches

Analytical

The approaches produced information about ADR risks compared to placebo or similar drugs as either odds ratios (OR), proportional reporting ratio (PRR) estimates, incidences (IC) and frequencies of ADRs. These parameters are built into the design and based on previous information or hypothesis. The studies were conducted on various patient populations, various medicines within the individual sub-groups, and different types of ADRs, different outcome measures, data sources and time periods. The purpose of the approaches made it possible to adjust the ADR estimate for known confounders and risk factors.

Observational

The approaches produced information about ADRs as estimates (OR, PRR, IC) or as single observations compared to placebo/similar medicines. Case reporting was the only approach that contributed new information about new ADRs in all three therapeutic cases.

Discussion

This review has several main findings:

First, analytical approaches ranging higher in the evidence hierarchy provided information about risks of already known or expected ADRs, while information about new and unknown ADRs was detected by case reports only, which range at the lowest level in the evidence hierarchy. Second, the studies primarily dealt with analyses of ADRs of type A and B, and only a few studies analysed type C and D. Third, similar approaches, both analytical and observational, were applied to all therapeutic cases. Fourth, the ADR cases provided similar results with regard to detecting new ADRs despite their connection to different time periods and organs attacked.

Methodological quality and capability of approaches

There is a general lack of standards in the field of ADRs, particularly because many ADRs are not detected until after marketing and the studies are based on selected patient groups, which makes it difficult to generalise the results to other patient groups. As previously argued in the literature, testing specific hypotheses in the analytical approaches makes it difficult to capture information about new and unknown ADRs [13]. Despite the fact that these types of studies rank high in the evidence hierarchy, the weaker design of the observational studies makes them more suitable for discovering previously undetected ADRs. Healthcare professionals have conventionally considered cohort and case-control studies to be well suited for post-marketing surveillance of ADRs, despite their lack of randomisation and lower position in the evidence hierarchy, level 4 and 5 respectively [14]. These studies primarily detected/analysed ADRs of type A and B and less frequently type C and D [14]. Thus, the approaches are not designed and therefore are not suitable for predicting new information about other ADRs that have not previously been detected or ADRs of the type C or D [14]. Case reports have provided data about patients, suspected ADRs, medicines involved and so on, but this information is often anecdotal in nature and collected retrospectively. However, it is interesting that despite their low rank in the evidence hierarchy, these reports provide new information about rare and previous undetected ADRs. Case reports may serve as whistleblowers, thereby initiating larger systematic analyses of patient populations or registering data to quantify the risk. A large majority of spontaneously reported ADRs are stored in databases hosted by regulatory agencies. Information about these observations is typically only released to the public in the form of press releases, insertions in product information or messages in national bulletins. If all these signals were published in the scientific literature or made public on the web pages of regulatory agencies, the number of spontaneous reports/case series would probably have been larger and added to the relative dominance of this design [93, 94]. The results confirm that spontaneous post-marketing reporting of ADRs is of great importance and that regulatory agencies must continue to encourage spontaneous reporting of ADRs [93, 94].

Alternative signal detection approaches

New ADR signals are often documented by only a small number of case reports, and systematic inclusion of data mining procedures in assessment of new ADR signals would probably contribute to earlier detection and quantification of serious ADR signals [95, 96]. However, data mining was not applied in the three therapeutic cases studied here. Examples of data mining are cumulative techniques, time scans and Poisson methods, proportional reporting ratios (PRRs) and Bayesian data mining [97]. These methods assess how much the observed reporting frequency of a given drug-event combination deviates from that expected, given statistical independence between drug and event. Methodological and practical experiences with data mining in signal detection are limited [97, 98].

Strengths and limitations of the study

The objective of this review was to analyse which information signal detection approaches have produced about new ADRs in selected and published therapeutic cases, rather than to perform a systematic review of the entire body of ADR literature covering all therapeutic groups. The choice of widely different therapeutic cases and the similar results obtained across therapeutic cases make us believe that the results qualitatively reflect the general, published experience on ADRs based on signal detection approaches. Findings across therapeutic cases were similar with respect to methodological approaches and time of publication, despite the fact that ADRs differed in nature and affected different organs. Although antibiotics have been marketed since the 1940s, it was not possible to search for literature before the mid-1960s due to the limitations of current databases. Lack of consistency in reporting ADRs, different methodologies used in the studies and their impact on the results are difficult to evaluate in this review.

Conclusion

Although the medicines were launched in different decades, approaches to the ADR studies were similar for all three therapeutic cases: antibiotics, NSAIDs and SSRIs. Descriptive as well as analytical designs were applied. Despite the fact that the analytical studies rank higher in the evidence hierarchy, only the descriptive case reports/spontaneous reports provided information about new and previously undetected ADRs. This review underscores the importance of systems for spontaneous reporting of ADRs. Therefore, spontaneous reporting should be encouraged further and the information in ADR databases should continuously be subjected to systematic analysis.

Acknowledgements

We thank The Danish Medicines Agency and the Hørslev Foundation for their financial support of the study.