Background

Chronic obstructive pulmonary disease (COPD) is a debilitating disorder that has become a major public health concern worldwide [1–3]. Guidelines for COPD management and treatment are predominantly based on results from double-blind, placebo-controlled, randomised controlled trials (RCTs). Generally considered to be the optimal study design to test the efficacy and safety of medical interventions [2], RCTs are designed to answer specific questions about treatments. This requires a uniform and well-characterised patient population, supervised care, careful monitoring, and control of factors that may confound or modify any potential effects [4]. However, this stringent selection means that RCT findings may be limited in the extent to which treatment effects can be extrapolated to a broad general patient population [4, 5]. For example, real-life patients with COPD tend to be older than trial participants, either because RCTs restrict the age range, or because they exclude patients with comorbidities [6], and the latter are sometimes excluded despite drug effectiveness being influenced by comorbidities [7].

Treatment of most patients with COPD occurs under very different conditions from RCTs, where a multitude of factors may influence the real-life effectiveness of therapeutic options [8]. This gap between community patients and RCTs leads to the so-called clinician’s fallacy, in which an inaccurate view of the nature and causes of a disease results from studying a small number of cases in clinical trials [9]. The process of care in clinical trials may also influence the assessment of treatment efficacy: for example, guidelines for COPD management recommend that patients with COPD are seen 1–2 times a year [10], while many RCTs improve adherence through much more frequent visits [11, 12]. Real-life adherence to treatment is not only low, but also influenced by side effects, which may in turn influence patients’ response to medication [13–15].

Long-acting bronchodilators are currently one of the first choices of maintenance medication for COPD according to guidelines. These can be used alone, together with other bronchodilators, or in combination with inhaled corticosteroids (ICS) [2, 10]. However, little is known about how representative participants of the RCTs testing bronchodilators are of the general COPD patient population. Despite this, there is a widespread and frequently quoted assumption that over 90 % of people treated for COPD would be ineligible to participate in RCTs [16–18]. To our knowledge, five studies have addressed this question and have reported eligibility ranges from 5 to 42 % [16, 19–22]. These studies were all limited, either by considering a low number of patients [16, 19, 21], by selecting patient populations that are likely not representative of community patients with COPD [16, 21, 22], or by only considering a limited range of selection criteria employed by RCTs [16, 19–21]. Individual studies show that the full range of criteria relevantly affect eligibility for participation in RCTs. There is therefore a need to combine the evidence provided by previous studies in order to give the full picture of eligibility, as well as to study any potential changes in trends [23].

A comprehensive assessment of the relevance of study findings to general patient populations requires thorough description of patient selection and clinical management [5]. The aim of this study was to determine the proportion of the general UK patient population with COPD that would be eligible for inclusion in recent RCTs testing inhaled long-acting bronchodilators. The objectives were; 1) to give an overview of inclusion and exclusion criteria applied in relevant clinical trials, 2) to describe the frequency of the characteristics and conditions of these selection criteria in a general COPD patient population, identified from a large database of anonymised medical records, and 3) to determine the percentage of patients with COPD in the database who would meet the eligibility criteria for RCTs of inhaled long-acting bronchodilator therapy.

Methods

Selection of RCTs and study population

We selected RCTs investigating the effects of inhaled long-acting bronchodilators in COPD from studies registered at https://clinicaltrials.gov/ through to 13 October 2014, using the criteria listed in Table 1. Briefly, eligible RCTs were phase III or phase IV trials in which spirometry parameters, COPD exacerbations, or COPD mortality were the primary efficacy outcome. For further details on search terms, see the online supplement.

Table 1 Criteria employed for selection of RCTs from ClinicalTrials.gov, and patients from OPCRD
Full size table

The population used to test the effects of selection criteria was patients in the Optimum Patient Care Research Database (OPCRD) [24], aged ≥40 years, with a confirmed diagnosis of COPD, as well as data on forced expiratory volume in 1 s (FEV1), modified Medical Research Council (mMRC) score [2] and full blood counts (Table 1). The index date was defined as the date of the last data extraction from the general practice.

Data source

The OPCRD is a quality-controlled, longitudinal, respiratory-focused database that contains de-identified data from general practices across the UK [24]. The database contains information about patient management in primary and secondary care and combines electronic patient records with linked patient-reported data, which are collected using disease-specific questionnaires. Routine clinical data are extracted from general practice management systems and include patient demographic characteristics, comorbidities, exacerbation history, mMRC score and current therapy. At the time of the study, the OPCRD contained 44,376 patients with a diagnostic Read code for COPD recorded who had data extracted from practice at least once from January 2011 to January 2015. The database has been approved by the Trent Multicentre Research Ethics Committee for clinical research use (approval reference 10/H0405/3). The study was approved by the Anonymised Data Ethics Protocols and Transparency committee, which is the independent scientific advisory committee for the OPCRD, and the study protocol was registered with the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (registration number ENCEPP/SDPP/7727).

Methods of analysis

To identify and analyse the data, a stepwise method was employed (see Additional file 1). Briefly, all inclusion and exclusion criteria reported in eligible RCTs were collected, and all published manuscripts and protocols were checked for additional selection criteria. Eligibility criteria were then divided into categories, which were translated into definitions of criteria compatible with the OPCRD database (Additional file 2: Tables S1 − S7). These criteria were applied to the database patient population with complete data on FEV1, full blood counts, and mMRC score, and the results were reported as mean (range) and/or median (interquartile range [IQR]). The percentage of OPCRD patients eligible for inclusion in each RCT was calculated for two reference populations: the full eligible population of patients with COPD, and a subpopulation with mMRC score ≥2, ie patients who have symptoms of moderate dyspnea and comprise a more specific target population for treatment with long-acting bronchodilators. Eligibility time-trends were studied by dividing the RCTs by start year using three 5-year periods, and differences were assessed using Kruskal-Wallis Test.

Results

Inclusion/exclusion criteria reported in RCTs

Using the selection process outlined in Fig. 1, 31 RCTs were studied (Table 2). These trials, which had start dates between February 1999 and July 2013, tested three long-acting muscarinic antagonists (LAMA; tiotropium, aclidinium and glycopyrronium), three long-acting β-agonists (LABA; formoterol, indacaterol and olodaterol), and three LABA/LAMA combinations (indacaterol + glycopyrronium, vilanterol + umeclidinium and tiotropium + olodaterol). Eighteen trials (58 %) were carried out in the last 5 years of the study period (2009–2013). FEV1 was the primary outcome in 29 (94 %) of the trials, while the remaining two [25, 26] studied reduction of exacerbation rates.

Fig. 1
figure 1

Selection of clinical trials. Flow chart showing the criteria and stepwise selection process of eligible randomised controlled trials (RCTs). Search parameters are outlined in the supplementary methods. Abbreviations: COPD = chronic obstructive pulmonary disease; FDC = fixed-dose combination; ICS = inhaled corticosteroids; LA-BD = long-acting bronchodilator; LABA = long-acting β-agonist; LAMA = long-acting muscarinic antagonist; PDE-4 = phosphodiesterase-4; RCT = randomised controlled trial

Full size image
Table 2 Selected RCTs, sorted by experimental drug tested, start date and publication date
Full size table

An overview of RCT eligibility criteria is shown in the supplement (Additional file 3: Table S8; Additional file 4: Table S9; Additional file 5: Table S10). Briefly, all trials included patients aged ≥40 years with a smoking history of ≥10 pack years. Patients with mild airflow limitation (%predicted FEV1 ≥ 80), which was found in 19 % of OPCRD patients, were excluded from all trials. Patients with severe airflow limitation (%predicted FEV1 < 30), found in 4 % of patients, were excluded from half of them, and most trials excluded patients with a recent history of exacerbations (n = 25). Other frequently applied COPD-related exclusion criteria included oxygen treatment (n = 23), recent participation in a pulmonary rehabilitation program (n = 16) and use of maintenance oral corticosteroids (n = 12). Finally, all trials excluded patients with asthma, all but one [27] excluded patients with concomitant pulmonary disease, and all excluded patients with other clinically significant diseases using diverse methodology.

Distribution of reported criteria in the OPCRD population

Using the selection criteria listed in Table 1, 36 893 eligible patients were identified in the OPCRD database (Fig. 2). Demographic and clinical characteristics of this population are shown in Table 3, and the distribution of other applied criteria is shown in the supplement (Additional file 6: Tables S11 − S17).

Fig. 2
figure 2

Selection of study population. Patient flow chart showing selection of study population from the Optimum Patient Care Research Database (OPCRD). Abbreviations: COPD = chronic obstructive pulmonary disease; FEV1 = forced expiratory volume in one second; mMRC = modified Medical Research Council score [2]

Full size image
Table 3 Characteristics of OPCRD patients with COPD that fulfil study criteria in Table 1 (n = 36 893)
Full size table

Substantial differences were identified between RCT participants and patients with COPD in the OPCRD database. OPCRD patients were on average 7 years older than RCT participants (71 and 64 years, respectively), and they were less frequently male (53 % vs 76 %). The %predicted mean FEV1 was also substantially lower for RCT participants than for OPCRD patients (49 and 63 %, respectively).

Of the selected OPCRD patients with COPD, almost half had mMRC score ≥2 [2], 69 % had recorded prescriptions of maintenance therapy and 50 % had a history of COPD exacerbation within the past year.

Proportion of OPCRD patients eligible for inclusion in RCTs

The overall median eligibility of OPCRD patients with COPD to participate in RCTs was 23 % (IQR 12–38), mean 24 % (range 3.5–58 %, Tables 4, 5 and 6). Studies of olodaterol [28, 29] showed the highest eligibility (55 and 58 %), while studies of combination therapies showed the lowest overall eligibility (13 %, Table 6), which was primarily due to all but one [30] being restricted to patients with evidence of COPD symptoms. As expected, the two studies of indacaterol + glycopyrronium that focused on reducing exacerbations in a patient population with a history of exacerbations in the past year [25, 26], showed the lowest inclusion of 3.5 and 3.9 %, respectively.

Table 4 Percentagea of total OPCRD patients with COPD eligible for RCTs testing tiotropium (n = 36 893)
Full size table
Table 5 Percentagea of OPCRD patients with COPD eligible for RCTs testing other single treatmentsb (n = 36 893)
Full size table
Table 6 Percentagea of total OPCRD patients with COPD eligible for RCTs testing combined treatmentsb (n = 36 893)
Full size table

The overall median eligibility (IQR) in the first 5-year period (1999–2003) was 16 % (10–24 %) when five RCTs testing tiotropium were carried out. This rose to 31 % (26–43 %) in the second 5-year period (2004–2008), when trials of other single therapies were carried out, and decreased to 18 % (8–38 %) in the final 5-year period (Fig. 3a). No significant difference was observed in overall eligibility across the 15-year period (p = 0.076). The relatively high mean eligibility in 2004–2008 could mainly be attributed to more relaxed selection criteria for the level of FEV1 in studies testing single therapies other than tiotropium (Tables 5 and 6).

Fig. 3
figure 3

Eligibility to participate in randomised controlled trials (RCT). Eligibility of patients from the Optimum Patient Care Research Database (OPCRD) to be included in individual randomised controlled trials (RCTs) by RCT start year. Data show fraction of; a total population of patients with chronic obstructive pulmonary disease (COPD) and b patients with COPD and modified Medical Research Council (mMRC) score ≥2. Data on trials investigating long-acting β-agonists (LABA) are indicated by ○, long-acting muscarinic antagonists (LAMA) by ▲, and LABA + LAMA combination therapy by ■

Full size image

A subanalysis of OPCRD patients with more severe dyspnea (mMRC score ≥2) [2] showed that median eligibility increased from 23 to 27 % (range 4.7–60 %) compared with the overall population (Additional file 6: Tables S18–S20). A small average increase in eligibility of 1.0 and 0.6 % was observed for RCTs testing tiotropium or other single therapies respectively, while the eligibility for RCTs testing combination therapies increased by an average of 7.5 % compared with the overall population (Fig. 3b).

Discussion

Using the selection criteria reported by 31 RCTs and applying them to a broad UK primary care population, this study showed that the overall median eligibility of patients with COPD to participate in RCTs of inhaled long-acting bronchodilators was 23 % (IQR 12–38). The highest eligibility was identified in two studies of olodaterol (55 and 58 %). Conversely, the lowest eligibility was identified in two trials of indacaterol + glycopyrronium that required a history of frequent exacerbations (3.5 and 3.9 %). Some variation was observed in eligibility over time (1999–2013) with a mean eligibility of 16, 31 and 18 % in the first, second and third 5-year periods respectively, although no significant difference was observed over the whole 15-year period. A subanalysis of patients with more severe dyspnea (mMRC score ≥2), who would likely be the patients treated in practice, showed an overall median eligibility of 27 %.

Five studies have previously investigated the eligibility of patients with COPD for participation in RCTs of bronchodilators [16, 19–22]. In agreement with our results, these studies reported that the majority of community patients with COPD would be excluded from RCTs, although the proportion varied substantially. Our data agree with the findings by Kruis et al [20] that RCT participants are on average younger, more likely to be male, and have lower %predicted FEV1 compared with the COPD patient population. Although previous studies have highlighted important limitations of the generalisability of COPD-related RCTs, they themselves suffer from substantial limitations that raise questions about the generalisabilty of their findings. The first is the representativeness of the chosen patient population. Four previous studies considered a low number of patients (110–696) [16, 19, 21, 22], which likely limits the accuracy of their estimates. Of these, two [21, 22] considered patients from hospital clinics, who likely have more severe disease on average than patients seen in primary care. A third study [16] identified patients with COPD in a postal survey of randomly selected adults in the community. However, the response rate was low (21 %), and many of those identified as having COPD had not previously been diagnosed, which was likely one of the main reasons for the low reported eligibility. The second important limitation of previously published studies is the range of eligibility criteria considered. Four of them [16, 19–21] considered a limited range of eligibility criteria. Of these, one [21] only considered criteria from a single RCT and another [19] did not specify how the criteria were chosen. Finally, only two previous studies [16, 20] provided information on eligibility for participation in individual RCTs.

Compared with the above, the current study has several strengths. In the UK all patients are registered with a General Practitioner (GP) who holds records that include demographic information, disease and comorbidity diagnoses as Read codes, prescribing information, test results, and information related to secondary care visits and hospitalisations. The OPCRD database is a large UK community database containing anonymised research-quality data focused on respiratory disease, derived from these GP records. Firstly, the database enabled us to assess the eligibility of patients for RCTs in a large and uniquely representative UK patient population with a diagnosis of COPD that meets the requirements of the Quality and Outcomes Framework (QOF), which is the UK system for the performance management and payment of general practitioners [31]. The results are therefore more likely to be truly representative of community patients with COPD than previously published studies. Secondly, we carried out an unbiased and comprehensive search among all trials registered at ClinicalTrials.gov and identified 579 studies testing long-acting bronchodilators in patients with COPD. From these, we selected 31 double-blind, placebo-controlled RCTs according to pre-specified criteria, extracted all reported selection criteria and translated them into definitions compatible with electronic medical records. This resulted in a range of selection criteria that was as close to the original RCTs as practically possible, which therefore likely provides better estimates of true eligibility. An example of this is that, while the study by Kruis et al [20] considered a large and likely representative COPD patient population (n = 3 508), the study only applied common inclusion criteria based on spirometry, smoking status and previous COPD exacerbations, but did not consider exclusion criteria such as COPD-related characteristics, the presence of asthma, atopy or other clinically relevant diseases. In the case of the UPLIFT trial [12], this resulted in a dramatic difference in eligibility, which was found to be 23 % in our analysis and 42 % in that by Kruis et al. [20] Conversely, Travers et al. [16] employed similar criteria to our study, but they also reported potential use of medication as one of the most common exclusion criteria. The current study did not exclude people on this basis and, in order to study the representativeness of the full COPD patient population, assumed that all patients would be capable of undergoing a washout.

Although information for this study was collected from both ClinicalTrials.gov and published literature, it is limited by the fact that not all criteria could be completely translated into definitions that match routine point-of-care data, and by potentially incomplete reporting of criteria in trial protocols. In addition to explicit selection criteria, clinicians recruiting patients for RCTs may use their own covert criteria to exclude patients who are more difficult to manage, are housebound, or have multiple comorbidities. Furthermore, some real-life data (eg spirometry) may be less accurately recorded in primary care than during RCTs. We selected patient records with a diagnostic Read code for COPD recorded after April 2008, when a post-bronchodilator FEV1/FVC <0.70 was introduced as part of the QOF diagnostic procedure for UK general practitioners and therefore available in all patients [31]. We also selected records with complete data on the applied criteria, and patients may therefore not be fully representative of all patients with COPD registered in the database. For example, eosinophil counts were missing in 7.9 % of extracted records (6.2 % of records with data on lung function and mMRC score). Another potential limitation is that exclusion criteria that refer to recent events (eg exacerbations or respiratory infections in the past 6 weeks) may be a temporary reason for exclusion that does not permanently exclude patients from RCT participation. To address this, we carried out a sensitivity analysis that included patients that would otherwise have been excluded by COPD-related criteria in the last 3 months, and found that this only increased eligibility by 1 %. Finally, this study focused on the representativeness of the patient population, but did not study other aspects that cause RCTs to differ from the real-life ecology of care [5].

Despite these potential limitations, we believe the current study provides the most comprehensive picture to date of the eligibility of real-life patients for participation in RCTs of inhaled long-acting bronchodilators. Our results show that, overall, around a quarter of community patients with COPD are eligible for RCT participation. Some studies represent less than 4 % of patients, leading to a high risk of “clinician’s fallacy”, while the most representative studies include over half of the real-life patient community.

Conclusions

This study combines an extensive range of RCT selection criteria with a large, representative COPD patient population to provide detailed information on eligibility of patients with COPD for participation in RCTs. The results highlight that interpretation of outcomes from RCTs of inhaled long-acting bronchodilators therapy in COPD must take into account that RCT participants are variably representative of real-life patients. In order to assess the relevance of the results of RCTs, it is essential that there is full and accurate reporting of trial selection criteria in published manuscripts and in clinical trial databases. This analysis also emphasises that, in addition to the results of RCTs, complementary information from effectiveness studies of real-life patients with COPD should be an important consideration for future guideline development.