Reassessing the Evidence Hierarchy in Asthma: Evaluating Comparative Effectiveness
Classical randomized controlled trials are the gold standard in medical evidence because of their high internal validity. However, their necessarily strict design can limit their external validity and the ability to extrapolate these data to real world patients. Therefore, alternatively designed studies may play a complementary role in evaluating the comparative effectiveness of therapies in nonidealized patients in more naturalistic, real world settings. Observational studies have high external validity and can evaluate real world outcomes. Their strength lies in hypothesis generation and testing and in identifying areas in which further clinical trials may be required. Pragmatic trials are designed to maximize applicability of trial results to usual care settings by relying on clinically important outcomes and enrolling a wide range of participants. A combination of these approaches is preferable and necessary.
KeywordsAsthma Guidelines Observational studies Pragmatic trials Randomized controlled trials Real world
Clinical Trial Acronyms
A Pragmatic Single-Blind RCT and Health Economic Evaluation of Leukotriene Receptor Antagonists in Primary Care at Steps Two and Three of the National Asthma Guidelines
Gaining Optimal Asthma Control
Investigation of Montelukast as a Partner Agent for Complementary Therapy
Management of Asthma in School-age Children on Therapy
Towards a Revolution in COPD Health
Understanding Potential Long-Term Impacts on Function With Tiotropium
Due to their high internal validity, classical randomized controlled trials (cRCTs) unequivocally sit atop the medical evidence hierarchy for examining management and therapeutic interventions [1, 2, 3]. To limit organizational constraints, costs, and patient dropouts, many cRCTs are of short duration, although 1-year cRCTs have been carried out [4, 5, 6]. Of particular importance, they are designed to include tightly controlled, well-characterized patient populations so as to minimize confounders and avoid loose causality relationships between an intervention and an outcome, seeking instead to identify clear cause and effect. For example, patients typically recruited to asthma cRCTs tend to have a clear-cut diagnosis; frequent use of rescue medication; substantial, concurrent reversibility to short-acting β2-agonists; high adherence with study drugs and procedures; good inhaler technique; and no (or negligible) comorbid illness. The resultant highly characterized trial population represents but a small subgroup of the broadly heterogeneous asthma population treated in everyday clinical practice.
For common chronic conditions (eg, asthma) that affect a wide, varied patient population, concerns exist regarding the external validity of cRCT data and the ability to broadly extrapolate these data to the real world patient population [7, 8, 9, 10•]. This paper examines the extent to which the findings of cRCTs in respiratory medicine, particularly with respect to asthma, can be generalized to the broader population. In addition, we consider complementary study designs and the role they play in supplementing the cRCT evidence base.
Gaps in the Evidence Base
Do Classical Randomized Controlled Trial Asthma Populations Represent Real Life Patients with Asthma?
Randomized controlled trials (RCTs) can be designed to show superiority, noninferiority, or equivalence and are used to evaluate the safety profile and efficacy of emerging therapies. Classical RCT design aims to maximize internal validity and to establish an unequivocal cause-and-effect relationship between an intervention and an outcome, and only a limited number of outcomes are evaluated. However, to establish causality, any potential confounding factor that may compromise results and their interpretation must be eliminated (as far as is realistically possible).
Comorbid and lifestyle factors present in real world patients with asthma who are frequently excluded from classical randomized controlled trial populations
Comorbid disease/lifestyle factor
Prevalence/degree of problem among patients with asthma
Rhinitis and rhinosinusitis
24%–94% (as measured in a range of European and American studies)
50%–100% (lifetime prevalence)
Anxiety and depression
25%–50% (prevalence in severe and difficult-to-control asthma)
Prevalence has increased concurrently with that of asthma over the past decades
Fivefold higher risk of GERD symptoms in individuals with asthma
Twofold higher risk of asthma in those with GERD
15%–35% (current smokers, wide international variations)
Real world inhaled corticosteroid adherence
Global Initiative for Asthma–Based Asthma Diagnostic Criteria Used in Classical Randomized Controlled Trials: Relevance in the Real World
The Global Initiative for Asthma (GINA) publishes management and therapeutic recommendations that draw heavily on cRCT evidence . To be included in cRCTs, patients must fulfill strict GINA-based diagnostic criteria to ensure that observed effects are not obscured by other actors (“confounders”) introduced, for example, by inclusion of patients with obstructive lung disease other than asthma (eg, chronic obstructive pulmonary disease [COPD]). GINA stipulates that when evaluating a patient’s response to asthma therapy, the following characteristics are consistent with an asthma diagnosis: an increase in forced expiratory volume in 1 s (FEV1) of ≥12% (and 200 mL) after administration of a bronchodilator (ie, indicative of reversible airflow limitation), an improvement in peak expiratory flow (PEF) of 60 L/min (or ≥20% of the prebronchodilator PEF) after inhalation of a bronchodilator, or diurnal variation in PEF of more than 20% (with twice-daily readings, >10%) .
Representative nature of cRCTs in asthma: percentage of real world patients with an asthma diagnosis who meet typical cRCT inclusion criteria
Criterion for inclusion in respiratory cRCT
Patients with clinical asthma eligible for cRCTs,%
Travers et al. 
Herland et al. 
Lung function diagnostic criteria outlined by GINA
Bronchodilator reversibility ≥12%
Peak flow variability ≥20%
Additional inclusion criteria typically used in asthma cRCTs
Bronchodilator reversibility ≥15%
FEV1 ≥50% and <80% of predicted
FEV1 ≥50% and <85% of predicted
Regular inhaled corticosteroid use
Nonsmoker or <10 pack-years of exposure to cigarette
Active symptoms or use of rescue drugs
FEV1 ≥50% predicted
A similar study was conducted in Norway to evaluate the extent to which a real life obstructive lung disease population met criteria commonly used in cRCTs . A minority of patients met the following commonly used asthma eligibility criteria: FEV1 50% to 85% of predicted (37.1% eligible), reversibility of 12% within the past year (14.9% eligible), absence of comorbidity (9.6% eligible), and nonsmoker (or ex-smoker with a nicotine burden <10 pack-years; 5.4% eligible). If patients were also required to be symptomatic and to have regular ICS usage, the percentage of eligible patients fell further to 3.3% (Table 2) . Thus, in asthma, the typical cRCT population represents a small minority of patients with asthma treated in real world everyday clinical practice.
This finding holds true in other areas of respiratory disease. For example, a recent prospective cohort study found that of patients treated for allergic rhinitis in everyday practice, only 7.4% of the 311 patients examined would have been eligible for major placebo-controlled RCTs of persistent and intermittent allergic rhinitis . The most common reasons for which the real world patients with allergic rhinitis would have been excluded from a cRCT were the absence of an allergy diagnosis based on skin testing and/or serum-specific IgE testing, insufficient disease severity, and the presence of comorbidities.
Applicability of Classical Randomized Controlled Trial Results in Specific Subgroups
Concerns surrounding the limited representative nature of cRCTs arise where there are data to suggest that cRCT findings may not hold true in particular subgroups of the population. Subgroups of particular note within the larger asthma population include those with poor inhaler technique, current smokers, obese patients, those with comorbid conditions (eg, rhinitis), and those with low adherence to therapy [11, 12, 13, 14, 22, 23, 24, 25, 26, 27, 28]. When considering inhaler handling, for example, data suggest a link between poor technique and poor asthma control, as improper use of pressurized metered-dose inhalers (pMDIs) for the delivery of ICS is associated with decreased asthma control .
Smoking and severity of rhinitis are also important determinants of asthma control, as patients with severe rhinitis and/or higher average cigarette use exhibit poorer control . Cigarette smoking is known to reduce the effect of ICS therapy [27, 28, 29, 30]. Other factors that have been shown to affect patients’ response to asthma therapy are obesity, likely through inflammatory mechanisms , and presence of comorbidities such as COPD and heart failure . It is often difficult in practice to differentiate between asthma and COPD, whereas only patients with a clear-cut diagnosis of asthma are included in cRCTs.
Therefore, the importance of real world factors in asthma (eg, smoking, comorbidities, patient adherence, and inhaler technique) should not be overlooked, as they may explain the wide gap between the level of asthma control that can be achieved in cRCTs  and the frequently disappointing results observed in observational studies carried out among less selected populations . However, data on the comparative effectiveness of therapies in these subgroups remain lacking. Appropriately designed studies to address these unanswered research questions may reveal differential effectiveness of therapeutic options in particular patient subgroups and could be used to help guide more tailored, individualized asthma management.
Other Gaps in the Evidence Base: Limited Outcome Evaluation, Duration of Trials, and Ethical Considerations
Another limitation of potential concern resulting from the high cost of cRCTs is that with few exceptions [4, 5, 6], they tend to be limited in the number of outcomes evaluated and short in duration, yet the resultant data are used to inform guidelines for asthma, a chronic disease requiring long-term management.
Gaps in the evidence base can also arise if ethical considerations prohibit the completion of a cRCT. The MASCOT trial, for example, was withdrawn because of an inability to identify eligible patients . MASCOT was designed to evaluate the use of combination ICS/long-acting β2-agonist (LABA) therapy in children of school age, but most patients who were potentially eligible were already receiving the study medication. Altering patients’ routine therapy when it equates to best standard of care is unethical and can present problems for trial recruitment.
Plugging the Gaps in the Evidence Base: Complementary Trial Designs
The need to look beyond asthma cRCTs when faced with gaps or limitations in the existing evidence base was recognized in the 2009 European Respiratory Society/American Thoracic Society Taskforce paper on asthma control and exacerbations . The Taskforce proposed the use of composite measures when evaluating asthma control and called for the measurement properties to be validated in clinical trials and in “large, prospective studies in ‘real-world’ settings (eg, trials designed pragmatically to reflect everyday clinical practice) to ensure they provide content validity as well as reflect clinically meaningful outcomes” . Similarly, the Cochrane Collaboration, when reviewing the effects of ICS use on linear growth in children, recognized the need for longer outcome periods than offered by typical cRCTs and specifically advised that “research efforts should concentrate on evaluating the long-term effects of inhaled steroids” . The 2008 Brussels Declaration on Asthma echoed this sentiment by stating in 1 of its 10 key points that there is a need to “include evidence from real world studies in treatment guidelines” . The Declaration’s rationale was that “asthma treatment guidelines are primarily based on evidence from large clinical trials that frequently assess lung function as the primary outcome. However, inflammatory biomarker levels, asthma exacerbations, and other outcomes may worsen regardless of lung function status” .
Sir Michael Rawlins [35••], chairman of the United Kingdom’s National Institute for Health and Clinical Excellence, added his voice to the debate in 2008 when he suggested that cRCTs should be complemented by a diversity of approaches that involve analyzing the totality of the evidence base. He argued that cRCTs “often miss the value of a therapeutic intervention and tend to be carried out in specific types of patients for relatively short periods of time” [35••]. He contrasted this approach with clinical practice “where treatments tend to be used on a long-term basis in a broad variety of patients who often have comorbid conditions” [35••].
Thus, although cRCTs are the cornerstone of medical evidence, there are specific areas in which their design (eg, strict inclusion/exclusion criteria, brevity of duration, limited outcome evaluation, interventional nature, control arm requirement) results in gaps in the full evidence base. Therefore, a role may exist for other study designs, such as pragmatic trials and observational studies, to provide data on the effectiveness and comparative effectiveness of therapies (ie, efficacy as evaluated in nonidealized patients in more naturalistic, real world settings).
The Role of Pragmatic Clinical Trials
Evaluating Real World Effectiveness
The term pragmatic trials was first used in 1967 by Schwartz and Lellouch  to describe trials designed to help choose between care options. They contrasted this with explanatory trials that were designed to test causal research hypotheses (eg, that an intervention causes a particular biological change). Schwartz and Lellouch  considered there to be a continuum rather than a dichotomy between explanatory and pragmatic trials, and characterized pragmatism as an attitude to trial design rather than a characteristic of the trial itself—an attitude that has come to be understood as one that maximizes applicability of trial results to usual care settings, relies on unequivocally important outcomes (eg, mortality and severe morbidity), and is tested in a wide range of participants [37, 38, 39, 40].
A trial using this approach for allergic rhinitis found that guideline-based treatment was more effective than free treatment choice . Thus, pragmatic clinical trial designs offer a means of testing a hypothesis in a more naturalistic, real world setting than cRCTs by modeling and reflecting everyday clinical practice in their scheduling (eg, longer treatment exposure) and in their approach to patient recruitment (eg, including patients with relevant comorbidities). While consenting patients are still assigned randomly to predefined study arms, pragmatic trials have broader inclusion criteria  than cRCTs and tend to be longer in duration [43, 44].
The international UPLIFT trial was a pragmatic trial designed to evaluate the long-term effect of tiotropium compared with placebo on lung function in patients with COPD . The UPLIFT design was more naturalistic than that used in cRCTs, as it allowed patients to continue taking their standard background therapy (any respiratory medications except anticholinergic drugs) throughout the 4-year trial. Moreover, UPLIFT used a true intention-to-treat approach when evaluating the mortality end point; thus, if patients discontinued study medication during the trial, they were still included in the final mortality analysis. It is unfortunate that the other trial end points were not treated in the same manner . Interestingly, UPLIFT found the rate of lung function decline in the “placebo” group (in which two thirds of patients were receiving an ICS and/or LABA through continuation of their standard background therapy) to be similar to that observed in the ICS, LABA, or ICS/LABA groups of the TORCH study, another long-term trial comparing these treatments with placebo .
Another pragmatic trial of note, commissioned by the United Kingdom government, is ELEVATE, an equivalence trial evaluating leukotriene receptor antagonists in primary care at steps 2 and 3 of the national asthma guidelines . Broad inclusion criteria were defined and effectiveness outcomes were measured over a 2-year outcome period; the primary outcome measure was asthma-related quality of life (QoL), a patient-oriented measure of effectiveness. Furthermore, the pragmatic trial design ensured continued patient participation in the study even if patients did not receive and complete the full prescribed regimen—a true intention-to-treat approach. As a result, the dropout rate was only 4% over 2 years in ELEVATE , which compares favorably with cRCT rates (eg, 25% in GOAL  and 16% in IMPACT ). Because patients continued to receive care at their usual practices, ELEVATE achieved high levels of complete data for the primary end point and high levels of clinical data from routine practice: more than 90% of patients supplied data at 2 years for the primary end point, and more than 95% for health care resource and asthma exacerbations. Previous cRCTs comparing step 2 and 3 asthma therapies have yielded inconsistent results on the relative efficacy of the treatment options available [48, 49, 50, 51, 52, 53, 54]. In ELEVATE, leukotriene receptor antagonists were equivalent to the comparators at 2 months with regard to QoL, and although equivalence in QoL was not shown at 2 years, there were no significant differences in secondary measures at either time point .
For seasonal allergic rhinitis, a cluster randomized trial in primary care showed that patients receiving a treatment following the international consensus on rhinitis demonstrated a large improvement compared with those receiving free treatment choice . In order to be closer to patients’ needs, the primary end point was QoL.
Evaluating Real World Adherence
Pragmatic trials also offer the possibility of evaluating adherence in a more naturalistic setting than cRCTs. Not only do cRCTs often demand unrepresentatively high levels of treatment adherence, but their inherently interventional nature can also artificially drive adherence.
One approach to more naturalistic adherence data capture is illustrated by a pragmatic trial designed to evaluate the impact of dosing regimen on adherence to asthma medication. Price et al.  compared adherence rates to once-daily and twice-daily mometasone furoate by capturing adherence data using patient self-report and dose counters. Patients with poor adherence remained in the study and thus were included in the final analysis. Using this approach, a discernible difference in adherence rates was measured between treatment arms, with greater adherence recorded for the once-daily regimen .
Evaluating Real World Cost-Effectiveness
To what extent a patient will benefit from the treatment and at what cost?
How much is the health care system willing to spend to prevent one morbid event ?
Understanding the most appropriate and meaningful data on which to base cost-effectiveness evaluations can also be challenging. A study designed to evaluate the external validity of published cost-effectiveness studies compared the data used in the published studies (typically based on cRCTs) with observational data from actual clinical practice. The authors concluded that cost-effectiveness evaluations based solely on cRCT data lack external validity and, as they do not represent patients in actual clinical practice, should not be used to inform prescribing policies . In light of this, treatment and health technology assessments should move away from analyses in carefully screened populations toward actual cost-effectiveness trials using real world clinical data .
Remaining Limitations of Pragmatic Trials
Nonetheless, however naturalistic the design of a pragmatic trial, it still requires patient consent and the involvement of “trial-minded” physicians. As such, pragmatic trials will still deal with a defined subgroup of the overall patient and physician populations, and their protocols still require closer monitoring of clinical and biological parameters and more frequent contact with health care professionals than occurs in standard clinical practice [9, 59, 60, 61].
As pragmatic trials are designed to study real world practice, they are less effective than efficacy trials, sacrificing internal validity to achieve generalizability . Poor design and/or execution (true also of some cRCTs) can bias results toward similar efficacy across treatment arms and therefore bias the study toward a finding of equivalence. Better designed pragmatic trials often include objective outcome measures (eg, survival, test results) and subjective measures (eg, QoL surveys), which, if broadly consistent, can diminish concerns about potential bias .
Additional challenges arise from the fact that the very characteristics of real world practice that pragmatic trials are designed to capture—including variable adherence, use of concomitant therapies, presence of comorbidities, changeable symptoms over time—tend to reduce measureable differences between therapies. Such concerns highlight the benefit of including both an intention-to-treat and a per-protocol analysis to allow regression to equivalence [63, 64]. Some of these shortcomings can be addressed—to varying degrees—by observational studies.
The Role of Observational Studies
As defined by the National Centre for Biotechnology Information, an observational study is a “type of nonrandomized study in which the investigators do not seek to intervene, instead simply observing the course of events” . As such, observational studies using clinical databases offer another method of studying the comparative effectiveness of outcomes as evaluated in real world patients in a noninterventional, naturalistic setting. They also provide a means to study, characterize, and better understand real world prescribing practices and adherence to guidelines in clinical practice. Observational studies involve accessing, collating, and analyzing information held in patient records and can be cross-sectional or longitudinal in design . Although they are limited by the lack of treatment randomization and potential bias through subjectivity of treatment choice, their use of routine clinical data gives them high external validity.
Prospective cohort studies provide important information, but they can be expensive to conduct and take many years to generate results; moreover, for logistical reasons, only a relatively small number of patients can be observed. This limits their power to detect differences in outcomes between subgroups, especially when considering relatively rare outcomes (eg, exacerbations, death). Conversely, retrospective studies look at events in the past (as recorded in patients’ clinical records), allowing the generation of more immediate results. Retrospective studies are also less restricted by patient numbers than prospective studies, as cohort definition can ensure sufficient numbers to demonstrate differences in treatment response (where such differences really exist). Well-designed database studies, while inherently retrospective, define patient cohorts and outcomes a priori based on a prespecified index event, such as a recorded treatment change (see later study examples), which can result in useful hypothesis testing. This contrasts with more skeptical views of databases being used inversely to suggest rather than answer questions.
The General Practice Research Database and the Doctors Independent Network (DIN-Link) Database in the United Kingdom are clinical databases that have been extensively used for research [66, 67, 68, 69]. They comprise patient records collected over years, not months, thereby allowing investigation of longitudinal treatment effects. These features are invaluable in hypothesis generation and testing and can also help refine the design and powering of RCTs .
Evaluating Guideline Implementation
Observational studies can help evaluate guideline implementation in everyday clinical practice. In the United States, an observational study was carried out using an integrated managed care database to characterize the patterns of care observed in patients prior to emergency department (ED) treatment of acute asthma . The study was motivated by the limited data on resource use prior to ED attendance and the recognition that better understanding of treatment patterns prior to an ED visit may help identify opportunities for improved interventions. The study explored adherence to guideline recommendations through evaluation of ICS therapy in the year prior to the ED visit and through quantification of short-acting β2-agonists and oral corticosteroids and rescue medications in the year before and in the month after the ED visit. Also investigated was the impact of the acute care intervention in the ED on altering the prescription of ICS and other asthma medications in the 2 months after the ED event. The study demonstrated a high dependence on rescue medications—short-acting β2-agonists and oral corticosteroids—in this population prior to ED attendance, and that the ED event resulted in only an incremental short-term improvement in ICS-containing controller treatment. Such characterization of prescribing patterns in real world clinical practice requires observational, noninterventional methods and cannot be achieved through cRCTs .
Evaluating Real World Influence of Inhaler Device Type
Another area of asthma management in which study alternatives to cRCTs can provide useful information is in the evaluation of inhaler type. Asthma cRCTs typically train recruited patients in inhaler technique and often require trial participants to be able to demonstrate and maintain proper inhalation technique throughout. However, in real world practice, many patients use their inhaler devices incorrectly, and proper inhaler technique is infrequently reinforced [14, 71, 72].
The REALITY study used the General Practice Research Database to evaluate the comparative effectiveness of different inhaler types as used in everyday asthma management [73, 74, 75]. Participating patients commenced or increased ICS therapy via a range of different inhaler types (pMDIs, breath-actuated metered-dose inhalers [BAIs], and dry powder inhalers). No requirements were placed on patients’ inhaler training beyond routine standard care. Significant differences in the odds of achieving successful asthma control were found for both BAI- and dry powder inhaler–treated patients compared with patients using pMDIs . Moreover, these differences had significant health economic implications, with BAIs being on average more cost-effective than pMDIs .
Evaluating Real World Effectiveness
Although well-designed observational studies have good external validity, they lack the internal validity of cRCTs. One of the main challenges of observational studies is ensuring adequate treatment group comparability, but expertise is growing and methodologies are continually evolving that assess baseline comparability of the study cohorts and also validate outcomes for consistency across multiple subgroups . When study cohorts are similar at baseline, outcome analysis can proceed with suitable statistical adjustments being made for any characteristics that are statistically or clinically significantly different between cohorts or are strongly predictive of the outcome [77, 78]. Where cross-sectional baseline data reveal substantial differences between groups, a matched cohort analysis (or other suitable methodology, such as propensity-based matching) should be used. Patients are matched on key demographic and clinical baseline characteristics to minimize any differences in baseline disease severity and to ensure that strongly confounding baseline effects are comparable across treatment groups, thus allowing study outcomes to be appropriately interpreted .
Oral corticosteroid prescriptions during baseline year
0, 1, 2, 3, ≥4
Mean SABA dose during the baseline year, μg/d
≥13 ± 5 y or
6–12 ± 3 y or
5 ± 1 y
For step-up population only: mean ICS dose prescribed during the baseline year, μg/d
For EF HFA-BDP vs CFC-BDP analysis only: number of asthma consultations without resulting in a prescription for oral corticosteroids
0, 1, 2, 3
Summary of co–primary outcomes: OR for achieving asthma control and rate ratio of exacerbationsa
CFC-BDP as the reference group (OR, 1.00)
QVAR vs BDP
EF HFA-BDP (N = 2,882)
EF HFA-BDP (N = 258)
Primary measure of asthma control, adjusted OR (95% CI)
Exacerbation during the outcome year, adjusted rate ratio (95% CI)
FP as the reference group (OR, 1.00)
QVAR vs FP
EF HFA-BDP (N = 1,319)
EF HFA-BDP (N = 250)
Primary measure of asthma control, adjusted OR (95% CI)
Exacerbation during the outcome year, adjusted rate ratio (95% CI)
QVAR vs BDPb (Fig. 1b)
EF HFA-BDP (N = 2,882)
CFC-BDP (N = 2,882)
EF HFA-BDP (N = 258)
FP (N = 258)
Distribution of patients by mean ICS dose prescribed at the index date,%
QVAR vs FP (Fig. 1b)
EF HFA-BDP (N = 1,319)
FP (N = 1,319)
EF HFA-BDP (N = 250)
FP (N = 250)
Distribution of patients by mean ICS dose prescribed at the index date,%
Remaining Limitations of Observational Studies
In addition to the difficulties in achieving comparability of treatment arms, another past criticism of observational studies has been their purported tendency to overestimate treatment effects . More recent reviews suggest that such concerns are largely unfounded. A pooled analysis of observational studies and cRCTs (taken from the Medline, Abridged Index Medicus, and Cochrane databases between 1985 and 1998), in which two or more treatments or interventions for the same condition were evaluated, found little evidence to suggest that treatment effects reported in the observational studies were consistently larger than or qualitatively different from those reported in the cRCTs. The analysis involved 136 reports across 19 diverse treatments. In only 2 of the 19 analyses of treatment effects did the combined magnitude of the effect in the observational studies lie outside the 95% CI for the combined magnitude reported in the cRCTs .
The knowledge of how to work with clinical databases and quality of practice-based patient data is continually improving as researchers collaborate and work with contributing practices to ensure that relevant, high-quality data are recorded. The matched cohort approach and validation of outcomes across subgroups that has emerged in recent observational research should help mitigate concerns around confounding of findings through differences in study populations. However, their real strength will continue to lie in hypothesis generation and testing, helping to identify areas in which further rigorous clinical trials are required.
To increase confidence in the results of database analyses, it is of the utmost importance to describe (a priori) in a detailed study protocol all planned analyses before they begin, exactly as for a cRCT. In that respect, the move toward greater ethical transparency in medical research, which requires increasing numbers of publicly and privately funded clinical trials to be registered and published in online study databases and centralized repositories (eg, http://www.clinicaltrials.gov), should also help improve communication of quality methodologies and gradually drive out poorly designed studies that can undermine the field of observational research.
The reporting of observational studies should follow the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines , while reporting of real life trials should follow the CONSORT (Consolidating Standards of Reporting Trials) guidelines for cluster randomized trials  and pragmatic trials .
Data from cRCTs represent the gold standard for evaluating treatment safety and efficacy, owing to rigorous trial design and strong internal validity. However, no study design is without its limitations, and questions exist concerning the generalizability of cRCT findings to the widely heterogeneous asthma population, and their accuracy over the longer term. Observational studies have high external validity and may assist in answering some of the questions that cRCTs have not yet answered or cannot answer. Contrastingly, their internal validity is often poor, but it can be improved by detailed a priori analysis planning and by grounding database analyses on rational hypotheses. Pragmatic clinical trials with appropriate quality checks are positioned between the two. As recently proposed by ARIA (Allergic Rhinitis and its Impact on Asthma) and GA2LEN (Global Allergy and Asthma European Network), a combination of all these approaches is probably needed because all have advantages and drawbacks and do not answer the same question .
The evidence base in asthma and respiratory medicine must be reviewed openly and broadly, and while cRCTs unequivocally lie at the core, quality contributions from well-designed pragmatic trials and observational studies also should be recognized and considered for the complementary role that they play.
Dr. Price has received grant support from the United Kingdom National Health Service.
Dr. Price has served as a consultant for Boehringer Ingelheim, GlaxoSmithKline, Mundipharma International Ltd., Merck & Co., Novartis, and Teva Pharmaceutical Industries Ltd.; has received grant support from Aerocrine, AstraZeneca, Boehringer Ingelheim, GlaxoSmithKline, Merck & Co., Mundipharma International Ltd., Novartis, Nycomed, Pfizer, Teva Pharmaceutical Industries Ltd., and Chiesi; has served on speakers’ bureaus for Teva Pharmaceutical Industries Ltd., AstraZeneca, Boehringer Ingelheim, Chiesi, GlaxoSmithKline, Merck & Co., Mundipharma International Ltd., and Pfizer; holds stock options in Akela Pharma; and is the owner of Research in Real Life Ltd.
Dr. van der Molen has served as a consultant for GlaxoSmithKline; received support for travel to meetings from GlaxoSmithKline; served on boards for GlaxoSmithKline, MSD Pharma, AstraZeneca, and Nycomed; and received grant support from AstraZeneca and MSD Pharma.
Dr. Roche has served on boards for Boehringer Ingelheim, GlaxoSmithKline, Mundipharma International Ltd., Chiesi, Nycomed, Hoffmann-La Roche, and Novartis; has received grant support from Boehringer Ingelheim and Pfizer; has served as a consultant for AstraZeneca; has received honoraria from Meda Pharmaceuticals and Teva Pharmaceutical Industries Ltd.; has received payment for manuscript preparation from Nycomed; has served on speakers’ bureaus for and has had travel/accommodations expenses covered/reimbursed by GlaxoSmthKline, Boehringer Ingelheim, Nycomed, Novartis, Mundipharma International Ltd., AstraZeneca, and Chiesi.
Dr. Hillyer has served as a consultant for Research in Real Life Ltd. and received payment for manuscript preparation from Teva Sante, Aerocrine, and Merck & Co.
Dr. Bousquet has received honoraria for serving on scientific/advisory boards, giving lectures during meetings, and participating in press conferences from Stallergenes, Actelion Pharmaceuticals Ltd., Almirall, AstraZeneca, Chiesi, GlaxoSmithKline, Merck & Co., MSD Pharma, Novartis, OM Pharma SA, Sanofi-Aventis, Schering-Plough, Teva Pharmaceutical Industries Ltd., and Uriach Pharma.
Ms. Chisholm reported no potential conflicts of interest relevant to this article.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
- 1.British Guideline on the Management of Asthma. A national clinical guideline. May 2008. Revised June 2009. Available at http://www.sign.ac.uk/guidelines/fulltext/101/index.html.
- 3.GINA report, Global Strategy for Asthma Management and Prevention. Available at http://www.ginasthma.org.
- 10.• Krishnan JA, Schatz M, Apter AJ. A call for action: comparative effectiveness research in asthma. J Allergy Clin Immunol. 2011;127:123–7. This is a discussion of comparative effectiveness research (and its challenges), how it complements traditional efficacy research in asthma, and how it helps provide a basis for rational decision making about individual patients’ care. PubMedCrossRefGoogle Scholar
- 21.Costa DJ, Amouyal M, Lambert P, et al. How representative are clinical study patients with allergic rhinitis in primary care? J Allergy Clin Immunol 2011; Jan 29 (Epub ahead of print).Google Scholar
- 32.Lenney W, Perry S, Price D. Clinical trials and tribulations: the MASCOT study. Thorax 2011; doi:10.1136/thx.2010.156885.
- 33.Sharek PJ, Bergman DA. Beclomethasone for asthma in children: effects on linear growth. Cochrane Database Syst Rev 2000; CD001282.Google Scholar
- 35.•• Rawlins MD, De Testimonio. On the evidence for decisions about the use of therapeutic interventions. In: Harveian Oration. Oct-2008 ed. London: Royal College of Physicians; 2008. As Chairman of the National Institute for Health and Clinical Excellence in the United Kingdom, Rawlins’ recognition of the importance of real world studies when considering the totality of the evidence base indicates a paradigm shift. Google Scholar
- 47.Bjermer L, Bisgaard H, Bousquet J, et al. Montelukast or salmeterol combined with an inhaled steroid in adult asthma: design and rationale of a randomized, double-blind comparative study (the IMPACT Investigation of Montelukast as a Partner Agent for Complementary Therapy-trial). Respir Med. 2000;94:612–21.PubMedCrossRefGoogle Scholar
- 53.Ducharme FM, Lasserson TJ, Cates CJ. Long-acting beta2-agonists versus anti-leukotrienes as add-on therapy to inhaled corticosteroids for chronic asthma. Cochrane Database Syst Rev 2006; CD003137.Google Scholar
- 60.Resch K-L. Pragmatic trials: propaganda or research tool? Focus Altern Complement Ther. 2005;10:181–3.Google Scholar
- 65.National Center for Biotechnology Information (NCBI) Bookshelf. Available at http://www.ncbi.nlm.nih.gov/books/NBK10370/.
- 73.Kemp L, Haughney J, Barnes N, et al. Cost-effectiveness analysis of corticosteroid inhaler devices in primary care asthma management: a real world observational study. ClinicoEconomics Outcomes Res. 2010;2:75–85.Google Scholar
- 75.Study of Treatment Effectiveness in Asthma and Rhinitis (STEAR). Merck, data on file.Google Scholar
- 77.Price D, Martin RJ, Barnes N, et al. Prescribing practices and asthma control with hydrofluoroalkane-beclomethasone and fluticasone: a real-world observational study. J Allergy Clin Immunol 2010, 126:511–518 e511-510.Google Scholar
- 78.Barnes N, Price D, Colice G, et al. Asthma control with extrafine-particle HFA-beclomethasone versus CFC-beclomethasone: a real-world observational study. Under consideration. Clin Exp Allergy 2011; Jul 14. doi:10.1111/j.1365-2222.2011.03820.x. [Epub ahead of print].