Background

The emergence of new evidence assessing the effectiveness of therapies for idiopathic pulmonary fibrosis (IPF) has created considerable interest and hope amongst patients, carers and clinicians. IPF is a devastating lung disease characterised by the deposition of excessive scar tissue within the lungs, which leads to breathlessness and ultimately respiratory failure and death. It has been well documented that the incidence of IPF is increasing but the reasons for this are unclear. High mortality rates are also well reported, and based on reported 5-year survival rates these would place IPF seventh on a list of fatal malignancies [1,2]. Until recently few treatment options have been available, so the focus of treatment for many patients has been symptom control and palliation [3].

The new optimism has stemmed from the fact pirfenidone, licensed in Europe in 2011 on the basis of evidence from 4 randomised controlled trials (RCTs) [4-6], has been shown in a recent multicentre RCT to slow the rate of progression of IPF [7]. Furthermore another agent (nintedanib) also met its primary end-point of reducing the annual rate of pulmonary function decline in two concurrent RCTs [8]. On the strength of these data the Food and Drug Administration (FDA) and the European Medicine Agency have recently approved both drugs for use in IPF. These treatments have the potential for offering new hope for patients and carers [9] and clinicians will be eager to offer patients the most appropriate treatment. It is therefore imperative to consider the weight of all the evidence for each of these treatments.

The purpose of this study was to systematically review the clinical effectiveness of treatments for IPF and present the findings of a network meta-analysis (NMA) of key outcomes. We followed the principles of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement [10].

Methods

The methods for this systematic review are described in a research protocol which is registered with the International Prospective Register of Systematic Reviews [11]. This is an update of a previous systematic review which also considered cost-effectiveness [12]. We identified articles by searches of MEDLINE, EMBASE and the Cochrane Library from database inception until May 2014 (for search strategies see Additional file 1: Table S1). No language restrictions were applied. Two independent reviewers screened titles and abstracts for inclusion and full text articles were retrieved for further scrutiny. These were reviewed by one reviewer and checked by a second to identify RCTs that included participants with a confirmed diagnosis of IPF. Eligible interventions were N-acetylcysteine (NAC) alone or in combination, pirfenidone or nintedanib, assessed on outcomes measuring indices of lung function/capacity, exercise performance, quality of life and adverse events.

Data extracted from studies included participant and study characteristics, intervention and comparator details and results. An assessment of the methodological quality of each included study was made [13]. Only published data were included. One reviewer undertook data extraction and quality assessment. These were checked by a second reviewer and differences in opinion were resolved through discussion with a third reviewer. We synthesised data in a narrative review and a Bayesian NMA [14,15]. In circumstances where randomised evidence between all relevant comparators is unavailable, network meta-analysis combines evidence from trials comparing different sets of treatments that form a connected evidence network through common comparators, in this case placebo. It retains within trial randomisation, allowing direct and indirect evidence to inform estimates of relative treatment effect in a single analysis. A vague prior distribution (a normal distribution with mean zero and variance 10,000) that contains little information relative to the likelihood was used to ensure that results were based on the data and not the choice of prior [16].

The NMA assessed five endpoints from 11 studies. An advantage of NMA is that where interventions have not been directly compared in RCTs the results of different trials may be used to estimate the relative treatment effect between two treatments through indirect comparison. In addition, the analysis allows for meta-regression to control for study level covariates which may be a source of heterogeneity between trials [17].

The decline in FVC may be measured in two ways: as a decline in FVC % predicted; and as an absolute change from baseline (litres). In addition, some trials report vital capacity, which can be assumed to be the same as FVC in IPF. We used the standardised mean difference (SMD) approach to convert these measurements to a common scale using Hedges’ adjusted g method to overcome potential small sample bias [18]. The SMDs were then converted to log odds ratios (logOR) using the method of Chinn [19] to facilitate interpretation.

Our base case used the FVC % predicted where reported, as by definition this includes adjustment for age and sex. We conducted sensitivity analyses using litres to examine variation in results arising from the different measurement scales, and controlling for baseline FVC through meta-regression.

The NMA was performed in WinBUGS using code adapted from Dias and colleagues [15], see Additional file 1: Appendix. Two chains were run for 50,000 simulations with a burn-in period of 20,000 and a thinning interval of 2, giving a final sample size of 30,000. Trace, Brooks-Gelman-Rubin, and density plots were examined to establish model convergence [20]. Fixed and random effects models were conducted with best model fit determined by the deviance information criterion [21].

Results

Searches yielded 1076 unique references and 67 of these were retrieved after initial screening. Of these we included 11 studies (Figure 1). Five RCTs evaluated the use of pirfenidone [4-7], three NAC [22-24], and three nintedanib. [8,25] Two of the NAC studies [22,23] are in fact one three arm RCT and as such there is not complete independence in the placebo arms. Table 1 provides summary descriptions of each study. The participants in all studies would likely be classed as mild to moderate IPF, with baseline FVC [26] ranging from approximately 68% to 89% of predicted values. Participants were around 64–68 years old, and were predominantly men. In most studies diagnosis was within the previous two years. The ratio of the forced expiratory volume in the first second to the FVC was not reported consistently across the studies. In three recent RCTs of pirfenidone [7] and nintedanib [7,8] this was required to be at least 0.80 and at least 0.70 for study inclusion respectively. For the RCT of NAC [22,23] this was required to be at least 0.65. Although there were subtle differences in trial inclusion criteria and diagnostic criteria, baseline characteristics were broadly similar and are deemed to be generally reflective of IPF patients with mild-to-moderate disease seen in clinical practice. As such we judged these populations to be similar enough to be combined in meta-analysis. Primary outcomes were based on the FVC in all but one trial [6]. With one exception [24], the studies had reasonable sample sizes and were adequately powered to test differences in their primary outcomes. Treatment duration ranged between eight and 16 months, with the majority following up until approximately 12 months.

Figure 1
figure 1

PRISMA flow-chart of included studies.

Full size image
Table 1 Characteristics of randomised controlled trials included in the meta-analysis
Full size table

Overall the studies were assessed as having a low or uncertain risk of bias based on the assessment of adequacy of concealment of allocations prior to assignment to groups. All but one trial [24] were assessed as having adequate randomisation procedures, and blinding was at an unclear or low risk of bias in all but one trial [24]. Most trials described using an intention to treat (ITT) analysis, although not all reported details. Several trials presented adjusted analyses incorporating a range of covariates and interactions [4,5,7,8,22,23]. As the adjustments in the analyses were to take account of within-study variability in the outcome associated with the covariates, it did not negate meta-analysis of studies with and without adjustment.

FVC

FVC outcome data used in the NMA can be seen in Table 2 and the evidence network in Figure 2. Trials were relatively homogeneous which validates the use of fixed effect models. When combining the different FVC treatment effects only pirfenidone and nintedanib produce a statistically significant slowing in the rate of FVC decline compared with placebo (ORs < 1 with 95% credible intervals [Crl] all less than 1) (Figure 3). These two treatments were compared indirectly using the placebo as the common comparator, and results indicate that nintedanib is statistically significantly better than pirfenidone in slowing FVC decline (OR 0.67, 95% CrI 0.51, 0.88). Random effects analysis of FVC supports the trends seen although the results were not statistically significant. Sensitivity analysis controlling for baseline FVC had little impact on results. Using FVC decline in litres, where reported, also gave similar results.

Table 2 Forced vital capacity outcomes reported in the included trials
Full size table
Figure 2
figure 2

Evidence network for forced vital capacity endpoint. Numbers on arrows refer to number of studies informing the comparison.

Full size image
Figure 3
figure 3

Results of the network meta-analysis for forced vital capacity.

Full size image

A number of trials reported FVC as a dichotomous outcome, using the proportion with a decline in FVC % predicted of 10% or more (see Table 2). In our NMA both treatments are associated with statistically significantly lower odds of a decline in FVC % predicted of ≥10% compared to placebo. The indirect comparison of nintedanib and pirfenidone indicates that pirfenidone is associated with slightly lower odds of a decline in FVC % predicted of ≥10% compared to nintedanib, but that this is non-significant (OR 1.21, 95% CrI 0.86, 1.72), Additional file 1: Figure S1. Although not significant this trend is divergent to the other FVC outcomes and may be related to differences in definitions in the studies. For example in one trial [7] this was the proportion with a reduction >10% in FVC or death. It is also uncertain whether studies reported this 10% decline as an absolute or a relative decline. Three trials reported results as an annualised decline in FVC but the network for this outcome was not complete for comparison.

The FVC treatment effect versus placebo in one pirfenidone trial [7] appears to be large when compared with other trials. Two scenario analyses, first to exclude this trial from the FVC % predicted model, and second to adjust the numbers with an FVC decline greater than 10% to remove all cause deaths, did not substantively alter the outputs of the NMA.

Acute exacerbations

Evidence of acute exacerbations were reported in some of the included trials but there were some differences in rates reported. One trial [25] only reported the incidence per 100 years which we converted using the study sample size to rate per year, and two trials [8] reported the proportion of participants with at least one event (Additional file 1: Table S2). Acute exacerbations are relatively rare occurrences and studies were not powered to detect a difference in these rates, therefore results should be interpreted cautiously (for the OR for acute exacerbation versus placebo, see Additional file 1: Figure S2). Both nintedanib and pirfenidone have favourable point estimates, however only the OR for nintedanib achieves significance. Owing to a number of uncertainties with these data no indirect comparison was undertaken.

Mortality

All-cause mortality and respiratory-related mortality were reported in trials of all treatments with the exception of inhaled NAC (Additional file 1: Table S3). The NMA ORs for all-cause mortality compared with placebo showed that both nintedanib and pirfenidone had favourable point estimates, however, only pirfenidone was statistically significant (Additional file 1: Figure S3). In indirect comparison pirfenidone is associated with lower odds of all-cause mortality compared to nintedanib, but this is not significant (OR 1.39, 95% CrI 0.70, 2.82). Similar results were seen for respiratory mortality, see Additional file 1: Table S4 and Figure S4. Studies were not powered to detect a difference on mortality and therefore the results should be considered cautiously.

Other outcomes

The included trials reported other outcomes less consistently and are therefore not appropriate for meta-analysis. No studies that reported it showed significant effects of treatment on diffusing capacity of the lung [4-6,22,23], and effects on the six-minute walk test were mixed across the trials with only three showing a positive effect [4,7,22]. Few studies reported dyspnoea and results were not significant [4,22,23]. On the St George’s Respiratory Questionnaire, a measure of quality of life, there were mixed results but overall it would appear that the treatments have little effect on quality of life.

Discussion

This study presents a comprehensive review of evidence of the effectiveness of three treatments for IPF and is a timely addition to the ongoing appraisal of recently published trials by patients and clinicians. Evidence for FVC shows a significantly slower decline in patients treated with pirfenidone or nintedanib compared to placebo. When compared indirectly, there was a slower decline in FVC for those treated with nintedanib than for pirfenidone. On endpoints thought more important for patients, such as mortality and acute exacerbations, there were mixed results. Nintedanib was significantly better than placebo for acute exacerbations and pirfenidone was significantly better than placebo for mortality and respiratory mortality. Indirect comparisons of mortality outcomes were not statistically significant.

Some aspects of the evidence included in the review should be taken into account when interpreting the findings. The study is limited by inconsistent reporting of the key outcomes in the trial publications and definitions of outcomes differed. It has been questioned whether the participants in one RCT (ASCEND) [7] differ from other trials [9]. Although our assessment of heterogeneity suggests no statistically significant heterogeneity, the rate of decline in FVC in the placebo group appears to tail off near to the end of the trial. On observation of placebo arms of all other included trials this is at a greater rate, although the slope of decline seen in the more recently published trials are more similar. One possible reason for this reduction may be related to the effects of the imputation of FVC data for patients who died, however, in scenario analyses to test the consistency of results we would suggest that the difference between groups is similar to the other pirfenidone trials. Finally, four of the trials did not report ITT analysis, which is often recommended as the least biased way to estimate intervention effects in RCTs [27]. It is possible therefore that treatment effects may be exaggerated in these trials [5,6,8,24].

On FVC, within individual trials, results have not been entirely consistent in their comparisons with placebo. However, through NMA results show statistically significant effects for both of pirfenidone and nintedanib. Both treatments have received authorisation from the FDA and in Europe. It is anticipated that the findings of our indirect comparisons between nintedanib and pirfenidone will be of use to inform decision making in the clinical setting. This is particularly important given that until recently there has been an unmet need. Few trials reported estimates of quality of life and good quality research to establish the quality of life of patients with IPF should be a priority. This would allow a fuller assessment of the effectiveness of these treatments. Additionally, these treatments do not stabilize or reverse the decline in IPF and more research into treatments with the potential to meet this aim is required.

Our review and NMA has been undertaken following recognised principles for undertaking a systematic review to ensure that our analyses are transparent and as unbiased as possible. Additionally we ensured that only the highest quality studies were included to limit uncertainty in the results. Our group has no direct vested interest in the pivotal trials. Limitations to our study are that the NMA for FVC assumes the different measures of FVC are equivalent and we converted results to odds ratios to provide meaningful results. Although recognised methodologies were used, these factors should be considered when interpreting the results. Assumptions of homogeneity, similarity, and consistency between direct and indirect evidence in our NMA were made [28], however, uncertainties remain. For example, there could be other sources of observed or unobserved heterogeneity which could impact on relative treatment effects.

Conclusions

Beneficial treatment effects were demonstrated across key outcomes for two included interventions. In indirect comparison nintedanib was associated with significantly better outcome on slowing the decline in FVC than pirfenidone and this finding was robust in sensitivity analyses. Mortality rates showed trends in favour of pirfenidone but these were not statistically significant. Our findings can be used to help inform treatment decisions for this population of patients who following the recent regulator’s approvals will benefit from greater access to these therapies.