Background

Idiopathic pulmonary fibrosis (IPF) is a debilitating respiratory condition for which there is no cure. IPF is characterised by aberrant wound healing in which excessive (and perhaps abnormal) extracellular matrix is deposited in the lung thereby distorting the architecture and disrupting function. This lung injury and scarring eventually leads to a decline in lung function which culminates in respiratory failure [1]. Shortness of breath on exercise and a chronic dry cough are the prominent symptoms [2]. IPF is known to affect males more than females and in particular affects people over 60 years of age. The prevalence of IPF is increasing, although the cause of this increase is uncertain, [3, 4] and with a poor prognosis (estimated mean survival of between 2–5 years) IPF has become an area of focus with recent UK national guidelines on the diagnosis and management of IPF published, [5] and international consensus guidelines [6] due to be updated in 2014.

IPF is a difficult condition to manage, particularly in the latter stages. Early and accurate diagnosis is important to maximise the potential for a better outcome but there is an unmet need with few recommended treatments [5]. In the UK all patients should be given best supportive care (BSC) from the point of diagnosis, which includes information and support, symptom relief, management of co-morbidities, withdrawal of ineffective therapies and end-of-life care. In addition individuals should be assessed for pulmonary rehabilitation if appropriate and have a clinical nurse specialist available to them [5]. For those without contraindications lung transplantation should be considered as this is the only treatment shown to improve survival [7]. However, with donor organs being in short supply there is a need for alternative treatments that aim to modify the disease and prolong survival. Few treatments are available to the clinician and patient currently, and the evidence for the effectiveness of such treatments is unclear. We aimed to evaluate the current state of the clinical and cost-effectiveness of treatments for people with IPF through three systematic reviews, a network meta-analysis (NMA) and economic modelling.

Methods

The a priori methods for this evidence synthesis are described in the research protocol which is registered with PROSPERO (reference: 42012002116). Search strategies were developed and applied to 11 electronic bibliographic databases (including the Cochrane library, MEDLINE and EMBASE) from inception to July 2013 with no language restrictions. Bibliographies of retrieved papers were screened and experts contacted to identify additional studies. Systematic reviews were undertaken of clinical effectiveness (including only randomised controlled trials [RCTs] and controlled clinical trials [CCTs]), economic evaluations and health related quality of life (HRQoL) studies. Eligible participants were those with a diagnosis of IPF and includable interventions were as deemed relevant by a clinical and patient advisory group. Best supportive care, placebo or any of the interventions were eligible as comparators and outcomes of relevance included measures of survival, measures of symptoms (breathlessness, cough), HRQoL, lung function, exercise performance, adverse events and measures of costs and cost-effectiveness. Studies reporting HRQoL in people with IPF were eligible for inclusion if they used either generic preference-based measures or the St Georges Respiratory Questionnaire (SGRQ) which is a disease specific instrument used in IPF. Other disease specific instruments were not eligible for inclusion as there are currently no methods to map results of these to utility measures required for economic evaluation.

Titles and abstracts were screened for potential eligibility by two reviewers using a pre-defined inclusion criteria, retrieved articles were assessed for eligibility, data were extracted and methodological quality assessed by one reviewer and checked by a second. Study quality was assessed using recognised methods [8–10]. For the review of clinical effectiveness we developed a check-list to assess the methodological quality of the studies based on the criteria recommended by the Centre for Reviews and Dissemination, [8] (Quality assessment/risk of bias section) and summarised the risk of bias (as per Cochrane collaboration recommendations [11]) within each study according to the risk of selection bias. We developed a check-list to assess the methodological quality of the cost effectiveness studies based on the check-list of Drummond and colleagues [9] and recommendations by Phillips and colleagues [10]. Data items extracted included study details (design, follow-up, funding), participant details (numbers, eligibility, characteristics), intervention details (including dose and duration of treatment), outcomes reported and results. Narrative syntheses were undertaken and in the review of clinical effectiveness meta-analysis was performed where appropriate with heterogeneity assessed. FVC was measured on two continuous scales and these were meta-analysed using the standardised mean difference (SMD). A NMA focusing on pharmacological treatments for IPF and assessing forced vital capacity (FVC) endpoints was undertaken [12].The NMA focused on FVC as it is correlated with disease progression [6] and was therefore relevant to the economic model. For FVC endpoints the NMA used the SMD in a Bayesian framework using code adapted from published sources [13]. Vague normal priors were used for the treatment effects and a vague uniform prior for the random effect standard deviation. Model code is provided in the Additional file 1. Fixed and random effects models were applied with best model fit determined using the deviance information criterion (DIC). The SMDs output from the NMA were then converted to log odds ratios using standard methods for ease of interpretation within the context of an NMA [14].

Quality assessment/risk of bias

  1. 1.

    Was the method used to generate random allocations adequate?

  2. 2.

    Was the allocation adequately concealed?

  3. 3.

    Were the groups similar at the outset of the study in terms of prognostic factors

  4. 4.

    Was the care provider blinded?

  5. 5.

    Was the patient blinded?

  6. 6.

    Were outcome assessors blinded to the treatment allocation?

  7. 7.

    i) Were there any unexpected imbalances in drop-outs between groups? ii) If so, were they explained or adjusted for?

  8. 8.

    Is there any evidence to suggest that the authors measured more outcomes than they reported?

  9. 9.

    i) Did the analysis include an intention to treat analysis? ii) If so, was this defined?

  10. 10.

    i) Did the analysis account for missing data? ii) If so, were the methods appropriate?

A decision-analytic model was developed to compare the cost-effectiveness of pharmacological interventions in patients with initially unprogressed IPF. The model perspective is that of the UK National Health Service and Personal Social Services. The model structure was informed by the available literature and expert opinion on the clinical progression of the disease. It uses four distinct health states: unprogressed IPF; progressed IPF; lung-transplant; and dead. Health states except death are associated with a HRQoL utility and a cost estimate. Progression is defined by an absolute decline in FVC per cent predicted of ≥10% from a baseline (recently-diagnosed) value, based on the included RCTs. Published sources were used to inform the probability of a lung transplant; survival after lung transplant; and all-cause mortality by age. Acute exacerbations are not modelled as separate health states but are associated with a cost and utility decrement. Model cycle length is one month and a lifetime horizon of 30 years was adopted to capture all clinically and economically important events. A half-cycle correction is applied. Key assumptions are that all patients are in in the unprogressed state initially; those experiencing a ≥10% absolute decline in FVC% predicted are considered to be in the progressed health state; and treatment has a constant effect on relative rate of FVC% decline, FVC% predicted was used as a proxy for disease severity when assigning utilities to the health states. In addition, where treatment costs for any individual treatment were not available an assumed cost was used and tested in threshold analysis.

Treatment effects were obtained from the NMA. Utility values from the systematic review of HRQoL are applied to the modelled health states to estimate the benefits measured as quality adjusted life years (QALYs). Costs are included for treatments, treatment monitoring, acute exacerbations, lung transplant and adverse events, based on the UK health system. Future costs and benefits are discounted at 3.5% per annum. The outcome is reported as cost per QALY gained against the next best alternative treatment using incremental cost effectiveness ratios (ICERs). The model examines uncertainty in deterministic and probabilistic sensitivity analyses. Model validation was undertaken by checking structure, calculations and data inputs. In addition the advisory group reviewed the structure and internal consistency was examined by varying input values. Model results were compared with trial outputs and other publications [15–17].

Results

Clinical effectiveness

Searches identified 905 unique references and 64 of these were retrieved after screening of titles and abstracts. Fourteen studies (13 RCTs and 1 CCT) were included (Figure 1).Four RCTs evaluated the use of pirfenidone, [15, 18, 19] three the use of n-acetylcysteine (alone or in combination), [20–22] one azathioprine, [23] one nintedanib, [16] one sildenafil, [24] one thalidomide, [25] one a pulmonary rehabilitation programme, [26] and one a disease management programme [27]. In addition one CCT of pulmonary rehabilitation was included [28]. This study was published in Polish and translation of key methods and results only were undertaken due to resource and time constraints. Therefore caution is recommended in interpreting our assessment of this study. No studies of palliative care interventions were identified that met the inclusion criteria. Study quality was generally good with a low risk of bias. Ten studies were undertaken in populations that would likely be classed as mild to moderate IPF [29]. The majority of these studies had reasonable sample sizes and duration of follow was between nine months and 16 months. Four studies were undertaken in populations that would be classed as moderate to severe IPF. Three of these were the non-pharmacological intervention studies, and one the drug sildenafil. Sample sizes were generally smaller in the non-pharmacological studies, and there was no long-term follow up. Across all studies the mean ages of participants ranged from approximately 54–69 years, the gender ratio of males to females was generally 3:1, and the duration of diagnosis tended to be between 1 and 3 years. In the ten studies in mild-to-moderate IPF the baseline FVC ranged between 65-90% and in the four studies in moderate-to-severe IPF this ranged from 55-70%. The populations were deemed to be reasonably similar to those seen in clinical practice by our advisory group. For further summary details see Table 1.

Figure 1
figure 1

Flow chart for the identification of studies in the clinical effectiveness review. a Two RCTs were reported in one publication; two further trials each had two linked publications. b One trial was published as a protocol only.

Full size image
Table 1 Study characteristics of included interventions
Full size table

Results for the clinical effectiveness of the five pharmacological interventions in patients with mild to moderate IPF were mixed. In clinical practice azathioprine is only used in restricted circumstances, in one RCT [23] azathioprine and prednisolone led to an improvement in survival compared with placebo and prednisolone when an age adjusted analysis was used. There was no effect on lung function. This trial had an unclear risk of bias, a small sample size, and it is uncertain whether all patients had a diagnosis of IPF based on current recommendations. Consequently, caution is recommended when interpreting results. Nintedanib 300 mg/day was more favourable than placebo on some FVC measures, acute exacerbations and mortality, however, the primary outcome of annual rate of decline in FVC was not statistically significant [16]. Treatment with NAC was evaluated in three studies, [20–22] in combination with azathioprine and prednisolone in two (triple therapy) and in an inhaled format in one. Study results were mixed and establishing the stand-alone effect of NAC is difficult due to the differences between the three studies. There was no benefit from triple therapy on FVC compared to placebo in one trial, however, there was a benefit on vital capacity (VC) when compared to azathioprine and prednisolone in another. The treatment effect of inhaled NAC was not statistically different from that of a control group (p = 0.05). The study using inhaled NAC had an unclear risk of bias which should be considered when interpreting results. Four RCTs [15, 18, 19] evaluated the use of pirfenidone and meta-analysis shows that pirfenidone appears to demonstrate a significant effect on FVC when compared to placebo (SMD 0.24, 95% CI 0.06, 0.41, p = 0.008). This should be cautiously interpreted as the outcomes pooled were different, the timing of assessment varied (from 48 weeks to 72 weeks) and there was moderate statistical heterogeneity (I2 = 45%). The effect of pirfenidone on secondary outcomes was more uncertain. Thalidomide was assessed in those with cough in a small crossover RCT [25]. HRQoL outcomes related to cough were improved with thalidomide compared to placebo. There is no evidence relating to any subgroups in any of the published studies.

One study [24] assessed sildenafil for those with moderate to severe IPF. No statistically significant benefit of sildenafil was seen on the primary outcome, a 20% improvement on the six minute walk test. Results on secondary outcomes were mixed with some favourable to sildenafil but others being not statistically significant.

Adverse events were generally mild to moderate and reasonably well balanced between the treatments and controls with the exception of thalidomide which led to a greater proportion of people experiencing at least one adverse event (77%) than the placebo treated participants (22%) [25]. Severe adverse events appeared to be more common in one study in those treated with triple therapy [21].

Ten studies of pharmacological interventions were included in NMA; the resulting evidence network is shown in Figure 2. Thalidomide was excluded as the focus of treatment is not on lung function. Inhaled NAC was considered separately from triple therapy owing to its different method of administration. Both direct and indirect evidence was used to assess the treatment effect compared to placebo. Only the fixed effect results for nintedanib and pirfenidone were statistically significant, odds ratios for reducing the rate of decline in FVC compared to placebo are shown in (Table 2). The random effects model did not demonstrate a better fit than the fixed effect model and there was no evidence of inconsistency within the evidence network. A head-to-head comparison of nintedanib versus pirfenidone suggested a trend favouring nintedanib, but this was not statistically significant and should be cautiously interpreted in the light of the various differences between the studies (Table 2). Further trial evidence could be used to test this further.

Figure 2
figure 2

Evidence network for FVC endpoint. Legend: The width of the lines connecting treatment pairs is proportional to the number of participants within each trial comparison.

Full size image
Table 2 NMA Fixed effects results, SMDs converted to log odds ratios for slowing the decline in FVC
Full size table

Three non-pharmacological intervention studies compared a pulmonary rehabilitation programme or disease management programme to control interventions in moderate-to-severe IPF. Results from the two pulmonary rehabilitation studies were inconclusive; some outcomes were favourable to pulmonary rehabilitation but not others. The risk of bias was uncertain and endpoints were only assessed immediately after the interventions finished. Limited evidence of the effectiveness of a disease management programme was demonstrated. This study had an uncertain risk of bias and follow-up was immediate.

Cost effectiveness and HRQoL systematic reviews

One economic evaluation was identified, which examined a testing strategy prior to treatment with triple therapy but did not examine the cost-effectiveness of treatment. The systematic review of HRQoL included 23 studies that examined HRQoL using either a generic preference-based tool (EuroQol five dimensions [EQ-5D], Short Form-36 [SF-36]) or a disease specific instrument (SGRQ) that could be mapped to utility for the economic model. Results varied between the studies, given the different populations under study and the different measures and time points of measurement, however, results generally showed that IPF has an adverse effect on HRQoL which increases with severity.

Cost-effectiveness of pharmacological treatments for IPF

The baseline risk of disease progression in the unprogressed state was taken from two of the pirfenidone trials included in the systematic review of clinical effectiveness [15] as the population most closely met the model definition of unprogressed IPF and the RCTs had large sample sizes. The probabilities of progression free survival were obtained from the Kaplan Meier survival curve published for the pooled placebo population of two RCTs [15]. Parametric survival curves were fitted to this curve in Stata using maximum likelihood estimation in order to extrapolate beyond the 72 weeks follow-up. Exponential, Weibull, loglogistic, lognormal and Gompertz parametric models were examined. Goodness of fit was assessed using the Akaike Information Criterion (AIC). The Weibull model was selected because of the balance of good fit (AIC) and face validity (comparison of predicted survival with known survival in IPF). In the progressed state the monthly probability of death was taken from the overall survival curve in a recent study reporting survival for those experiencing a 10% or greater decline in FVC [30]. This study was an observational study but followed a larger sample than other published studies that differentiate survival by FVC decline and was deemed the most appropriate for the evaluation. Five parametric survival curves were fitted using the distributions noted above. The exponential distribution was selected (using AIC and face validity) to extrapolate beyond the five years of observed data.Probabilities of acute exacerbations in both unprogressed and progressed health states, and probabilities of lung transplant and survival from lung transplant, were taken from published sources.

HRQoL utility values were applied to each of the alive health states. These were not differentiated by treatment; the impact of treatment on utility was assumed to occur because of delay to disease progression which the model accounts for. The HRQoL values used for the unprogressed and progressed IPF health states were taken from EQ-5D values reported in two trials included in the systematic reviews of clinical effectiveness and HRQoL [21, 24]. These trials were conducted by the same clinical network and it is likely that the estimates of EQ-5D are consistent and can therefore be contrasted in our economic model. The utility values applied in the model can be seen in Table 3. Utility associated with lung transplant is taken from a UK-based study [31] which assessed HRQoL using the EQ-5D. We weighted the utility to account for a greater proportion of single lung transplants in the IPF population than the proportion seen in the study, based on clinical opinion. Utility data for acute exacerbations were not identified in the literature and therefore a utility decrement was applied that was in line with decrements seen in those with asthma and chronic obstructive pulmonary disease (Table 3). Sensitivity analyses were used to test these utility decrements.

Table 3 EQ-5D utility values by model health state
Full size table

Five types of cost are considered in the economic model. The costs associated with each treatment were made up of the costs of the drug and the monitoring costs associated with the treatment. Dose information and unit costs were taken from published sources where available [32, 33]. Unit costs and data sources were unavailable for nintedanib and an assumed cost was used which was also subject to full sensitivity analysis. Hospital admission costs arising from acute exacerbations were estimated from NHS reference costs for all treatments as no treatment-specific acute exacerbation costs were available (£1361.04) [34]. Ongoing non-pharmacological treatment costs for management of the condition were included and covered annual home oxygen costs (£824.30) and the costs of long-term oxygen monitoring (£173.94) for the progressed IPF state. The costs associated with lung transplant were calculated using NHS reference costs [34] excluding outpatient procedures (£35,468.61). Costs of adverse events were attributed to the pharmacological interventions. Based on the incidence of serious adverse events seen in the trials included in the systematic review of clinical effectiveness, costs were applied for each event per patient in the first cycle of the model [34].

The model base-case results show increased survival for five of the treatments compared with BSC, at increased cost (see Table 4). The combination of azathioprine and prednisolone is dominated by BSC (treatment is more costly and less effective than BSC). Triple therapy is associated with an ICER of £41,811 per QALY gained when compared to BSC. Inhaled NAC is associated with an ICER of £5,037 per QALY gained when compared to BSC. Sildenafil, pirfenidone and nintedanib are not cost-effective at a willingness to pay (WTP) threshold of £30,000/QALY when compared to BSC. Therefore only one treatment, inhaled NAC, is cost-effective at a £30,000 WTP threshold, but its treatment effect is not statistically significant in the RCT, a small study with undetermined risk of bias.

Table 4 Summary of base case results
Full size table

Deterministic and probabilistic sensitivity analyses tested uncertainty in model parameter values, including costs and probabilities of acute exacerbation and lung transplant. Treatment effects and utilities were also varied in sensitivity analyses. The parameters were varied between the 2.5th and 97.5th percentiles of the mean value and analyses found that results were generally robust but were particularly sensitive to changes in the value of the treatment effect parameters.

The monthly cost of nintedanib was assumed (£3,274). Results demonstrate that, given a WTP of £30,000 per QALY, nintedanib must cost less than £736 per month to be considered as the cost-effective treatment option compared to BSC and pirfenidone.

Discussion

We systematically reviewed evidence for the clinical effectiveness of six pharmacological interventions and two non-pharmacological interventions for IPF. Participants in most of these studies would likely be classed as having mild-to-moderate IPF and generally were similar to those seen in clinical practice. There was a range of treatments under investigation in these trials with only one treatment, pirfenidone, providing evidence from more than one trial that was suitable for a formal meta-analysis to be undertaken. The outcomes reported in these studies differed however, and therefore caution is required when considering the results of the meta-analysis and the narrative synthesis of each of the included studies. In a network meta-analysis of the pharmacological treatments only pirfenidone and nintedanib had a statistically significant treatment effect, reducing the rate of decline in FVC compared to placebo. One pharmacological treatment was excluded from the network meta-analysis as the focus of treatment was not on lung function but on the symptom cough. In this study thalidomide appeared to improve cough, and quality of life compared with a placebo treatment. Results of three studies investigating two non-pharmacological treatments show mixed results and it is therefore unclear how effective these interventions are. There are differences between the studies in terms of the interventions, the participants, and the outcomes reported together with study design issues (e.g. short follow-up) and uncertain risk of bias that may account for some of the results seen.

Evidence from systematic reviews of cost effectiveness and HRQoL identified one economic evaluation of limited relevance and 23 HRQoL studies. These latter studies varied in their populations and study methods but generally showed that IPF has an adverse effect on HRQoL compared to population norms, and that HRQoL diminishes as IPF progresses. A new decision analytic health economic model was developed to assess the cost-effectiveness of six pharmacological treatments for IPF. Results show increased survival for five of the treatments compared with best supportive care, at increased cost. Only one treatment, inhaled NAC, is cost-effective at a WTP threshold of £30,000 but no statistically significant treatment effect was seen in the RCT or our NMA. Probabilistic sensitivity analysis showed that inhaled NAC has a 65% probability of being cost-effective if a decision threshold of £20,000 per QALY gained is used. Although pirfenidone and nintedanib achieve a statistically significant treatment effect in NMA they each have a probability of 0% of being cost-effective at a threshold of £30,000 per QALY. This is based on an assumed cost for nintedanib. A sensitivity analysis indicated that nintedanib must cost less than £736 per month to be considered cost-effective.

The past few years have seen an increasing interest in the management of IPF, with pharmacological companies evaluating a range of potential interventions, and a number of influential bodies producing guidelines. However, this systematic review demonstrates that at present there are few treatments which have any effect on surrogate outcomes which can be linked through evidence to patient related outcomes such as mortality. The findings of our research also suggest that under current willingness to pay thresholds only one treatment is likely to be cost effective, although, general recommendations cannot be made due to limitations in the evidence base. In terms of a cure it is considered that lung transplantation is the only intervention available which has curative intent. However, no evidence on lung transplant was eligible for inclusion in this evidence synthesis and so this could not be evaluated formally. There is also a scarcity of studies on interventions for symptom management and palliative care in IPF despite this being a recommended approach in recent clinical guidance [5].

No previous systematic reviews have included all potentially relevant treatments for IPF, and there has only been limited economic evaluation previously. Our results are useful to clinicians and patients, and complement recent national guidance by NICE in the UK [5]. In addition to standard synthesis we undertook a network meta-analysis to compare the pharmacological therapies to a common comparator. Our results show that only two treatments (nintedanib and pirfenidone) significantly slow the decline in FVC compared to placebo under a fixed effect model. However, with few studies it was not possible to fully explore heterogeneity within these data and the results should be cautiously interpreted. We also undertook an illustrative analysis comparing nintedanib with pirfenidone through an indirect comparison. While this showed a trend favouring nintedanib, it was not statistically significant and should be interpreted with caution until such time that a more complete analysis can be undertaken on more robust data. We identified a number of ongoing trials of potential relevance. Our evidence synthesis has highlighted the current evidence base in which these new trials can be contextualised once they report.

There were numerous differences between the studies included in this review. However we applied a rigorous approach to the inclusion, quality assessment and data synthesis of the studies (laid out in a research protocol), to ensure that our work was as unbiased as possible. Our research was guided by an advisory group from its initiation, in particular to ensure that the interventions included were appropriate to current or future management in the NHS. We ensured that only the highest quality studies were included to limit uncertainty in the results.

Our study has several limitations. The meta-analysis and NMA used the standardised mean difference to express findings from studies on a common scale. In this case we combined mean change in FVC% predicted with absolute change in FVC, albeit the former is adjusted for certain baseline characteristics, and this should be considered when interpreting the results. Many of the included studies compared treatments to placebo. Few direct comparisons were identified and results of an indirect comparison via the NMA approach are presented. However there are known limitations to the use of indirect comparisons which should also be considered in interpreting our findings [35]. The economic model assumes that treatments have a constant effect on the relative rate of FVC% decline compared to BSC, but treatment may in fact become less effective with time or as the condition progresses. (This would make the treatments less cost-effective than shown in our results.) Finally, because of limitations in the data the absolute decline in FVC% predicted was used as the measure of disease progression in the model. It is possible that use of this measure may introduce bias because the starting FVCs of patients (which might vary widely) is not taken into account.

Conclusions

This evidence synthesis reports on the effectiveness of a range of interventions for IPF and complements recent UK guidance [5]. The current evidence suggests that there are currently few treatments which are clinically and cost-effective. Pirfenidone and nintedanib offer the potential for hope to sufferers and their clinicians, however, their cost-effectiveness is likely to be prohibitive. This research has thoroughly examined the current evidence and can be seen as a platform from which the clinical importance of newer treatments can be assessed when ongoing trials report. The systematic review has highlighted the need for further research into interventions to help alleviate or control symptoms of this debilitating condition, in particular pulmonary rehabilitation programmes and thalidomide. Given the results of our study and the weaknesses of the inhaled NAC trial, a well-designed RCT of inhaled NAC should also be considered; our search of ongoing RCTs failed to identify any such studies currently underway.