Introduction

Lung cancer is the most common cancer in the world with 1.8 million new cases diagnosed in 2012 [1]. Lung cancer was attributable to 5.4% of the total number of deaths in the European Union (2013), equating to more than a quarter of a million people (268,744 people) or 55.2 deaths per 100,000 inhabitants [2]. Overall, the prognosis for long-term survival with lung cancer is poor. Net survival for adults (aged 15 to 99) in England and Wales in 2010–2011 was 32.1% at 1 year, 9.5% at 5 years and 4.9% at 10 years [3]. The main cause of lung cancer is smoking, so primary prevention remains the priority [4].

However, prognosis is related to stage at diagnosis. The 1-year survival rate is over 80% when lung cancer is diagnosed in stage I, but under 20% when diagnosed in stage IV [5], this being due to the ability to treat surgically with curative intent at early stages. Also few lung cancers present in their early stages, 25% in the UK National Lung Cancer Audit annual report 2016 [6]. Together, these facts suggest that there may be an opportunity to use secondary prevention by screening to increase the number of cancers identified at an early stage.

Over several decades, a number of potential screening tests for lung cancer have been investigated including chest x-ray (CXR) and sputum cytology. Neither of these has been found to be effective [7]. As computed tomography (CT) has developed, offering improved images at lower radiation dosage, so low-dose CT (LDCT) has become the test offering the greatest potential for effective and cost-effective screening for lung cancer with much research devoted to investigating this [7].

The National Lung Screening Trial (NLST) has been particularly influential comparing LDCT screening with CXR [8]. 53,454 persons at high risk for lung cancer in 33 US medical centres were randomised from August 2002 to April 2004, 26,722 to three annual screenings of LDCT and 26,732 to single-view posteroanterior CXR. All participants were followed to 31/12/2009. Investigators concluded that “screening with LDCT reduces mortality from lung cancer”. The US Preventive Services Task Force agreed with this and in 2013 changed their negative recommendation for screening for lung cancer screening to a positive one [9].

In the UK, population lung cancer screening is currently not carried out by the NHS, based on the recommendation of the UK National Screening Committee in July 2006 when they last assessed whether lung cancer screening should be recommended for adult cigarette smokers. They concluded that it should not be recommended but should be reviewed in 2015/2016. The research we report was commissioned to update the systematic review which underpinned the last guidance [10, 11].

Methods

Our objective was to evaluate the clinical effectiveness of screening programmes for lung cancers with LDCT in high-risk populations using a systematic review, meta-analysis and network meta-analysis of RCTs. The wider project also considered cost-effectiveness. The systematic review was registered (PROSPERO CRD42016048530). All aspects of the work were undertaken in accordance with a pre-specified protocol [12] with some minor recorded exceptions. These involved an expansion of the range of outcomes we abstracted data on, searching some different websites to those originally specified and being more precise about what constituted poor study quality in the investigation of heterogeneity. The work was commissioned by the NIHR on behalf of the UK NHS to inform a future decision by the UK National Screening Committee on LDCT screening for lung cancer. The complete health technology assessment has recently been published [13].

We searched MEDLINE, MEDLINE In-Process, Embase, PsycINFO (all via Ovid), Web of Science (Thomson Reuters), CDSR and CENTRAL (via The Cochrane Library) and CINAHL (EBSCO) from 2004 to January 2017. (Additional file 1: Table S1. MEDLINE search strategy). Literature prior to 2004 was identified via the 2006 health technology assessment by the Aberdeen Health Technology Assessment Group [10] which this research project was commissioned to up-date. Other published and unpublished literature was identified from systematic searches of electronic sources, citation chasing, consultation with experts in the field and reference checking of relevant systematic reviews.

In the main systematic review and meta-analysis, we included LDCT lung cancer screening programme RCTs involving populations at high risk of lung cancer. Any definition of high risk was eligible. LDCT screening programmes included both single and multiple rounds. The eligible comparators were no screening or other imaging technology screening programmes (such as CXR). RCTs evaluating the effectiveness of CXR but not LDCT were also included in the network meta-analysis. The outcomes of interest for this analysis were lung cancer mortality and all-cause mortality.

Two researchers independently screened the titles and abstracts of all reports identified by the search strategy. Full-text papers were subsequently obtained and screened in the same way. Data extraction and quality assessment were undertaken by one researcher and checked by a second. The risk of bias of included studies was assessed using the Cochrane Risk of Bias tool [14]. We also considered underpowered sample size for important outcomes and significant baseline differences between study arms on important characteristics.

All data were tabulated and primarily considered in a narrative review. DerSimonian and Laird random-effects models meta-analyses were performed to pool the estimates of effect [15]. We restricted the meta-analysis to RCTs with at least 5 years follow-up consistent with the primary outcome in NLST. A random-effects approach was pre-specified as part of the protocol development process; a fixed approach was not favoured as it was thought highly unlikely that chance alone would account for differences between the results of included studies. Statistical heterogeneity was assessed using the I2 statistic. Based on the advice in the Cochrane handbook, 30% to 50% was categorised as moderate heterogeneity; 50% upwards as substantial heterogeneity [16]. We considered the following factors for the exploration of heterogeneity: quality of trials (particularly adequacy of randomisation), nature of interventions (e.g. frequency of LDCT screening), and nature of control groups (e.g. best available care such as CXR screening or usual care).

Network meta-analysis was performed to assess the relative effectiveness of three screening strategies (LDCT, usual care and CXR). A multivariate random-effects meta-analysis using restricted maximum likelihood approach was performed [17]. We estimated the relative ranking probability of each intervention and obtained the treatment hierarchy of competing interventions using rankogram, surface under the cumulative ranking curve and mean ranks [18]. The probability was estimated using a Bayesian model with flat priors, under the assumption that the posterior distribution of the parameter estimates was approximated by a normal distribution with mean and variance equal to the frequentist estimates and variance–covariance matrix [17]. In order to assess the presence of inconsistency, both consistency and inconsistency models were fit for data. We used the design-by-treatment model to check the assumption of consistency in the entire network [19]. This provides a robust approach to assess the consistency of the network being constructed.

Statistical analyses were performed using the ‘metan’ ‘mvmeta’ and ‘network’ commands in Stata 14 (StataCorp LLC, Texas) [20, 21].

Results

From 9655 records identified in the searches, four RCTs were included in the meta-analysis of mortality data [8, 22,23,24,25] and a further two RCTs in the network meta-analysis [26,27,28] (Fig. 1). One further RCT was used in a sensitivity analysis of the network meta-analysis [29].

Fig. 1
figure 1

PRISMA diagram

Full size image

The characteristics of the included studies are shown in Table 1. Concerning the LDCT trials, most were conducted in Europe and the USA. There was variation between the LDCT programmes, but typically they involved 3–5 rounds of screening over 3 to 6.5 years. The nature of high-risk participants also varied but was usually defined in terms of age and current and past smoking history. Of the trials NLST stands apart, not just in terms of size with over 50,000 participants, but by LDCT being compared to CXR screening rather than no screening [8].

Table 1 Characteristics of included studies
Full size table

Looking at the 12 included RCTs in the qualitative review (Additional file 1: Table S2) reveals that although only four RCTs currently contribute mortality data to the direct meta-analysis, there are others [ITALUNG [30], German Lung Cancer Screening Intervention trial (LUSI) [31], NEderlands Leuvens Longkanker Screenings ONderzoek (NELSON) [32, 33]] which started between 2001 to 2010 and will be maturing in the near future. In addition, the UK Lung Cancer Screening trial (UKLS) [34] has indicated that it will incorporate its mortality data with the NELSON study.

The two additional trials [26,27,28] for the network meta-analysis compared intensive screening with CXR and sputum cytology over 3 to 6 years with usual care involving occasional CXR examination. The frequency of screening in the intervention arms was much more frequent than the LDCT RCTs, with CXR examinations two or three times a year. The RCTs were done in the Czech Republic [26, 27] and the USA [28] in the 1970s and consequently benefit from long follow up. A third RCT of CXR screening conducted in the USA in the 1990s, Prostate, Lung, Colorectal and Ovarian cancer screening trial (PLCO) [29], could not be included because the majority of subjects were low risk. We did however include a post-hoc high-risk sub-group analysis of this trial in a sensitivity analysis as this subgroup (NLST-eligible subgroup involving high-risk participants) was relevant to our research question. It compared four annual rounds of CXR screening with no screening.

The majority of the LDCT included trials were judged to be of moderate to high quality, although allocation concealment was consistently poorly addressed. One RCT, Multicentric Italian Lung Detection project (MILD) [24], was however judged to be of much poorer quality with a particularly marked risk of bias arising from lack of clarity about randomisation, accompanied by marked imbalances in some of the baseline characteristics (Additional file 1: Table S3).

The additional RCTs for the network meta-analysis were of slightly poorer methodological quality than most of the LDCT RCTs, with greater lack of clarity about loss to follow-up and absence of power calculations. A mitigating factor may be that standards for reporting RCTs were not well established in the 1970s when the studies were conducted with the first Consolidated Statement of Reporting of Trials version being published in 1996 [35]. The PLCO main trial [29] was of similar quality to the LDCT RCTs, but the NLST sub-group study admitted very limited power to detect small differences in mortality and was only able to demonstrate baseline equivalence for a small number of characteristics (Table 2).

Table 2 Quality assessment of included studies
Full size table

The direct meta-analysis showed that LDCT screening was associated with a statistically non-significant decrease in lung cancer mortality (pooled RR 0.94, 95% CI 0.74 to 1.19; p = 0.62) with up to 9.80 years of follow-up when compared with controls (Fig. 2). A moderate level of heterogeneity was observed in the magnitude of effects (I2 = 43.3%), given which the results should be treated with caution. A range of potential sources for heterogeneity were investigated. When removing the poor quality trial (MILD) [24], sensitivity analysis demonstrated a statistically significant decrease in lung cancer mortality (pooled RR 0.85, 95% CI 0.74 to 0.98; p = 0.02) in favour of LDCT screening compared with controls (Additional file 1: Figure S1). A considerable reduction in heterogeneity was observed (I2 = 6.9%).

Fig. 2
figure 2

Lung cancer mortality—overall results

Full size image

The direct meta-analysis also showed that, compared with controls, LDCT screening demonstrated a statistically non-significant increase in all-cause mortality outcome (pooled RR 1.01, 95% CI 0.87 to 1.16; p = 0.95) with up to 9.80 years of follow-up (Fig. 3). Likewise, given the substantial heterogeneity (I2 = 57.0%) detected between studies, the results from this pooled analysis should be treated with caution. We also investigated the potential sources of heterogeneity. When removing the low-quality trial (MILD) [24], sensitivity analysis showed that LDCT screening demonstrated a borderline statistically non-significant decrease in all-cause mortality (pooled RR 0.95, 95% CI 0.89 to 1.00; p = 0.05) compared with controls (Additional file 1: Figure S2). The level of heterogeneity was also considerably reduced (I2 = 0%), suggesting that variation in trial quality could be a potential source of heterogeneity between studies.

Fig. 3
figure 3

All-cause mortality—overall results

Full size image

Network meta-analysis assessed the relative effectiveness of LDCT, usual care and CXR screening. The main network consisted of three RCTs comparing LDCT with usual care [22,23,24]; one trial comparing LDCT with CXR [8, 25]; and two trials comparing CXR with usual care [26,27,28]. A further RCT of CXR vs usual care was included in a sensitivity analysis [29]. The estimated relative risk of lung cancer mortality of LDCT compared with usual care was 0.95 (95% CI 0.82 to 1.11) (Additional file 1: Table S5). LDCT was ranked first according to the estimated surface under the cumulative ranking curve values, with a 74.8% probability of being the best intervention in terms of lung cancer mortality reduction. Usual care had a 74.7% probability of being the second best strategy among the three interventions. However, CXR screening had a 99.7% probability of being the worst intervention in terms of lung cancer mortality reduction. Both consistency and inconsistency models were fit for lung cancer mortality data. By applying the design-by-treatment model, we did not find any evidence of inconsistency. The global test for inconsistency gave a p value of 0.29, indicating no evidence of inconsistency.

In the sensitivity analysis, the estimated relative risk of lung cancer mortality comparing LDCT with usual care was 0.93 (95% CI 0.76 to 1.14) (Additional file 1: Table S6). Based on the estimated surface under the cumulative ranking curve values, LDCT screening was ranked first—it had a 75.3% probability of being the best intervention in terms of lung cancer mortality reduction. Usual care had a 68.3% probability of being the second best strategy among the three interventions. Similarly, CXR screening had an 87.7% probability of being the worst intervention in terms of the lung cancer mortality outcome. Again there was no evidence of inconsistency (p = 0.18) (Fig. 4).

Fig. 4
figure 4

Network meta-analysis rankogram

Full size image

Discussion

The main findings of the systematic review and meta-analysis of RCTs comparing LDCT screening programmes with usual care (no screening) or other imaging screening programme (such as chest X-ray (CXR)) are a statistically non-significant decrease in lung cancer mortality (pooled RR 0.94, 95% CI 0.74 to 1.19) and a statistically non-significant increase in all-cause mortality outcome (pooled RR 1.01, 95% CI 0.87 to 1.16). The network meta-analysis agrees with the direct meta-analysis that LDCT is the most effective screening option between LDCT, CXR and usual care, but also raises the possibility that CXR screening may be less effective than usual care. This in turn has implications for the interpretation of the largest RCT which compares LDCT with CXR rather than usual care, the most relevant comparator in trying to decide whether LDCT screening should be introduced.

There is thus considerable uncertainty about the effect of LDCT screening on mortality, particularly the size of the effect. Contributors to this uncertainty beyond the relevance of the main included RCT, are the width of the 95% CI (which include no effect) and the statistical heterogeneity between the included study results. Further, although a RR of 0.94 looks like a useful effect, the rarity of the outcomes in the trials needs to be taken into account (4.7 lung cancer deaths per 100 persons over an 8-year period was found in the Detection And screening of early lung cancer with Novel imaging Technology and molecular Essays (DANTE) RCT [22], which identified the highest lung cancer risk of death in the RCTs contributing data on mortality). This translates to a number needed to screen to avoid one lung cancer death of 357 (95% CI 82 to − 113 [negative value indicates screening increases lung cancer deaths; the confidence interval includes infinity, equivalent to a risk reduction of 0, so the point estimate is encompassed by the interval although apparently lying outside it]) emphasising the considerable number of participants who need to be screened multiply over a period of at least five years to achieve one less lung cancer death even in high-risk populations. The uncertainty is confirmed if the evidence is GRADED, with downgrading for imprecision, inconsistency and indirectness.

The research we report was undertaken by an experienced health technology assessment group, working to a pre-specified protocol, informed by comments from a steering group with wide expertise. The technology assessment as a whole was informed by a patient and public involvement exercise. No members of the research team had any connection with the research teams doing any of the included RCTs. Although likely to increase in the foreseeable future, our main limitation was the small number of included studies. This limited our ability to investigate the heterogeneity between study results and to explore other important phenomena like publication bias. We also did not have opportunity to systematically contact each of the original research teams which may have helped fill some of the gaps in details about the RCTs, particularly randomisation methods. We were conscious that selected enquiry of particular studies might in itself introduce bias.

Our findings are not consistent with recent guidelines by the US Preventive Services Task Force [9] nor with recent expert comment in the UK [37] both of which are strongly supportive of the need to implement LDCT screening for lung cancer. They emphasise the results of the NLST trial pointing to its size and apparent conclusive results. However, we are not alone in pointing to the need to consider the whole evidence base [7, 38]. We have not ignored trials which in normal circumstances would be considered sufficiently important in terms of size and quality to be taken into account in deciding policy. Method of analysis does not appear to be the issue as our pooled estimate of effect on lung cancer mortality just including DANTE, Danish Lung Cancer Screening Trial (DLST) and NLST (excluding MILD) (RR 0.85, 95% CI 0.74 to 0.98) is very similar to the pooled estimate relied on by the US Preventative Task Force, RR 0.81 (95% CI 0.72 to 0.91) [9]. A particular contribution of this study has been to consider the implications of NLST using CXR screening as the comparator. The NLST authors argued qualitatively that CXR screening has no effect using the PLCO NLST high-risk sub-group results and so that NLST gives a valid estimate of LDCT screening versus no screening. We used a formal network meta-analysis, which suggests that CXR screening may possibly be ineffective, so adding a further note of caution about whether NLST may be overestimating the effect of LDCT screening for lung cancer.

Conclusions

On balance, the evidence does not yet clearly support the case for population LDCT screening and any final decision should await the results of trials in progress. Our recommendation is that the evidence should be re-reviewed when the trials in progress are reported. This should include additional detailed investigation about study quality of all RCTs as we have provisionally identified this as a potential explanatory factor of the heterogeneity between the four RCTs published so far. Up-to-date reviews of other outcomes and cost-effectiveness modelling will also be important aspects of further research. These have already been conducted as part of this project and are reported in a recent publication [13].