Background

Lung cancer is a multifactorial disease with high incidence and mortality [1, 2]. Environmental exposures including tobacco smoking [3,4,5,6], occupational exposures [7, 8], and air pollution [9,10,11], as well as heritable genetics, contribute to lung cancer risk [12]. Unfortunately, a majority of lung cancer cases are diagnosed at a late disease stage with a poor 5-year survival rate of less than 5%. There is, therefore, an urgent and unmet need to detect lung cancer early when prevention or earlier intervention is possible [1]. Polygenic risk scores (PRSs) have emerged as a promising tool for predicting cancer risk from genome-wide association studies (GWAS) [13,14,15,16]. Lung cancer PRSs and their potential clinical utility have been explored in European and Chinese populations [2, 17,18,19,20]. Lung cancer PRSs may provide additional information for personalized risk stratification and prediction beyond non-genetic risk factors [17,18,19,20].

As PRS prediction moves towards clinical implementation in personalized medicine, accurate and unbiased PRS predictions for any single individual are needed. Unfortunately, to date, the predictive accuracy of PRS has been mostly evaluated at the population level using cohort-level metrics of prediction R2 and area under the curve (AUC), with its precision at a single individual level remaining largely unexplored [21, 22]. Moreover, whether individual-level PRS instability would influence the subsequent clinical utilization of PRS-based risk stratification and prediction is also of great interest to discover [21,22,23,24,25]. In recent studies, the individual level uncertainty in PRS estimation on various traits including height and body mass index, and diseases including breast cancer, hypertension, and dementia have been assessed in the British population using UK Biobank data [21, 22]. Another study showed that post-traumatic stress disorder and type 2 diabetes PRSs estimated among populations from different ancestries have very modest correlations at the individual level [24]. To our knowledge, there has been no study investigating the stability of lung cancer PRS for individuals and its potential impact on subsequent prediction for downstream clinical applications.

Using data from the International Lung Cancer Consortium (ILCCO) [16], we estimated lung cancer PRSs and constructed corresponding confidence/credible interval (CI) for each individual using two statistical approaches: (1) the weighted sum of 16 GWAS-derived significant single nucleotide polymorphisms (SNP) loci that have been validated in European descent population and the CI through bootstrapping method (PRS-16-CV) and (2) LDpred2 and the CI through posteriors sampling (PRS-Bayes) [21]. We further evaluated the impact of individual-level PRS uncertainty on PRS-based ranking, risk stratification, and prediction in conjunction with non-genetic risk factors of age, gender, and smoking history. Our research shows that the uncertainty of lung cancer PRS at the individual level greatly impacts the subsequent performance of individual risk stratification and prediction, highlighting the importance of cautious clinical interpretation and implementation in precision medicine.

Methods

Study population

We conducted our study in 30,060 participants of European ancestry (17,166 lung cancer cases and 12,894 controls) from 25 lung cancer OncoArray studies of ILCCO including both population-based and hospital-based case-control studies. The basic characteristics of these studies are summarized in Table S1 (Fig. 1, Additional file 1: Table S1). The eligibility criteria of studies to be included in ILCCO were that they had a study protocol for subject recruitment and a structured questionnaire for baseline lifestyle information. A detailed description of each study was described in the consortium flagship paper [16, 26]. Baseline demographics of age, gender, self-reported race/ethnicity, smoking history, and lung cancer histology information were collected and adjusted for in multivariable logistic regression analyses.

Fig. 1
figure 1

Overview of the study. The study was conducted in 17,166 lung cancer cases and 12,894 controls with European ancestry from the International Lung Cancer Consortium (ILCCO). The lung cancer PRSs and corresponding confidence/credible interval were constructed using two statistical approaches for each individual—(1) the weighted sum of 16 GWAS-derived significant SNP loci that have been validated in European descent population and the confidence interval through the bootstrapping method (PRS-16-CV) and (2) LDpred2 and the credible interval through posteriors sampling (PRS-Bayes). The individual-level PRS uncertainty was characterized and the impact on subsequent risk stratification and prediction were evaluated

Full size image

Genotyping, imputation, and quality control

Genotyping was performed at the Center for Inherited Disease Research using the OncoArray platform with 533,631 SNPs from Illumina and imputation was conducted based on 1000G phase 3 reference panel [27]. Standard QC procedures were applied to keep variants with genotype calling rate > 95%, minor allele frequency > 1%, and deviation from Hardy-Weinberg equilibrium with P-value > 10−10 in controls. A total of 5,097,871 variants remained in our statistical analyses. To adjust for subtle population stratification, we performed a principal component analysis using PLINK v1.9 [28] and adopted the top 10 principal components (PCs) following reference [29].

PRS estimation and confidence/credible interval (CI) construction

PRS-16-CV was calculated as the weighted sum of 16 GWAS-derived common SNP loci that have been validated in European descent populations [18]. Effect sizes were estimated through five-fold cross-validation after adjusting for age, gender, smoking status, and 10 PCs. Confidence intervals (CI) for PRS-16-CV were constructed using 1000 bootstrap samples within each fold. Therefore, each individual would have their own individual-level PRS distribution characterized by the PRS-16-CV bootstrapped mean and CI for downstream risk stratification and prediction. Cross-validation and bootstrapping were implemented using R v4.1.0. In addition, we calculated the PRS-16 based on the same 16 SNP loci with effect sizes directly extracted from the GWAS catalog and previous literature (Additional file 1: Table S2-3), and no confidence interval was provided for the PRS-16 [16, 18, 30,31,32,33].

On the other hand, PRS-Bayes was estimated using a Bayesian approach leveraging the LDpred2 framework, and PRS-Bayes credible interval was constructed via posterior distribution sampling using the method proposed by Ding et al. [21, 34]. More specifically, we utilized the same training data in PRS-16-CV to estimate a posterior distribution of effect sizes, given the observed genotype and lung cancer status. For each variant, a sample of effect size estimates was drawn from the posterior distribution of the causal effect size using Markov chain Monte Carlo. A credible interval of the PRS-Bayes estimator was constructed by aggregating the number of effect allele copies weighted by each of the drawn estimates.

Characterization of PRS uncertainty at the individual level

We characterized the individual level uncertainty in PRS estimation with standard deviation (s.d.) and a pre-specified confidence/credible level of CI. With a pre-specified confidence/credible level p, we derived an individual p-level CI of PRS by obtaining the empirical (1-p)/2 and (1+p)/2 quantiles from the individual level PRS distribution.

Impact of individual-level PRS uncertainty on PRS-based risk stratification

Having obtained PRSs and corresponding CI, we stratified individuals into different PRS risk subgroups based on the relationship between their own PRS CI and the population level threshold (Fig. 2). Individuals with their PRS CI above a pre-specified population-level threshold t at the upper tail (e.g., t = 90th percentile) were classified as certainly high genetic risk and similarly for individuals with PRS CI below the population level threshold t at the lower tail (e.g., t = 10th percentile) as certainly low genetic risk. Individuals whose CI covered the population level threshold were considered uncertain. As a comparison, we classified individuals based on their estimated PRS mean and population threshold without taking individual-level certainty into account.

Fig. 2
figure 2

Risk stratification based on PRS confidence/credible interval (CI). The large distribution illustrates the PRS distribution at the population level, and the four small ones refer to individual PRS distributions for participants with different PRS-based risks of lung cancer. The dashed horizontal lines indicate the population level thresholds for risk stratification. Individuals with their PRS CI above a pre-specified population-level threshold t at the upper tail (e.g., t = 90th percentile) were classified as certainly high genetic risk, and similarly for individuals with PRS CI below the population level threshold t at the lower tail (e.g., t = 10th percentile) as certainly low genetic risk. Individuals whose CI covered the population level threshold were considered uncertain

Full size image

To evaluate the degree of consistency for PRS-based risk stratification, we counted the number of overlapped individuals that are concordantly identified as the same PRS-based risk by different PRS estimators. Sensitivity analyses were conducted by changing the confidence/credible level p and risk threshold t at the population level. In addition to the two PRS approaches, we also assessed the concordance of PRS-based risk stratification among 22 lung cancer PRS in the PGS catalog.

Impact of individual level PRS uncertainty on relative risk and risk prediction

We applied multivariable logistic regression to discover the effect of PRS-based risk subgroups on lung cancer risk controlling for age, gender, and smoking history in the individuals who were identified with certainty. Stratified analyses by gender, smoking status, and lung cancer histology were similarly conducted. Lung cancer risk prediction models were constructed by integrating both PRS risk groups and other non-genetic baseline covariates, such as age, gender, and smoking history. The prediction model performance was evaluated using five-fold cross-validation based on the metric of AUC. To investigate the potential impact of individual-level uncertainty on risk prediction, we constructed a risk prediction model within the individuals identified with certainty (certainly high vs. low risk) and compared the model performance constructed in the high (n = 3006) and low (n = 3006) risk deciles identified by PRS-16-CV mean without taking individual level uncertainty into account under the exact same model specification.

Results

Table 1 summarizes the baseline characteristics of 30,060 participants that were included in the study. Ever smokers were significantly enriched in lung cancer cases with a significantly longer median smoking pack-years (39 pack-years) compared to controls (13 pack-years). Among lung cancer patients, 76.4% of the cases were non-small cell lung cancer (NSCLC) with an enrichment of lung adenocarcinoma (38.1%). Three PRSs (PRS-16, PRS-16-CV, and PRS-Bayes) of lung cancer risk were calculated for each individual and statistically higher PRS mean (mPRS) were consistently observed in lung cancer patients compared to controls (mPRS-16 in lung cancer cases: 1.26, controls: 1.21, P-value < 2.2e−16; mPRS-16-CV in lung cancer cases: 1.21, controls: 1.16, P-value < 2.2e−16; mPRS-Bayes in lung cancer cases: − 0.04, controls: − 0.11, P-value < 2.2e−16), suggesting a potentially higher genetic risk in lung cancer patients (Additional file 2: Fig. S1).

Table 1 Baseline demographic information of the study population
Full size table

Characterization of lung cancer PRS uncertainty at the individual level

The variability of lung cancer PRS at the individual level was evaluated by leveraging the CIs constructed in PRS-16-CV and PRS-Bayes. A considerable variation of PRS point estimates was observed for both methods. On average, a larger standard deviation (s.d.) of individual-level PRS distribution was observed in PRS-Bayes (mean s.d.: 0.88, 95% CI of s.d.: 0.68–1.11) compared to PRS-16-CV (mean s.d.: 0.12, 95% CI of s.d.: 0.09–0.15), suggesting a larger variability using PRS-Bayes at the individual level. For illustration purposes, we show the individual-level PRS distribution for both methods in 100 individuals (Fig. 3).

Fig. 3
figure 3

Individual-level distribution of PRS-16-CV and PRS-Bayes. A Individual-level PRS distributions obtained from PRS-16-CV. B Individual-level PRS distributions obtained from PRS-Bayes. For illustration purposes, here we only show the individual-level PRS distributions for 100 participants

Full size image

Impact of Individual-level PRS uncertainty on lung cancer risk stratification

All individuals were stratified into PRS deciles and at a confidence/credible level of 95%. Using PRS-16-CV, only 25.0% (751/3006) of individuals with PRS point estimates in the lowest decile of PRS and 16.8% (505/3006) in the highest decile have their entire 95% CI fully contained in the lowest and highest decile, respectively, while PRS-Bayes was unable to find any eligible individual (Table 2). We further conducted sensitivity analysis by changing the population level threshold t and the confidence/credible level p. We varied the range of the confidence/credible level p from 0 to 100% and fixed the threshold at t = 10th or 5th percentile for the low-risk population and at t = 90th or 95th percentile for the high-risk population. The proportion of certainty was negatively correlated with the confidence/credible level p for both the high-risk and low-risk classification (Additional file 2: Fig. S2). A consistently lower percentage of certain classifications was observed in PRS-Bayes than in PRS-16-CV across all confidence/credible levels. Similar relationships between the proportion and the confidence/credible level p were observed across subgroups of gender, histology, and smoking status (Additional file 2: Fig. S3). The proportion of certainty decreases as more stringent threshold t and confidence/credible level p are specified.

Table 2 PRS-based rankings identified by PRS-16-CV and PRS-Bayes
Full size table

Furthermore, the individual-level uncertainty greatly impacts PRS-based rankings. Each individual would have a distribution of the rankings for each PRS point estimate obtained from both PRS-16-CV and PRS-Bayes. We calculated the mean and range of the rankings for each individual. Substantial variability was also observed in the rankings identified by both methods for each PRS decile (Table 2). A wider range of rankings was observed for PRS-Bayes compared to using PRS-16-CV. The minimal range of the rankings was observed in the lowest decile (mean ranking 7th, range: 0–66th) identified by PRS-16-CV, and individuals above the 90th percentile of PRS can be anywhere from the 25th–100th percentile. In contrast, individuals in each PRS decile can be anywhere from the 0th–100th using PRS-Bayes when their CI is taken into consideration.

To answer the question of whether individuals would be commonly assigned into the same genetic risk categories, we counted the number of overlapped individuals that were commonly identified using different PRS estimators of PRS-16, PRS-16-CV, and PRS-Bayes (Table 3). Overall, the degree of overlap decreases as the population level threshold increases. PRS-16 and PRS-16-CV identified a fair number of overlapped individuals as the same SNP loci were utilized. Two thousand four hundred seventy (82%) individuals were concordantly classified as high risk (> 90th percentile) using PRS-16 and PRS-16-CV, while PRS-Bayes agreed with either PRS-16-CV or PRS-16 on 19% of high-risk (> 90th percentile) stratification. In addition, we assessed the concordance of 22 lung cancer PGSs available in the PGS catalog, and similar modest correlations in PRS-based risk stratification were observed using PGS estimate (Additional file 2: Fig. S4-7). Taken together, substantial disagreement was observed for lung cancer PRS at the individual level using different PRS estimators.

Table 3 Overlap of commonly identified individuals using different PRS estimators
Full size table

Impact of individual-level uncertainty on relative risk of PRS deciles on lung cancer

Next, we evaluated its impact on lung cancer risk in individuals identified by PRS-16-CV taking individual-level uncertainty into account. In contrast to risk stratification based on PRS mean, only individuals in the top and lowest deciles were able to be identified with certainty when taking variance in PRS estimates into account; hence, the relative risk was only evaluated in the two PRS-based risk subgroups. An increased effect size for PRS CI (OR = 2.73, 95% CI: 2.12–3.50, P-value = 4.13 × 10−15) was compared to using PRS mean (OR = 2.23, 95% CI: 1.99–2.49, P-value = 5.70 × 10−46) (Table 4). Similar improvement was observed in stratified analyses by gender, lung cancer histology, and smoking status (Table 5). The largest increase in relative risk of lung cancer from OR = 2.63 to OR = 4.22 was observed in never smokers, suggesting a potentially larger genetic contribution to lung cancer and thus more susceptible to individual-level PRS uncertainty.

Table 4 Impact of individual-level uncertainty on relative risk of PRS deciles on lung cancer risk
Full size table
Table 5 Impact of individual-level uncertainty on the relative risk of PRS deciles on lung cancer risk in stratified analyses by gender, lung cancer histology, and smoking status
Full size table

Impact of individual-level PRS uncertainty on lung cancer risk prediction

Finally, we evaluated the impact of the individual-level PRS uncertainty on lung cancer risk prediction by including the two PRS-based risk subgroups identified by PRS-16-CV. Prediction models were constructed based on PRS-based risk subgroups and non-genetic risk factors of age, gender, and smoking history. An improved prediction model performance was consistently observed in all models when considering individual-level uncertainty compared to using PRS mean solely (Table 6). The discriminative performance improved as we gradually incorporated non-genetic predictors into the model in a sequence of age, gender, and smoking information. The highest AUC (0.73, 95% CI = 0.72–0.74) was achieved by the model whose predictors were a combination of PRS-based risk categories and non-genetic predictors of age, gender, and detailed smoking pack years. Taking individual-level uncertainty of genetic risk into account in conjunction with non-genetic risk factors improves lung cancer risk prediction.

Table 6 Impact of individual-level uncertainty on lung cancer risk prediction performance
Full size table

Discussion

Our work explored lung cancer PRS uncertainty for individuals in populations with European ancestry using both GWAS-derived SNPs and LDpred2 and demonstrated that individual-level PRS uncertainty greatly impacts PRS-based rankings, risk stratification, and ultimately prediction of lung cancer risk. The substantial recent interest in translating lung cancer PRS and other predictive tools like deep learning models from low radiation dose chest computed tomography (LDCT) for future lung cancer risk prediction necessitates a careful assessment of individual-level uncertainty to truly accomplish personalized risk assessment [35, 36]. Taking individual-level uncertainty into PRS-based risk stratification and prediction provides more accurate risk stratification and improves the ability to identify high-risk subjects and recommend LDCT screening with more certainty. It is imperative to develop more stable PRS and account for uncertainty at the individual level in risk stratification to help ensure reliable clinical applications of PRS in real-world settings.

A critical concern of PRS application is delivering inaccurate risk estimates at the individual level, and wrongly categorizing an individual as low or high genetic risk based on unstable PRS estimates. The downstream effect could be inappropriate or even contradictory medical advice or clinical decisions [25]. Our study shows substantial disagreement in risk categorization using different lung cancer PRSs (PRS-16, PRS-16-CV, and PRS-Bayes), suggesting that individuals that were identified as very high risk by one PRS method may not be classified as such by another. Similar modest correlations were observed for another 22 lung cancer PRS in the PGS catalog. This issue is not unique to lung cancer, as large discordances have also been found in breast cancer, hypertension, and dementia using approaches to calculate PRS in a white British population [22]. More complex traits or diseases and different ancestries between the discovery and target population of GWAS may lead to even more profound inconsistency at the individual level [24]. PRS needs to be reliable and reproducible if it is going to inform personalized decision-making in clinical settings.

Comparing the two PRS generative methods, a much larger variability was observed in PRS-Bayes, resulting in no individuals can be identified with sufficient certainty. Ding et al. also found only a limited proportion of individuals are classified as high risk with certainty across 13 traits in UK Biobank [21]. PRS-Bayes includes a much greater number of non-zero weight SNPs (~2000 SNPs) with small effect sizes and this may explain a higher proportion of SNP-based heritability compared to PRS-16-CV (16 SNPs). In the meanwhile, the numerous SNPs with small effect sizes may contribute to a higher variance in PRS estimates. PRS-Bayes may perform better in terms of lung cancer risk prediction at the population level as the accuracy is determined by the proportion of phenotypic variance explained by variants included and the improved population prediction error. On the other hand, PRS-16-CV is relatively parsimonious and contains only those that have been robustly validated SNPs and have a minor individual-level variance. Additionally, we constructed and evaluated the individual-level PRS uncertainty and population-level prediction accuracy using nine experimentally validated SNPs that are associated with lung cancer risk [37] (Additional file 1: Table S4). More individuals can be identified with certainty while similar prediction accuracy was achieved (Additional file 1: Table S5-6). This may suggest that experimentally validated fine-mapped variants may be more likely to be biologically causal variants and thus have stable effects and PRS risk stratification performance at the individual level for lung cancer prediction given the similar population-level prediction accuracy. A trade-off between these methods may be manifested by the compromise between prediction performance at the population level and stability at the individual level, and this needs to be considered cautiously when developing new PRS methods tailoring for different purposes and applications [21].

Given the individual-level PRS uncertainty, it is imperative to cautiously interpret and implement it in clinical applications. From our analyses, using the PRS-based risk subgroup alone resulted in the lowest AUC, and including other non-genetic factors of age, gender, and smoking history largely improved the lung cancer risk prediction performance. PRS stand-alone is imperfect and the use of PRS as a covariate or effect modifier in conjunction with non-genetic risk factors may inform better outcomes [22, 38]. Despite modest improvement in prediction accuracy using PRS-based risk subgroups taking individual-level uncertainty into account, likely due to the limited sample sizes of eligible individuals, it attains both PRS stability and predictive accuracy given the data. Also, it reflects the situation that it is challenging for the current PRS approaches to balance reproducible PRS for individuals with sufficient certainty and high prediction accuracy at the population level. It is crucial and pressing to develop guidelines in constructing PRS to minimize the extent to which individuals could be provided with imprecise or contradictory clinical advice and intervention as well as PRS reporting standards in light of the patient and primary care providers’ perspective [25].

There are several improvements and future directions for the present study. First, more statistical approaches to construct lung cancer PRS, e.g., regularization-based methods, can be included [39]. Second, we did not assess the potential interactions between lung cancer PRS and non-genetic risk factors when taking individual-level uncertainty into account, and we would expect more uncertainty arising from environmental factors as well as from the gene-environment interactions in predicting the risk of lung cancer [40]. Further research is needed to assess the impact on performance when using commonly applied LD clumping methods for SNP selection, particularly with varying parameters. Lastly, our study was conducted in a population of European ancestry; thus, the generalizability of the results in other populations of different genetic ancestries is a concern given the discrepancy of genetic structures for causal variants, allele frequencies, and LD patterns across ancestries. To realize the equitable and reliable potential of PRS, it is necessary to carry out larger, multi-ancestry GWAS and further investigate the uncertainty at the individual level in PRS.

Conclusions

Our study characterized the uncertainty of lung cancer PRS at the individual level and evaluated its impact on subsequent PRS-based ranking, risk stratification, and prediction in populations with European ancestry. It is imperative to develop reproducible PRS, reliable PRS-based clinical applications as well as guidelines to report and communicate PRS to patients and their primary physicians.