Skip to main content
SpringerLink
Log in

Prediction of visceral pleural invasion in lung cancer on CT: deep learning model achieves a radiologist-level performance with adaptive sensitivity and specificity to clinical needs

  • Chest
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To develop and validate a preoperative CT-based deep learning model for the prediction of visceral pleural invasion (VPI) in early-stage lung cancer.

Methods

In this retrospective study, dataset 1 (for training, tuning, and internal validation) included 676 patients with clinical stage IA lung adenocarcinomas resected between 2009 and 2015. Dataset 2 (for temporal validation) included 141 patients with clinical stage I adenocarcinomas resected between 2017 and 2018. A CT-based deep learning model was developed for the prediction of VPI and validated in terms of discrimination and calibration. An observer performance study and a multivariable regression analysis were performed.

Results

The area under the receiver operating characteristic curve (AUC) of the model was 0.75 (95% CI, 0.67–0.84), which was comparable to those of board-certified radiologists (AUC, 0.73–0.79; all p > 0.05). The model had a higher standardized partial AUC for a specificity range of 90 to 100% than the radiologists (all p < 0.05). The high sensitivity cutoff (0.245) yielded a sensitivity of 93.8% and a specificity of 31.2%, and the high specificity cutoff (0.448) resulted in a sensitivity of 47.9% and a specificity of 86.0%. Two of the three radiologists provided highly sensitive (93.8% and 97.9%) but not specific (48.4% and 40.9%) diagnoses. The model showed good calibration (p > 0.05), and its output was an independent predictor for VPI (adjusted odds ratio, 1.07; 95% CI, 1.03–1.11; p < 0.001).

Conclusions

The deep learning model demonstrated a radiologist-level performance. The model could achieve either highly sensitive or highly specific diagnoses depending on clinical needs.

Key Points

• The preoperative CT-based deep learning model demonstrated an expert-level diagnostic performance for the presence of visceral pleural invasion in early-stage lung cancer.

• Radiologists had a tendency toward highly sensitive, but not specific diagnoses for the visceral pleural invasion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

AUC:

Area under the receiver operating characteristic curve

CI:

Confidence interval

IQR:

Interquartile range

NPV:

Negative predictive value

OR:

Odds ratio

PPV:

Positive predictive value

VPI:

Visceral pleural invasion

References

  1. Rami-Porta R, Bolejack V, Crowley J et al (2015) The IASLC Lung Cancer Staging Project: proposals for the revisions of the T descriptors in the forthcoming eighth edition of the TNM classification for lung cancer. J Thorac Oncol 10:990–1003

    Article  Google Scholar 

  2. Kamigaichi A, Tsutani Y, Kagimoto A et al (2020) Comparing segmentectomy and lobectomy for clinical stage IA solid-dominant lung cancer measuring 2.1-3 cm. Clin Lung Cancer. https://doi.org/10.1016/j.cllc.2020.04.015

  3. Smith CB, Swanson SJ, Mhango G, Wisnivesky JP (2013) Survival after segmentectomy and wedge resection in stage I non-small-cell lung cancer. J Thorac Oncol 8:73–78

    Article  Google Scholar 

  4. Tsutani Y, Miyata Y, Nakayama H et al (2014) Segmentectomy for clinical stage IA lung adenocarcinoma showing solid dominance on radiology. Eur J Cardiothorac Surg 46:637–642

    Article  Google Scholar 

  5. Zhong C, Fang W, Mao T, Yao F, Chen W, Hu D (2012) Comparison of thoracoscopic segmentectomy and thoracoscopic lobectomy for small-sized stage IA lung cancer. Ann Thorac Surg 94:362–367

    Article  Google Scholar 

  6. Lopez Guerra JL, Gomez DR, Lin SH et al (2013) Risk factors for local and regional recurrence in patients with resected N0-N1 non-small-cell lung cancer, with implications for patient selection for adjuvant radiation therapy. Ann Oncol 24:67–74

    Article  CAS  Google Scholar 

  7. Takizawa H, Kondo K, Kawakita N et al (2018) Autofluorescence for the diagnosis of visceral pleural invasion in non-small-cell lung cancer. Eur J Cardiothorac Surg 53:987–992

    Article  Google Scholar 

  8. Mizuno T, Arimura T, Kuroda H, Sakakura N, Yatabe Y, Sakao Y (2018) P2.16-34 visceral pleural invasion is closely associated with nodal spread in cStage IA lung adenocarcinoma. J Thorac Oncol 13:S845

    Article  Google Scholar 

  9. Ahn SY, Park CM, Jeon YK et al (2017) Predictive CT features of visceral pleural invasion by T1-sized peripheral pulmonary adenocarcinomas manifesting as subsolid nodules. AJR Am J Roentgenol 209:561–566

    Article  Google Scholar 

  10. Hsu JS, Han IT, Tsai TH et al (2016) Pleural tags on CT scans to predict visceral pleural invasion of non-small cell lung cancer that does not abut the pleura. Radiology 279:590–596

    Article  Google Scholar 

  11. Imai K, Minamiya Y, Ishiyama K et al (2013) Use of CT to evaluate pleural invasion in non-small cell lung cancer: measurement of the ratio of the interface between tumor and neighboring structures to maximum tumor diameter. Radiology 267:619–626

    Article  Google Scholar 

  12. Yang S, Yang L, Teng L et al (2018) Visceral pleural invasion by pulmonary adenocarcinoma ≤3 cm: the pathological correlation with pleural signs on computed tomography. J Thorac Dis 10:3992–3999

    Article  Google Scholar 

  13. Ehteshami Bejnordi B, Veta M, Johannes van Diest P et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199–2210

    Article  Google Scholar 

  14. Hwang EJ, Park S, Jin KN et al (2019) Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2:e191095

    Article  Google Scholar 

  15. Ding Y, Sohn JH, Kawczynski MG et al (2019) A deep learning model to predict a diagnosis of Alzheimer disease by using (18)F-FDG PET of the brain. Radiology 290:456–464

    Article  Google Scholar 

  16. Kim H, Goo JM, Kim YT, Park CM (2019) CT-defined visceral pleural invasion in T1 lung adenocarcinoma: lack of relationship to disease-free survival. Radiology 292:741–749

    Article  Google Scholar 

  17. Moons KG, Altman DG, Reitsma JB et al (2015) Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 162:W1–W73

  18. Goldstraw P, Chansky K, Crowley J et al (2016) The IASLC Lung Cancer Staging Project: proposals for revision of the TNM stage groupings in the forthcoming (eighth) edition of the TNM classification for lung cancer. J Thorac Oncol 11:39–51

    Article  Google Scholar 

  19. Travis WD, Brambilla E, Rami-Porta R et al (2008) Visceral pleural invasion: pathologic criteria and use of elastic stains: proposal for the 7th edition of the TNM classification for lung cancer. J Thorac Oncol 3:1384–1390

    Article  Google Scholar 

  20. Venkatraman ES (2000) A permutation test to compare receiver operating characteristic curves. Biometrics 56:1134–1138

    Article  CAS  Google Scholar 

  21. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    Article  CAS  Google Scholar 

  22. McClish DK (1989) Analyzing a portion of the ROC curve. Med Decis Making 9:190–195

    Article  CAS  Google Scholar 

  23. McNemar Q (1947) Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12:153–157

    Article  CAS  Google Scholar 

  24. Leisenring W, Alonzo T, Pepe MS (2000) Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics 56:345–351

    Article  CAS  Google Scholar 

  25. Tammemagi MC, Ten Haaf K, Toumazis I et al (2019) Development and validation of a multivariable lung cancer risk prediction model that includes low-dose computed tomography screening results: a secondary analysis of data from the National Lung Screening Trial. JAMA Netw Open 2:e190204

    Article  Google Scholar 

  26. Walsh CG, Sharman K, Hripcsak G (2017) Beyond discrimination: a comparison of calibration methods and clinical usefulness of predictive models of readmission risk. J Biomed Inform 76:9–18

    Article  Google Scholar 

  27. Iizuka S, Kawase A, Oiwa H, Ema T, Shiiya N, Funai K (2019) A risk scoring system for predicting visceral pleural invasion in non-small lung cancer patients. Gen Thorac Cardiovasc Surg 67:876–879

    Article  Google Scholar 

  28. Hsu JS, Jaw TS, Yang CJ et al (2017) Convex border of peripheral non-small cell lung cancer on CT images as a potential indicator of pleural invasion. Medicine (Baltimore) 96:e7323

    Article  Google Scholar 

  29. Brown LM, Louie BE, Jackson N, Farivar AS, Aye RW, Vallieres E (2016) Recurrence and survival after segmentectomy in patients with prior lung resection for early-stage non-small cell lung cancer. Ann Thorac Surg 102:1110–1118

    Article  Google Scholar 

  30. Karimi D, Dou H, Warfield SK, Gholipour A (2019) Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. arXiv preprint arXiv:1912.02911

  31. Johnson JM, Khoshgoftaar TM (2019) Survey on deep learning with class imbalance. J Big Data 6:27. https://doi.org/10.1186/s40537-019-0192-5

    Article  Google Scholar 

Download references

Acknowledgments

We sincerely thank Myunghee Lee and Ju Young Jeong for their assistance in the data acquisition.

Funding

This study was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Science and ICT (grant number: NRF-2020R1C1C1003684), Republic of Korea.

Author information

Authors and Affiliations

  1. Department of Radiology, Seoul National University Hospital, 101, Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Hyewon Choi, Hyungjin Kim, Wonju Hong, Jongsoo Park, Eui Jin Hwang, Chang Min Park & Jin Mo Goo

  2. Department of Radiology, Seoul National University College of Medicine, 101, Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Hyungjin Kim, Chang Min Park & Jin Mo Goo

  3. Institute of Radiation Medicine, Seoul National University Medical Research Center, 101, Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Chang Min Park & Jin Mo Goo

  4. Cancer Research Institute, Seoul National University, 101, Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Chang Min Park, Young Tae Kim & Jin Mo Goo

  5. Department of Thoracic and Cardiovascular Surgery, Seoul National University College of Medicine, 101, Daehak-ro, Jongno-gu, Seoul, 03080, South Korea

    Young Tae Kim

Authors
  1. Hyewon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyungjin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wonju Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jongsoo Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eui Jin Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chang Min Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Young Tae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jin Mo Goo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hyungjin Kim.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Hyungjin Kim.

Conflict of interest

Activities related to the present article: none.

Activities not related to the present article: H.K. received a research grant from Lunit. E.J.H. and C.M.P. received research grants from Lunit and Coreline Soft. J.M.G. received research grants from Lunit, INFINITT Healthcare, and DONGKOOK Pharmaceutical.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

Some study subjects or cohorts have been previously reported (Radiology. 2019 Sep;292(3):741–749.; Radiology. 2019 Mar;290(3):807–813.; Eur Radiol. 2019 Nov;29(11):6069–6079.; Lung Cancer. 2019 Aug;134:151–157.; J Thorac Oncol. 2018 Dec;13(12):1864–1872.).

Methodology

• retrospective

• diagnostic or prognostic study

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 29 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, H., Kim, H., Hong, W. et al. Prediction of visceral pleural invasion in lung cancer on CT: deep learning model achieves a radiologist-level performance with adaptive sensitivity and specificity to clinical needs. Eur Radiol 31, 2866–2876 (2021). https://doi.org/10.1007/s00330-020-07431-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-020-07431-2

Keywords

Search

Navigation