Skip to main content

Deep learning algorithm for surveillance of pneumothorax after lung biopsy: a multicenter diagnostic cohort study



Pneumothorax is the most common and potentially life-threatening complication arising from percutaneous lung biopsy. We evaluated the performance of a deep learning algorithm for detection of post-biopsy pneumothorax in chest radiographs (CRs), in consecutive cohorts reflecting actual clinical situation.


We retrospectively included post-biopsy CRs of 1757 consecutive patients (1055 men, 702 women; mean age of 65.1 years) undergoing percutaneous lung biopsies from three institutions. A commercially available deep learning algorithm analyzed each CR to identify pneumothorax. We compared the performance of the algorithm with that of radiology reports made in the actual clinical practice. We also conducted a reader study, in which the performance of the algorithm was compared with those of four radiologists. Performances of the algorithm and radiologists were evaluated by area under receiver operating characteristic curves (AUROCs), sensitivity, and specificity, with reference standards defined by thoracic radiologists.


Pneumothorax occurred in 17.5% (308/1757) of cases, out of which 16.6% (51/308) required catheter drainage. The AUROC, sensitivity, and specificity of the algorithm were 0.937, 70.5%, and 97.7%, respectively, for identification of pneumothorax. The algorithm exhibited higher sensitivity (70.2% vs. 55.5%, p < 0.001) and lower specificity (97.7% vs. 99.8%, p < 0.001), compared with those of radiology reports. In the reader study, the algorithm exhibited lower sensitivity (77.3% vs. 81.8–97.7%) and higher specificity (97.6% vs. 81.7–96.0%) than the radiologists.


The deep learning algorithm appropriately identified pneumothorax in post-biopsy CRs in consecutive diagnostic cohorts. It may assist in accurate and timely diagnosis of post-biopsy pneumothorax in clinical practice.

Key Points

• A deep learning algorithm can identify chest radiographs with post-biopsy pneumothorax in multicenter consecutive cohorts reflecting actual clinical situation.

• The deep learning algorithm has a potential role as a surveillance tool for accurate and timely diagnosis of post-biopsy pneumothorax.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6



American College of Chest Physicians


Area under the receiver operating characteristic curve


British Thoracic Society


Confidence interval


Chest radiograph


Interquartile range


Negative predictive value


Positive predictive value


Percutaneous transthoracic needle biopsy


  1. 1.

    Callister ME, Baldwin DR, Akram AR et al (2015) British Thoracic Society guidelines for the investigation and management of pulmonary nodules. Thorax 70(Suppl 2):ii1–ii54

    Article  Google Scholar 

  2. 2.

    Rivera MP, Mehta AC, Wahidi MM (2013) Establishing the diagnosis of lung cancer: diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:e142S–e165S

    Article  Google Scholar 

  3. 3.

    Manhire A, Charig M, Clelland C et al (2003) Guidelines for radiologically guided lung biopsy. Thorax 58:920–936

    CAS  Article  Google Scholar 

  4. 4.

    Yoon SH, Park CM, Lee KH et al (2019) Analysis of complications of percutaneous transthoracic needle biopsy using CT-guidance modalities in a multicenter cohort of 10568 biopsies. Korean J Radiol 20:323–331

    Article  Google Scholar 

  5. 5.

    Schnell J, Beer M, Eggeling S et al (2018) Management of spontaneous pneumothorax and post-interventional pneumothorax: German S3 guideline. Respiration.

  6. 6.

    Heerink WJ, de Bock GH, de Jonge GJ, Groen HJ, Vliegenthart R, Oudkerk M (2017) Complication rates of CT-guided transthoracic lung biopsy: meta-analysis. Eur Radiol 27:138–148

    CAS  Article  Google Scholar 

  7. 7.

    Lim WH, Park CM, Yoon SH et al (2018) Time-dependent analysis of incidence, risk factors and clinical significance of pneumothorax after percutaneous lung biopsy. Eur Radiol 28:1328–1337

    Article  Google Scholar 

  8. 8.

    Ehteshami Bejnordi B, Veta M, Johannes van Diest P et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199–2210

    Article  Google Scholar 

  9. 9.

    Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118

    CAS  Article  Google Scholar 

  10. 10.

    Gulshan V, Peng L, Coram M et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316:2402–2410

    Article  Google Scholar 

  11. 11.

    Hwang EJ, Park S, Jin KN et al (2019) Development and validation of a deep learning-based automatic detection algorithm for active pulmonary tuberculosis on chest radiographs. Clin Infect Dis 69:739–747

    Article  Google Scholar 

  12. 12.

    Lakhani P, Sundaram B (2017) Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284:574–582

    Article  Google Scholar 

  13. 13.

    Nam JG, Park S, Hwang EJ et al (2019) Development and validation of deep learning-based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology 290:218–228

    Article  Google Scholar 

  14. 14.

    Annarumma M, Withey SJ, Bakewell RJ, Pesce E, Goh V, Montana G (2019) Automated triaging of adult chest radiographs with deep artificial neural networks. Radiology 291:196–202

    Article  Google Scholar 

  15. 15.

    Rajpurkar P, Irvin J, Ball RL et al (2018) Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med 15:e1002686

    Article  Google Scholar 

  16. 16.

    Singh R, Kalra MK, Nitiwarangkul C et al (2018) Deep learning in chest radiography: detection of findings and presence of change. PLoS One 13:e0204155

    Article  Google Scholar 

  17. 17.

    Taylor AG, Mielke C, Mongan J (2018) Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: a retrospective study. PLoS Med 15:e1002697

    Article  Google Scholar 

  18. 18.

    Cicero M, Bilbily A, Colak E et al (2017) Training and validating a deep convolutional neural network for computer-aided detection and classification of abnormalities on frontal chest radiographs. Invest Radiol 52:281–287

    Article  Google Scholar 

  19. 19.

    Hwang EJ, Park S, Jin KN et al (2019) Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2:e191095

    Article  Google Scholar 

  20. 20.

    Park SH, Han K (2018) Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 286:800–809

    Article  Google Scholar 

  21. 21.

    Collins CD, Lopez A, Mathie A, Wood V, Jackson JE, Roddie ME (1995) Quantification of pneumothorax size on chest radiographs using interpleural distances: regression analysis based on volume measurements from helical CT. AJR Am J Roentgenol 165:1127–1130

    CAS  Article  Google Scholar 

  22. 22.

    Choi BG, Park SH, Yun EH, Chae KO, Shinn KS (1998) Pneumothorax size: correlation of supine anteroposterior with erect posteroanterior chest radiographs. Radiology 209:567–569

    CAS  Article  Google Scholar 

  23. 23.

    MacDuff A, Arnold A, Harvey J, BTS Pleural Disease Guideline Group (2010) Management of spontaneous pneumothorax: British Thoracic Society pleural disease guideline 2010. Thorax 65(Suppl 2):ii18–ii31

  24. 24.

    Baumann MH, Strange C, Heffner JE et al (2001) Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement. Chest 119:590–602

    CAS  Article  Google Scholar 

  25. 25.

    DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    CAS  Article  Google Scholar 

  26. 26.

    Leisenring W, Alonzo T, Pepe MS (2000) Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics 56:345–351

    CAS  Article  Google Scholar 

  27. 27.

    Park S, Lee SM, Kim N et al (2019) Application of deep learning-based computer-aided detection system: detecting pneumothorax on chest radiograph after biopsy. Eur Radiol 29:5341–5348

    Article  Google Scholar 

Download references


Lunit Inc. provided technical supports in the present study, for analyses of chest radiographs with deep learning algorithm. The funding sources and Lunit Inc. did not have any role either in the design of the study or in the acquisition, analyses, and interpretation of the data and in the manuscript preparation.


This study has received funding by the Seoul National University Hospital Research fund (grant number 03-2019-0190) and the Seoul Research & Business Development Program (grant number FI170002).

Author information



Corresponding author

Correspondence to Chang Min Park.

Ethics declarations


The scientific guarantor of this publication is Chang Min Park.

Conflict of interest

The authors of this manuscript declare relationships with Lunit Inc.

Jin Mo Goo and Chang Min Park report grants from the Lunit Inc., outside the present study.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the institutional review board.

Ethical approval

Institutional review board approval was obtained.


• Retrospective

• Diagnostic or prognostic study

• Multicenter study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hwang, E.J., Hong, J.H., Lee, K.H. et al. Deep learning algorithm for surveillance of pneumothorax after lung biopsy: a multicenter diagnostic cohort study. Eur Radiol 30, 3660–3671 (2020).

Download citation


  • Thoracic radiography
  • Pneumothorax
  • Needle biopsy
  • Artificial intelligence
  • Deep learning