Segmentation of organs from chest X-ray images is an essential task for an accurate and reliable diagnosis of lung diseases and chest organ morphometry. In this study, we investigated the benefits of augmenting state-of-the-art deep convolutional neural networks (CNNs) for image segmentation with organ contour information and evaluated the performance of such augmentation on segmentation of lung fields, heart, and clavicles from chest X-ray images.
Three state-of-the-art CNNs were augmented, namely the UNet and LinkNet architecture with the ResNeXt feature extraction backbone, and the Tiramisu architecture with the DenseNet. All CNN architectures were trained on ground-truth segmentation masks and additionally on the corresponding contours. The contribution of such contour-based augmentation was evaluated against the contour-free architectures, and 20 existing algorithms for lung field segmentation.
The proposed contour-aware segmentation improved the segmentation performance, and when compared against existing algorithms on the same publicly available database of 247 chest X-ray images, the UNet architecture with the ResNeXt50 encoder combined with the contour-aware approach resulted in the best overall segmentation performance, achieving a Jaccard overlap coefficient of 0.971, 0.933, and 0.903 for the lung fields, heart, and clavicles, respectively.
In this study, we proposed to augment CNN architectures for CXR segmentation with organ contour information and were able to significantly improve segmentation accuracy and outperform all existing solution using a public chest X-ray database.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Chen S, Zhong S, Yao L, Shang Y, Suzuki K (2016) Enhancement of chest radiographs obtained in the intensive care unit through bone suppression and consistent processing. Phys Med Biol 61:2283
Candemir S, Antani S (2019) A review on lung boundary detection in chest X-rays. Int J Comput Assist Radiol Surg 14:563–576
Miniati M, Coppini G, Monti S, Bottai M, Paterni M, Ferdeghini E (2011) Computer-aided recognition of emphysema on digital chest radiography. Eur J Radiol 80:169–175
Candemir S, Jaeger S, Lin W, Xue Z, Antani S, Thoma G (2016) Automatic heart localization and radiographic index computation in chest X-rays. In: SPIE medical imaging, 9785
Finnegan R, Dowling J, Koh E-S, Tang S, Otton J, Delaney G, Batumalai V, Luo C, Atluri P, Satchithanandha A (2019) Feasibility of multi-atlas cardiac segmentation from thoracic planning CT in a probabilistic framework. Phys Med Biol 64:085006
Gordienko Y, Gang P, Hui J, Zeng W, Kochura Y, Alienin O, Rokovyi O, Stirenko S (2018) Deep learning with lung segmentation and bone shadow exclusion techniques for chest x-ray analysis of lung cancer. In: International conference on ICAFS, pp 638–647
Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K, Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR Am J Roentgenol 174:71–4
van Ginneken B, Stegmann MB, Loog M (2006) Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal 10:19–40
Vittitoe NF, Vargas-Voracek R, Floyd CF Jr (1998) Identification of lung regions in chest radiographs using Markov random field modeling. Med Phys 25:976–85
Shi Z, Zhou P, He L, Nakamura T, Yao Q, Itoh H (2009) Lung segmentation in chest radiographs by means of Gaussian kernel-based FCM with spatial constraints. In: 6th international conference on fuzzy systems and knowledge discovery (FSKD). IEEE, pp 428–432
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Computer vision and pattern recognition (CVPR). IEEE, pp 248–55
Wang C (2017) Segmentation of multiple structures in chest radiographs using multi-task fully convolutional networks. In: Scandinavian conference on image analysis (SCIA). Volume 10270 of Lecture notes in computer science. Springer, Cham, pp 282–289
Hwang S, Park S (2017) Accurate lung segmentation via network-wise training of convolutional networks. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, vol 10553 lecture notes in computer science. Springer, Cham, pp 92–99
Dai W, Dong N, Wang Z, Liang X, Zhang H, Xing EP (2018) SCAN: structure correcting adversarial network for organ segmentation in chest X-rays. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, volume 11045 of lecture notes in computer science. Springer, Berlin, pp 263–73
Bi L, Feng D, Kim J (2018) Dual-path adversarial learning for fully convolutional network (FCN)-based medical image segmentation. Vis Comput 34:1043–52
Novikov AA, Lenis D, Major D, Hladůvka J, Wimmer M, Bühler K (2018) Fully convolutional architectures for multi-class segmentation in chest radiographs. IEEE Trans Med Imaging 37:1865–76
Mittal A, Hooda R, Sofat S (2018) LF-SegNet: a fully convolutional encoder-decoder network for segmenting lung fields from chest radiographs. Wirel Pers Commun 101:511–29
Frid-Adar M, Ben-Cohen A, Amer R, Greenspan H (2018) Improving the segmentation of anatomical structures in chest radiographs using U-Net with an ImageNet pre-trained encoder. In: Image analysis for moving organ. breast, and thoracic images, volume 11040 of lecture notes in computer science. Springer, Cham, pp 159–68
Bonheur S, Stern D, Payer C, Pienn M, Olschewski H, Urschler M (2019) Matwo-capsnet: a multi-label semantic segmentation capsules network. MICCAI 2019:664–672
Ngo T A, Carneiro G (205) Lung segmentation in chest radiographs using distance regularized level set and deep-structured learning and inference. In: International conference on image processing (ICIP). IEEE, pp 2140–2143
Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A (2017) A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imaging 36:1550–60
Cui Y, Zhang G, Liu Z, Xiong Z, Hu J (2018) A deep learning algorithm for one-step contour aware nuclei segmentation of histopathological images. arXiv, p 1803.02786
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention (MICCAI), volume 9351 of lecture notes in computer science. Springer, Cham, pp 234–241
Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Computer vision and pattern recognition (CVPR). IEEE, pp 5987–5995
Chaurasia A, Culurciello E (2017) LinkNet: exploiting encoder representations for efficient semantic segmentation. In: Visual communications and image processing (VCIP). IEEE
Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation. In: Computer vision and pattern recognition workshops (CVPRW). IEEE, pp 1175–1783
Huang G, Liu Z, van der Maaten L, Weinberger K Q (2017) Densely connected convolutional networks. In: Computer vision and pattern recognition (CVPR). IEEE, pp 2261–2269
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Computer vision and pattern recognition (CVPR). IEEE, pp 3431–3440
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) UNet++: a nested U-Net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support (DLMIA / ML-CDS), volume 11045 of lecture notes in computer science. Springer, Cham, pp 3–11
Wang CW, Huang CT, Lee JH, Li CH, Chang SW, Siao MJ, Lai TM, Ibragimov B, Vrtovec T, Ronneberger O, Fischer P, Cootes TF, Lindner C (2016) A benchmark for comparison of dental radiography analysis algorithms. Med Image Anal 31:63–76
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Computer vision and pattern recognition (CVPR). IEEE, pp 770–788
Shaikh M, Anand G, Acharya G, Amrutkar A, Varghese A, Krishnamurthi G (2018) Brain tumor segmentation using dense fully convolutional neural network. Brainlesion: Glioma. In: Multiple sclerosis, stroke and traumatic brain injuries (BrainLes), volume 10670 of lecture notes in computer Science. Springer, Cham, pp 309–319
Xu C, Xu L, Brahm G, Zhang, Li S (2018) Mutgan: simultaneous segmentation and quantification of myocardial infarction without contrast agents via joint adversarial learning. In: Conference on Medical Image Computing and Computer-Assisted Intervention, pp 525–534, 2018
Arik SO, Ibragimov B, Xing L (2017) Fully automated quantitative cephalometry using convolutional neural networks. J Med Imaging 4:014501
Milletari F, Navab N, Ahmadi S-A (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 3D vision (3DV). IEEE, pp 565–571
DSBowl2018 (2018) 2018 data science bowl. https://www.kaggle.com/c/data-science-bowl-2018/overview. Accessed 15 Oct 2019
KiTS2019 (2019) 2019 kidney and kidney tumor segmentation challenge. https://kits19.grand-challenge.org/. Accessed 15 Oct 2019
Isensee F, Petersen J, Kohl S A A, Jager PF, Maier-Hein KH (2019) Nnu-net: breaking the spell on successful medical image segmentation. arXiv:1904.08128
Kingma DP, Ba LJ (2015) Adam: a method for stochastic optimization. In: 3rd International conference on learning representations (ICLR)
Shi Y, Qi F, Xue Z, Chen L, Ito K, Matsuo H, Shen D (2008) Segmenting lung fields in serial chest radiographs using both population-based and patient-specific shape statistics. IEEE Trans Med Imaging 27:481–94
Li X, Luo S, Hu Q, Li J, Wang D, Chiong F (2016) Automatic lung field segmentation in X-ray radiographs using statistical shape and appearance models. J Med Imaging Health Inform 6:338–48
Dawoud A (2010) Fusing shape information in lung segmentation in chest radiographs. In: Image analysis and recognition—ICIAR 2010, volume 6112 of lecture notes in computer science. Springer, Berlin, Heidelberg, pp 70–78
Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Zhiyun X, Karargyris A, Antani S, Thoma G, McDonald CJ (2014) Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 33:577–90
Ibragimov B, Likar B, Pernuš F, Vrtovec T (2012) A game-theorteic framework for landmark-based image segmentation. IEEE Trans. Med. Imaging 31:1761–76
Seghers D, Loeckx D, Maes F, Vandermeulen D, Suetens P (2007) Minimal shape and intensity cost path segmentation. IEEE Trans Med Imaging 26:1115–29
Wu G, Zhang X, Luo S, Hu Q (2015) Lung segmentation based on customized active shape model from digital radiography chest images. J Med Imaging Health Inform 5:184–91
Yang W, Liu Y, Lin L, Yun Z, Lu Z, Feng Q, Chen W (2018) Lung field segmentation in chest radiographs from boundary maps by a structured edge detector. IEEE J Biomed Health Inform 22:842–51
Ibragimov B, Likar B, Pernuš F, Vrtovec T (2016) Accurate landmark-based segmentation by incorporating landmark misdetections. In: 13th international symposium on biomedical imaging (ISBI). IEEE, pp 1072–1075
Chondro P, Yao C-Y, Ruan S-J, Chien L-C (2018) Low order adaptive region growing for lung segmentation on plain chest radiographs. Neurocomputing 275:1002–11
Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of CVPR 2017
RSNA (2018) Kaggle pneumonia detection challenge. https://www.kaggle.com/c/rsna-pneumonia-detection-challenge. Accessed 15 Oct 2019
Karargyris A, Siegelman J, Tzortzis D, Jaeger S, Candemir S, Xue Z, Santosh K, Vajda S, Antani S, Folio L, Thoma G (2016) Combination of texture and shape features to detect pulmonary abnormalities in digital chest X-rays. Int J Comput Assist Radiol Surg 11:99–106
Zheng G, Chu C, Belavy DL, Ibragimov B, Korez R, Vrtovec T, Hutt H, Everson R, Meakin J, Andrade IL, Glocker B, Chen H, Qi Dou Q, Heng PA, Wang C, Forsberg D, Neubert A, Fripp J, Urschler M, Stern D, Wimmer M, Novikov AA, Cheng H, Armbrecht G, Felsenberg D, Li S (2017) Evaluation and comparison of 3d intervertebral disc localization and segmentation methods for 3d t2mr data: a grand challenge. Med Image Anal 35:327–344
Cong J, Zheng Y, Xue W, Cao B, Li S (2019) Ma-shape: modality adaptation shape regression for left ventricle segmentation on mixed MR and CT images. IEEE Access 7:16584–16593
Kholiavchenko M (2019) https://github.com/dirtmaxim/ca-ml-organ-segmentation. Accessed 15 Oct 2019
Jeager S, Candemir S, Antani S, Wang Y, Lu P, G Thoma (2014) Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 4:475–477
This work was supported by the Russian Science Foundation (Grant No. 18-71-10072), Russian Foundation for Basic Research (Grant No. 18-47-160015), and Slovenian Research Agency—ARRS (Grant No. P2-0232). Due to the requirements of the Russian Science Foundation, we explicitly state that the methodology development and implementation were solely supported by the Russian Science Foundation (Grant No. 18-71-10072).
Conflict of interest
The authors declare that they have no conflict of interest.
This study has not been supported by any industrial company and does not serve to promote any commercial product. Anonymized publicly available databases of CXR were used in the conducted experiments.
For this type of study, formal consent is not required.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kholiavchenko, M., Sirazitdinov, I., Kubrak, K. et al. Contour-aware multi-label chest X-ray organ segmentation. Int J CARS 15, 425–436 (2020). https://doi.org/10.1007/s11548-019-02115-9
- Image segmentation
- Convolutional neural networks
- Deep learning architectures
- Chest X-ray (CXR) images
- JSRT database