Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Contour-aware multi-label chest X-ray organ segmentation

Abstract

Purpose

Segmentation of organs from chest X-ray images is an essential task for an accurate and reliable diagnosis of lung diseases and chest organ morphometry. In this study, we investigated the benefits of augmenting state-of-the-art deep convolutional neural networks (CNNs) for image segmentation with organ contour information and evaluated the performance of such augmentation on segmentation of lung fields, heart, and clavicles from chest X-ray images.

Methods

Three state-of-the-art CNNs were augmented, namely the UNet and LinkNet architecture with the ResNeXt feature extraction backbone, and the Tiramisu architecture with the DenseNet. All CNN architectures were trained on ground-truth segmentation masks and additionally on the corresponding contours. The contribution of such contour-based augmentation was evaluated against the contour-free architectures, and 20 existing algorithms for lung field segmentation.

Results

The proposed contour-aware segmentation improved the segmentation performance, and when compared against existing algorithms on the same publicly available database of 247 chest X-ray images, the UNet architecture with the ResNeXt50 encoder combined with the contour-aware approach resulted in the best overall segmentation performance, achieving a Jaccard overlap coefficient of 0.971, 0.933, and 0.903 for the lung fields, heart, and clavicles, respectively.

Conclusion

In this study, we proposed to augment CNN architectures for CXR segmentation with organ contour information and were able to significantly improve segmentation accuracy and outperform all existing solution using a public chest X-ray database.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Chen S, Zhong S, Yao L, Shang Y, Suzuki K (2016) Enhancement of chest radiographs obtained in the intensive care unit through bone suppression and consistent processing. Phys Med Biol 61:2283

  2. 2.

    Candemir S, Antani S (2019) A review on lung boundary detection in chest X-rays. Int J Comput Assist Radiol Surg 14:563–576

  3. 3.

    Miniati M, Coppini G, Monti S, Bottai M, Paterni M, Ferdeghini E (2011) Computer-aided recognition of emphysema on digital chest radiography. Eur J Radiol 80:169–175

  4. 4.

    Candemir S, Jaeger S, Lin W, Xue Z, Antani S, Thoma G (2016) Automatic heart localization and radiographic index computation in chest X-rays. In: SPIE medical imaging, 9785

  5. 5.

    Finnegan R, Dowling J, Koh E-S, Tang S, Otton J, Delaney G, Batumalai V, Luo C, Atluri P, Satchithanandha A (2019) Feasibility of multi-atlas cardiac segmentation from thoracic planning CT in a probabilistic framework. Phys Med Biol 64:085006

  6. 6.

    Gordienko Y, Gang P, Hui J, Zeng W, Kochura Y, Alienin O, Rokovyi O, Stirenko S (2018) Deep learning with lung segmentation and bone shadow exclusion techniques for chest x-ray analysis of lung cancer. In: International conference on ICAFS, pp 638–647

  7. 7.

    Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K, Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR Am J Roentgenol 174:71–4

  8. 8.

    van Ginneken B, Stegmann MB, Loog M (2006) Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal 10:19–40

  9. 9.

    Vittitoe NF, Vargas-Voracek R, Floyd CF Jr (1998) Identification of lung regions in chest radiographs using Markov random field modeling. Med Phys 25:976–85

  10. 10.

    Shi Z, Zhou P, He L, Nakamura T, Yao Q, Itoh H (2009) Lung segmentation in chest radiographs by means of Gaussian kernel-based FCM with spatial constraints. In: 6th international conference on fuzzy systems and knowledge discovery (FSKD). IEEE, pp 428–432

  11. 11.

    Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Computer vision and pattern recognition (CVPR). IEEE, pp 248–55

  12. 12.

    Wang C (2017) Segmentation of multiple structures in chest radiographs using multi-task fully convolutional networks. In: Scandinavian conference on image analysis (SCIA). Volume 10270 of Lecture notes in computer science. Springer, Cham, pp 282–289

  13. 13.

    Hwang S, Park S (2017) Accurate lung segmentation via network-wise training of convolutional networks. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, vol 10553 lecture notes in computer science. Springer, Cham, pp 92–99

  14. 14.

    Dai W, Dong N, Wang Z, Liang X, Zhang H, Xing EP (2018) SCAN: structure correcting adversarial network for organ segmentation in chest X-rays. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, volume 11045 of lecture notes in computer science. Springer, Berlin, pp 263–73

  15. 15.

    Bi L, Feng D, Kim J (2018) Dual-path adversarial learning for fully convolutional network (FCN)-based medical image segmentation. Vis Comput 34:1043–52

  16. 16.

    Novikov AA, Lenis D, Major D, Hladůvka J, Wimmer M, Bühler K (2018) Fully convolutional architectures for multi-class segmentation in chest radiographs. IEEE Trans Med Imaging 37:1865–76

  17. 17.

    Mittal A, Hooda R, Sofat S (2018) LF-SegNet: a fully convolutional encoder-decoder network for segmenting lung fields from chest radiographs. Wirel Pers Commun 101:511–29

  18. 18.

    Frid-Adar M, Ben-Cohen A, Amer R, Greenspan H (2018) Improving the segmentation of anatomical structures in chest radiographs using U-Net with an ImageNet pre-trained encoder. In: Image analysis for moving organ. breast, and thoracic images, volume 11040 of lecture notes in computer science. Springer, Cham, pp 159–68

  19. 19.

    Bonheur S, Stern D, Payer C, Pienn M, Olschewski H, Urschler M (2019) Matwo-capsnet: a multi-label semantic segmentation capsules network. MICCAI 2019:664–672

  20. 20.

    Ngo T A, Carneiro G (205) Lung segmentation in chest radiographs using distance regularized level set and deep-structured learning and inference. In: International conference on image processing (ICIP). IEEE, pp 2140–2143

  21. 21.

    Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A (2017) A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imaging 36:1550–60

  22. 22.

    Cui Y, Zhang G, Liu Z, Xiong Z, Hu J (2018) A deep learning algorithm for one-step contour aware nuclei segmentation of histopathological images. arXiv, p 1803.02786

  23. 23.

    Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention (MICCAI), volume 9351 of lecture notes in computer science. Springer, Cham, pp 234–241

  24. 24.

    Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Computer vision and pattern recognition (CVPR). IEEE, pp 5987–5995

  25. 25.

    Chaurasia A, Culurciello E (2017) LinkNet: exploiting encoder representations for efficient semantic segmentation. In: Visual communications and image processing (VCIP). IEEE

  26. 26.

    Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation. In: Computer vision and pattern recognition workshops (CVPRW). IEEE, pp 1175–1783

  27. 27.

    Huang G, Liu Z, van der Maaten L, Weinberger K Q (2017) Densely connected convolutional networks. In: Computer vision and pattern recognition (CVPR). IEEE, pp 2261–2269

  28. 28.

    Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Computer vision and pattern recognition (CVPR). IEEE, pp 3431–3440

  29. 29.

    Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) UNet++: a nested U-Net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support (DLMIA / ML-CDS), volume 11045 of lecture notes in computer science. Springer, Cham, pp 3–11

  30. 30.

    Wang CW, Huang CT, Lee JH, Li CH, Chang SW, Siao MJ, Lai TM, Ibragimov B, Vrtovec T, Ronneberger O, Fischer P, Cootes TF, Lindner C (2016) A benchmark for comparison of dental radiography analysis algorithms. Med Image Anal 31:63–76

  31. 31.

    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Computer vision and pattern recognition (CVPR). IEEE, pp 770–788

  32. 32.

    Shaikh M, Anand G, Acharya G, Amrutkar A, Varghese A, Krishnamurthi G (2018) Brain tumor segmentation using dense fully convolutional neural network. Brainlesion: Glioma. In: Multiple sclerosis, stroke and traumatic brain injuries (BrainLes), volume 10670 of lecture notes in computer Science. Springer, Cham, pp 309–319

  33. 33.

    Xu C, Xu L, Brahm G, Zhang, Li S (2018) Mutgan: simultaneous segmentation and quantification of myocardial infarction without contrast agents via joint adversarial learning. In: Conference on Medical Image Computing and Computer-Assisted Intervention, pp 525–534, 2018

  34. 34.

    Arik SO, Ibragimov B, Xing L (2017) Fully automated quantitative cephalometry using convolutional neural networks. J Med Imaging 4:014501

  35. 35.

    Milletari F, Navab N, Ahmadi S-A (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 3D vision (3DV). IEEE, pp 565–571

  36. 36.

    DSBowl2018 (2018) 2018 data science bowl. https://www.kaggle.com/c/data-science-bowl-2018/overview. Accessed 15 Oct 2019

  37. 37.

    KiTS2019 (2019) 2019 kidney and kidney tumor segmentation challenge. https://kits19.grand-challenge.org/. Accessed 15 Oct 2019

  38. 38.

    Isensee F, Petersen J, Kohl S A A, Jager PF, Maier-Hein KH (2019) Nnu-net: breaking the spell on successful medical image segmentation. arXiv:1904.08128

  39. 39.

    Kingma DP, Ba LJ (2015) Adam: a method for stochastic optimization. In: 3rd International conference on learning representations (ICLR)

  40. 40.

    Shi Y, Qi F, Xue Z, Chen L, Ito K, Matsuo H, Shen D (2008) Segmenting lung fields in serial chest radiographs using both population-based and patient-specific shape statistics. IEEE Trans Med Imaging 27:481–94

  41. 41.

    Li X, Luo S, Hu Q, Li J, Wang D, Chiong F (2016) Automatic lung field segmentation in X-ray radiographs using statistical shape and appearance models. J Med Imaging Health Inform 6:338–48

  42. 42.

    Dawoud A (2010) Fusing shape information in lung segmentation in chest radiographs. In: Image analysis and recognition—ICIAR 2010, volume 6112 of lecture notes in computer science. Springer, Berlin, Heidelberg, pp 70–78

  43. 43.

    Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Zhiyun X, Karargyris A, Antani S, Thoma G, McDonald CJ (2014) Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 33:577–90

  44. 44.

    Ibragimov B, Likar B, Pernuš F, Vrtovec T (2012) A game-theorteic framework for landmark-based image segmentation. IEEE Trans. Med. Imaging 31:1761–76

  45. 45.

    Seghers D, Loeckx D, Maes F, Vandermeulen D, Suetens P (2007) Minimal shape and intensity cost path segmentation. IEEE Trans Med Imaging 26:1115–29

  46. 46.

    Wu G, Zhang X, Luo S, Hu Q (2015) Lung segmentation based on customized active shape model from digital radiography chest images. J Med Imaging Health Inform 5:184–91

  47. 47.

    Yang W, Liu Y, Lin L, Yun Z, Lu Z, Feng Q, Chen W (2018) Lung field segmentation in chest radiographs from boundary maps by a structured edge detector. IEEE J Biomed Health Inform 22:842–51

  48. 48.

    Ibragimov B, Likar B, Pernuš F, Vrtovec T (2016) Accurate landmark-based segmentation by incorporating landmark misdetections. In: 13th international symposium on biomedical imaging (ISBI). IEEE, pp 1072–1075

  49. 49.

    Chondro P, Yao C-Y, Ruan S-J, Chien L-C (2018) Low order adaptive region growing for lung segmentation on plain chest radiographs. Neurocomputing 275:1002–11

  50. 50.

    Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of CVPR 2017

  51. 51.

    RSNA (2018) Kaggle pneumonia detection challenge. https://www.kaggle.com/c/rsna-pneumonia-detection-challenge. Accessed 15 Oct 2019

  52. 52.

    Karargyris A, Siegelman J, Tzortzis D, Jaeger S, Candemir S, Xue Z, Santosh K, Vajda S, Antani S, Folio L, Thoma G (2016) Combination of texture and shape features to detect pulmonary abnormalities in digital chest X-rays. Int J Comput Assist Radiol Surg 11:99–106

  53. 53.

    Zheng G, Chu C, Belavy DL, Ibragimov B, Korez R, Vrtovec T, Hutt H, Everson R, Meakin J, Andrade IL, Glocker B, Chen H, Qi Dou Q, Heng PA, Wang C, Forsberg D, Neubert A, Fripp J, Urschler M, Stern D, Wimmer M, Novikov AA, Cheng H, Armbrecht G, Felsenberg D, Li S (2017) Evaluation and comparison of 3d intervertebral disc localization and segmentation methods for 3d t2mr data: a grand challenge. Med Image Anal 35:327–344

  54. 54.

    Cong J, Zheng Y, Xue W, Cao B, Li S (2019) Ma-shape: modality adaptation shape regression for left ventricle segmentation on mixed MR and CT images. IEEE Access 7:16584–16593

  55. 55.

    Kholiavchenko M (2019) https://github.com/dirtmaxim/ca-ml-organ-segmentation. Accessed 15 Oct 2019

  56. 56.

    Jeager S, Candemir S, Antani S, Wang Y, Lu P, G Thoma (2014) Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 4:475–477

Download references

Acknowledgements

This work was supported by the Russian Science Foundation (Grant No. 18-71-10072), Russian Foundation for Basic Research (Grant No. 18-47-160015), and Slovenian Research Agency—ARRS (Grant No. P2-0232). Due to the requirements of the Russian Science Foundation, we explicitly state that the methodology development and implementation were solely supported by the Russian Science Foundation (Grant No. 18-71-10072).

Author information

Correspondence to B. Ibragimov.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This study has not been supported by any industrial company and does not serve to promote any commercial product. Anonymized publicly available databases of CXR were used in the conducted experiments.

Informed consent

For this type of study, formal consent is not required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kholiavchenko, M., Sirazitdinov, I., Kubrak, K. et al. Contour-aware multi-label chest X-ray organ segmentation. Int J CARS 15, 425–436 (2020). https://doi.org/10.1007/s11548-019-02115-9

Download citation

Keywords

  • Image segmentation
  • Convolutional neural networks
  • Deep learning architectures
  • Chest X-ray (CXR) images
  • JSRT database