Skip to main content
SpringerLink
Log in

Deep architectures for high-resolution multi-organ chest X-ray image segmentation

  • Recent Advances in Deep Learning for Medical Image Processing
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Chest X-ray images (CXRs) are the most common radiological examination tool for screening and diagnosis of cardiac and pulmonary diseases. The automatic segmentation of anatomical structures in CXRs is critical for many clinical applications. However, existing deep models work on severely down-sampled images (commonly \(256\times 256\) pixels), reducing the quality of the contours of the resulting segmentation and negatively affecting the possibilities of such methods to be effectively used in a real environment. In this paper, we study multi-organ (clavicles, lungs, and hearts) segmentation, one of the most important problems in semantic understanding of CXRs. We completely avoid down-sampling in images up to \(1024\times 1024\) (as in the JSRT dataset), and we diminish its impact in higher resolutions via network architecture simplification without a significant loss in the accuracy. To do so, we propose four different convolutional models by introducing structural changes to the baselines employed (U-Net and InvertedNet) as well as by integrating several techniques barely used by CXRs segmentation algorithms, such as instance normalization and atrous convolution. We also compare single-class and multi-class strategies to elucidate which approach is the most convenient for this problem. Our best proposal, X-Net+, outperforms nine state-of-the-art methods on clavicles and lungs obtaining a Dice similarity coefficient of 0.938 and 0.978, respectively, employing a tenfold cross-validation protocol. The same architecture yields comparable results to the state of the art in heart segmentation with a Dice value of 0.938. Finally, its reduced version, RX-Net+, obtains similar results but with a significant reduction in memory usage and training time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Search performed the September 8, 2018, using the keywords [TITLE-ABS-KEY (chest AND X-ray AND segmentation) OR TITLE-ABS-KEY (chest AND radiograph AND segmentation) AND NOT TITLE-ABS-KEY (computed AND tomography)].

  2. According to the International Agency for Research on Cancer, lung cancer was the most common cause of cancer death in 2015 with 1.69 million deaths.

References

  1. Staffan S, Ostensen H, Pettersson H (2003) The WHO manual of diagnostic imaging: radiographic technique and projections, vol 2. World Health Organization, Geneva

    Google Scholar 

  2. Daffner RH, Hartman M (2013) Clinical radiology: the essentials. Lippincott Williams & Wilkins, Philadelphia

    Google Scholar 

  3. Rigby D-M, Hacking L (2018) Interpreting the chest radiograph. Anaesth Intensive Care 19(2):50–54

    Google Scholar 

  4. Van Ginneken B, Ter Haar Romeny BM, Viergever MA (2001) Computer-aided diagnosis in chest radiography: a survey. IEEE Trans Med Imaging 20(12):1228–1241

    Google Scholar 

  5. NHS England (2016) Diagnostic imaging dataset annual statistical release 2015/2016

  6. Laserson J, Lantsman CD, Cohen-Sfady M, Tamir I, Goz E, Brestel C, Bar S, Atar M, Elnekave E (2018) TextRay: mining clinical reports to gain a broad understanding of chest X-rays. In: MICCAI, pp 553–561

  7. Schilham AMR, van Ginneken B, Loog M (2006) A computer-aided diagnosis system for detection of lung nodules in chest radiographs with an evaluation on a public database. Med Image Anal 10(2):247–258

    Google Scholar 

  8. Mittal A, Hooda R, Sofat S (2017) Lung field segmentation in chest radiographs: a historical review, current status, and expectations from deep learning. IET Image Process 11(11):937–952

    Google Scholar 

  9. Tsakok MT, Gleeson FV (2018) The chest radiograph in heart disease. Medicine 46(8):453–457

    Google Scholar 

  10. Kapoor S, Tiwari A, Kapoor S (2008) Primary tumours and tumorous lesions of clavicle. Int Orthop 32(6):829

    Google Scholar 

  11. Gómez O, Ibáñez O, Valsecchi A, Cordón O, Kahana T (2018) 3D–2D silhouette-based image registration for comparative radiography-based forensic identification. Pattern Recognit 83:469–480

    Google Scholar 

  12. Bruno MA, Walker EA, Abujudeh HH (2015) Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 35(6):1668–1676

    Google Scholar 

  13. Robinson PJ, Wilson D, Coral A, Murphy A, Verow P (1999) Variation between experienced observers in the interpretation of accident and emergency radiographs. Br J Radiol 72(856):323–330

    Google Scholar 

  14. Brady A, Laoide RÓ, McCarthy P, McDermott R (2012) Discrepancy and error in radiology: concepts, causes and consequences. Ulst Med J 81(1):3

    Google Scholar 

  15. Toriwaki J-I, Suenaga Y, Negoro T, Fukumura T (1973) Pattern recognition of chest X-ray images. Comput Vis Graph 2(3):252–271

    Google Scholar 

  16. Wechsler H, Sklansky J (1977) Finding the rib cage in chest radiographs. Pattern Recognit 9(1):21–30

    Google Scholar 

  17. Zhu Y, Prummer S, Wang P, Chen T, Comaniciu D, Ostermeier M (2009) Dynamic layer separation for coronary DSA and enhancement in fluoroscopic sequences. In: MICCAI, pp 877–884

  18. Ronneberger O, Fischer P, Brox T(2015) U-Net: convolutional networks for biomedical image segmentation. In: MICCAI, pp 234–241

  19. Ter Bram Van Ginneken BM, Romeny H, Viergever MA (2001) Computer-aided diagnosis in chest radiography: a survey. IEEE Trans Med Imaging 20(12):1228–1241

    Google Scholar 

  20. Shao Y, Gao Y, Guo Y, Shi Y, Yang X, Shen D (2014) Hierarchical lung field segmentation with joint shape and appearance sparse learning. IEEE Trans Med Imaging 33(9):1761–1780

    Google Scholar 

  21. Yang W, Liu Y, Lin L, Yun Z, Zhentai L, Feng Q, Chen W (2018) Lung field segmentation in chest radiographs from boundary maps by a structured edge detector. IEEE J Biomed Health Inform 22(3):842–851

    Google Scholar 

  22. Boussaid H, Kokkinos I, Paragios N (2014) Discriminative learning of deformable contour models. In: ISBI, IEEE, pp 624–628

  23. Hogeweg L, Sánchez CI, de Jong PA, Maduskar P, van Ginneken B (2012) Clavicle segmentation in chest radiographs. Med Image Anal 16(8):1490–1502

    Google Scholar 

  24. Novikov AA, Lenis D, Major D, Hladuvka J, Wimmer M, Bühler K (2018) Fully convolutional architectures for multiclass segmentation in chest radiographs. IEEE Trans Med Imaging 37(8):1865–1876

    Google Scholar 

  25. Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K, Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR Am J Roentgenol 174(1):71–74

    Google Scholar 

  26. Van Ginneken B, Stegmann MB, Loog M (2006) Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal 10(1):19–40

    Google Scholar 

  27. Ulyanov D, Vedaldi A, Lempitsky VS (2016) Instance normalization: the missing ingredient for fast stylization. arxiv:1607.08022

  28. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2016) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. arxiv:1606.00915,

  29. Withey DJ, Koles ZJ (2007) Medical image segmentation: methods and software. In: NFSI-ICFBI, pp 140–143

  30. Waseem Khan M (2014) A survey: image segmentation techniques. Int J Future Comput Commun 3(2):89

    Google Scholar 

  31. Smistad E, Falch TL, Bozorgi M, Elster AC, Lindseth F (2015) Medical image segmentation on GPUs: a comprehensive review. Med Image Anal 20(1):1–18

    Google Scholar 

  32. Mesejo P, Ibáñez O, Cordón O, Cagnoni S (2016) A survey on image segmentation using metaheuristic-based deformable models: state of the art and critical analysis. Appl Soft Comput 44:1–29

    Google Scholar 

  33. Cabezas M, Oliver A, Lladó X, Freixenet J, Cuadra MB (2011) A review of atlas-based segmentation for magnetic resonance brain images. Comput Methods Programs Biomed 104(3):e158–e177

    Google Scholar 

  34. Peng B, Zhang L, Zhang D (2013) A survey of graph theoretical approaches to image segmentation. Pattern Recognit 46(3):1020–1038

    Google Scholar 

  35. Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J (2017) A review on deep learning techniques applied to semantic segmentation. arXiv:1704.06857

  36. Chondro P, Yao C-Y, Ruan S-J, Chien L-C (2018) Low order adaptive region growing for lung segmentation on plain chest radiographs. Neurocomputing 275:1002–1011

    Google Scholar 

  37. Siang Tan K, Mat Isa NA (2011) Color image segmentation using histogram thresholding: fuzzy C-means hybrid approach. Pattern Recognit 44(1):1–15

    MATH  Google Scholar 

  38. Bi L, Feng D, Kim J (2018) Dual-path adversarial learning for fully convolutional network (FCN)-based medical image segmentation. Vis Comput 34(6):1043–1052

    Google Scholar 

  39. Goh SK, Abbass HA, Tan KC, Al-Mamun A, Thakor N, Bezerianos A, Li J (2018) Spatio-spectral representation learning for electroencephalographic gait-pattern classification. IEEE Trans Neural Syst Rehabilit Eng 26(9):1858–1867

    Google Scholar 

  40. Pang S, José J, del Coz Z, Yu OL, Díez J (2018) Deep learning and preference learning for object tracking: a combined approach. Neural Process Lett 47(3):859–876

    Google Scholar 

  41. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  42. Li J, Struzik Z, Zhang L, Cichocki A (2015) Feature learning from incomplete EEG with denoising autoencoder. Neurocomputing 165:23–31

    Google Scholar 

  43. Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587

  44. Wang C (2017) Segmentation of multiple structures in chest radiographs using multi-task fully convolutional networks. In: Sharma P, Bianchi FM (eds) SCIA, pp 282–289

  45. Mittal A, Hooda R, Sofat S (2018) LF-SegNet: a fully convolutional encoder–decoder network for segmenting lung fields from chest radiographs. Wirel Pers Commun 101(1):511–529

    Google Scholar 

  46. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. ICML 1:448–456

    Google Scholar 

  47. Jaeger S, Candemir S, Antani S, Wáng Y-XJ, Pu-Xuan L, Thoma G (2014) Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 4(6):475

    Google Scholar 

  48. Hooda R, Mittal A, Sofat S (2018) An efficient variant of fully-convolutional network for segmenting lung fields from chest radiographs. Wirel Pers Commun 101(3):1559–1579

    Google Scholar 

  49. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  50. Sørensen T (1948) A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on Danish commons. K Dan Vidensk Selsk 5:1–34

    Google Scholar 

  51. Denil M, Shakibi B, Dinh L, De Freitas N et al (2013) Predicting parameters in deep learning. In: Advances in neural information processing systems, pp 2148–2156

  52. Han S, Liu X, Mao H, Pu J, Pedram A, Horowitz MA, Dally WJ (2016) EIE: efficient inference engine on compressed deep neural network. In: ISCA, pp 243–254

  53. Kim Y-D, Park E, Yoo S, Choi T, Yang L, Shin D (2015) Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv:1511.06530

  54. Baiying L, Shan H, Ran L, Cheng B, Hang L, Yi-Hong C, Jie-Zhi C (2018) Segmentation of breast anatomy for automated whole breast ultrasound images with boundary regularized convolutional encoder–decoder network. Neurocomputing 321:178–186

    Google Scholar 

  55. Liu J, Cai J, Chellamuthu K, Bagheri M, Lu L, Summers RM (2018) Cascaded coarse-to-fine convolutional neural networks for pericardial effusion localization and segmentation on ct scans. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 1092–1095

  56. Gong K, Liang X, Li Y, Chen Y, Yang M, Lin L (2018) Instance-level human parsing via part grouping network. arXiv:1808.00157

  57. Grana C, Borghesani D, Cucchiara R (2010) Optimized block-based connected components labeling with decision trees. IEEE Trans Image Process 19(6):1596–1609

    MathSciNet  MATH  Google Scholar 

  58. Simonyan K, Vedaldi A, Zisserman A (2013) Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv:1312.6034

  59. Litjens G, Sánchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, Kaa CH-VD, Bult P, Van Ginneken B, Van Der Laak J (2016) Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 6:26286

    Google Scholar 

  60. Qi CR, Hao S, Mo K, Guibas LJ (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. Proc Comput Vis Pattern Recognit IEEE 1(2):4

    Google Scholar 

  61. Beauchemin M, Thomson KPB, Edwards G (1998) On the Hausdorff distance used for the evaluation of segmentation results. Can J Remote Sens 24(1):3–8

    Google Scholar 

  62. Jaccard P (1912) The distribution of the flora in the alpine zone. 1. New Phytol 11(2):37–50

    Google Scholar 

  63. Lathuilière S, Mesejo P, Alameda-Pineda X, Radu H (2018) A comprehensive analysis of deep regression. arXiv:1803.08450

  64. Gehan EA (1965) A generalized Wilcoxon test for comparing arbitrarily singly-censored samples. Biometrika 52(1–2):203–223

    MathSciNet  MATH  Google Scholar 

  65. Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. In: ECCV, pp 94–108

  66. Wu Z, Valentini-Botinhao C, Watts O, King S (2015) Deep neural networks employing multi-task learning and stacked bottleneck features for speech synthesis. In: ICASSP, pp 4460–4464

  67. Myles H, Douglas AW (1999) Nonparametric statistical methods. Wiley, New York

    MATH  Google Scholar 

  68. Xiong J, Shao Y, Ma J, Ren Y, Wang Q, Zhao J (2017) Lung field segmentation using weighted sparse shape composition with robust initialization. Med Phys 44(11):5916–5929

    Google Scholar 

  69. Milletari F, Navab N, Ahmadi S-A (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 3DV, IEEE, pp 565–571

  70. Luo J, Zhang H, Zhou H, Xie C, Wu J, Lin W (2019) ThiNet: pruning CNN filters for a thinner net. IEEE Trans Pattern Anal Mach Intell 41(10):2525–2538

    Google Scholar 

Download references

Acknowledgements

This research was supported by the Spanish Ministerio de Economía y Competividad under the NEWSOCO project [Grant Number TIN2015-67661-P], including European Development Regional Funds (EDRF). This work was also supported by the Spanish Ministry of Science, Innovation and Universities, and European Regional Development Funds (ERDF) under grant EXASOCO (PGC2018-101216-B-I00). Mr. Gómez’s work was supported by Spanish MECD FPU Grant [Grant Number FPU14/02380]. Pablo Mesejo is funded by the European Commission H2020-MSCA-IF-2016 through the Skeleton-ID Marie Curie Individual Fellowship [Grant Number 746592]. We acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research.

Author information

Authors and Affiliations

  1. Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain

    Oscar Gómez, Pablo Mesejo, Oscar Ibáñez, Andrea Valsecchi & Oscar Cordón

  2. Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain

    Oscar Gómez, Pablo Mesejo, Oscar Ibáñez, Andrea Valsecchi & Oscar Cordón

  3. Perception Team, Inria Grenoble Rhône-Alpes, Grenoble, France

    Pablo Mesejo

  4. Panacea Cooperative Research, Ponferrada, Spain

    Pablo Mesejo, Oscar Ibáñez & Andrea Valsecchi

Authors
  1. Oscar Gómez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pablo Mesejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oscar Ibáñez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea Valsecchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Oscar Cordón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oscar Gómez.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gómez, O., Mesejo, P., Ibáñez, O. et al. Deep architectures for high-resolution multi-organ chest X-ray image segmentation. Neural Comput & Applic 32, 15949–15963 (2020). https://doi.org/10.1007/s00521-019-04532-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04532-y

Keywords

Search

Navigation