Robust Segmentation of Challenging Lungs in CT Using Multi-stage Learning and Level Set Optimization

  • Neil BirkbeckEmail author
  • Michal Sofka
  • Timo Kohlberger
  • Jingdan Zhang
  • Jens Wetzl
  • Jens Kaftan
  • S. Kevin Zhou


Automatic segmentation of lung tissue in thoracic CT scans is useful for diagnosis and treatment planning of pulmonary diseases. Unlike healthy lung tissue that is easily identifiable in CT scans, diseased lung parenchyma is hard to segment automatically due to its higher attenuation, inhomogeneous appearance, and inconsistent texture. We overcome these challenges through a multi-layer machine learning approach that exploits geometric structures both within and outside the lung (e.g., ribs, spine). In the coarsest layer, a set of stable landmarks on the surface of the lung are detected through a hierarchical detection network (HDN) that is trained on hundreds of annotated CT volumes. These landmarks are used to robustly initialize a coarse statistical model of the lung shape. Subsequently, a region-dependent boundary refinement uses a discriminative appearance classifier to refine the surface, and finally a region-driven level set refinement is used to extract the fine-scale detail. Through this approach we demonstrate robustness to a variety of lung pathologies.


Interstitial Lung Disease Input Volume Landmark Point Healthy Lung Statistical Shape Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abe Y, Hanai K, Nakano M, Ohkubo Y, Hasizume T, Kakizaki T, Nakamura M, Niki N, Eguchi K, Fujino T, Moriyama N (2005) A computer-aided diagnosis (CAD) system in lung cancer screening with computed tomography. Anticancer Res 25:483–488Google Scholar
  2. 2.
    Adams H, Bernard M, McConnochie K (1991) An appraisal of CT pulmonary density mapping in normal subjects. Clin Radiol 43(4):238–42CrossRefGoogle Scholar
  3. 3.
    Armato III S, Sensakovic W (2004) Automated lung segmentation for thoracic CT: Impact on computer-aided diagnosis. Acad Radiol 11(9):1011–1021CrossRefGoogle Scholar
  4. 4.
    Brown MS, McNitt-Gray MF, Mankovich NJ, Goldin JG, Hiller J, Wilson LS, Aberle DR (1997) Method for segmenting chest CT image data using an anatomical model: preliminary results. IEEE Trans Med Imag 16(6):828–839CrossRefGoogle Scholar
  5. 5.
    Chan T, Vese L (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277CrossRefzbMATHGoogle Scholar
  6. 6.
    Cootes T, Hill A, Taylor C, Haslam J (1994) Use of active shape models for locating structures in medical images. Image Vis Comput 12(6):355–365CrossRefGoogle Scholar
  7. 7.
    Cremers D, Rousson M, Deriche R (2007) A review of statistical approaches to level set segmentation: Integrating color, texture, motion and shape. Int J Comput Vis 72(2):195–215CrossRefGoogle Scholar
  8. 8.
    Haider C, Bartholmai B, Holmes D, Camp J, Robb R (2005) Quantitative characterization of lung disease. Comput Med Imag Graph 29(7):555–563CrossRefGoogle Scholar
  9. 9.
    Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern 3(6):610–621CrossRefGoogle Scholar
  10. 10.
    Hedlund L, Anderson R, Goulding P, Beck J, Effmann E, Putman C (1982) Two methods for isolating the lung area of a CT scan for density information. Radiology 144(2):353–357Google Scholar
  11. 11.
    Hu S, Hoffman E, Reinhardt J (2002) Automatic lung segmentation for accurate quantitation of volumetric X-ray CT images. IEEE Trans Med Imag 20(6):490–498CrossRefGoogle Scholar
  12. 12.
    Hua P, Song Q, Sonka M, Hoffman E, Reinhardt J (2011) Segmentation of pathological and diseased lung tissue in CT images using a graph-search algorithm. International Symposium on Biomedical Imaging, Chicago, IL, March 2011Google Scholar
  13. 13.
    International Consensus Statement (2000) Idiopathic pulmonary fibrosis: diagnosis and treatment. American Thoracic Society (ATS) and the European Respiratory Society (ERS), vol 161, pp 646–664Google Scholar
  14. 14.
    International Consensus Statement (2002) American thoracic society/European respiratory society international multidisciplinary consensus classification of the idiopathic interstitial pneumonias. Am J Respir Crit Care Med 165:277–304CrossRefGoogle Scholar
  15. 15.
    Kanazawa K, Kawata Y, Niki N, Satoh H, Ohmatsu H, Kakinuma R, Kaneko M, Moriyama N, Eguchi K (1998) Computer-aided diagnosis for pulmonary nodules based on helical CT images. Comput Med Imag Graph 22(2):157–167CrossRefGoogle Scholar
  16. 16.
    King TE (2005) Clinical advances in the diagnosis and therapy of the interstitial lung diseases. Am J Respir Crit Care Med 172(3):268–279 (2005)Google Scholar
  17. 17.
    Kockelkorn T, van Rikxoort E, Grutters J, van Ginneken B (2010) Interactive lung segmentation in CT scans with severe abnormalities. In: International Symposium on Biomedical Imaging, Rotterdam, Netherlands, pp 564–567, 14–17 April 2010Google Scholar
  18. 18.
    Kohlberger T, Sofka M, Zhang J, Birkbeck N, Wetzl J, Kaftan J, Declerck J, Zhou S (2011) Automatic multi-organ segmentation using learning-based segmentation and level set optimization. In: Mediacl Image Computing and Computer-assisted Iintervention 2011Google Scholar
  19. 19.
    Kohlberger T, Uzunbaş MG, Alvino C, Kadir T, Slosman DO, Funka-Lea G (2009) Organ segmentation with level sets using local shape and appearance priors. In: Proceedings of MICCAI: Part II. pp 34–42. MICCAI ’09Google Scholar
  20. 20.
    Korfiatis P, Kalogeropoulou C, Karahaliou A, Kazantzi A, Skiadopoulos S, Costaridou L (2008) Texture classification-based segmentation of lung affected by interstitial pneumonia in high-resolution CT. Med phys 35:5290Google Scholar
  21. 21.
    Ling H, Zhou SK, Zheng Y, Georgescu B, Suehling M, Comaniciu D (2008) Hierarchical, learning-based automatic liver segmentation. In: CVPR, pp 1–8, IEEE Computer Society, Los Alamitos, CA, USA, 2008Google Scholar
  22. 22.
    Prasad M, Brown M, Ahmad S, Abtin F, Allen J, da Costa I, Kim H, McNitt-Gray M, Goldin J (2008) Automatic segmentation of lung parenchyma in the presence of diseases based on curvature of ribs. Acad. Radiol. 15(9):1173–1180CrossRefGoogle Scholar
  23. 23.
    Pu J, Roos J, Yi C, Napel S, Rubin G, Paik D (2008) Adaptive border marching algorithm: Automatic lung segmentation on chest CT images. Comput Med Imag Graph 32(6):452–462CrossRefGoogle Scholar
  24. 24.
    Reeves AP, Kostis WJ (2000) Computer-aided diagnosis for lung cancer. Radiol Clin North Am. 38(3):497–509CrossRefGoogle Scholar
  25. 25.
    Robinson P, Krell L (1979) Pulmonary tissue attenuation with computed tomography: comparison of inspiration and expiration scans. J Comput Assist Tomogr 3(6):740–748Google Scholar
  26. 26.
    Silva A, Silva JS, Santos BS, Ferreira C (2001) Fast pulmonary contour extraction in X-ray CT images: A methodology and quality assessment. In: SPIE Conf on Medical Imaging: Physiology and Function from Multidimensional Images. vol 4321, pp 216–224Google Scholar
  27. 27.
    Sluimer I, Schilham A, Prokop M, van Ginneken B (2006) Computer analysis of computed tomography scans of the lung: a survey. IEEE Trans Med Imag 25(4):385–405CrossRefGoogle Scholar
  28. 28.
    Sluimer I, Prokop M, van Ginneken B (2005) Toward automated segmentation of the pathological lung in CT. IEEE Trans Med Imag 24(8):1025–1038CrossRefGoogle Scholar
  29. 29.
    Sofka M, Ralovich K, Birkbeck N, Zhang J, Zhou S (2011) Integrated detection network (IDN) for pose and boundary estimation in medical images. In: Proceedings of the 8th International Symposium on Biomedical Imaging (ISBI 2011). Chicago, IL, 30 March–2 April 2011Google Scholar
  30. 30.
    Sofka M, Wetzl J, Birkbeck N, Zhang J, Kohlberger T, Kaftan J, Declerck J, Zhou S (2011) Multi-stage learning for robust lung segmentation in challenging CT volumes. In: Medical Image Computing and Computer-Assisted Intervention, MICCAI ’11, Toronto, Canada, 18–22 September 2011Google Scholar
  31. 31.
    Sofka M, Zhang J, Zhou S, Comaniciu D: Multiple object detection by sequential Monte Carlo and hierarchical detection network. In: IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, 13–18 June 2010Google Scholar
  32. 32.
    Sun S, McLennan G, Hoffman EA, Beichel R (2010) Model-based segmentation of pathological lungs in volumetric ct data. In: The Third International Workshop on Pulmonary Image Analysis, Beijing, 20 September 2010Google Scholar
  33. 33.
    Tu Z (2005) Probabilistic boosting-tree: learning discriminative models for classification, recognition, and clustering. In: International Conference on Computer Vision, ICCV 2005, pp 1589–1596, vol 2,Beijing, 17–21 October 2005Google Scholar
  34. 34.
    Uchiyama Y, Katsuragawa S, Abe H, Shiraishi J, Li F, Li Q, Zhang CT, Suzuki K, Doi K (2003) Quantitative computerized analysis of diffuse lung disease in high-resolution computed tomography. Med Phys 30:2440–2454CrossRefGoogle Scholar
  35. 35.
    Ukil S, Reinhardt J (2009) Anatomy-guided lung lobe segmentation in X-ray CT images. IEEE Trans Med Image 28(2):202–214CrossRefGoogle Scholar
  36. 36.
    Ukil S, Reinhardt JM (2005) Smoothing lung segmentation surfaces in three-dimensional X-ray CT images using anatomic guidance. Acad. Radiol 12(12):1502–1511CrossRefGoogle Scholar
  37. 37.
    Uppaluri R, Hoffman EA, Sonka M, Hartley PG, Hunninghake GW, McLennan G (1999) Computer recognition of regional lung disease patterns. Am J Respir Crit Care Med 160(2):648–654CrossRefGoogle Scholar
  38. 38.
    Uppaluri R, Mitsa T, Sonka M, Hoffman EA, Mclennan G (1997) Quantification of pulmonary emphysema from lung computed tomography images. Am J Respir Crit Care Med 156:248–254CrossRefGoogle Scholar
  39. 39.
    Wang J, Li F, Li Q (2009) Automated segmentation of lungs with severe interstitial lung disease in CT. Med phys 36:4592Google Scholar
  40. 40.
    Wang J, Lib F, Doib K, Lia Q (2009) A novel scheme for detection of diffuse lung disease in MDCT by use of statistical texture features. In: SPIE 7260, 27 February 2009Google Scholar
  41. 41.
    Zhang L, Hoffman E, Reinhardt J (2005) Atlas-driven lung lobe segmentation in volumetric X-ray CT images. IEEE Trans Med Image 25(1):1–16CrossRefzbMATHGoogle Scholar
  42. 42.
    Zheng Y, Barbu A, Georgescu B, Scheuering M, Comaniciu D (2007) Fast automatic heart chamber segmentation from 3D CT data using marginal space learning and steerable features. In: IEEE International Conference on Computer Vision, ICCV 2007, pp 1–8, Rio de Janeiro, 14–21 October 2007Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Neil Birkbeck
    • 1
    Email author
  • Michal Sofka
    • 1
  • Timo Kohlberger
    • 1
  • Jingdan Zhang
    • 1
  • Jens Wetzl
    • 1
  • Jens Kaftan
    • 2
  • S. Kevin Zhou
    • 1
  1. 1.Siemens Corp., Corporate TechnologyPlainsboroUSA
  2. 2.Siemens Molecular ImagingOxfordUK

Personalised recommendations