Progressive and Multi-path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images

  • Adam P. Harrison
  • Ziyue XuEmail author
  • Kevin George
  • Le Lu
  • Ronald M. Summers
  • Daniel J. Mollura
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


Pathological lung segmentation (PLS) is an important, yet challenging, medical image application due to the wide variability of pathological lung appearance and shape. Because PLS is often a pre-requisite for other imaging analytics, methodological simplicity and generality are key factors in usability. Along those lines, we present a bottom-up deep-learning based approach that is expressive enough to handle variations in appearance, while remaining unaffected by any variations in shape. We incorporate the deeply supervised learning framework, but enhance it with a simple, yet effective, progressive multi-path scheme, which more reliably merges outputs from different network stages. The result is a deep model able to produce finer detailed masks, which we call progressive holistically-nested networks (P-HNNs). Using extensive cross-validation, our method is tested on a multi-institutional dataset comprising 929 CT scans (848 publicly available), of pathological lungs, reporting mean dice scores of 0.985 and demonstrating significant qualitative and quantitative improvements over state-of-the art approaches.


Progressive and multi-path convolutional neural networks Holistically nested neural networks Pathological lung segmentation 


  1. 1.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_49 CrossRefGoogle Scholar
  2. 2.
    Depeursinge, A., Vargas, A., Platon, A., Geissbuhler, A., Poletti, P.A.: Mller, H.: Building a reference multimedia database for interstitial lung diseases. Comput. Med. Imaging Graph. 36(3), 227–238 (2012)CrossRefGoogle Scholar
  3. 3.
    El-Baz, A., Beache, G.M., Gimel’farb, G.L., Suzuki, K., Okada, K., Elnakib, A., Soliman, A., Abdollahi, B.: Computer-aided diagnosis systems for lung cancer: Challenges and methodologies. Int. J. Biomed. Imaging 2013, 1–46 (2013)Google Scholar
  4. 4.
    Gill, G., Beichel, R.R.: Segmentation of lungs with interstitial lung disease in CT Scans: A TV-L1 based texture analysis approach. In: Bebis, G., et al. (eds.) ISVC 2014. LNCS, vol. 8887, pp. 511–520. Springer, Cham (2014). doi: 10.1007/978-3-319-14249-4_48 CrossRefGoogle Scholar
  5. 5.
    Hosseini-Asl, E., Zurada, J.M., Gimelfarb, G., El-Baz, A.: 3-d lung segmentation by incremental constrained nonnegative matrix factorization. IEEE Trans. Biomed. Eng. 63(5), 952–963 (2016)CrossRefGoogle Scholar
  6. 6.
    Karwoski, R.A., Bartholmai, B., Zavaletta, V.A., Holmes, D., Robb, R.A.: Processing of ct images for analysis of diffuse lung disease in the lung tissue research consortium. In: Proceedings of SPIE 6916, Medical Imaging 2008: Physiology, Function, and Structure from Medical Images (2008)Google Scholar
  7. 7.
    Lin, G., Milan, A., Shen, C., Reid, I.: RefineNet: Multi-path refinement networks for high-resolution semantic segmentation, November 2016. arXiv:1611.06612
  8. 8.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE CVPR, pp. 3431–3440 (2015)Google Scholar
  9. 9.
    Mansoor, A., Bagci, U., Xu, Z., Foster, B., Olivier, K.N., Elinoff, J.M., Suffredini, A.F., Udupa, J.K., Mollura, D.J.: A generic approach to pathological lung segmentation. IEEE Trans. Med. Imaging 33(12), 2293–2310 (2014)CrossRefGoogle Scholar
  10. 10.
    Merkow, J., Marsden, A., Kriegman, D., Tu, Z.: Dense volume-to-volume vascular boundary detection. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 371–379. Springer, Cham (2016). doi: 10.1007/978-3-319-46726-9_43 CrossRefGoogle Scholar
  11. 11.
    Nogues, I., Lu, L., Wang, X., Roth, H., Bertasius, G., Lay, N., Shi, J., Tsehay, Y., Summers, R.M.: Automatic lymph node cluster segmentation using holistically-nested neural networks and structured optimization in CT images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 388–397. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_45 CrossRefGoogle Scholar
  12. 12.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  13. 13.
    Roth, H.R., Lu, L., Farag, A., Sohn, A., Summers, R.M.: Spatial aggregation of holistically-nested networks for automated pancreas segmentation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 451–459. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_52 CrossRefGoogle Scholar
  14. 14.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale visual recognition. In: ICLR (2015)Google Scholar
  15. 15.
    Sofka, M., Wetzl, J., Birkbeck, N., Zhang, J., Kohlberger, T., Kaftan, J., Declerck, J., Zhou, S.K.: Multi-stage learning for robust lung segmentation in challenging CT volumes. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6893, pp. 667–674. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23626-6_82 CrossRefGoogle Scholar
  16. 16.
    Wang, J., Li, F., Li, Q.: Automated segmentation of lungs with severe interstitial lung disease in ct. Med. Phys. 36(10), 4592–4599 (2009)CrossRefGoogle Scholar
  17. 17.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: The IEEE International Conference on Computer Vision (ICCV), December 2015Google Scholar
  18. 18.
    Zhou, Y., Xie, L., Shen, W., Fishman, E., Yuille, A.: Pancreas segmentation in abdominal ct scan: A coarse-to-fine approach. CoRR/abs/1612.08230 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Adam P. Harrison
    • 1
  • Ziyue Xu
    • 1
    Email author
  • Kevin George
    • 1
  • Le Lu
    • 1
  • Ronald M. Summers
    • 1
  • Daniel J. Mollura
    • 1
  1. 1.National Institutes of HealthBethesdaUSA

Personalised recommendations