Skip to main content
SpringerLink
Log in

Automatic Grading of Emphysema by Combining 3D Lung Tissue Appearance and Deformation Map Using a Two-Stream Fully Convolutional Neural Network

  • Conference paper
  • First Online:
Machine Learning in Medical Imaging (MLMI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13583))

Included in the following conference series:

  • 1113 Accesses

Abstract

We propose and develop an end-to-end learning pipeline incorporating lung tissue appearance and deformation pattern during two phase respiratory CT chest imaging. We formulate the task as a voxel level classification and quantification problem for the emphysematous areas of lung parenchyma on CT imaging and develop a two-stream fully convolutional neural network which fuses together lung tissue pattern and deformation map in a unified framework. The Jacobian matrix, computed between inspiratory and expiratory images, is used as a surrogate measurement of lung deformation. The DenseVNet architecture is adopted for memory efficiency and accuracy on smaller structure segmentation.

The proposed model is trained from inspiratory and expiratory CT images, where the Jacobian matrix calculates the volumetric deformation map of the lung during respiration from inspiration to expiration: the former is used for tissue appearance of the lung while both together provide motion characteristics. Our ablation study shows that incorporating tissue appearance and lung motion accurately classifies all patients with different types and severity of emphysema against the non-emphysema cases. In addition, the proposed method outperforms previous state-of-the-art methods in grading and severity calculation of emphysema, which enables automatic image-based emphysema grading for the first time. To the best of our knowledge, our work is the first proposed framework to incorporate both tissue appearance and deformation map in a deep learning platform for disease diagnosis in lung using CT imaging, which can be intuitively extended to other moving organs and diseases.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fernandes, L., Gulati, N., Mesquita, A.M., Sardesai, M., Fernandes, Y.: Quantification of emphysema in chronic obstructive pulmonary disease by volumetric computed tomography of lung. Indian J. Chest Dis. Allied Sci. 57, 155–160 (2015)

    Article  Google Scholar 

  2. Omori, H., Fujimoto, K., Katoh, T.: Computed-tomography findings of emphysema: correlation with spirometric values. Curr. Opin. Pulm. Med. 14, 110–114 (2008)

    Article  Google Scholar 

  3. Wille, M.M.W., et al.: Visual assessment of early emphysema and interstitial abnormalities on CT is useful in lung cancer risk analysis. Eur. Radiol. 26(2), 487–494 (2015). https://doi.org/10.1007/s00330-015-3826-9

    Article  Google Scholar 

  4. Lynch, D.A., Austin, J.H., Hogg, J.C., Grenier, P.A., Kauczor, H.U., et al.: CT-definable subtypes of chronic obstructive pulmonary disease: a statement of the fleischner society. Radiology 277, 192–205 (2015)

    Article  Google Scholar 

  5. Hohberger, L.A., Schroeder, D.R., Bartholmai, B.J., Yang, P., Wendt, C.H., et al.: Correlation of regional emphysema and lung cancer: a lung tissue research consortium-based study. J. Thorac. Oncol. 9, 639–645 (2014)

    Article  Google Scholar 

  6. Haruna, A., Muro, S., Nakano, Y., Ohara, T., Hoshino, Y., et al.: CT scan findings of emphysema predict mortality in COPD. Chest 138, 635–640 (2010)

    Article  Google Scholar 

  7. Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A., Mougiakakou, S.: Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans. Med. Imaging 35, 1207–1216 (2016)

    Article  Google Scholar 

  8. Gao, M., Bagci, U., Lu, L., Wu, A., Buty, M., et al.: Holistic classification of CT attenuation patterns for interstitial lung diseases via deep convolutional neural networks. Comput. Methods Biomech. Biomed. Eng. Imaging Visual. 6, 1–6 (2018)

    Article  Google Scholar 

  9. Ørting, S. N., Petersen, J., Thomsen, L. H., Wille, M. M. W., and Bruijne, M. d.: Detecting emphysema with multiple instance learning. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 510–513. (2018)

    Google Scholar 

  10. Christodoulidis, S., Anthimopoulos, M., Ebner, L., Christe, A., Mougiakakou, S.: Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J. Biomed. Health Inform. 21, 76–84 (2017)

    Article  Google Scholar 

  11. Negahdar, M., Beymer, D.: Lung tissue characterization for emphysema differential diagnosis using deep convolutional neural networks. In: SPIE Medical Imaging, vol. 10950, pp. 109503R. San Diego, CA (2019)

    Google Scholar 

  12. Bermejo-Peláez, D., Estepar, R.S.J., Ledesma-Carbayo, M.J.: Emphysema classification using a multi-view convolutional network. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 519–522 (2018)

    Google Scholar 

  13. Fischer, A.M., Varga-Szemes, A., van Assen, M., Griffith, L.P., Sahbaee, P. et al.: Comparison of artificial intelligence–based fully automatic chest CT emphysema quantification to pulmonary function testing. Am. J. Roentgenol. 214(5), 1065–1071 (2020)

    Google Scholar 

  14. Kauczor, H.-U., Wielpütz, M.O., Jobst, B.J., Weinheimer, O., Gompelmann, D., et al.: Computed tomography imaging for novel therapies of chronic obstructive pulmonary disease. J. Thorac. Imaging 34, 202–213 (2019)

    Article  Google Scholar 

  15. Negahdar, M., Amini, A.A.: Regional lung strains via a volumetric mass conserving optical flow model. In: IEEE International Symposium on Biomedical Imaging (ISBI): From Nano to Macro, pp. 1475–1478. Barcelona, Spain (2012)

    Google Scholar 

  16. Negahdar, M., Dunlap, N., Zacarias, A., Civelek, A.C., Woo, S.Y. et al.: Comparison of indices of regional lung function from 4-D X-ray CT: Jacobian vs. strain of deformation. In: IEEE International Symposium on Biomedical Imaging (ISBI): From Nano to Macro, pp. 648–651. San Francisco, CA, USA (2013)

    Google Scholar 

  17. Negahdar, M., Fasola, C.E., Yu, A.S., von Eyben, R., Yamamoto, T., et al.: Noninvasive pulmonary nodule elastometry by CT and deformable image registration. Radiother. Oncol. 115, 35–40 (2015)

    Article  Google Scholar 

  18. Joe Yue-Hei, N., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R. et al.: Beyond short snippets: deep networks for video classification. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4694–4702 (2015)

    Google Scholar 

  19. Wang, L., Ouyang, W., Wang, X., Lu, H.: Visual tracking with fully convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3119–3127 (2015)

    Google Scholar 

  20. Jain, S.D., Xiong, B., Grauman, K.: FusionSeg: learning to combine motion and appearance for fully automatic segmentation of generic objects in videos. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117–2126 (2017)

    Google Scholar 

  21. Li, S., Seybold, B., Vorobyov, A., Lei, X., Kuo, C.C.J.: Unsupervised video object segmentation with motion-based bilateral networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., et al. (eds.) Computer Vision – ECCV 2018, pp. 215-231. Cham (2018)

    Google Scholar 

  22. Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., et al.: Towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks. In: Medical Image Computing and Computer Assisted Intervention − MICCAI 2017, pp. 728-736. Cham (2017)

    Google Scholar 

  23. Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 3D Vision (3DV), 2016 Fourth International Conference on, pp. 565–571 (2016)

    Google Scholar 

  24. Negahdar, M., Beymer, D., Syeda-Mahmood, T.: Automated volumetric lung segmentation of thoracic CT images using fully convolutional neural network. In: SPIE Medical Imaging, vol. 10575 (2018)

    Google Scholar 

  25. Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., et al.: Automatic multi-organ segmentation on abdominal CT with dense V-networks. IEEE Trans. Med. Imaging 37, 1822–1834 (2018)

    Article  Google Scholar 

  26. Ghavami, N., Hu, Y., Gibson, E., Bonmati, E., Emberton, M., et al.: Automatic segmentation of prostate MRI using convolutional neural networks: investigating the impact of network architecture on the accuracy of volume measurement and MRI-ultrasound registration. Med. Image Anal. 58, 101558 (2019)

    Article  Google Scholar 

  27. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.: elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29, 196–205 (2010)

    Article  Google Scholar 

  28. Kalogeiton, V., Weinzaepfel, P., Ferrari, V., Schmid, C.: Action tubelet detector for spatio-temporal action localization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4415–4423 (2017)

    Google Scholar 

  29. Singh, G., Saha, S., Sapienza, M., Torr, P., Cuzzolin, F.: Online real-time multiple spatiotemporal action localisation and prediction. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3657–3666 (2017)

    Google Scholar 

  30. Mohamed Hoesein, F.A., van Rikxoort, E., van Ginneken, B., de Jong, P.A., Prokop, M., et al.: Computed tomography-quantified emphysema distribution is associated with lung function decline. Eur. Respir. J. 40, 844–850 (2012)

    Google Scholar 

  31. Bailey, K.L.: The importance of the assessment of pulmonary function in COPD. Med. Clin. North Am. 96, 745–752 (2012)

    Article  Google Scholar 

  32. Wille, M.M.W., Thomsen, L.H., Dirksen, A., Petersen, J., Pedersen, J.H., Shaker, S.B.: Emphysema progression is visually detectable in low-dose CT in continuous but not in former smokers. Eur. Radiol. 24(11), 2692–2699 (2014). https://doi.org/10.1007/s00330-014-3294-7

    Article  Google Scholar 

  33. Heussel, C.P., Herth, F.J.F., Kappes, J., Hantusch, R., Hartlieb, S., et al.: Fully automatic quantitative assessment of emphysema in computed tomography: comparison with pulmonary function testing and normal values. Eur. Radiol. 19, 2391–2402 (2009)

    Article  Google Scholar 

  34. Bartholmai, B., Karwoski, R., Zavaletta, V., Robb, R., Holmes, D.: The lung tissue research consortium: an extensive open database containing histological, clinical, and radiological data to study chronic lung disease. In: The Insight Journal—2006 MICCAI Open Science Workshop (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Genentech, South San Francisco, CA, USA

    Mohammadreza Negahdar

Authors
  1. Mohammadreza Negahdar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammadreza Negahdar .

Editor information

Editors and Affiliations

  1. Xi'an Jiaotong University, Xi'an, China

    Chunfeng Lian

  2. Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China

    Xiaohuan Cao

  3. Istanbul Technical University, Istanbul, Turkey

    Islem Rekik

  4. Rensselaer Polytechnic Institute, Troy, NY, USA

    Xuanang Xu

  5. ShanghaiTech University, Pudong, China

    Zhiming Cui

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Negahdar, M. (2022). Automatic Grading of Emphysema by Combining 3D Lung Tissue Appearance and Deformation Map Using a Two-Stream Fully Convolutional Neural Network. In: Lian, C., Cao, X., Rekik, I., Xu, X., Cui, Z. (eds) Machine Learning in Medical Imaging. MLMI 2022. Lecture Notes in Computer Science, vol 13583. Springer, Cham. https://doi.org/10.1007/978-3-031-21014-3_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21014-3_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21013-6

  • Online ISBN: 978-3-031-21014-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation