Skip to main content

Postoperative glioma segmentation in CT image using deep feature fusion model guided by multi-sequence MRIs

Abstract

Objectives

Computed tomography (CT) and magnetic resonance imaging (MRI) are the most commonly selected methods for imaging gliomas. Clinically, radiotherapists always delineate the CT glioma region with reference to multi-modal MR image information. On this basis, we develop a deep feature fusion model (DFFM) guided by multi-sequence MRIs for postoperative glioma segmentation in CT images.

Methods

DFFM is a multi-sequence MRI–guided convolutional neural network (CNN) that iteratively learns the deep features from CT images and multi-sequence MR images simultaneously by utilizing a multi-channel CNN architecture, and then combines these two deep features together to produce the segmentation result. The whole network is optimized together via a standard back-propagation. A total of 59 CT and MRI datasets (T1/T2-weighted FLAIR, T1-weighted contrast-enhanced, T2-weighted) of postoperative gliomas as tumor grade II (n = 24), grade III (n = 18), or grade IV (n = 17) were included. Dice coefficient (DSC), precision, and recall were used to measure the overlap between automated segmentation results and manual segmentation. The Wilcoxon signed-rank test was used for statistical analysis.

Results

DFFM showed a significantly (p < 0.01) higher DSC of 0.836 than U-Net trained by single CT images and U-Net trained by stacking the CT and multi-sequence MR images, which yielded 0.713 DSC and 0.818 DSC, respectively. The precision values showed similar behavior as DSC. Moreover, DSC and precision values have no significant statistical difference (p > 0.01) with difference grades.

Conclusions

DFFM enables the accurate automated segmentation of CT postoperative gliomas of profit guided by multi-sequence MR images and may thus improve and facilitate radiotherapy planning.

Key Points

• A fully automated deep learning method was developed to segment postoperative gliomas on CT images guided by multi-sequence MRIs.

• CT and multi-sequence MR image integration allows for improvements in deep learning postoperative glioma segmentation method.

• This deep feature fusion model produces reliable segmentation results and could be useful in delineating GTV in postoperative glioma radiotherapy planning.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. Code source: https://github.com/tfnfyd110/DFFM

Abbreviations

CNNs:

Convolutional neural networks

DFFM:

Deep feature fusion model

FLAIR:

Fluid-attenuated inversion recovery

GTV:

Gross tumor volume

T1CE:

T1-weighted contrast-enhanced

U-Net_CT:

U-Net trained by single CT images

U-Net_CT&mMRIs:

U-Net trained by stacking the CT and multi-sequence MR images

References

  1. Goodenberger ML, Jenkins RB (2012) Genetics of adult glioma. Cancer Genet 205(12):613–621

    CAS  Article  Google Scholar 

  2. Louis DN, Ohgaki H, Wiestler OD et al (2007) The 2007 WHO classification of tumours of the central nervous system. Acta Neuropathol 114(2):97–109

  3. Evans PM (2008) Anatomical imaging for radiotherapy. Phys Med Biol 53(12):R151–R191

    Article  Google Scholar 

  4. Warmuth-Metz M (2003) Postoperative imaging after brain tumor resection. In: Westphal M, Tonn JC, Ram Z (eds) Local therapies for glioma present status and future developments. Springer, Vienna, pp 13–20

  5. Yahyanejad S, Granton PV, Lieuwes NG et al (2015) Complementary use of bioluminescence imaging and contrast-enhanced micro-computed tomography in an orthotopic brain tumor model. Mol Imaging. https://doi.org/10.2310/7290.2014.00038

  6. Lee CH, Murtha A, Murtha A, Bistritz A, Greiner R (2005) Segmenting brain tumors with conditional random fields and support vector machines. In: Liu Y, Jiang T, Zhang C (eds) Computer Vision for Biomedical Image Applications. CVBIA 2005. Lecture Notes in Computer Science, vol 3765. Springer, Berlin, Heidelberg, pp 469–478

  7. Li H, Song M, Fan Y (2010) Segmentation of brain tumors in multi-parametric MR images via robust statistic information propagation. In: Kimmel R, Klette R, Sugimoto A (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6495. Springer, Berlin, pp 606–617

  8. Zikic D, Glocker B, Konukoglu E et al (2012) Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. MICCAI 2012. Lecture Notes in Computer Science, vol 7512. Springer, Berlin, pp 369–376

  9. Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2014) Appearance-and context-sensitive features for brain tumor segmentation. Proceedings of MICCAI BRATS Challenge, pp 20–26

  10. Liang S, Tang F, Huang X et al (2019) Deep-learning-based detection and segmentation of organs at risk in nasopharyngeal carcinoma computed tomographic images for radiotherapy planning. Eur Radiol 29(4):1961–1967

    Article  Google Scholar 

  11. Huang X, Wang J, Tang F, Zhong T, Zhang Y (2018) Metal artifact reduction on cervical CT images by deep residual learning. Biomed Eng Online 17(1):175

    Article  Google Scholar 

  12. Shen D, Wu G, Suk H (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19(1):221–248

    CAS  Article  Google Scholar 

  13. Zikic D, Ioannou Y, Brown M, Criminisi A (2014) Segmentation of brain tumor tissues with convolutional neural networks. Proceedings of MICCAI BRATS Challenge, pp 036–039

  14. Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251

    Article  Google Scholar 

  15. Havaei M, Davy A, Warde-Farley D et al (2017) Brain tumor segmentation with deep neural net-works. Med Image Anal 35:18–31

    Article  Google Scholar 

  16. Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi A, Bakas S, Kuijf H, Menze B, Reyes M (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2017. Lecture Notes in Computer Science, vol 10670. Springer, Cham, pp 178–190

  17. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems (NIPS), pp 1097–1105

    Google Scholar 

  18. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  19. Menze BH, Jakab A, Bauer S et al (2015) The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  20. Bakas S, Akbari H, Sotiras A et al (2017) Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci Data 4:170117

    Article  Google Scholar 

  21. Jones D (1994) ICRU report 50-prescribing, recording and reporting photon beam therapy. Med Phys 21(6):833–834

  22. Landberg T, Chavaudra J, Dobbs J et al (1999) ICRU report 62-prescribing, recording and reporting photon beam therapy (supplement to ICRU report 50). J ICRU os32(1):1–52

    Google Scholar 

  23. National Comprehensive Cancer Network (2018) Clinical Practice Guidelines in Oncology for central nervous system cancers Version 1. National Comprehensive Cancer Network. Available via https://www.nccn.org/professionals/physician_gls/pdf/cns.pdf. Accessed 15 Mar 2018

  24. Tustison NJ, Avants BB, Cook PA et al (2010) N4ITK: improved N3 bias correction. IEEE Trans Med Imaging 29(6):1310–1320

    Article  Google Scholar 

  25. Lyu S, Simoncelli EP (2008) Nonlinear image representation using divisive normalization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, pp 1–8

    Google Scholar 

  26. Pinto N, Cox DD, DiCarlo JJ (2008) Why is real-world visual object recognition hard? PLoS Comput Biol 4(1):e27

    Article  Google Scholar 

  27. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012. MICCAI 2012. Lecture Notes in Computer Science, vol 7512. Springer, Berlin, pp 234–241

  28. Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8694.Springer, Cham, pp 94–108

  29. Lin TY, Goyal P, Girshick R, He K, Dollar P (2017) Focal loss for dense object detection. In: The IEEE International Conference on Computer Vision (ICCV). IEEE, Piscataway, pp 2999–3007

    Chapter  Google Scholar 

  30. He K, Zhang X, Ren S, Jian S (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: The IEEE International Conference on Computer Vision (ICCV). IEEE, Piscataway, pp 1026–1034

    Chapter  Google Scholar 

  31. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  32. Chen L, Price RA Jr, Wang L et al (2004) MRI-based treatment planning for radiotherapy: dosimetric verification for prostate IMRT. Int J Radiat Oncol Biol Phys 60(2):636–647

  33. Walker A, Liney G, Metcalfe P, Holloway L (2014) MRI distortion: considerations for MRI based radiotherapy treatment planning. Australas Phys Eng Sci Med 37(1):103–113

    Article  Google Scholar 

  34. Khoo VS, Dearnaley DP, Finnigan DJ, Padhani A, Tanner SF, Leach MO (1997) Magnetic resonance imaging (MRI): considerations and applications in radiotherapy treatment planning. Radiother Oncol 42(1):1–15

    CAS  Article  Google Scholar 

  35. Belhawi SM, Hoefnagels FW, Baaijen JC et al (2011) Early postoperative MRI overestimates residual tumour after resection of gliomas with no or minimal enhancement. Eur Radiol 21(7):1526–1534

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the reviewers for their fruitful comments.

Funding

This study has received funding by the National Natural Science Foundation of China under Grant Nos. 61671230 and 31271067, the Science and Technology Program of Guangdong Province under Grant No. 2017A020211012, the Guangdong Provincial Key Laboratory of Medical Image Processing under Grant No. 2014B030301042, and the Science and Technology Program of Guangzhou under Grant No. 201607010097.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Yu Zhang or Linghong Zhou.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Yu Zhang.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• experimental

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 22 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tang, F., Liang, S., Zhong, T. et al. Postoperative glioma segmentation in CT image using deep feature fusion model guided by multi-sequence MRIs. Eur Radiol 30, 823–832 (2020). https://doi.org/10.1007/s00330-019-06441-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-019-06441-z

Keywords

  • Glioma
  • Machine learning
  • Magnetic resonance imaging
  • Radiotherapy