Skip to main content
SpringerLink
Log in

Diffusion and perfusion MRI radiomics obtained from deep learning segmentation provides reproducible and comparable diagnostic model to human in post-treatment glioblastoma

  • Magnetic Resonance
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

Deep learning-based automatic segmentation (DLAS) helps the reproducibility of radiomics features, but its effect on radiomics modeling is unknown. We therefore evaluated whether DLAS can robustly extract anatomical and physiological MRI features, thereby assisting in the accurate assessment of treatment response in glioblastoma patients.

Methods

A DLAS model was trained on 238 glioblastomas and validated on an independent set of 98 pre- and 86 post-treatment glioblastomas from two tertiary hospitals. A total of 1618 radiomics features from contrast-enhanced T1-weighted images (CE-T1w) and histogram features from apparent diffusion coefficient (ADC) and cerebral blood volume (CBV) mapping were extracted. The diagnostic performance of radiomics features and ADC and CBV parameters for identifying treatment response was tested using area under the curve (AUC) from receiver operating characteristics analysis. Feature reproducibility was tested using a 0.80 cutoff for concordance correlation coefficients.

Results

Reproducibility was excellent for ADC and CBV features (ICC, 0.82–0.99) and first-order features (pre- and post-treatment, 100% and 94.1% remained), but lower for texture (79.0% and 69.1% remained) and wavelet-transformed (81.8% and 74.9% remained) features of CE-T1w. DLAS-based radiomics showed similar performance to human-performed segmentations in internal validation (AUC, 0.81 [95% CI, 0.64–0.99] vs. AUC, 0.81 [0.60–1.00], p = 0.80), but slightly lower performance in external validation (AUC, 0.78 [0.61–0.95] vs. AUC, 0.65 [0.46–0.84], p = 0.23).

Conclusion

DLAS-based feature extraction showed high reproducibility for first-order features from anatomical and physiological MRI, and comparable diagnostic performance to human manual segmentations in the identification of pseudoprogression, supporting the utility of DLAS in quantitative MRI analysis.

Key Points

• Deep learning-based automatic segmentation (DLAS) enables fast and robust feature extraction from diffusion- and perfusion-weighted MRI.

• DLAS showed high reproducibility in first-order feature extraction from anatomical, diffusion, and perfusion MRI across two centers.

• DLAS-based radiomics features showed comparable diagnostic accuracy to manual segmentations in post-treatment glioblastoma.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

3D:

Three-dimensional

ADC:

Apparent diffusion coefficient

AUC:

Area under the curve

CBV:

Cerebral blood volume

CCC:

Concordance correlation coefficient

CE-T1w:

Contrast-enhanced T1-weighted

DICE:

Dice similarity coefficient

DLAS:

Deep learning-based automatic segmentation

TP:

Tumor progression

References

  1. Wen PY, Chang SM, Van den Bent MJ, Vogelbaum MA, Macdonald DR, Lee EQ (2017) Response assessment in neuro-oncology clinical trials. J Clin Oncol 35:2439–2449

    Article  CAS  Google Scholar 

  2. Kickingereder P, Isensee F, Tursunova I et al (2019) Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. Lancet Oncol 20:728–740

    Article  Google Scholar 

  3. Parmar C, Velazquez ER, Leijenaar R et al (2014) Robust radiomics feature quantification using semiautomatic volumetric segmentation. Plos One 9

  4. Pavic M, Bogowicz M, Wurms X et al (2018) Influence of inter-observer delineation variability on radiomics stability in different tumor sites. Acta Oncol 57:1070–1074

    Article  Google Scholar 

  5. Kumar V, Gu YH, Basu S et al (2012) Radiomics: the process and the challenges. Magn Reson Imaging 30:1234–1248

    Article  Google Scholar 

  6. Lambin P, Leijenaar RTH, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762

    Article  Google Scholar 

  7. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446

    Article  Google Scholar 

  8. Park JE, Kim HS (2018) Radiomics as a quantitative imaging biomarker: practical considerations and the current standpoint in neuro-oncologic studies. Nucl Med Mol Imaging 52:99–108

    Article  Google Scholar 

  9. Menze BH, Jakab A, Bauer S et al (2015) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34:1993–2024

    Article  Google Scholar 

  10. Ellingson BM, Cloughesy TF, Zaw T et al (2012) Functional diffusion maps (fDMs) evaluated before and after radiochemotherapy predict progression-free and overall survival in newly diagnosed glioblastoma. Neuro Oncol 14:333–343

    Article  CAS  Google Scholar 

  11. Law M, Young RJ, Babb JS et al (2008) Gliomas: predicting time to progression or survival with cerebral blood volume measurements at dynamic susceptibility-weighted contrast-enhanced perfusion MR imaging. Radiology 247:490–498

    Article  Google Scholar 

  12. Kickingereder P, Burth S, Wick A et al (2016) Radiomic profiling of glioblastoma: identifying an imaging predictor of patient survival with improved performance over established clinical and radiologic risk models. Radiology 280:880–889

    Article  Google Scholar 

  13. Bae S, Choi YS, Ahn SS et al (2018) Radiomic MRI phenotyping of glioblastoma: improving survival prediction. Radiology 289:797–806

    Article  Google Scholar 

  14. Kim JY, Park JE, Jo Y et al (2018) Incorporating diffusion- and perfusion-weighted MRI into a radiomics model improves diagnostic performance for pseudoprogression in glioblastoma patients. Neuro Oncol. https://doi.org/10.1093/neuonc/noy133

  15. Kickingereder P, Gotz M, Muschelli J et al (2016) Large-scale radiomic profiling of recurrent glioblastoma identifies an imaging predictor for stratifying anti-angiogenic treatment response. Clin Cancer Res 22:5765–5771

    Article  CAS  Google Scholar 

  16. Louis DN, Perry A, Reifenberger G et al (2016) The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol 131:803–820

    Article  Google Scholar 

  17. Nolden M, Zelzer S, Seitel A et al (2013) The medical imaging interaction toolkit: challenges and advances : 10 years of open-source development. Int J Comput Assist Radiol Surg 8:607–620

    Article  Google Scholar 

  18. Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage 54:2033–2044

    Article  Google Scholar 

  19. Shinohara RT, Sweeney EM, Goldsmith J et al (2014) Statistical normalization techniques for magnetic resonance imaging. Neuroimage Clin 6:9–19

  20. Kang D, Park JE, Kim YH et al (2018) Diffusion radiomics as a diagnostic model for atypical manifestation of primary central nervous system lymphoma: development and multicenter external validation. Neuro Oncol. https://doi.org/10.1093/neuonc/noy021

  21. Zwanenburg A, Leger S, Vallières M, Löck S (2016) Image biomarker standardisation initiative. arXiv preprint arXiv:1612.07003

  22. Weisskoff RM, Boxerman JL, Sorensen AG et al (1994) Simultaneous blood volume and permeability mapping using a single Gd-based contrast injection. In: Proceedings of the Society of Magnetic Resonance, Second Annual Meeting; 1994 Aug 6–12; San Francisco, Calif.; Berkeley, Calif: Society of Magnetic Resonance, 279

  23. Chung WJ, Kim HS, Kim N, Choi CG, Kim SJ (2013) Recurrent glioblastoma: optimum area under the curve method derived from dynamic contrast-enhanced T1-weighted perfusion MR imaging. Radiology 269:561–568

    Article  Google Scholar 

  24. Kim HS, Suh CH, Kim N, Choi CG, Kim SJ (2014) Histogram analysis of intravoxel incoherent motion for differentiating recurrent tumor from treatment effect in patients with glioblastoma: initial clinical experience. AJNR Am J Neuroradiol 35:490–497

    Article  CAS  Google Scholar 

  25. Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26:297–302

    Article  Google Scholar 

  26. Kursa MB (2014) Robustness of Random Forest-based gene selection methods. BMC Bioinformatics 15(1):8

  27. Loh WY (2011) Classification and regression trees. Wiley Interdisciplinary Reviews: Data Min Knowl Discov 1(1):14-23

  28. Lin L, Dou Q, Jin YM et al (2019) Deep learning for automated contouring of primary tumor volumes by MRI for nasopharyngeal carcinoma. Radiology 291:677–686

    Article  Google Scholar 

  29. Norman B, Pedoia V, Majumdar S (2018) Use of 2D U-Net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology 288:177–185

    Article  Google Scholar 

  30. Barboriak DP, Zhang Z, Desai P et al (2019) Interreader variability of dynamic contrast-enhanced MRI of recurrent glioblastoma: the multicenter ACRIN 6677/RTOG 0625 study. Radiology 290:467–476

    Article  Google Scholar 

  31. Newitt DC, Zhang Z, Gibbs JE et al (2019) Test-retest repeatability and reproducibility of ADC measures by breast DWI: results from the ACRIN 6698 trial. J Magn Reson Imaging 49:1617–1628

    Article  Google Scholar 

  32. Ellingson BM, Bendszus M, Boxerman J et al (2015) Consensus recommendations for a standardized brain tumor imaging protocol in clinical trials. Neuro Oncol 17:1188–1198

    Article  Google Scholar 

  33. Baessler B, Weiss K, Pinto Dos Santos D (2019) Robustness and reproducibility of radiomics in magnetic resonance imaging: a phantom study. Invest Radiol 54:221–228

    Article  Google Scholar 

  34. Milletari F, Navab N, Ahmadi S-A (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV) (pp. 565-571). IEEE

  35. Estienne T, Lerousseau M, Vakalopoulou M et al (2020) Deep learning-based concurrent brain registration and tumor segmentation. Front Comput Neurosci 14:17

    Article  Google Scholar 

  36. Zwanenburg A, Vallieres M, Abdalah MA et al (2020) The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295:328–338

    Article  Google Scholar 

  37. Zhao B, Tan Y, Tsai WY et al (2016) Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci Rep 6:23428

    Article  CAS  Google Scholar 

  38. Park CM (2019) Can artificial intelligence fix the reproducibility problem of radiomics? Radiology. https://doi.org/10.1148/radiol.2019191154:191154

Download references

Funding

This research was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP) (grant number: NRF-2020R1A2B5B01001707 and NRF-2020R1A2C4001748).

Author information

Author notes

  1. Ji Eun Park and Sungwon Ham contributed equally to this work.

Authors and Affiliations

  1. Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, 43 Olympic-ro 88, Songpa-Gu, Seoul, 05505, South Korea

    Ji Eun Park, Ho Sung Kim & Namkug Kim

  2. Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea

    Sungwon Ham, Jihye Yun & Namkug Kim

  3. Department of Clinical Epidemiology and Biostatistics, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea

    Seo Young Park

  4. Health Innovation Big Data Center, Asan Institute for Life Science, Asan Medical Center, Seoul, South Korea

    Hyunna Lee

  5. Department of Radiology, Seoul National University College of Medicine, Seoul, 03080, South Korea

    Seung Hong Choi

Authors
  1. Ji Eun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sungwon Ham
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ho Sung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seo Young Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jihye Yun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hyunna Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Seung Hong Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Namkug Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ho Sung Kim.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Namkug Kim.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise (Seo Young Park, 8 years of experienced statistician).

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

The external 33 post-treatment glioblastoma patients were in part of a previously reported study (Kim JY et al Neuro-Oncology, Volume 21, Issue 3, March 2019, Pages 404–414, https://doi.org/10.1093/neuonc/noy133). This prior article dealt with development of multiparametric MRI radiomics model using manual segmentation whereas in this manuscript we report reproducibility and accuracy from DLAS-obtained radiomics features and focused on CE-T1w imaging. The method is totally different that the previous study used feature selection in order to create the model and only 12 features were used. Meanwhile what we did here was using entire 1618 extracted features and used random forest classifier to calculate diagnostic performance, with the aim to measure the effect of segmentation to subsequent feature extraction. Also, we created the DLAS model in this study while previous report did manual segmentation.

Methodology

• retrospective

• cross-sectional study

• multicentre study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 340 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, J.E., Ham, S., Kim, H.S. et al. Diffusion and perfusion MRI radiomics obtained from deep learning segmentation provides reproducible and comparable diagnostic model to human in post-treatment glioblastoma. Eur Radiol 31, 3127–3137 (2021). https://doi.org/10.1007/s00330-020-07414-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-020-07414-3

Keywords

Search

Navigation