Skip to main content

Advertisement

SpringerLink
Log in

Reliability of Semi-Automated Segmentations in Glioblastoma

  • Original Article
  • Published:
Clinical Neuroradiology Aims and scope Submit manuscript

Abstract

Purpose

In glioblastoma, quantitative volumetric measurements of contrast-enhancing or fluid-attenuated inversion recovery (FLAIR) hyperintense tumor compartments are needed for an objective assessment of therapy response. The aim of this study was to evaluate the reliability of a semi-automated, region-growing segmentation tool for determining tumor volume in patients with glioblastoma among different users of the software.

Methods

A total of 320 segmentations of tumor-associated FLAIR changes and contrast-enhancing tumor tissue were performed by different raters (neuroradiologists, medical students, and volunteers). All patients underwent high-resolution magnetic resonance imaging including a 3D-FLAIR and a 3D-MPRage sequence. Segmentations were done using a semi-automated, region-growing segmentation tool. Intra- and inter-rater-reliability were addressed by intra-class-correlation (ICC). Root-mean-square error (RMSE) was used to determine the precision error. Dice score was calculated to measure the overlap between segmentations.

Results

Semi-automated segmentation showed a high ICC (> 0.985) for all groups indicating an excellent intra- and inter-rater-reliability. Significant smaller precision errors and higher Dice scores were observed for FLAIR segmentations compared with segmentations of contrast-enhancement. Single rater segmentations showed the lowest RMSE for FLAIR of 3.3 % (MPRage: 8.2 %). Both, single raters and neuroradiologists had the lowest precision error for longitudinal evaluation of FLAIR changes.

Conclusions

Semi-automated volumetry of glioblastoma was reliably performed by all groups of raters, even without neuroradiologic expertise. Interestingly, segmentations of tumor-associated FLAIR changes were more reliable than segmentations of contrast enhancement. In longitudinal evaluations, an experienced rater can detect progressive FLAIR changes of less than 15 % reliably in a quantitative way which could help to detect progressive disease earlier.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Stupp R, Hegi ME, Mason WP, van den Bent MJ, Taphoorn MJ, Janzer RC, Ludwin SK, Allgeier A, Fisher B, Belanger K, Hau P, Brandes AA, Gijtenbeek J, Marosi C, Vecht CJ, Mokhtari K, Wesseling P, Villa S, Eisenhauer E, Gorlia T, Weller M, Lacombe D, Cairncross JG, Mirimanoff RO. Effects of radiotherapy with concomitant and adjuvant temozolomide versus radiotherapy alone on survival in glioblastoma in a randomised phase III study: 5-year analysis of the EORTC-NCIC trial. Lancet Oncol. 2009;10:459–66.

    Article  CAS  PubMed  Google Scholar 

  2. Stupp R, Mason W, van den Bent MJ, Weller M, Fisher BM, Taphoorn MJB, Belanger K, Brandes AA, Marosi C, Bogdahn U, Curschmann J, Janzer RC, Ludwin SK, Gorlia T, Allgeier A, Lacombe D, Cairncross G, Eisenhauer E, Mirimanoff RO. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. Cancer Radiother. 2005;9(3):196–7.

    Article  Google Scholar 

  3. Sughrue ME, Sheean T, Bonney PA, Maurer AJ, Teo C. Aggressive repeat surgery for focally recurrent primary glioblastoma: outcomes and theoretical framework. Neurosurg Focus. 2015;38:E11.

    Article  PubMed  Google Scholar 

  4. Chinot OL, Wick W, Mason W, Henriksson R, Saran F, Nishikawa R, Carpentier AF, Hoang-Xuan K, Kavan P, Cernea D, Brandes AA, Hilton M, Abrey L, Cloughesy T. Bevacizumab plus radiotherapy-temozolomide for newly diagnosed glioblastoma. N Engl J Med. 2014;370:709–22.

    Article  CAS  PubMed  Google Scholar 

  5. Woernle CM, Péus D, Hofer S, Rushing EJ, Held U, Bozinov O, Krayenbühl N, Weller M, Regli L. Efficacy of surgery and further treatment of progressive glioblastoma. World Neurosurg. 2015;84:301–7.

    Article  PubMed  Google Scholar 

  6. Franceschi E, Bartolotti M, Tosoni A, Bartolini S, Sturiale C, Fioravanti A, Pozzati E, Galzio R, Talacchi A, Volpin L, Morandi L, Danieli D, Ermani M, Brandes AA. The effect of re-operation on survival in patients with recurrent glioblastoma. Anticancer Res. 2015;35:1743–8.

    PubMed  Google Scholar 

  7. Wen PY, Macdonald DR, Reardon DA, Cloughesy TF, Sorensen AG, Galanis E, DeGroot J, Wick W, Gilbert MR, Lassman AB, Tsien C, Mikkelsen T, Wong ET, Chamberlain MC, Stupp R, Lamborn KR, Vogelbaum MA, van den Bent MJ, Chang SM. Updated response assessment criteria for high-grade gliomas: response assessment in neuro-oncology working group. J Clin Oncol. 2010;28(11):1963–72.

    Article  PubMed  Google Scholar 

  8. Macdonald DR, Cascino TL, Schold SC, Cairncross JG. Response criteria for phase II studies of supratentorial malignant glioma. J Clin Oncol. 1990;8:1277–80.

    Article  CAS  PubMed  Google Scholar 

  9. Sorensen AG, Batchelor TT, Wen PY, Zhang W-T, Jain RK. Response criteria for glioma. Nat Clin Pract Oncol. 2008;5:634–44.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, Weber MA, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp C, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SM, Ryan M, Sarikaya D, Schwartz L, Shin HC, Shotton J, Silva CA, Sousa N, Subbanna NK, Szekely G, Taylor TJ, Thomas OM, Tustison NJ, Unal G, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Van Leemput K. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging. 2015;34(10):1993-2024.

  11. Rana M, Modrow D, Keuchel J, Chui C, Rana M, Wagner M, Gellrich N-C. Development and evaluation of an automatic tumor segmentation tool: a comparison between automatic, semi-automatic and manual segmentation of mandibular odontogenic cysts and tumors. J Craniomaxillofac Surg. 2015;43:355–9.

    Article  PubMed  Google Scholar 

  12. Gordillo N, Montseny E, Sobrevilla P. State of the art survey on MRI brain tumor segmentation. Magn Reson Imaging. 2013;31:1426–38.

    Article  PubMed  Google Scholar 

  13. WMA General Assembly. World Medical Association Declaration of Helsinki. 1964.

  14. Hojjatoleslami SA, Kittler J. Region growing: a new approach. IEEE Trans Image Process. 1998;7:1079–84.

    Article  CAS  PubMed  Google Scholar 

  15. Hojjatoleslami SA, Kruggel F. Segmentation of large brain lesions. IEEE Trans Med Imaging. 2001;20:666–9.

    Article  CAS  PubMed  Google Scholar 

  16. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46.

    Article  Google Scholar 

  17. Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8:23–34.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6:284–90.

    Article  Google Scholar 

  19. Glüer CC, Blake G, Lu Y, Blunt BA, Jergas M, Genant HK. Accurate assessment of precision errors: how to measure the reproducibility of bone densitometry techniques. Osteoporos Int. 1995;5:262–70.

    Article  PubMed  Google Scholar 

  20. Shepherd JA, Lu Y. A generalized least significant change for individuals measured on different DXA systems. J Clin Densitom. 2007;10:249–58.

    Article  PubMed  Google Scholar 

  21. Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945;26:297–302.

    Article  Google Scholar 

  22. Galluzzi L, Aaronson SA, Abrams J, Alnemri ES, Andrews DW, Baehrecke EH, Bazan NG, Blagosklonny MV, Blomgren K, Borner C, Bredesen DE, Brenner C, Castedo M, Cidlowski JA, Ciechanover A, Cohen GM, De Laurenzi V, De Maria R, Deshmukh M, Dynlacht BD, El-Deiry WS, Flavell RA, Fulda S, Garrido C, Golstein P, Gougeon M-L, Green DR, Gronemeyer H, Hajnóczky G, Hardwick JM, Hengartner MO, Ichijo H, Jäättelä M, Kepp O, Kimchi A, Klionsky DJ, Knight RA, Kornbluth S, Kumar S, Levine B, Lipton SA, Lugli E, Madeo F, Malomi W, Marine J-CW, Martin SJ, Medema JP, Mehlen P, Melino G, Moll UM, Morselli E, Nagata S, Nicholson DW, Nicotera P, Nuñez G, Oren M, Penninger J, Pervaiz S, Peter ME, Piacentini M, Prehn JHM, Puthalakath H, Rabinovich GA, Rizzuto R, Rodrigues CMP, Rubinsztein DC, Rudel T, Scorrano L, Simon H-U, Steller H, Tschopp J, Tsujimoto Y, Vandenabeele P, Vitale I, Vousden KH, Youle RJ, Yuan J, Zhivotovsky B, Kroemer G. Guidelines for the use and interpretation of assays for monitoring cell death in higher eukaryotes. Cell Death Differ. 2009;16:1093–107.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Chow DS, Qi J, Guo X, Miloushev VZ, Iwamoto FM, Bruce JN, Lassman AB, Schwartz LH, Lignelli A, Zhao B, Filippi CG. Semiautomated volumetric measurement on postcontrast MR imaging for analysis of recurrent and residual disease in glioblastoma multiforme. AJNR Am J Neuroradiol. 2014;35:498–503.

    Article  CAS  PubMed  Google Scholar 

  24. Radbruch A, Lutz K, Wiestler B, Bäumer P, Heiland S, Wick W, Bendszus M. Relevance of T2 signal changes in the assessment of progression of glioblastoma according to the Response Assessment in Neurooncology criteria. Neuro Oncol. 2012;14:222–9.

    Article  CAS  PubMed  Google Scholar 

  25. Porz N, Bauer S, Pica A, Schucht P, Beck J, Verma RK, Slotboom J, Reyes M, Wiest R. Multi-modal glioblastoma segmentation: man versus machine. PLoS One. 2014;9(5):e96873.

Download references

Acknowledgments

We thank the group of medical students and volunteers for taking part in this study.

Ethical Standards

This study was approved by the local ethics committee at the Klinikum rechts der Isar of the Technical University of Munich, Germany, in accordance with the ethical standards of the 1964 Declaration of Helsinki and its later amendments [13].

Conflict of Interest

Brainlab (Feldkirchen, Germany) provided the segmentation software used in this study for research purpose.

Author information

Authors and Affiliations

  1. Department of Neuroradiology, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str. 22, 81675, Munich, Germany

    T Huber, G Alber, S Bette, T Boeckh-Behrens, E Alberts, C Zimmer & J S Bauer

  2. Department of Neurosurgery, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str. 22, 81675, Munich, Germany

    J Gempt & F Ringel

Authors
  1. T Huber
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G Alber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S Bette
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T Boeckh-Behrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J Gempt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. F Ringel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. E Alberts
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. C Zimmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. J S Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T Huber.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huber, T., Alber, G., Bette, S. et al. Reliability of Semi-Automated Segmentations in Glioblastoma. Clin Neuroradiol 27, 153–161 (2017). https://doi.org/10.1007/s00062-015-0471-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00062-015-0471-2

Keywords

Search

Navigation