Abstract
In this paper, we propose a UNet-VAE deep neural network architecture for brain tumor segmentation and survival prediction. UNet-VAE architecture has shown great success in brain tumor segmentation in the multimodal brain tumor segmentation (BraTS) 2018 challenge. In this work, we utilize the UNet-VAE to extract high dimension features, then fuse with hand-crafted texture features to perform survival prediction. We apply the proposed method to the BraTS 2019 validation dataset for both tumor segmentation and survival prediction. The tumor segmentation result shows dice score coefficient (DSC) of 0.759, 0.90, and 0.806 for enhancing tumor (ET), whole tumor (WT), and tumor core (TC), respectively. For the feature fusion-based survival prediction method, we achieve 56.4% classification accuracy with mean square error (MSE) 101577, and 51.7% accuracy with MSE 70590 for training and validation, respectively. In testing phase, the proposed method for tumor segmentation achieves average DSC of 0.81328, 0.88616, and 0.84084 for ET, WT, and TC, respectively. Moreover, the model offers accuracy of 0.439 with MSE of 449009.135 for overall survival prediction in testing phase.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ostrom, Q.T., Gittleman, H., Truitt, G., Boscia, A., Kruchko, C., Barnholtz-Sloan, J.S.: CBTRUS statistical report primary brain and other central nervous system tumors diagnosed in the United States in 2011–2015. Neuro-oncology. 20 suppl. 4, iv1–iv86 (2018)
Bakas, S. et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:1811.02629 (2018)
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. data 4, 170117 (2017)
Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2014)
Mustaqeem, A., Javed, A., Fatima, T.: An efficient brain tumor detection algorithm using watershed & thresholding based segmentation. Int. J. Image Graph. Signal Process. 4(10), 34 (2012)
Pei, L., Bakas, S., Vossough, A., Reza, S.M., Davatzikos, C., Iftekharuddin, K.M.: Longitudinal brain tumor segmentation prediction in MRI using feature and label fusion. Biomed. Signal Process. Control 55, 101648 (2020)
Pei, L., Reza, S.M., Li, W., Davatzikos, C., Iftekharuddin, K.M.: Improved brain tumor segmentation by utilizing tumor growth model in longitudinal brain MRI. In: Medical Imaging 2017: Computer-Aided Diagnosis, vol. 10134, p. 101342L. International Society for Optics and Photonics (2017)
Pei, L., Reza, S.M., Iftekharuddin, K.M.: Improved brain tumor growth prediction and segmentation in longitudinal brain MRI. In: 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 421–424. IEEE (2015)
Prastawa, M., Bullitt, E., Ho, S., Gerig, G.: A brain tumor segmentation framework based on outlier detection. Med. Image Anal. 8(3), 275–283 (2004)
Ho, S., Bullitt, E., Gerig, G.: Level-set evolution with region competition: automatic 3-D segmentation of brain tumors. In: null, p. 10532. Citeseer (2002)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436 (2015)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT press, Cambridge (2016)
Mohsen, H., El-Dahshan, E.-S.A., El-Horbaty, E.-S.M., Salem, A.-B.M.: Classification using deep learning neural networks for brain tumors. Future Comput. Inform. J. 3(1), 68–71 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Shboul, Z.A., Alam, M., Vidyaratne, L., Pei, L., Elbakary, M.I., Iftekharuddin, K.M.: Feature-guided deep radiomics for glioblastoma patient survival prediction (in English). Front. Neurosci. 13(966) (2019). Original Research
Bakas, S. et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. The Cancer Imaging Archive (2017)
Bakas, S. et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. The Cancer Imaging Archive, vol. 286 (2017)
Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Acknowledgements
This work was partially funded through NIH/NIBIB grant under award number R01EB020683. This work is also partially supported in part by NSF under grant CNS-1828593.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pei, L., Vidyaratne, L., Monibor Rahman, M., Shboul, Z.A., Iftekharuddin, K.M. (2020). Multimodal Brain Tumor Segmentation and Survival Prediction Using Hybrid Machine Learning. In: Crimi, A., Bakas, S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2019. Lecture Notes in Computer Science(), vol 11993. Springer, Cham. https://doi.org/10.1007/978-3-030-46643-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-46643-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-46642-8
Online ISBN: 978-3-030-46643-5
eBook Packages: Computer ScienceComputer Science (R0)
