Abstract
Brain Tumor Segmentation from magnetic resonance imaging (MRI) is a critical technique for early diagnosis. However, rather than having complete four modalities as in BraTS dataset, it is common to have missing modalities in clinical scenarios. We design a brain tumor segmentation algorithm that is robust to the absence of any modality. Our network includes a channel-independent encoding path and a feature-fusion decoding path. We use self-supervised training through channel dropout and also propose a novel domain adaptation method on feature maps to recover the information from the missing channel. Our results demonstrate that the quality of the segmentation depends on which modality is missing. Furthermore, we also discuss and visualize the contribution of each modality to the segmentation results. Their contributions are along well with the expert screening routine.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)
Chen, L., Wu, Y., DSouza, A.M., Abidin, A.Z., Wismüller, A., Xu, C.: MRI tumor segmentation with densely connected 3D CNN. In: Medical Imaging 2018: Image Processing, vol. 10574, p. 105741F. International Society for Optics and Photonics (2018)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International Conference on Machine Learning, pp. 1180–1189 (2015)
Havaei, M., et al.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)
Havaei, M., Guizard, N., Chapados, N., Bengio, Y.: HeMIS: hetero-modal image segmentation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 469–477. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_54
Hoo-Chang, S., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285 (2016)
Isensee, F., et al.: Brain Tumor Segmentation Using Large Receptive Field Deep Convolutional Neural Networks. In: Maier-Hein, geb. Fritzsche, K., Deserno, geb. Lehmann, T., Handels, H., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2017. Informatik aktuell, pp. 86–91. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-54345-0_24
Jin, Q., Meng, Z., Sun, C., Wei, L., Su, R.: RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans. arXiv preprint arXiv:1811.01328 (2018)
Kao, P.-Y., Ngo, T., Zhang, A., Chen, J., Manjunath, B.: Brain tumor segmentation and tractographic feature extraction from structural MR images for overall survival prediction. arXiv preprint arXiv:1807.07716 (2018)
Le, T.H.N., Gummadi, R., Savvides, M.: Deep recurrent level set for segmenting brain tumors. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 646–653. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_74
Manders, J., Marchiori, E., van Laarhoven, T.: Simple domain adaptation with class prediction uncertainty alignment. arXiv preprint arXiv:1804.04448 (2018)
Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Robnik-Šikonja, M., Kononenko, I.: Explaining classifications for individual instances. IEEE Trans. Knowl. Data Eng. 20(5), 589–600 (2008)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 178–190. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75238-9_16
Zhang, Y., Brady, M., Smith, S.: Segmentation of brain MR images through a hidden markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging 20(1), 45–57 (2001)
Zhang, Y., Funkhouser, T.: Deep depth completion of a single RGB-D image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 175–185 (2018)
Zheng, S., et al.: Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1529–1537 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Shen, Y., Gao, M. (2019). Brain Tumor Segmentation on MRI with Missing Modalities. In: Chung, A., Gee, J., Yushkevich, P., Bao, S. (eds) Information Processing in Medical Imaging. IPMI 2019. Lecture Notes in Computer Science(), vol 11492. Springer, Cham. https://doi.org/10.1007/978-3-030-20351-1_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-20351-1_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20350-4
Online ISBN: 978-3-030-20351-1
eBook Packages: Computer ScienceComputer Science (R0)