Abstract
Deep Learning (DL)-based segmentation methods have been quite successful in various medical imaging applications. The main bottleneck of these methods is the scarcity of quality-labelled samples needed for their training. The lack of labelled training data is often addressed by augmentation methods, which aim to synthesise realistic samples with corresponding labels. While the synthesis of realistic samples remains a challenging task, little is known about the impact of fine detail in synthetic data on the performance of DL-based segmentation models. In this work, we investigate whether, and to what extent, the high-frequency (HF) detail in synthetic brain MR images (MRIs) impacts the performance of DL-based segmentation methods. To assess the impact of HF detail, we generate two synthetic datasets, with and without HF detail and train corresponding segmentation models to evaluate the impact on their performance. The results obtained demonstrate that the presence of HF detail in synthetic brain MRIs, used during training, significantly improve the Dice score up to 1.73% for Gray Matter (GM), 1.34% for White Matter (WM) and 4.41% for Cerebrospinal Fluid (CSF); and therefore justify the need for synthesising realistic-looking MRIs.
Keywords
- Data augmentation
- Brain MRI
- Generative adversarial network
- Realistic brain MRI synthesis
This work was funded in part through an Australian Department of Industry, Energy and Resources CRC-P project between CSIRO, Maxwell Plus and I-Med Radiology Network.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner,MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). For up-to-date information, see www.adni-info.org.
- 2.
The pytorch implementation of vox2vox taken from https://github.com/enochkan/vox2vox.
References
Acosta, O., et al.: Automated voxel-based 3D cortical thickness measurement in a combined Lagrangian-Eulerian PDE approach using partial volume maps. Med. Image Anal. 13(5), 730–743 (2009)
Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)
Avants, B.B., Tustison, N., Song, G.: Advanced normalization tools (ANTS). Insight J 2(365), 1–35 (2009)
Chen, C., et al.: Realistic adversarial data augmentation for MR image segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 667–677. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_65
Cirillo, M.D., Abramian, D., Eklund, A.: Vox2vox: 3D-GAN for brain tumour segmentation. arXiv preprint arXiv:2003.13653 (2020)
Coupé, P., et al.: AssemblyNet: a large ensemble of CNNs for 3D whole brain MRI segmentation. Neuroimage 219, 117026 (2020)
Eaton-Rosen, Z., Bragman, F., Ourselin, S., Cardoso, M.J.: Improving data augmentation for medical image segmentation (2018)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680 (2014)
Henschel, L., Conjeti, S., Estrada, S., Diers, K., Fischl, B., Reuter, M.: FastSurfer-a fast and accurate deep learning based neuroimaging pipeline. NeuroImage 117012 (2020)
Huang, S.G., Chung, M.K., Qiu, A., Initiative, A.D.N.: Fast mesh data augmentation via chebyshev polynomial of spectral filtering. arXiv preprint arXiv:2010.02811 (2020)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Jack Jr., C.R., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging: Off. J. Int. Soc. Magn. Reson. Med. 27(4), 685–691 (2008)
Jog, A., Carass, A., Roy, S., Pham, D.L., Prince, J.L.: MR image synthesis by contrast learning on neighborhood ensembles. Med. Image Anal. 24(1), 63–76 (2015)
Keong, C.C., Wei, H.E.T.: Synthesis of 3D MRI brain images with shape and texture generative adversarial deep neural networks. IEEE Access 9, 64747–64760 (2021)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
Lee, J., Kim, E., Lee, S., Lee, J., Yoon, S.: FickleNet: weakly and semi-supervised semantic image segmentation using stochastic inference. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5267–5276 (2019)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roy, A.G., Conjeti, S., Navab, N., Wachinger, C., Initiative, A.D.N., et al.: QuickNAT: a fully convolutional network for quick and accurate segmentation of neuroanatomy. Neuroimage 186, 713–727 (2019)
Rusak, F., et al.: 3D brain MRI GAN-based synthesis conditioned on partial volume maps. In: Burgos, N., Svoboda, D., Wolterink, J.M., Zhao, C. (eds.) SASHIMI 2020. LNCS, vol. 12417, pp. 11–20. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59520-3_2
Sun, L., Chen, J., Xu, Y., Gong, M., Yu, K., Batmanghelich, K.: Hierarchical amortized training for memory-efficient high resolution 3D GAN. arXiv preprint arXiv:2008.01910 (2020)
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J.N., Wu, Z., Ding, X.: Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation. Med. Image Anal. 63, 101693 (2020)
Uzunova, H., Ehrhardt, J., Jacob, F., Frydrychowicz, A., Handels, H.: Multi-scale GANs for memory-efficient generation of high resolution medical images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 112–120. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_13
Van Leemput, K., Maes, F., Vandermeulen, D., Suetens, P.: Automated model-based bias field correction of MR images of the brain. IEEE Trans. Med. Imaging 18(10), 885–896 (1999)
Van Leemput, K., Maes, F., Vandermeulen, D., Suetens, P.: Automated model-based tissue classification of MR images of the brain. IEEE Trans. Med. Imaging 18(10), 897–908 (1999)
Wang, J., Chen, Y., Wu, Y., Shi, J., Gee, J.: Enhanced generative adversarial network for 3D brain MRI super-resolution. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3627–3636 (2020)
Weiner, M.W., et al.: The Alzheimer’s disease neuroimaging initiative 3: continued innovation for clinical trial improvement. Alzheimer’s Dementia 13(5), 561–571 (2017)
Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review. Med. Image Anal. 58, 101552 (2019)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
Zhao, A., Balakrishnan, G., Durand, F., Guttag, J.V., Dalca, A.V.: Data augmentation using learned transformations for one-shot medical image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8543–8553 (2019)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Zuo, L., et al.: Synthesizing realistic brain MR images with noise control. In: Burgos, N., Svoboda, D., Wolterink, J.M., Zhao, C. (eds.) SASHIMI 2020. LNCS, vol. 12417, pp. 21–31. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59520-3_3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Rusak, F. et al. (2021). Detail Matters: High-Frequency Content for Realistic Synthetic MRI Generation. In: Svoboda, D., Burgos, N., Wolterink, J.M., Zhao, C. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2021. Lecture Notes in Computer Science(), vol 12965. Springer, Cham. https://doi.org/10.1007/978-3-030-87592-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-87592-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87591-6
Online ISBN: 978-3-030-87592-3
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
