Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11037)


Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of the largest challenges facing machine learning in medical imaging, namely the small incidence of pathological findings, and the restrictions around sharing of patient data.


Generative models GAN Image synthesis Deep learning Brain tumor Magnetic resonance imaging MRI Segmentation 


  1. 1.
    Ashburner, J., Friston, K.J.: Unified segmentation. Neuroimage 26(3), 839–851 (2005)CrossRefGoogle Scholar
  2. 2.
    Carlini, N., Liu, C., Kos, J., Erlingsson, Ú., Song, D.: The secret sharer: measuring unintended neural network memorization & extracting secrets. arXiv preprint arXiv:1802.08232 (2018)
  3. 3.
    Chartsias, A., Joyce, T., Giuffrida, M.V., Tsaftaris, S.A.: Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans. Med. Imaging 37(3), 803–814 (2018)CrossRefGoogle Scholar
  4. 4.
    Costa, P.: End-to-end adversarial retinal image synthesis. IEEE Trans. Med. Imaging 37(3), 781–791 (2018)CrossRefGoogle Scholar
  5. 5.
    Dar, S.U.H., Yurt, M., Karacan, L., Erdem, A., Erdem, E., Çukur, T.: Image synthesis in multi-contrast MRI with conditional generative adversarial networks. arXiv preprint arXiv:1802.01221 (2018)
  6. 6.
    Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)CrossRefGoogle Scholar
  7. 7.
    Frangi, A.F., Tsaftaris, S.A., Prince, J.L.: Simulation and synthesis in medical imaging. IEEE Trans. Med. Imaging 37(3), 673–679 (2018)CrossRefGoogle Scholar
  8. 8.
    Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Synthetic data augmentation using GAN for improved liver lesion classification. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2018)Google Scholar
  9. 9.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  10. 10.
    Hu, Y., et al.: Intraoperative organ motion models with an ensemble of conditional generative adversarial networks. In: Descoteaux, M., et al. (eds.) MICCAI 2017 Part II. LNCS, vol. 10434, pp. 368–376. Springer, Cham (2017). Scholar
  11. 11.
    Iglesias, J.E., Liu, C.-Y., Thompson, P.M., Tu, Z.: Robust brain extraction across datasets and comparison with publicly available methods. IEEE Trans. Med. Imaging 30(9), 1617–1634 (2011)CrossRefGoogle Scholar
  12. 12.
    Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  13. 13.
    Mahapatra, D., Bozorgtabar, B., Hewavitharanage, S., Garnavi, R.: Image super resolution using generative adversarial networks and local saliency maps for retinal image analysis. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017 Part III. LNCS, vol. 10435, pp. 382–390. Springer, Cham (2017). Scholar
  14. 14.
    Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)CrossRefGoogle Scholar
  15. 15.
    Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)Google Scholar
  16. 16.
    Nie, D., et al.: Medical image synthesis with context-aware generative adversarial networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017 Part III. LNCS, vol. 10435, pp. 417–425. Springer, Cham (2017). Scholar
  17. 17.
    Christopher, G.S., et al.: A large-scale comparison of cortical thickness and volume methods for measuring Alzheimer’s disease severity. NeuroImage: Clin. 11, 802–812 (2016)CrossRefGoogle Scholar
  18. 18.
    Shin, H.-C., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016)CrossRefGoogle Scholar
  19. 19.
    Tustison, N.J., et al.: Large-scale evaluation of ANTS and FreeSurfer cortical thickness measurements. Neuroimage 99, 166–179 (2014)CrossRefGoogle Scholar
  20. 20.
    Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. arXiv preprint arXiv:1709.00382 (2017)
  21. 21.
    Yang, D., et al.: Automatic liver segmentation using an adversarial image-to-image network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017 Part III. LNCS, vol. 10435, pp. 507–515. Springer, Cham (2017). Scholar
  22. 22.
    Zhang, Y., Yang, L., Chen, J., Fredericksen, M., Hughes, D.P., Chen, D.Z.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017 Part III. LNCS, vol. 10435, pp. 408–416. Springer, Cham (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.NVIDIA CorporationSanta ClaraUSA
  2. 2.MGH & BWH Center for Clinical Data ScienceBostonUSA
  3. 3.Mayo ClinicRochesterUSA

Personalised recommendations