Abstract
Although generative adversarial networks (GANs) have shown promise in medical imaging, they have four main limitations that impede their utility: computational cost, data requirements, reliable evaluation measures, and training complexity. Our work investigates each of these obstacles in a novel application of StyleGAN2-ADA to high-resolution medical imaging datasets. Our dataset is comprised of liver-containing axial slices from non-contrast and contrast-enhanced computed tomography (CT) scans. Additionally, we utilized four public datasets composed of various imaging modalities. We trained a StyleGAN2 network with transfer learning (from the Flickr-Faces-HQ dataset) and data augmentation (horizontal flipping and adaptive discriminator augmentation). The network’s generative quality was measured quantitatively with the Fréchet Inception Distance (FID) and qualitatively with a visual Turing test given to seven radiologists and radiation oncologists.
The StyleGAN2-ADA network achieved a FID of 5.22 (±0.17) on our liver CT dataset. It also set new record FIDs of 10.78, 3.52, 21.17, and 5.39 on the publicly available SLIVER07, ChestX-ray14, ACDC, and Medical Segmentation Decathlon (brain tumors) datasets. In the visual Turing test, the clinicians rated generated images as real 42% of the time, approaching random guessing. Our computational ablation study revealed that transfer learning and data augmentation stabilize training and improve the perceptual quality of the generated images. We observed the FID to be consistent with human perceptual evaluation of medical images. Finally, our work found that StyleGAN2-ADA consistently produces high-quality results without hyperparameter searches or retraining.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aleef, T.A., Spadinger, I.T., Peacock, M.D., Salcudean, S.E., Mahdavi, S.S.: Rapid treatment planning for low-dose-rate prostate brachytherapy with TP-GAN. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 581–590. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_56
Anderson, B.M., et al.: Automated contouring of contrast and noncontrast computed tomography liver images with fully convolutional networks. Adv. Radiat. Oncol. 6, 100464 (2021). https://doi.org/10.1016/j.adro.2020.04.023
Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37, 2514–2525 (2018). https://doi.org/10.1109/TMI.2018.2837502
Borji, A.: Pros and cons of GAN evaluation measures. Comput. Vis. Image Underst. 179, 41–65 (2019). https://doi.org/10.1016/j.cviu.2018.10.009
Chen, J., Wei, J., Li, R.: TarGAN: target-aware generative adversarial networks for multi-modality medical image translation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 24–33. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_3
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR 2009, pp. 248–255. IEEE (2009). https://doi.org/10.1109/CVPR.2009.5206848
Fetty, L., et al.: Latent space manipulation for high-resolution medical image synthesis via the StyleGAN. Z. Med. Phys. 30, 305–314 (2020). https://doi.org/10.1016/j.zemedi.2020.05.001
Heimann, T., et al.: Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans. Med. Imaging 28, 1251–1265 (2009). https://doi.org/10.1109/TMI.2009.2013851
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS 2017, pp. 6629–6640. Curran Associates Inc. (2017)
Jiang, Y., Zheng, Y., Jia, W., Song, S., Ding, Y.: Synthesis of contrast-enhanced spectral mammograms from low-energy mammograms using cGAN-based synthesis network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12907, pp. 68–77. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87234-2_7
Jung, E., Luna, M., Park, S.H.: Conditional GAN with an attention-based generator and a 3D discriminator for 3D medical image generation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 318–328. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_31
Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., Aila, T.: Training generative adversarial networks with limited data. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) NeurIPS 2020, vol. 33, pp. 12104–12114. Curran Associates, Inc. (2020)
Karras, T., et al.: Alias-free generative adversarial networks. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) NeurIPS 2021, vol. 34, pp. 852–863. Curran Associates, Inc. (2021)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: CVPR 2019, pp. 4396–4405. IEEE (2019). https://doi.org/10.1109/CVPR.2019.00453
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: CVPR 2020, pp. 8107–8116. IEEE (2020). https://doi.org/10.1109/CVPR42600.2020.00813
Lowekamp, B., Chen, D., Ibanez, L., Blezek, D.: The design of simpleitk. Front. Neuroinform. 7, 45 (2013). https://doi.org/10.3389/fninf.2013.00045
Lučić, M., Kurach, K., Michalski, M., Gelly, S., Bousquet, O.: Are GANs created equal? A large-scale study. In: Bengio, S., et al. (eds.) NeurIPS 2018, vol. 31. Curran Associates, Inc. (2018)
Luo, Y., et al.: 3D transformer-GAN for high-quality PET reconstruction. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 276–285. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_27
Marcus, D.S., Olsen, T.R., Ramaratnam, M., Buckner, R.L.: The extensible neuroimaging archive toolkit: an informatic platform for managing, exploring, and sharing neuroimaging data. Neuroinformatics 5, 11–33 (2007). https://doi.org/10.1385/ni:5:1:11
Montero, A., Bonet-Carne, E., Burgos-Artizzu, X.P.: Generative adversarial networks to improve fetal brain fine-grained plane classification. Sensors 21, 7975 (2021). https://doi.org/10.3390/s21237975
Pang, T., Wong, J.H.D., Ng, W.L., Chan, C.S.: Semi-supervised GAN-based radiomics model for data augmentation in breast ultrasound mass classification. Comput. Methods Programs Biomed. 203, 106018 (2021). https://doi.org/10.1016/j.cmpb.2021.106018
Pocevičiūtė, M., Eilertsen, G., Lundström, C.: Unsupervised anomaly detection in digital pathology using GANs. In: ISBI 2021, pp. 1878–1882 (2021). https://doi.org/10.1109/ISBI48211.2021.9434141
Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 146–157. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_12
Segal, B., Rubin, D.M., Rubin, G., Pantanowitz, A.: Evaluating the clinical realism of synthetic chest X-rays generated using progressively growing GANs. SN Comput. Sci. 2(4), 1–17 (2021). https://doi.org/10.1007/s42979-021-00720-7
Simpson, A.L., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. CoRR abs/1902.09063 (2019). https://doi.org/10.48550/arXiv.1902.09063
Skandarani, Y., Jodoin, P.M., Lalande, A.: GANs for medical image synthesis: an empirical study. CoRR abs/2105.05318 (2021). https://doi.org/10.48550/arXiv.2105.05318
Tronchin, L., Sicilia, R., Cordelli, E., Ramella, S., Soda, P.: Evaluating GANs in medical imaging. In: Engelhardt, S., et al. (eds.) DGM4MICCAI/DALI -2021. LNCS, vol. 13003, pp. 112–121. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88210-5_10
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.: Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: CVPR 2017, pp. 3462–3471. IEEE (2017). https://doi.org/10.1109/CVPR.2017.369
Xun, S., et al.: Generative adversarial networks in medical image segmentation: a review. Comput. Biol. Med. 140, 105063 (2022). https://doi.org/10.1016/j.compbiomed.2021.105063
Acknowledgements
This work was supported by the Tumor Measurement Initiative through the MD Anderson Strategic Initiative Development Program (STRIDE). We thank the NIH Clinical Center for the ChestX-ray14 dataset.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
This image was generated by the baseline StyleGAN2 model (10.43 FID). It was chosen to demonstrate the noise artifacts contained in many of the images generated by the model.
The average generator loss (with standard deviation bars) across training. We see that augmentation not only significantly decreases the loss, but also leads to more stable convergence.
Randomly selected images generated by the pretrained StyleGAN2-ADA model (5.06 FID).
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Woodland, M. et al. (2022). Evaluating the Performance of StyleGAN2-ADA on Medical Images. In: Zhao, C., Svoboda, D., Wolterink, J.M., Escobar, M. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2022. Lecture Notes in Computer Science, vol 13570. Springer, Cham. https://doi.org/10.1007/978-3-031-16980-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-16980-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16979-3
Online ISBN: 978-3-031-16980-9
eBook Packages: Computer ScienceComputer Science (R0)
