Abstract
Generative adversarial networks have already found widespread use for the formation of artificial, but realistic images of a wide variety of content, including medical imaging. Mostly they are considered to be used for expanding and augmenting datasets in order to improve accuracy of neural networks classification. In this paper we discuss the problem of evaluating the quality of computer tomography images of lung cancer, which is characterized by small size of nodules, synthesized using two different generative adversarial network, architectures – for 2D and 3D dimensions. We select the set of metrics for estimating the quality of the generated images, including Visual Turing Test, FID and MRR metrics; then we carry out a problem-oriented modification of the Turing test in order to adapt it both to the actually obtained images and to resource constraints. We compare the constructed GANs using the selected metrics; and we show that such a parameter as the size of the generated image is very important in the development of the GAN architecture. We consider that with this work we have for the first time shown that for small neo-plasms, direct scaling of the corresponding solutions used to generate large neo-plasms (for example, gliomas) is ineffective. Developed assessment methods have shown that additional techniques like MIP and special combinations of metrics are required to generate small neoplasms. In addition, an important conclusion can be considered that it is very important to use GAN networks not only, as is usually the case, for augmentation and expansion of the datasets, but for direct use in clinical practice by radiologists.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Goodfellow I.J., et al.: Generative Adversarial Nets. arXiv:1406.2661v1 [stat.ML] 10 Jun 2014
Tschuchnig, M.E., Oostingh, G.J., Gadermayr, M.: Generative adversarial networks in digital pathology: a survey on trends and future potential. Patterns 1(6), 100089 (2020)
Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review. Med. Image Anal. 58, 101552 (2019)
Kang, H., et al.: Visual and quantitative evaluation of amyloid brain PET image synthesis with generative adversarial network. Appl. Sci. 10, 2628 (2020)
Bargsten, L., Schlaefer, A.: SpeckleGAN: a generative adversarial network with an adaptive speckle layer to augment limited training data for ultrasound image processing. Int. J. Comput. Assist. Radiol. Surg. 15(9), 1427–1436 (2020). https://doi.org/10.1007/s11548-020-02203-1
Levy, J.J., et al.: A large-scale internal validation study of unsupervised virtual trichrome staining technologies on nonalcoholic steatohepatitis liver biopsies. Mod. Pathol. 34(4), 808–822 (2020). https://doi.org/10.1038/s41379-020-00718-1
Borji, A.: Pros and xons of GAN evaluation measures. arXiv:1802.03446v5 [cs.CV] 24 Oct 2018
Kazeminia, S., et al.: GANs for medical image analysis. arXiv.org > cs > arXiv:1809.06222v3. 9 Oct 2019
Wang, T., et al.: A review on medical imaging synthesis using deep learning and its clinical applications. J. Appl. Clin. Vedical Phys. 22(1), 11–36 (2021)
Chuquicusma, M.J.M., Hussein, S., Burt, J., Bagci, U.: How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2018)
Onishi, Y., et al.: Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. Hindawi BioMed. Res. Int. 2019(6051939), 1–9 (2019)
Wang, Y., Zhou, L., Wang, M., et al.: Combination of generative adversarial network and convolutional neural network for automatic subcentimeter pulmonary adenocarcinoma classification. Quant. Imaging Med. Surg. 10(6), 1249–1264 (2020)
Han, C., et al.: Synthesizing diverse lung nodules wherever massively: 3D multi-conditional GAN-based CT image augmentation for object detection. In: 2019 International Conference on 3D Vision (3DV), Quebec City, QC, Canada, pp. 729–737 (2019). https://doi.org/10.1109/3DV.2019.00085
Shi, H., Lu, J., Zhou, Q.: A novel data augmentation method using style-based GAN for robust pulmonary nodule segmentation. In: 2020 Chinese Control and Decision Conference (CCDC), Hefei, China, pp. 2486-2491 (2020)
Jin, D., Xu, Z., Tang, Y., Harrison, A.P., Mollura, D.J.: CT-realistic lung nodule simulation from 3D conditional generative adversarial networks for robust lung segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. LNCS, vol. 11071, pp. 732–740. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_81
Gao, C., et al.: Augmenting LIDC dataset using 3D generative adversarial networks to improve lung nodule detection. Medical imaging 2019: computer-aided diagnosis. In: International Society for Optics and Photonics, vol. 10950 (2019)
Mirsky, Y. et al.: CT-GAN: malicious tampering of 3D medical imagery using deep learning. In: 28th {USENIX}. Security Symposium ({USENIX} Security 2019), pp. 461–478 (2019)
Han, C., et al.: Learning more with less: conditional PGGAN based data augmentation for brain metastases detection using highly-rough annotation on MR images. In: Proceedings of ACM International Conference on Information and Knowledge Management (CIKM) (2019)
Zhang, J., Xia, Y., Zeng, H., Zhang, Y.: NODULe: combining constrained multi-scale LoG filters with densely dilated 3D deep convolutional neural network for pulmonary nodule detection. Neurocomputing 317, 159–167 (2018)
Zheng, S., Guo J., Cui,, X., Veldhuis, R.N.J., Matthijs, O., van Ooijen, P.M.A.: Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection. arXiv:1904.05956 [cs.CV] 10 Jun 2019
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. arXiv:1812.04948v3 [cs.NE] 29 Mar 2019
Chang, A., Suriyakumar, V.M., Moturu, A., et al.: Using generative models for pediatric wbMRI. Medical Imaging with Deep Learning, pp. 1–7 (2020)
Hillis, S.L., Chakraborty, D.P., Orton, C.G.: ROC or FROC? It depends on the research question. Med. Phys. 44(5), 1603–1606 (2017)
Ghosal, S.S., Sarkar, I., Hallaoui, I.E.: Lung nodule classification using convolutional autoencoder and clustering augmented learning method (CALM). http://ceur-ws.org/Vol-2551/paper-05.pdf. Accessed 05 Feb 2021
Salimans, I., Goodfellow, W., Zaremba, V., Cheung, A., Radford, X.: Chen. Improved techniques for training GANs. In: Advances in Neural Information Processing Systems (NIPS), pp. 2234–2242 (2016)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Haarburger C., et al.: Multiparametric magnetic resonance image synthesis using generative adversarial networks. In: Eurographics Workshop on Visual Computing for Biology and Medicine (2019)
Xu, Q., et al.: An empirical study on evaluation metrics of generative adversarial networks. arXiv. arXiv:1806.07755 (2018)
Lopez-Paz, D., Oquab, M.: Revisiting classifier two-sample tests. arXiv. arXiv:1610.06545 (2016)
Gretton, A., Borgwardt, K.M., Rasch, M.J., Schölkopf, B., Smola, A.: A kernel two-sample test. J. Mach. Learn. Res. 13, 723–773 (2012)
Dziugaite, G.K., Roy, D.M., Ghahramani, Z.: Training generative neural networks via maximum mean discrepancy optimization. arXiv preprint arXiv:1505.03906 (2015)
Dowson, D.C., Landau, B.V.: The Fréchet distance between multivariate normal distributions. J. Multivar. Anal. 12(3), 450–455 (1982)
Li, Y., Fan, Y.: DeepSEED: 3D squeeze-and-excitation encoder-decoder convnets for pulmonary nodule detection. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE (2020)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 [cs.CV] 10 Apr 2015
Armato, S.G.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scan. Med. Phys. 38(2), 915–931 (2011)
Feeman, T.G.: The Mathematics of Medical Imaging: A Beginner’s Guide. Springer Undergraduate Texts in Mathematics and Technology. Springer, New York (2010).978-0387927114
Onishi, Y., et al.: Multiplanar analysis for pulmonary nodule classification in CT images using deep convolutional neural network and generative adversarial networks. Int. J. Comput. Assist. Radiol. Surg. 15(1), 173–178 (2019). https://doi.org/10.1007/s11548-019-02092-z
Gwet, K.L.: Computing inter-rater reliability and its variance in the presence of high agreement. Br. J. Math. Stat. Psychol. 61, 29–48 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Semiletov, A. et al. (2021). Comparative Evaluation of Lung Cancer CT Image Synthesis with Generative Adversarial Networks. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12744. Springer, Cham. https://doi.org/10.1007/978-3-030-77967-2_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-77967-2_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77966-5
Online ISBN: 978-3-030-77967-2
eBook Packages: Computer ScienceComputer Science (R0)