Abstract
Deep learning algorithms produces state-of-the-art results for different machine learning and computer vision tasks. To perform well on a given task, these algorithms require large dataset for training. However, deep learning algorithms lack generalization and suffer from over-fitting whenever trained on small dataset, especially when one is dealing with medical images. For supervised image analysis in medical imaging, having image data along with their corresponding annotated ground-truths is costly as well as time consuming since annotations of the data is done by medical experts manually. In this paper, we propose a new Generative Adversarial Network for Medical Imaging (MI-GAN). The MI-GAN generates synthetic medical images and their segmented masks, which can then be used for the application of supervised analysis of medical images. Particularly, we present MI-GAN for synthesis of retinal images. The proposed method generates precise segmented images better than the existing techniques. The proposed model achieves a dice coefficient of 0.837 on STARE dataset and 0.832 on DRIVE dataset which is state-of-the-art performance on both the datasets.
Similar content being viewed by others
References
de Vos, B.D., Wolterink, J.M., de Jong, P.A., Viergever, M.A., and Išgum, I.: 2D image classification for 3D anatomy localization: employing deep convolutional neural networks. In: Medical imaging 2016: Image processing, vol. 9784, international society for optics and photonics, 2016.
Cai, Y., Landis, M., Laidley, D.T., Kornecki, A., Lum, A., and Li, S., Multi-modal vertebrae recognition using transformed deep convolution network. Comput. Med. Imaging Graph. 51:11–19, 2016.
Chen, H., Ni, D., Qin, J., Li, S., Yang, X., Wang, T., and Heng, P.A., Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J. of Biomedical and Health Informatics 19 (5):1627–1636, 2015.
Kumar, A., Sridar, P., Quinton, A., Kumar, R.K., Feng, D., Nanan, R., and Kim, J.: Plane identification in fetal ultrasound images using saliency maps and convolutional neural networks. In: Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on, IEEE, pp. 791–794, 2016.
Ghesu, F.C., Georgescu, B., Mansi, T., Neumann, D., Hornegger, J., and Comaniciu, D.: An artificial agent for anatomical landmark detection in medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 229–237, 2016.
Baumgartner, C.F., Kamnitsas, K., Matthew, J., Smith, S., Kainz, B., and Rueckert, D.: Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 203–211, 2016.
Kong, B., Zhan, Y., Shin, M., Denny, T., and Zhang, S.: Recognizing end-diastole and end-systole frames via deep temporal regression network. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 264–272, 2016.
Barbu, A., Lu, L., Roth, H., Seff, A., and Summers, R.M., An analysis of robust cost functions for cnn in computer-aided diagnosis. Comput. Methods Biomechanics Biomedical Eng.: Imaging & Vis. 6(3):253–258, 2018.
Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Kim, L., and Summers, R.M., Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5):1170–1181, 2016.
Teramoto, A., Fujita, H., Yamamuro, O., and Tamaki, T., Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique. Med. Phys. 43(6Part1):2821–2827, 2016.
Becker, C., Rigamonti, R., Lepetit, V., and Fua, P.: Supervised feature learning for curvilinear structure segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 526–533, 2013.
Makhzani, A., and Frey, B.J.: PixelGAN autoencoders. In: Advances in Neural Information Processing Systems, pp. 1972–1982, 2017.
Ganin, Y., and Lempitsky, V.: N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms. In: Asian Conference on Computer Vision, Springer, pp. 536–551, 2014.
Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A., van Ginneken, B., and Sánchez, C.I., A survey on deep learning in medical image analysis. Med. Image Anal. 42:60–88, 2017.
Gatys, L.A., Ecker, A.S., and Bethge, M.: A neural algorithm of artistic style, arXiv:1508.06576
Ulyanov, D., Lebedev, V., Vedaldi, A., and Lempitsky, V.S.. In: ICML, pp. 1349–1357, 2016.
Johnson, J., Alahi, A., and Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, Springer, pp. 694–711, 2016.
Kirbas, C., and Quek, F., A review of vessel extraction techniques and algorithms. ACM Comput. Surveys (CSUR) 36(2):81–121, 2004.
Lesage, D., Angelini, E.D., Bloch, I., and Funka-Lea, G., A review of 3D vessel lumen segmentation techniques: models, features and extraction schemes. Med. Image Anal. 13(6):819–845, 2009.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680, 2014.
Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, Vol. 4, p. 12, 2017.
Ding, C., Xia, Y., and Li, Y.: Supervised segmentation of vasculature in retinal images using neural networks. In: Orange technologies (ICOT), 2014 IEEE International Conference on, IEEE, pp. 49–52, 2014.
Radford, A., Metz, L., and Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv:1511.06434
Mirza, M., and Osindero, S.: Conditional generative adversarial nets, arXiv:1411.1784
Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1486–1494, 2015.
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5769–5779, 2017.
Peng, H., Hawrylycz, M., Roskams, J., Hill, S., Spruston, N., Meijering, E., and Ascoli, G.A., Bigneuron: large-scale 3d neuron reconstruction from optical microscopy images. Neuron 87(2):252–256, 2015.
Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976, 2017.
Zhang, J., Chen, L., Zhuo, L., Liang, X., and Li, J., An Efficient Hyperspectral Image Retrieval Method: Deep Spectral-Spatial Feature Extraction with DCGAN and Dimensionality Reduction Using t-SNE-based NM Hashing. Remote Sens. 10(2):271, 2018.
Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., and Fergus, R.: Regularization of neural networks using dropconnect. In: International Conference on Machine Learning, pp. 1058–1066, 2013.
Wang, X., and Gupta, A.: Generative image modeling using style and structure adversarial networks. In: European Conference on Computer Vision, Springer, pp. 318–335, 2016.
Mao, X., Shen, C., and Yang, Y.-B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in neural information processing systems, pp. 2802–2810, 2016.
Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251, 2017.
Ronneberger, O., Fischer, P., and Brox, T.: U-NET: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, Springer, pp. 234–241, 2015.
Zhao, H., Li, H., and Cheng, L.: Synthesizing Filamentary Structured Images with GANs, arXiv:1706.02185
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L., Deeplab: Semantic image segmentation with deep convolutional nets, Atrous Convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4):834–848, 2018.
Li, Y., Wang, N., Shi, J., Liu, J., and Hou, X., Revisiting batch normalization for practical domain adaptation. Pattern Recogn. 80:109–117, 2018.
Zhang, W., Li, W., Yan, J., Yu, L., and Pan, C., Adaptive threshold selection for background removal in fringe projection profilometry. Opt. Lasers Eng. 90:209–216, 2017.
Son, J., Park, S.J., and Jung, K.-H.: Retinal vessel segmentation in fundoscopic images with generative adversarial networks, arXiv:1706.09318
Maninis, K.-K., Pont-Tuset, J., Arbeláez, P., and Van Gool, L.: Deep retinal image understanding. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 140–148, 2016.
Farokhian, F., Yang, C., Demirel, H., Wu, S., and Beheshti, I., Automatic parameters selection of Gabor filters with the imperialism competitive algorithm with application to retinal vessel segmentation. Biocybernetics Biomedical Eng. 37(1):246–254, 2017.
Xie, S., and Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp. 1395–1403, 2015.
Funding
No funding declared.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
Talha Iqbal declares that he has no conflict of interest. Hazrat Ali declares that he has no conflict of interest.
Additional information
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Not applicable.
This article is part of the Topical Collection on Advanced Computational Intelligence and Soft Computing in Medical Imaging
Rights and permissions
About this article
Cite this article
Iqbal, T., Ali, H. Generative Adversarial Network for Medical Images (MI-GAN). J Med Syst 42, 231 (2018). https://doi.org/10.1007/s10916-018-1072-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-018-1072-9