Skip to main content

Generative Adversarial Network for Medical Images (MI-GAN)

Journal of Medical Systems volume 42, Article number: 231 (2018) Cite this article

Abstract

Deep learning algorithms produces state-of-the-art results for different machine learning and computer vision tasks. To perform well on a given task, these algorithms require large dataset for training. However, deep learning algorithms lack generalization and suffer from over-fitting whenever trained on small dataset, especially when one is dealing with medical images. For supervised image analysis in medical imaging, having image data along with their corresponding annotated ground-truths is costly as well as time consuming since annotations of the data is done by medical experts manually. In this paper, we propose a new Generative Adversarial Network for Medical Imaging (MI-GAN). The MI-GAN generates synthetic medical images and their segmented masks, which can then be used for the application of supervised analysis of medical images. Particularly, we present MI-GAN for synthesis of retinal images. The proposed method generates precise segmented images better than the existing techniques. The proposed model achieves a dice coefficient of 0.837 on STARE dataset and 0.832 on DRIVE dataset which is state-of-the-art performance on both the datasets.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. http://cecas.clemson.edu/~ahoover/stare/

  2. https://www.isi.uu.nl/Research/Databases/DRIVE/

References

  1. de Vos, B.D., Wolterink, J.M., de Jong, P.A., Viergever, M.A., and Išgum, I.: 2D image classification for 3D anatomy localization: employing deep convolutional neural networks. In: Medical imaging 2016: Image processing, vol. 9784, international society for optics and photonics, 2016.

  2. Cai, Y., Landis, M., Laidley, D.T., Kornecki, A., Lum, A., and Li, S., Multi-modal vertebrae recognition using transformed deep convolution network. Comput. Med. Imaging Graph. 51:11–19, 2016.

    Article  Google Scholar 

  3. Chen, H., Ni, D., Qin, J., Li, S., Yang, X., Wang, T., and Heng, P.A., Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J. of Biomedical and Health Informatics 19 (5):1627–1636, 2015.

    Article  Google Scholar 

  4. Kumar, A., Sridar, P., Quinton, A., Kumar, R.K., Feng, D., Nanan, R., and Kim, J.: Plane identification in fetal ultrasound images using saliency maps and convolutional neural networks. In: Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on, IEEE, pp. 791–794, 2016.

  5. Ghesu, F.C., Georgescu, B., Mansi, T., Neumann, D., Hornegger, J., and Comaniciu, D.: An artificial agent for anatomical landmark detection in medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 229–237, 2016.

  6. Baumgartner, C.F., Kamnitsas, K., Matthew, J., Smith, S., Kainz, B., and Rueckert, D.: Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 203–211, 2016.

  7. Kong, B., Zhan, Y., Shin, M., Denny, T., and Zhang, S.: Recognizing end-diastole and end-systole frames via deep temporal regression network. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 264–272, 2016.

  8. Barbu, A., Lu, L., Roth, H., Seff, A., and Summers, R.M., An analysis of robust cost functions for cnn in computer-aided diagnosis. Comput. Methods Biomechanics Biomedical Eng.: Imaging & Vis. 6(3):253–258, 2018.

    Google Scholar 

  9. Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Kim, L., and Summers, R.M., Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5):1170–1181, 2016.

    Article  Google Scholar 

  10. Teramoto, A., Fujita, H., Yamamuro, O., and Tamaki, T., Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique. Med. Phys. 43(6Part1):2821–2827, 2016.

    Article  Google Scholar 

  11. Becker, C., Rigamonti, R., Lepetit, V., and Fua, P.: Supervised feature learning for curvilinear structure segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 526–533, 2013.

  12. Makhzani, A., and Frey, B.J.: PixelGAN autoencoders. In: Advances in Neural Information Processing Systems, pp. 1972–1982, 2017.

  13. Ganin, Y., and Lempitsky, V.: N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms. In: Asian Conference on Computer Vision, Springer, pp. 536–551, 2014.

  14. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A., van Ginneken, B., and Sánchez, C.I., A survey on deep learning in medical image analysis. Med. Image Anal. 42:60–88, 2017.

    Article  Google Scholar 

  15. Gatys, L.A., Ecker, A.S., and Bethge, M.: A neural algorithm of artistic style, arXiv:1508.06576

  16. Ulyanov, D., Lebedev, V., Vedaldi, A., and Lempitsky, V.S.. In: ICML, pp. 1349–1357, 2016.

  17. Johnson, J., Alahi, A., and Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, Springer, pp. 694–711, 2016.

  18. Kirbas, C., and Quek, F., A review of vessel extraction techniques and algorithms. ACM Comput. Surveys (CSUR) 36(2):81–121, 2004.

    Article  Google Scholar 

  19. Lesage, D., Angelini, E.D., Bloch, I., and Funka-Lea, G., A review of 3D vessel lumen segmentation techniques: models, features and extraction schemes. Med. Image Anal. 13(6):819–845, 2009.

    Article  Google Scholar 

  20. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680, 2014.

  21. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, Vol. 4, p. 12, 2017.

  22. Ding, C., Xia, Y., and Li, Y.: Supervised segmentation of vasculature in retinal images using neural networks. In: Orange technologies (ICOT), 2014 IEEE International Conference on, IEEE, pp. 49–52, 2014.

  23. Radford, A., Metz, L., and Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv:1511.06434

  24. Mirza, M., and Osindero, S.: Conditional generative adversarial nets, arXiv:1411.1784

  25. Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1486–1494, 2015.

  26. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5769–5779, 2017.

  27. Peng, H., Hawrylycz, M., Roskams, J., Hill, S., Spruston, N., Meijering, E., and Ascoli, G.A., Bigneuron: large-scale 3d neuron reconstruction from optical microscopy images. Neuron 87(2):252–256, 2015.

    CAS  Article  Google Scholar 

  28. Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976, 2017.

  29. Zhang, J., Chen, L., Zhuo, L., Liang, X., and Li, J., An Efficient Hyperspectral Image Retrieval Method: Deep Spectral-Spatial Feature Extraction with DCGAN and Dimensionality Reduction Using t-SNE-based NM Hashing. Remote Sens. 10(2):271, 2018.

    Article  Google Scholar 

  30. Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., and Fergus, R.: Regularization of neural networks using dropconnect. In: International Conference on Machine Learning, pp. 1058–1066, 2013.

  31. Wang, X., and Gupta, A.: Generative image modeling using style and structure adversarial networks. In: European Conference on Computer Vision, Springer, pp. 318–335, 2016.

  32. Mao, X., Shen, C., and Yang, Y.-B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in neural information processing systems, pp. 2802–2810, 2016.

  33. Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251, 2017.

  34. Ronneberger, O., Fischer, P., and Brox, T.: U-NET: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, Springer, pp. 234–241, 2015.

  35. Zhao, H., Li, H., and Cheng, L.: Synthesizing Filamentary Structured Images with GANs, arXiv:1706.02185

  36. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L., Deeplab: Semantic image segmentation with deep convolutional nets, Atrous Convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4):834–848, 2018.

    Article  Google Scholar 

  37. Li, Y., Wang, N., Shi, J., Liu, J., and Hou, X., Revisiting batch normalization for practical domain adaptation. Pattern Recogn. 80:109–117, 2018.

    Article  Google Scholar 

  38. Zhang, W., Li, W., Yan, J., Yu, L., and Pan, C., Adaptive threshold selection for background removal in fringe projection profilometry. Opt. Lasers Eng. 90:209–216, 2017.

    Article  Google Scholar 

  39. Son, J., Park, S.J., and Jung, K.-H.: Retinal vessel segmentation in fundoscopic images with generative adversarial networks, arXiv:1706.09318

  40. Maninis, K.-K., Pont-Tuset, J., Arbeláez, P., and Van Gool, L.: Deep retinal image understanding. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 140–148, 2016.

  41. Farokhian, F., Yang, C., Demirel, H., Wu, S., and Beheshti, I., Automatic parameters selection of Gabor filters with the imperialism competitive algorithm with application to retinal vessel segmentation. Biocybernetics Biomedical Eng. 37(1):246–254, 2017.

    Article  Google Scholar 

  42. Xie, S., and Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp. 1395–1403, 2015.

Download references

Funding

No funding declared.

Author information

Affiliations

  1. Department of Electrical Engineering, COMSATS University Islamabad, Abbottabad Campus, Islamabad, Pakistan

    Talha Iqbal & Hazrat Ali

Authors
  1. Talha Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hazrat Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hazrat Ali.

Ethics declarations

Conflict of interests

Talha Iqbal declares that he has no conflict of interest. Hazrat Ali declares that he has no conflict of interest.

Additional information

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Not applicable.

This article is part of the Topical Collection on Advanced Computational Intelligence and Soft Computing in Medical Imaging

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Iqbal, T., Ali, H. Generative Adversarial Network for Medical Images (MI-GAN). J Med Syst 42, 231 (2018). https://doi.org/10.1007/s10916-018-1072-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-018-1072-9

Keywords

  • GAN
  • Medical imaging
  • Style transfer
  • Deep learning
  • Retinal images