Abstract
Age-Related Macular Degeneration (AMD) is an asymptomatic retinal disease which may result in loss of vision. There is limited access to high-quality relevant retinal images and poor understanding of the features defining sub-classes of this disease. Motivated by recent advances in machine learning we specifically explore the potential of generative modeling, using Generative Adversarial Networks (GANs) and style transferring, to facilitate clinical diagnosis and disease understanding by feature extraction. We design an analytic pipeline which first generates synthetic retinal images from clinical images; a subsequent verification step is applied. In the synthesizing step we merge GANs (DCGANs and WGANs architectures) and style transferring for the image generation, whereas the verified step controls the accuracy of the generated images. We find that the generated images contain sufficient pathological details to facilitate ophthalmologists’ task of disease classification and in discovery of disease relevant features. In particular, our system predicts the drusen and geographic atrophy sub-classes of AMD. Furthermore, the performance using CFP images for GANs outperforms the classification based on using only the original clinical dataset. Our results are evaluated using existing classifier of retinal diseases and class activated maps, supporting the predictive power of the synthetic images and their utility for feature extraction. Our code examples are available online. (https://github.com/huckiyang/EyeNet-GANs).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer. CoRR, abs/1703.07511 2 (2017)
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Universal style transfer via feature transforms. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30, pp. 386–396. Curran Associates, Inc. (2017)
Salehinejad, H., Valaee, S., Dowdell, T., Colak, E., Barfett, J.: Generalization of deep neural networks for chest pathology classification in x-rays using generative adversarial networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 990–994. IEEE (2018)
Klein, R., Klein, B.E., Knudtson, M.D., Meuer, S.M., Swift, M., Gangnon, R.E.: Fifteen-year cumulative incidence of age-related macular degeneration: the beaver dam eye study. Ophthalmology 114, 253–262 (2007)
Green, W.R., McDonnell, P.J., Yeo, J.H.: Pathologic features of senile macular degeneratlon. Ophthalmology 92, 615–627 (1985)
Gheorghe, A., Mahdi, L., Musat, O.: Age-related macular degeneration. Rom. J. Ophthalmol. 59, 74–77 (2015)
Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: CVPR (2016)
Yang, C.H.H., et al.: A novel hybrid machine learning model for auto-classification of retinal diseases. arXiv preprint arXiv:1806.06423 (2018)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223 (2017)
Pumarola, A., Agudo, A., Martinez, A.M., Sanfeliu, A., Moreno-Noguer, F.: GANimation: Anatomically-Aware Facial Animation from a Single Image. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 835–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_50
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: CVPR, vol. 2, p. 6 (2017)
Qian, X., et al.: Pose-normalized image generation for person re-identification. arXiv preprint arXiv:1712.02225 (2017)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
Li, Y., Liu, M.Y., Li, X., Yang, M.H., Kautz, J.: A closed-form solution to photorealistic image stylization. arXiv preprint arXiv:1802.06474 (2018)
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3462–3471. IEEE (2017)
Staal, J., Abràmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. TMI 23, 501–509 (2004)
Retina Image Bank: A project from the American Society of Retina Specialists. http://imagebank.asrs.org/about. Accessed 30 June 2018
Beers, A., et al.: High-resolution medical image synthesis using progressively grown generative adversarial networks. arXiv preprint arXiv:1805.03144 (2018)
Guibas, J.T., Virdi, T.S., Li, P.S.: Synthetic medical images from dual generative adversarial networks. arXiv preprint arXiv:1709.01872 (2017)
Kaur, J., Mittal, D.: Segmentation and measurement of exudates in fundus images of the retina for detection of retinal disease. J. Biomed. Eng. Med. Imaging 2, 27 (2015)
Yang, C.H.H., et al.: Auto-classification of retinal diseases in the limit of sparse data using a two-streams machine learning model. arXiv preprint arXiv:1808.05754 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)
Coleman, H.R., Chan, C.C., Ferris, F.L., Chew, E.Y.: Age-related macular degeneration. Lancet 372, 1835–1845 (2008)
Mewis, L., Young, S.E.: Breast carcinoma metastatic to the choroid: analysis of G7 patients. Ophthalmology 89, 147–151 (1982)
Crick, R.P., Khaw, P.T.: A Textbook of Clinical Ophthalmology: A Practical Guide to Disorders of the Eyes and Their Management. World Scientific, Singapore (1998)
Akram, I., Rubinstein, A.: Common retinal signs. an overview. Optometry Today (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, YC. et al. (2019). Synthesizing New Retinal Symptom Images by Multiple Generative Models. In: Carneiro, G., You, S. (eds) Computer Vision – ACCV 2018 Workshops. ACCV 2018. Lecture Notes in Computer Science(), vol 11367. Springer, Cham. https://doi.org/10.1007/978-3-030-21074-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-21074-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-21073-1
Online ISBN: 978-3-030-21074-8
eBook Packages: Computer ScienceComputer Science (R0)