Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning

  • Xiangji Pan
  • Kai Jin
  • Jing Cao
  • Zhifang Liu
  • Jian Wu
  • Kun You
  • Yifei Lu
  • Yufeng Xu
  • Zhaoan Su
  • Jiekai Jiang
  • Ke Yao
  • Juan YeEmail author
Retinal Disorders



To automatically detect and classify the lesions of diabetic retinopathy (DR) in fundus fluorescein angiography (FFA) images using deep learning algorithm through comparing 3 convolutional neural networks (CNNs).


A total of 4067 FFA images from Eye Center at the Second Affiliated Hospital of Zhejiang University School of Medicine were annotated with 4 kinds of lesions of DR, including non-perfusion regions (NP), microaneurysms, leakages, and laser scars. Three CNNs including DenseNet, ResNet50, and VGG16 were trained to achieve multi-label classification, which means the algorithms could identify 4 retinal lesions above at the same time.


The area under the curve (AUC) of DenseNet reached 0.8703, 0.9435, 0.9647, and 0.9653 for detecting NP, microaneurysms, leakages, and laser scars, respectively. For ResNet50, AUC was 0.8140 for NP, 0.9097 for microaneurysms, 0.9585 for leakages, and 0.9115 for laser scars. And for VGG16, AUC was 0.7125 for NP, 0.5569 for microaneurysms, 0.9177 for leakages, and 0.8537 for laser scars.


Experimental results demonstrate that DenseNet is a suitable model to automatically detect and distinguish retinal lesions in the FFA images with multi-label classification, which lies the foundation of automatic analysis for FFA images and comprehensive diagnosis and treatment decision-making for DR.


Diabetic retinopathy Fundus fluorescein angiography Deep learning Multi-label classification 


Funding information

This work was financially supported by Zhejiang Provincial Key Research and Development Plan (grant number 2019C03020), the Natural Science Foundation of China (grant number 81670888), and the Natural Science Foundation of China (grant number 81870635).

Compliance with ethical standards

This study was conducted in compliance with the principles of the Declaration of Helsinki and was approved by the Ethics Committees of the Second Affiliated Hospital of Zhejiang University School of Medicine. Written informed consents were obtained from all subjects for the publication of this study and accompanying images.

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Huang Y (2017) International Diabetes Federation (IDF) (2017) IDF diabetes atlas. 8th edition. Accessed 9 Jun 2019
  2. 2.
    Li Z, Keel S, Liu C, He Y, Meng W, Scheetz J, Lee PY, Shaw J, Ting D, Wong TY, Taylor H, Chang R, He M (2018) An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care 41:2509–2516. CrossRefPubMedGoogle Scholar
  3. 3.
    Lamoke F, Shaw S, Yuan J, Ananth S, Duncan M, Martin P, Bartoli M (2015) Increased oxidative and nitrative stress accelerates aging of the retinal vasculature in the diabetic retina. PLoS One 10:e0139664. CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Dodo Y, Murakami T, Uji A, Yoshitake S, Yoshimura N (2015) Disorganized retinal lamellar structures in nonperfused areas of diabetic retinopathy. Invest Ophthalmol Vis Sci 56:2012–2020. CrossRefPubMedGoogle Scholar
  5. 5.
    Unoki N, Nishijima K, Sakamoto A, Kita M, Watanabe D, Hangai M, Kimura T, Kawagoe N, Ohta M, Yoshimura N (2007) Retinal sensitivity loss and structural disturbance in areas of capillary nonperfusion of eyes with diabetic retinopathy. Am J Ophthalmol 144:755–760. CrossRefPubMedGoogle Scholar
  6. 6.
    Kozak I, El-Emam SY, Cheng L, Bartsch DU, Chhablani J, Freeman WR, Arevalo JF (2014) Fluorescein angiography versus optical coherence tomography-guided planning for macular laser photocoagulation in diabetic macular edema. Retina 34:1600–1605. CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Zhang XH, Chutatape O, Ieee (2004) Detection and classification of bright lesions in color fundus images. Icip: 2004 international conference on image processing 1–5: 139–142Google Scholar
  8. 8.
    van Ginneken B (2017) Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol 10:23–32. CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N (2017) Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One 12:e0174944. CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Bejnordi BE, Zuidhof G, Balkenhol M, Hermsen M, Bult P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J (2017) Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imaging (Bellingham) 4:044504. CrossRefGoogle Scholar
  11. 11.
    Mercan C, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG (2018) Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE Trans Med Imaging 37:316–325. CrossRefPubMedGoogle Scholar
  12. 12.
    Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Corrigendum: dermatologist-level classification of skin cancer with deep neural networks. Nature 546:686. CrossRefPubMedGoogle Scholar
  13. 13.
    Abramoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M (2016) Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci 57:5200–5206. CrossRefPubMedGoogle Scholar
  14. 14.
    Vidal-Alaball J, Royo Fibla D (2019) Artificial intelligence for the detection of diabetic retinopathy in primary care: protocol for algorithm development. JMIR Res Protoc 8:e12539. CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE (2018) Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Invest Dermatol 138:1529–1538. CrossRefPubMedGoogle Scholar
  16. 16.
    Arslan Tuncer S, Akilotu B, Toraman S (2019) A deep learning-based decision support system for diagnosis of OSAS using PTT signals. Med Hypotheses 127:15–22. CrossRefPubMedGoogle Scholar
  17. 17.
    Kamel M, Belkassim S, Mendonca AM, Campilho A, Ieee; Ieee; Ieee I (2001) A neural network approach for the automatic detection of microaneurysms in retinal angiograms. Ijcnn'01: International Joint Conference on Neural Networks 1–4, Proceedings: 2695–2699Google Scholar
  18. 18.
    Hafez M, Azeem SA, Ieee; Ieee; Ieee I (2002) Using adaptive edge technique for detecting microaneurysms in fluorescein angiograms of the ocular fundus. 11th Ieee Mediterranean Electrotechnical Conference, Proceedings: 479–483Google Scholar
  19. 19.
    Tavakoli M, Shahri RP, Pourreza H, Mehdizadeh A, Banaee T, Toosi MHB (2013) A complementary method for automated detection of microaneurysms in fluorescein angiography fundus images to assess diabetic retinopathy. Pattern Recogn 46:2740–2753. CrossRefGoogle Scholar
  20. 20.
    Frame AJ, Undrill PE, Cree MJ, Olson JA, McHardy KC, Sharp PF, Forrester JV (1998) A comparison of computer based classification methods applied to the detection of microaneurysms in ophthalmic fluorescein angiograms. Comput Biol Med 28:225–238. CrossRefPubMedGoogle Scholar
  21. 21.
    Tan JH, Fujita H, Sivaprasad S, Bhandary SV, Rao AK, Kuang CC, Acharya UR (2017) Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci 420:S0020025517308927CrossRefGoogle Scholar
  22. 22.
    Lam C, Yu C, Huang L, Rubin D (2018) Retinal lesion detection with deep learning using image patches. Invest Ophthalmol Vis Sci 59:590–596. CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Lu W, Tong Y, Yu Y, Xing Y, Chen C, Shen Y (2018) Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images. Transl Vis Sci Technol 7:41. CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316:2402–2410. CrossRefPubMedGoogle Scholar
  25. 25.
    Cao W, Czarnek N, Shan J, Li L (2018) Microaneurysm detection using principal component analysis and machine learning methods. IEEE Trans Nanobiosci 17:191–198. CrossRefGoogle Scholar
  26. 26.
    Khojasteh P, Aliahmad B, Kumar DK (2018) Fundus images analysis using deep features for detection of exudates, hemorrhages and microaneurysms. BMC Ophthalmol 18:288. CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2020

Authors and Affiliations

  1. 1.Department of OphthalmologyThe Second Affiliated Hospital of Zhejiang University, College of MedicineHangzhouChina
  2. 2.College of Computer Science and TechnologyZhejiang UniversityHangzhouChina

Personalised recommendations