Skip to main content
SpringerLink
Log in

Comparing Deep Feature Extraction Strategies for Diabetic Retinopathy Stage Classification from Fundus Images

  • Research Article-computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Diabetic retinopathy (DR) is the damage to the micro-vascular system in the retina, due to prolonged diabetes mellitus. Diagnosis and treatment of DR entail screening of retinal fundus images of diabetic patients. The manual inspection of pathological changes in retinal images is a skill-based task that involves lots of effort and time. Therefore, computer-aided detection and diagnosis of DR have been extensively explored for the past few decades. In recent years with the development of different benchmark deep convolutional neural networks (CNN), deep learning and machine learning have been efficiently and effectively adapted to different DR classification tasks. The success of CNNs largely relies on how good they are in extracting discriminative features from the fundus images. However, to the best of our knowledge, till date no study has been conducted to evaluate the feature extraction capabilities of all the benchmark CNNs to support the DR classification tasks and to find the best training-hyper-parameters for each of them in fundus retinal image-based DR classification tasks. In this work, we try to find the best benchmark CNN, which can be used as the backbone feature extractor for the DR classification tasks using fundus retinal images. We also aim to find the optimal hyper-parameters for training each of the benchmark CNN family, particularly when they are applied to the DR gradation tasks using retinal image datasets with huge class-imbalance and limited samples of higher severity classes. To address the cause, we conduct a detailed comprehensive comparative study on the performances of almost all the benchmark CNNs and their variants proposed during 2014 to 2019, for the DR gradation tasks on common standard retinal datasets. We have also conducted a comprehensive optimal training hyper-parameter search for each of the benchmark CNN family for the fundus image-based DR classification tasks. The benchmark CNNs are transfer learned and end-to-end trained in an incremental fashion on a class-balanced dataset curated from the train set of the EyePACS dataset. The benchmark models are evaluated on APTOS, MESSIDOR-1, and MESSIDOR-2 datasets to test their cross-dataset generalization. Experimental results show that features extracted by EfficientNetB1 have outperformed features of all the other CNN models in DR classification tasks on all three test datasets. MobileNet-V3-Large also shows promising performance on MESSIDOR-1 dataset. The success of EfficientNetB1 and MobileNet-V3-Large indicates that comparatively shallower and light-weighted CNNs tend to extract more discriminative and expressive features from fundus images for DR stage detection. In future, researchers can explore different preprocessing and post-processing techniques and incorporate novel architectural components on these networks to further improve the classification accuracy and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

References:

  1. Sussman, E.J.; Tsiaras, W.G.; Soper, K.A.: Diagnosis of diabetic eye disease. JAMA Ophthalmol. 247(23), 3231–3234 (1982)

    Google Scholar 

  2. Keenan, T.D.; Johnston, R.L.; Donachie, P.H.; Sparrow, J.M.; Stratton, I.M.; Scanlon, P.: United kingdom national ophthalmology database study: diabetic retinopathy; report 1: prevalence of centre-involving diabetic macular edema and other grades of maculopathy and retinopathy in hospital eye services. Eye (London) 27, 1397–1404 (2013)

    Article  Google Scholar 

  3. Klein, R.; Klein, B.E.; Moss, S.E.; Davis, M.D.; DeMets, D.L.: The wisconsin epidemiologic study of diabetic retinopathy. ii. prevalence and risk of diabetic retinopathy when age at diagnosis is less than 30 years. Arch. Ophthalmol. 102, 520–526 (1984)

    Article  Google Scholar 

  4. Klein, R.; Klein, B.E.; Moss, S.E.; Davis, M.D.; DeMets, D.L.: The Wisconsin epidemiologic study of diabetic retinopathy III. prevalence and risk of diabetic retinopathy when age at diagnosis is 30 or more years. Arch. Ophthalmol. 102, 527–53 (1984)

    Article  Google Scholar 

  5. Zachariah, S.; Wykes, W.; Yorston, D.: Grading diabetic retinopathy (DR) using the Scottish grading protocol. Community Eye Health 28(92), 72–73 (2015)

    Google Scholar 

  6. Adarsh, P. and Jeyakumari, D.: “Multiclass SVM-based automated diagnosis of diabetic retinopathy,” In Proceedings of International Conference on Communication and Signal Processing (ICCSP 2013), pp. 206–210, 2013, doi: https://doi.org/10.1109/iccsp.2013.6577044.

  7. Casanova, R.; Saldana, S.; Chew, E.Y.; Danis, R.P.; Greven, C.M.; Ambrosius, W.T.: Application of random forests methods to diabetic retinopathy classification analyses. PLoS One 9(6), 985–987 (2014). https://doi.org/10.1371/journal.pone.0098587

    Article  Google Scholar 

  8. Carrera, E., González, A., and Carrera, R.: “Automated detection of diabetic retinopathy using SVM,” 2017, doi: https://doi.org/10.1109/INTERCON.2017.8079692.

  9. Costa, P.; Campilho, A.: Convolutional bag of words for diabetic retinopathy detection from eye fundus images. IPSJ Trans. Computer Vision Appl. 9, 10 (2017). https://doi.org/10.1186/s41074-017-0023-6

    Article  Google Scholar 

  10. Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.W.M.; van Ginneken, B.; Sánchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). https://doi.org/10.1016/j.media.2017.07.005

    Article  Google Scholar 

  11. Pratt, H.; Coenen, F.; Broadbent, D.M.; Harding, S.P.; Zheng, Y.: Convolutional neural networks for diabetic retinopathy. Proc. Computer Sci. 90, 200–205 (2016). https://doi.org/10.1016/j.procs.2016.07.014

    Article  Google Scholar 

  12. Gulshan, V.; Peng, L.; Coram, M.; Stumpe, M.C.; Wu, D.; Narayanaswamy, A.; Venugopalan, S.; Widner, K.; Madams, T.; Cuadros, J.; Kim, R.; Raman, R.; Nelson, P.C.; Mega, J.L.; Webster, D.R.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA Ophthalmol. 316(22), 2402–2410 (2016). https://doi.org/10.1001/jama.2016.17216

    Article  Google Scholar 

  13. Doshi, D., Shenoy, A., Sidhpura, D. and Gharpure, P.: “Diabetic retinopathy detection using deep convolutional neural networks,” 2016 International Conference on Computing, Analytics and Security Trends (CAST), Pune, pp. 261–266, 2016, doi: https://doi.org/10.1109/CAST.2016.7914977.

  14. Gargeya, R.; Leng, T.: Automated identification of diabetic retinopathy using deep learning. Ophthalmology 124(7), 962–969 (2017). https://doi.org/10.1016/j.ophtha.2017.02.008

    Article  Google Scholar 

  15. Kanungo, Y. S., Srinivasan, B., and Choudhary, S.: “Detecting diabetic retinopathy using deep learning,” 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, pp. 801–804, 2017, doi: https://doi.org/10.1109/RTEICT.2017.8256708.

  16. García, G., Gallardo, J., Mauricio, A., López, J., and Del Carpio, C. (2017)“Detection of Diabetic Retinopathy Based on a Convolutional Neural Network Using Retinal Fundus Images. In: A. Lintas, S. Rovetta, P. Verschure, A. Villa (eds) Artificial Neural Networks and Machine Learning – ICANN 2017, Lecture Notes in Computer Science. Springer, Cham

  17. Li, X., Pang, T., Xiong, B., Liu, W., Liang, P., and Wang, T.: “Convolutional neural networks based transfer learning for diabetic retinopathy fundus image classification,” 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Shanghai, pp. 1–11, 2017, doi: https://doi.org/10.1109/CISP-BMEI.2017.8301998

  18. Wan, S.; Liang, Y.; Zhang, Y.: Deep convolutional neural networks for diabetic retinopathy detection by image classification. Comput. Electr. Eng. 72, 274–282 (2018). https://doi.org/10.1016/j.compeleceng.2018.07.042

    Article  Google Scholar 

  19. Chen, Y-W., Wu, T-Y., Wong W-H., and Lee, C-Y.: “Diabetic retinopathy detection based on deep convolutional neural networks,” 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, pp. 1030–1034, 2018, doi: https://doi.org/10.1109/ICASSP.2018.8461427.

  20. Lam, C., Yi, D., Guo, M., and Lindsey, T.: “Automated detection of diabetic retinopathy using deep learning,” In Proceedings of AMIA Joint Summits on Translational Science, vol. 2017, pp. 147–155, May. 2018.

  21. Mateen, M.; Wen, J.; Song Nasrullah, S.; Huang, Z.: “Fundus Image Classification Using VGG-19 Architecture with PCA and SVD. Symmetry 11, 1 (2019). https://doi.org/10.3390/sym11010001

    Article  Google Scholar 

  22. Saxena, G.; Verma, D.K.; Paraye, A.; Rajan, A.; Rawat, A.: “Improved and robust deep learning agent for preliminary detection of diabetic retinopathy using public datasets. Intell. -Based Med. 3(4), 87 (2020). https://doi.org/10.1016/j.ibmed.2020.100022

    Article  Google Scholar 

  23. Zhang, Z.: “Deep-learning-based early detection of diabetic retinopathy on fundus photography using EfficientNet,” In Proceedings of the 4th International Conference on Innovation in Artificial Intelligence (ICIAI 2020), Association for Computing Machinery, New York, NY, USA, pp. 70–74, 2020, doi: https://doi.org/10.1145/3390557.3394303.

  24. Zeiler, M.D.; Fergus, R.: Visualizing and understanding convolutional Networks. In: Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. (Eds.) Computer vision – ECCV 2014 ECCV 2014 Lecture Notes in Computer Science 8689. Springer (2014)

    Google Scholar 

  25. Kaggle diabetic retinopathy detection competition: EyePACS dataset, Available: https://www.kaggle.com/c/diabetic-retinopathy-detection/data. [Accessed: 2020–06–12].

  26. Kaggle APTOS 2019 Blindness Detection competition, Available: https://www.kaggle.com/c/aptos2019-blindness-detection/data. [Accessed: 2020–06–12].

  27. Decencière, E., Zhang, X., Cazuguel, G., Lay, B., Cochener, B., Trone, C., Gain, P., Ordonez, R., Massin, P., Erginay, A., Charton, B., and Klein, J. C.: “Feedback on a publicly distributed database: the Messidor database,” Image Analysis and Stereology, vol. 33(3), pp. 231–234, Aug. 2014, doi: https://doi.org/10.5566/ias.1155. Available: https://www.adcis.net/en/third-party/messidor/. [Accessed: 2022–10–28]

  28. Abràmoff, M. D., Folk, J. C., Han, D. P. et al.: “Automated analysis of retinal images for detection of referable diabetic retinopathy,” JAMA Ophthalmology, vol. 131(3), pp. 351–357, 2013. Available: https://www.adcis.net/en/third-party/messidor2/. [Accessed: 2022–10–28]

  29. Simonyan, K. and Zisserman, A.: “Very deep convolutional networks for large-scale image recognition. http://arxiv.org/abs/math/1409.1556, 2014.

  30. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Ribinovich, A.: “Going deeper with convolution,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015, pp. 1–9, Boston, MA, 2015.

  31. He, K., Zhang, X. Ren, S. and Sun, J.: “Deep residual learning for image recognition,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, pp. 770–778, Las Vegas, NV, 2016.

  32. Szegedy, C., Vanhoucke, V., Ioffe, S. et al.: “Rethinking the inception architecture for computer vision,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, pp. 2818–2826, Las Vegas, NV, 2016.

  33. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alexander, A. A.: “Inception-v4, inception-ResNet and the impact of residual connections on learning,” In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI'17), AAAI Press, pp. 4278–4284, 2017.

  34. Chollet, F.: "Xception: deep learning with depthwise separable convolutions," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 1800–1807, 2017, doi: https://doi.org/10.1109/CVPR.2017.195.

  35. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto M., and Adam, H.: “MobileNets: efficient convolutional neural networks for mobile vision applications, http://arxiv.org/abs/math/1704.04861, 2017.

  36. Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K. Q.: “Densely connected convolutional networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 2261–2269, 2017, doi: https://doi.org/10.1109/CVPR.2017.243.

  37. Xie, S., Girshick, R., Dollár, P., Tu Z. and He K.: “Aggregated residual transformations for deep neural networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 5987–5995, 2017, doi: https://doi.org/10.1109/CVPR.2017.634.

  38. Zoph, B. Vasudevan, V. Shlens, J. and Le, Q. V.: “Learning transferable architectures for scalable image recognition,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 8697–8710, 2018, doi: https://doi.org/10.1109/CVPR.2018.00907.

  39. Hu, J., Shen, L. and Sun, G.: "Squeeze-and-excitation networks," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 7132–7141, 2018, doi: https://doi.org/10.1109/CVPR.2018.00745.

  40. Zhang, X., Zhou, X., Lin, M., and Sun, J.: "ShuffleNet: an extremely efficient convolutional neural network for mobile devices," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6848–6856, 2018, doi: https://doi.org/10.1109/CVPR.2018.00716

  41. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. and Chen, L.: "MobileNetV2: inverted residuals and linear bottlenecks," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520, 2018, doi: https://doi.org/10.1109/CVPR.2018.00474.

  42. Tan, M., and Le, Q.: “EfficientNet: rethinking model scaling for convolutional neural networks,” In Proceedings of the 36th International Conference on Machine Learning (ICML 2019), Long Beach, pp. 6105–6114, 9–15 June 2019.

  43. Howard, A. et al.: "Searching for MOBILENETV3," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1314–1324, 2019, doi: https://doi.org/10.1109/ICCV.2019.00140.

  44. Deng, J., Dong, W., Socher, R., Li, L., Kai, L., and Li, F-F.:, “ImageNet: A large-scale hierarchical image database,” The IEEE Conference on Computer Vision and Pattern Recognition (CVPR-2009), Miami, FL, pp. 248–255, 2009, doi: https://doi.org/10.1109/CVPR.2009.5206848.

  45. Chen, C.; Zhang, Q.; Kashani, M.H.; Jun, C.; Bateni, S.M.; Band, S.S.; Dash, S.S.; Chau, K.W.: Forecast of rainfall distribution based on fixed sliding window long short-term memory. Engineering Applications of Comput. Fluid Mech. 16(1), 248–261 (2022). https://doi.org/10.1080/19942060.2021.2009374

    Article  Google Scholar 

  46. Chen, W.; Sharifrazi, D.; Liang, G.; Band, S.S.; Chau, K.W.; Mosavi, A.: Accurate discharge coefficient prediction of streamlined weirs by coupling linear regression and deep convolutional gated recurrent unit. Eng. Appl. Comput. Fluid Mech. 16(1), 965–976 (2022). https://doi.org/10.1080/19942060.2022.2053786

    Article  Google Scholar 

  47. Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  48. Krizhevsky, A.; Sutskever, I.; Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2012). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  49. J. J. Staal, M. D. Abramoff, M. Niemeijer, M. A. Viergever, B. V. Ginneken, “Ridge based vessel segmentation in color images of the retina,” IEEE Trans. Med. Imag., vol. 23, pp. 501–509, 2004. Available: https://drive.grand-challenge.org/DRIVE/. [Accessed 2022.10.25].

  50. T. K. Kauppi, V. Kamarainen, J. K. Lensu, L. Sorri, I. Uusitalo, H. Kälviäinen, H. J. Pietilä, “DIARETDB0: Evaluation database and methodology for diabetic retinopathy algorithms,” Technical Report, 2006. Available: https://www.it.lut.fi/project/imageret/diaretdb0/. [Accessed 2022.10.25].

  51. T. K. Kauppi, V. Kamarainen, J. K. Lensu, L. Sorri, A. Raninen, R. Voutilainen, I. Uusitalo, H. Kälviäinen, H. J. Pietilä, “DIARETDB1: Diabetic Retinopathy Database and Evaluation Protocol,” Technical Report, 2007. Available: https://www.it.lut.fi/project/imageret/diaretdb1/. [Accessed 2022.10.25].

  52. T. Köhler, A. Budai, M. Kraus, J. Odstrcilik, G. Michelson, J. Hornegger, “Automatic no-reference quality assessment for retinal fundus images using vessel segmentation,” 26th IEEE International Symposium on Computer-Based Medical Systems, Porto, 2013. Available: https://www5.cs.fau.de/research/ data/fundus-images/. [Accessed 2022.10.25].

  53. E. Decencière, G. Cazuguel, X. Zhang, G. Thibault, J.-C. Klein, F. Meyer, B. Marcotegui, G. Quellec, M. Lamard, R. Danno, D. Elie, P. Massin, Z. Viktor, A. Erginay, B. Laÿ, A. Chabouis, “TeleOphta: Machine learning and image processing methods for teleophthalmology,” IRBM, vol. 34(2), pp. 196–203, 2013, doi: https://doi.org/10.1016/j.irbm.2013.01.010. Available: https://www.adcis.net/en/third-party/e-ophtha/. [Accessed 2022.10.25].

  54. P. Porwal, S. Pachade, R. Kamble, M. Kokare, G. Deshmukh, V. Sahasrabuddhe, and F. Meriaudeau, “Indian diabetic retinopathy image dataset (idrid): A database for diabetic retinopathy screening research,” Data, vol. 3, no. 3, p. 25, 2018. Available: https://ieee-dataport.org/open-access/indian-diabetic-retinopathy-image-dataset-idrid/. [Accessed 2022.10.25].

  55. Mukherjee, N., and Sengupta, S.: “comparing different preprocessing techniques for the classification tasks in diabetic retinopathy from fundus images,” In Proceedings of 2nd International Conference on Advanced Computing and Applications (ICACA-2021), March, 2021.

  56. B. Graham, "Kaggle diabetic retinopathy detection competition report," University of Warwick, Aug 6 2015.

  57. van der Maaten, L.J.P.; Hinton, G.E.: Visualizing high-dimensional data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  58. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh and D. Batra, “Grad-CAM: visual explanations from deep networks via gradient-based localization,” 2017 IEEE International Conference on Computer Vision (ICCV), pp. 618–626, 2017, doi: https://doi.org/10.1109/ICCV.2017.74.

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Bengal Institute of Technology, Kolkata, India

    Nilarun Mukherjee

  2. Aliah University, Kolkata, India

    Souvik Sengupta

Authors
  1. Nilarun Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Souvik Sengupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NM and SS wrote the main manuscript text and prepared figures. All authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Nilarun Mukherjee.

Ethics declarations

Conflict of interest

Nilarun Mukherjee and Souvik Sengupta declare that they have no conflict of interest.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by the any of the authors.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mukherjee, N., Sengupta, S. Comparing Deep Feature Extraction Strategies for Diabetic Retinopathy Stage Classification from Fundus Images. Arab J Sci Eng 48, 10335–10354 (2023). https://doi.org/10.1007/s13369-022-07547-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-022-07547-1

Keywords

Search

Navigation