Skip to main content

Advertisement

SpringerLink
Log in

Clinically applicable deep learning-based decision aids for treatment of neovascular AMD

  • Retinal Disorders
  • Published:
Graefe's Archive for Clinical and Experimental Ophthalmology Aims and scope Submit manuscript

Abstract

Purpose

Anti-vascular endothelial growth factor (Anti-VEGF) therapy is currently seen as the standard for treatment of neovascular AMD (nAMD). However, while treatments are highly effective, decisions for initial treatment and retreatment are often challenging for non-retina specialists. The purpose of this study is to develop convolutional neural networks (CNN) that can differentiate treatment indicated presentations of nAMD for referral to treatment centre based solely on SD-OCT. This provides the basis for developing an applicable medical decision support system subsequently.

Methods

SD-OCT volumes of a consecutive real-life cohort of 1503 nAMD patients were analysed and two experiments were carried out. To differentiate between no treatment class vs. initial treatment nAMD class and stabilised nAMD vs. active nAMD, two novel CNNs, based on SD-OCT volume scans, were developed and tested for robustness and performance. In a step towards explainable artificial intelligence (AI), saliency maps of the SD-OCT volume scans of 24 initial indication decisions with a predicted probability of > 97.5% were analysed (score 0–2 in respect to staining intensity). An AI benchmark against retina specialists was performed.

Results

At the first experiment, the area under curve (AUC) of the receiver-operating characteristic (ROC) for the differentiation of patients for the initial analysis was 0.927 (standard deviation (SD): 0.018), for the second experiment (retreatment analysis) 0.865 (SD: 0.027). The results were robust to downsampling (¼ of the original resolution) and cross-validation (tenfold). In addition, there was a high correlation between the AI analysis and expert opinion in a sample of 102 cases for differentiation of patients needing treatment (κ = 0.824). On saliency maps, the relevant structures for individual initial indication decisions were the retina/vitreous interface, subretinal space, intraretinal cysts, subretinal pigment epithelium space, and the choroid.

Conclusion

The developed AI algorithms can define and differentiate presentations of AMD, which should be referred for treatment or retreatment with anti-VEGF therapy. This may support non-retina specialists to interpret SD-OCT on expert opinion level. The individual decision of the algorithm can be supervised by saliency maps.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

AMD:

Age-related macular degeneration

AUC:

Area under curve

BCVA:

Best corrected visual acuity

BM:

Bruch’s membrane

CATT trial:

Comparison of Age-related Macular Degeneration Treatment Trials: Lucentis-Avastin Trial

CNN:

Convolutional neural network

CNV:

Choroidal neovascularization

FA:

Fluorescein angiography

ILM:

Inner limiting membrane

IVAN trial:

Inhibition of VEGF in Age-related choroidal Neovascularisation trial

GPU:

Graphics processing unit

LSTM:

Long short-term memory

M:

Mean score

nAMD:

Neovascular age-related macular degeneration

PRN:

Pro re nata

RC:

reading centre

RPE:

Retinal pigment epithelium

ROC:

Receiver operating characteristic

ReLU:

Rectified linear unit

SD:

Standard deviation

SD-OCT:

Spectral domain optical coherence tomography

tanh:

Hyperbolic tangent

TNR:

True negative rate

TPR:

True positive rate

VEGF:

Vascular endothelial growth factor

References

  1. Brown DM, Kaiser PK, Michels M et al (2006) Ranibizumab versus verteporfin for neovascular age-related macular degeneration. N Engl J Med 355:1432–1444. https://doi.org/10.1056/NEJMoa062655

    Article  CAS  PubMed  Google Scholar 

  2. Stasch-Bouws J, Eller-Woywod SM, Schmickler S et al (2020) IVOM quality assurance in Westfalen-Lippe : Structure of quality assurance and results of the pilot study Q-VERA. Ophthalmologe 117:336–342. https://doi.org/10.1007/s00347-019-01030-3

    Article  CAS  PubMed  Google Scholar 

  3. Brinkmann CK, Chang P, Schick T et al (2019) Baseline diagnostics and initial treatment decision for anti-vascular endothelial growth factor treatment in retinal diseases : Comparison between results by study physician and reading centers (ORCA/OCEAN study). Ophthalmologe 116:753–765. https://doi.org/10.1007/s00347-018-0805-y

    Article  PubMed  Google Scholar 

  4. Gulshan V, Peng L, Coram M et al (2016) Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 316:2402. https://doi.org/10.1001/jama.2016.17216

    Article  PubMed  Google Scholar 

  5. Lee CS, Tyring AJ, Deruyter NP et al (2017) Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed Opt Express 8:3440–3448. https://doi.org/10.1364/BOE.8.003440

    Article  PubMed  PubMed Central  Google Scholar 

  6. Burlina PM, Joshi N, Pekala M et al (2017) Automated Grading of Age-Related Macular Degeneration From Color Fundus Images Using Deep Convolutional Neural Networks. JAMA Ophthalmol 135:1170–1176. https://doi.org/10.1001/jamaophthalmol.2017.3782

    Article  PubMed  PubMed Central  Google Scholar 

  7. Abràmoff MD, Lavin PT, Birch M et al (2018) Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 1:39. https://doi.org/10.1038/s41746-018-0040-6

    Article  PubMed  PubMed Central  Google Scholar 

  8. De Fauw J, Ledsam JR, Romera-Paredes B et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24:1342–1350. https://doi.org/10.1038/s41591-018-0107-6

    Article  CAS  PubMed  Google Scholar 

  9. Russakoff DB, Lamin A, Oakley JD et al (2019) Deep Learning for Prediction of AMD Progression: A Pilot Study. Invest Ophthalmol Vis Sci 60:712. https://doi.org/10.1167/iovs.18-25325

    Article  PubMed  Google Scholar 

  10. Wang Y, Tu D, Du J et al (2019) Classification of Subcortical Vascular Cognitive Impairment Using Single MRI Sequence and Deep Learning Convolutional Neural Networks. Front Neurosci 13:627. https://doi.org/10.3389/fnins.2019.00627

    Article  PubMed  PubMed Central  Google Scholar 

  11. Maetschke S, Antony B, Ishikawa H et al (2019) A feature agnostic approach for glaucoma detection in OCT volumes. PLoS ONE 14:e0219126. https://doi.org/10.1371/journal.pone.0219126

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Li Z, Guo C, Nie D et al (2020) Deep learning for detecting retinal detachment and discerning macular status using ultra-widefield fundus images. Commun Biol 3:15. https://doi.org/10.1038/s42003-019-0730-x

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Kermany DS, Goldbaum M, Cai W et al (2018) Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell 172:1122-1131.e9. https://doi.org/10.1016/j.cell.2018.02.010

    Article  CAS  PubMed  Google Scholar 

  14. Liefers B, Colijn JM, González-Gonzalo C et al (2020) A Deep Learning Model for Segmentation of Geographic Atrophy to Study Its Long-Term Natural History. Ophthalmology 127:1086–1096. https://doi.org/10.1016/j.ophtha.2020.02.009

    Article  PubMed  Google Scholar 

  15. Schmidt-Erfurth U, Waldstein SM, Klimscha S et al (2018) Prediction of Individual Disease Conversion in Early AMD Using Artificial Intelligence. Invest Ophthalmol Vis Sci 59:3199–3208. https://doi.org/10.1167/iovs.18-24106

    Article  CAS  PubMed  Google Scholar 

  16. Yim J, Chopra R, Spitz T et al (2020) Predicting conversion to wet age-related macular degeneration using deep learning. Nat Med 26:892–899. https://doi.org/10.1038/s41591-020-0867-7

    Article  CAS  PubMed  Google Scholar 

  17. Lee CS, Baughman DM, Lee AY (2016) Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration. Ophthalmol Retina 1:322–327. https://doi.org/10.1016/j.oret.2016.12.009

    Article  Google Scholar 

  18. Schmidt-Erfurth U, Bogunovic H, Sadeghipour A et al (2018) Machine Learning to Analyze the Prognostic Value of Current Imaging Biomarkers in Neovascular Age-Related Macular Degeneration. Ophthalmol Retina 2:24–30. https://doi.org/10.1016/j.oret.2017.03.015

    Article  PubMed  Google Scholar 

  19. Rothaus K, Farecki M-L, Mussinghoff P et al (2019) Analysis of the “Portal” Care Model - Examination of the Outcome Quality of IVOM Therapy with Regard to Latency Periods in Exudative AMD. Klin Monbl Augenheilkd 238:293–301. https://doi.org/10.1055/a-0982-5294

    Article  PubMed  Google Scholar 

  20. IVAN Study Investigators, Chakravarthy U, Harding SP et al (2012) Ranibizumab versus bevacizumab to treat neovascular age-related macular degeneration: one-year findings from the IVAN randomized trial. Ophthalmology 119:1399–1411. https://doi.org/10.1016/j.ophtha.2012.04.015

    Article  Google Scholar 

  21. CATT Research Group, Martin DF, Maguire MG et al (2011) Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med 364:1897–1908. https://doi.org/10.1056/NEJMoa1102673

    Article  Google Scholar 

  22. Mazzaferri J, Beaton L, Hounye G et al (2017) Open-source algorithm for automatic choroid segmentation of OCT volume reconstructions. Sci Rep 7:42112. https://doi.org/10.1038/srep42112

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Bradski G (2000) The OpenCV Library: Resampling using pixel area relation. https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html. Accessed 17 Oct 2021

  24. Lim G, Bellemo V, Xie Y et al (2020) Different fundus imaging modalities and technical factors in AI screening for diabetic retinopathy: a review. Eye Vis Lond Engl 7:21. https://doi.org/10.1186/s40662-020-00182-7

    Article  Google Scholar 

  25. Dozat T (2016) Incorporating Nesterov Momentum into Adam. https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ. Accessed 17 Oct 2021

  26. Chollet F (2015) Keras. https://keras.io. Accessed 17 Oct 2021

  27. Abadi M, Agarwal A, Barham P, Brevdo, E (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org. Accessed 17 Oct 2021

  28. Simonyan K, Vedaldi, Andrea, Zisserman, Andrew (2014) Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv:1312.6034. Accessed 17 Oct 2021

  29. Kotikalapudi R (2017) What is Saliency? https://raghakot.github.io/keras-vis/visualizations/saliency/#what-is-saliency. Accessed 17 Oct 2021

  30. Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 39:561–577

    Article  CAS  Google Scholar 

  31. Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382. https://doi.org/10.1037/h0031619

    Article  Google Scholar 

  32. Mishra Z, Ganegoda A, Selicha J et al (2020) Automated Retinal Layer Segmentation Using Graph-based Algorithm Incorporating Deep-learning-derived Information. Sci Rep 10:9541. https://doi.org/10.1038/s41598-020-66355-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Maloca PM, Lee AY, de Carvalho ER et al (2019) Validation of automated artificial intelligence segmentation of optical coherence tomography images. PLoS ONE 14:e0220063. https://doi.org/10.1371/journal.pone.0220063

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Kurmann T, Yu S, Márquez-Neila P et al (2019) Expert-level Automated Biomarker Identification in Optical Coherence Tomography Scans. Sci Rep 9:13605. https://doi.org/10.1038/s41598-019-49740-7

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Schlegl T, Waldstein SM, Bogunovic H et al (2018) Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. Ophthalmology 125:549–558. https://doi.org/10.1016/j.ophtha.2017.10.031

    Article  PubMed  Google Scholar 

  36. Yan Q, Weeks DE, Xin H et al (2020) Deep-learning-based Prediction of Late Age-Related Macular Degeneration Progression. Nat Mach Intell 2:141–150. https://doi.org/10.1038/s42256-020-0154-9

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

H. Faatz, MD 1,6, as a retina specialist, he was involved in analysing the saliency maps. B. Heimes-Bussmann, MD 1,6, M. Ziegler, MD 1,6, as retina specialists, they were involved in benchmarking the AI analysis of the initial indication and in analysing the saliency maps.

Funding

This study was funded by Novartis Pharma GmbH, Nuernberg, Germany. The sponsor or funding organization had no role in the design or conduct of this research.

Author information

Authors and Affiliations

  1. Department of Ophthalmology, St. Franziskus-Hospital, Hohenzollernring 74, 48145, Muenster, Germany

    Matthias Gutfleisch, Georg Spital, Albrecht Lommatzsch, Kai Rothaus & Daniel Pauleikhoff

  2. M3 Macula Monitor Muenster GmbH & Co KG, Muenster, Germany

    Matthias Gutfleisch, Georg Spital, Albrecht Lommatzsch, Kai Rothaus & Daniel Pauleikhoff

  3. Westphalia DataLab GmbH, Muenster, Germany

    Oliver Ester, Sökmen Aydin & Martin Quassowski

  4. Department of Ophthalmology, University Duisburg-Essen, Essen, Germany

    Albrecht Lommatzsch & Daniel Pauleikhoff

  5. Achim Wessing Institute of Ophthalmic Diagnostic, University Duisburg-Essen, Essen, Germany

    Albrecht Lommatzsch & Daniel Pauleikhoff

  6. NIHR Biomedical Resource Centre, UCL Institute of Ophthalmology and Moorfields Eye Hospital NHS Trust, London, United Kingdom

    Adam Michael Dubis

Authors
  1. Matthias Gutfleisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oliver Ester
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sökmen Aydin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Martin Quassowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Georg Spital
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Albrecht Lommatzsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kai Rothaus
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Adam Michael Dubis
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Daniel Pauleikhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthias Gutfleisch.

Ethics declarations

Ethics approval

All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Approval for the study was obtained from the local ethics committee at the University of Muenster.

Informed consent

For this type of study, formal consent is not required.

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript. O. Ester, S. Aydin, M. Quassowski and K. Rothaus declare they have no financial interests. M. Gutfleisch received speaker honoraria from Novartis, Bayer and Zeiss outside the submitted work. G. Spital received speaker honoraria from Zeiss, OD-OS and Allergan outside the submitted work. A. Lommatzsch received speaker honoraria from Novartis, Bayer and Zeiss outside the submitted work. A. M. Dubis has a patent OCT analysis technology pending, and a patent prediction method from retinal imaging pending outside the submitted work. D. Pauleikhoff received speaker honoraria from Bayer, Novartis, Zeiss and Allergan outside the submitted work.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gutfleisch, M., Ester, O., Aydin, S. et al. Clinically applicable deep learning-based decision aids for treatment of neovascular AMD. Graefes Arch Clin Exp Ophthalmol 260, 2217–2230 (2022). https://doi.org/10.1007/s00417-022-05565-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00417-022-05565-1

Keywords

Search

Navigation