Computer-aided detection systems for retinal fluid could be beneficial for disease monitoring and management by chronic age-related macular degeneration (AMD) and diabetic retinopathy (DR) patients, to assist in disease prevention via early detection before the disease progresses to a “wet AMD” pathology or diabetic macular edema (DME), requiring treatment. We propose a proof-of-concept AI-based app to help predict fluid via a “fluid score”, prevent fluid progression, and provide personalized, serial monitoring, in the context of predictive, preventive, and personalized medicine (PPPM) for patients at risk of retinal fluid complications.
The app comprises a convolutional neural network–Vision Transformer (CNN-ViT)–based segmentation deep learning (DL) network, trained on a small dataset of 100 training images (augmented to 992 images) from the Singapore Epidemiology of Eye Diseases (SEED) study, together with a CNN-based classification network trained on 8497 images, that can detect fluid vs. non-fluid optical coherence tomography (OCT) scans. Both networks are validated on external datasets.
Internal testing for our segmentation network produced an IoU score of 83.0% (95% CI = 76.7–89.3%) and a DICE score of 90.4% (86.3–94.4%); for external testing, we obtained an IoU score of 66.7% (63.5–70.0%) and a DICE score of 78.7% (76.0–81.4%). Internal testing of our classification network produced an area under the receiver operating characteristics curve (AUC) of 99.18%, and a Youden index threshold of 0.3806; for external testing, we obtained an AUC of 94.55%, and an accuracy of 94.98% and an F1 score of 85.73% with Youden index.
We have developed an AI-based app with an alternative transformer-based segmentation algorithm that could potentially be applied in the clinic with a PPPM approach for serial monitoring, and could allow for the generation of retrospective data to research into the varied use of treatments for AMD and DR. The modular system of our app can be scaled to add more iterative features based on user feedback for more efficient monitoring. Further study and scaling up of the algorithm dataset could potentially boost its usability in a real-world clinical setting.
This is a preview of subscription content,to check access.
Access this article
Mitchell P, et al. Age-related macular degeneration. Lancet. 2018;392(10153):1147–59.
Cheung N, Mitchell P, Wong TY. Diabetic retinopathy. Lancet. 2010;376(9735):124–36.
Daruich A, et al. Central serous chorioretinopathy: Recent findings and new physiopathology hypothesis. Prog Retin Eye Res. 2015;48:82–118.
Hayreh SS, Zimmerman MB. Branch retinal vein occlusion: Natural history of visual outcome. JAMA Ophthalmol. 2014;132(1):13–22.
Pichi, F. and P. Neri, Complications in uveitis. p. X, 288 p. 89 illus., 73 illus. in color. online resource.
Huang D, et al. Optical coherence tomography. Science. 1991;254(5035):1178–81.
Schmidt-Erfurth U, et al. AI-based monitoring of retinal fluid in disease activity and under therapy. Prog Retin Eye Res. 2022;86:100972.
Virgili, G., et al., Optical coherence tomography (OCT) for detection of macular oedema in patients with diabetic retinopathy. Cochrane Database Syst Rev, 2015. 1: p. CD008081.
Schmidt-Erfurth U, et al. Guidelines for the management of neovascular age-related macular degeneration by the European Society of Retina Specialists (EURETINA). Br J Ophthalmol. 2014;98(9):1144–67.
Group CR, et al. Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med. 2011;364(20):1897–908.
Silva R, et al. Treat-and-extend versus monthly regimen in neovascular age-related macular degeneration: Results with ranibizumab from the TREND study. Ophthalmology. 2018;125(1):57–65.
Flaxel CJ, et al. Age-related macular degeneration preferred practice pattern(R). Ophthalmology. 2020;127(1):P1–65.
Keenan TDL, et al. Retinal specialist versus artificial intelligence detection of retinal fluid from OCT: Age-related eye disease study 2: 10-Year follow-on study. Ophthalmology. 2021;128(1):100–9.
Sarhan MH, et al. Machine learning techniques for ophthalmic data processing: A review. IEEE J Biomed Health Inform. 2020;24(12):3338–50.
Bogunovic H, et al. RETOUCH: The retinal OCT fluid detection and segmentation benchmark and challenge. IEEE Trans Med Imaging. 2019;38(8):1858–74.
De Fauw J, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med. 2018;24(9):1342–50.
Lin, M., et al., Recent advanced deep learning architectures for retinal fluid segmentation on optical coherence tomography images. Sensors (Basel), 2022. 22(8).
Rim TH, et al. Computer-aided detection and abnormality score for the outer retinal layer in optical coherence tomography. Br J Ophthalmol. 2022;106(9):1301–7.
Rim TH, et al. Detection of features associated with neovascular age-related macular degeneration in ethnically distinct data sets by an optical coherence tomography: Trained deep learning algorithm. Br J Ophthalmol. 2021;105(8):1133–9.
Golubnitschaja O, et al. Medicine in the early twenty-first century: Paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7:23.
Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.
Vaswani, A., et al., Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, Curran Associates Inc.: Long Beach, California, USA. p. 6000–6010.
Dosovitskiy, A., et al., An image is worth 16x16 words: Transformers for image recognition at scale. 2021.
Han, K., et al., A survey on visual transformer. ArXiv, 2020. abs/2012.12556.
Raghu, M., et al. Do vision transformers see like convolutional neural networks? 2021. arXiv:2108.08810.
Karimi, D., S. Vasylechko, and A. Gholipour Convolution-free medical image segmentation using transformers. 2021. arXiv:2102.13645.
Chen, J., et al. TransUNet: Transformers make strong encoders for medical image segmentation. 2021. arXiv:2102.04306.
Luo, X., et al. Semi-supervised medical image segmentation via cross teaching between CNN and transformer. 2021. arXiv:2112.04894.
Kermany DS, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018;172(5):1122-1131 e9.
Fuchs P, et al. Artificial intelligence in the management of anti-VEGF treatment: The Vienna fluid monitor in clinical practice. Ophthalmologe. 2022;119(5):520–4.
Yu HJ, et al. Home monitoring of age-related macular degeneration: Utility of the ForeseeHome device for detection of neovascularization. Ophthalmol Retina. 2021;5(4):348–56.
Liu Y, Holekamp NM, Heier JS. Prospective, longitudinal study: Daily self-imaging with home OCT for neovascular age-related macular degeneration. Ophthalmol Retina. 2022;6(7):575–85.
Skelly, A., et al., Treat and extend treatment interval patterns with anti-VEGF therapy in nAMD patients. Vision (Basel), 2019. 3(3).
Hasler PW, Flammer J. Predictive, preventive and personalised medicine for age-related macular degeneration. EPMA J. 2010;1(2):245–51.
Golubnitschaja O. Time for new guidelines in advanced diabetes care: Paradigm change from delayed interventional approach to predictive, preventive & personalized medicine. EPMA J. 2010;1(1):3–12.
Majithia S, et al. Cohort profile: The Singapore epidemiology of eye diseases study (SEED). Int J Epidemiol. 2021;50(1):41–52.
Hassan B, et al. Deep learning based joint segmentation and characterization of multi-class retinal fluid lesions on OCT scans for clinical use in anti-VEGF therapy. Comput Biol Med. 2021;136:104727.
Mantel I, et al. Automated quantification of pathological fluids in neovascular age-related macular degeneration, and its repeatability using deep learning. Translational Vision Science & Technology. 2021;10(4):17–17.
Wilson M, et al. Validation and clinical applicability of whole-volume automated segmentation of optical coherence tomography in retinal disease using deep learning. JAMA Ophthalmol. 2021;139(9):964–73.
Ranftl, R., A. Bochkovskiy, and V. Koltun Vision transformers for dense prediction. 2021. arXiv:2103.13413.
Hu, J., L. Shen, and G. Sun. Squeeze-and-excitation networks. in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018.
Lee, Y. and J. Park CenterMask : Real-time anchor-free instance segmentation. 2019. arXiv:1911.06667.
Wang, S., et al., U-Net using stacked dilated convolutions for medical image segmentation. ArXiv, 2020. abs/2004.03466.
Oktay, O., et al. Attention U-net: Learning where to look for the pancreas. 2018. arXiv:1804.03999.
Chakravarthy U, et al. Automated identification of lesion activity in neovascular age-related macular degeneration. Ophthalmology. 2016;123(8):1731–6.
Kwon JM, et al. Validation of deep-learning-based triage and acuity score using a large national dataset. PLoS ONE. 2018;13(10):e0205836.
Alryalat, S.A., et al., Deep learning prediction of response to anti-VEGF among diabetic macular edema patients: Treatment response analyzer system (TRAS). Diagnostics (Basel), 2022. 12(2).
Lanzetta P, Loewenstein A, Vision Academy Steering. Fundamental principles of an anti-VEGF treatment regimen: Optimal application of intravitreal anti-vascular endothelial growth factor therapy of macular diseases. Graefes Arch Clin Exp Ophthalmol. 2017;255(7):1259–73.
Granstam E, et al. Switching anti-VEGF agent for wet AMD: evaluation of impact on visual acuity, treatment frequency and retinal morphology in a real-world clinical setting. Graefes Arch Clin Exp Ophthalmol. 2021;259(8):2085–93.
Ran AR, et al. Three-dimensional multi-task deep learning model to detect glaucomatous optic neuropathy and myopic features from optical coherence tomography scans: A retrospective multi-centre study. Front Med (Lausanne). 2022;9:860574.
This study was funded by SingHealth Duke-NUS AM (AM-NHIC/JMT010/2020/SRDUKAMR20M0).
This retrospective study was deemed exempt from institutional review board (IRB) review by the SingHealth Centralised Institutional Review Board (CIRB). This study was conducted in adherence to the tenets of the Declaration of Helsinki.
Consent to participate
Written informed consent was obtained from the participants of the original studies involved.
Consent for publication
This article has been approved for publication by the principal investigator (PI) of this study, T.H.R.
T.H.R. was a former scientific adviser and owns stock of Medi Whale. All other authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Gerald Liew and Ching-Yu Cheng are co-last authors.
Below is the link to the electronic supplementary material.
About this article
Cite this article
Quek, T.C., Takahashi, K., Kang, H.G. et al. Predictive, preventive, and personalized management of retinal fluid via computer-aided detection app for optical coherence tomography scans. EPMA Journal 13, 547–560 (2022). https://doi.org/10.1007/s13167-022-00301-5