Skip to main content
SpringerLink
Log in

Predictive, preventive, and personalized management of retinal fluid via computer-aided detection app for optical coherence tomography scans

EPMA Journal Aims and scope Submit manuscript

Cite this article

Abstract

Aims

Computer-aided detection systems for retinal fluid could be beneficial for disease monitoring and management by chronic age-related macular degeneration (AMD) and diabetic retinopathy (DR) patients, to assist in disease prevention via early detection before the disease progresses to a “wet AMD” pathology or diabetic macular edema (DME), requiring treatment. We propose a proof-of-concept AI-based app to help predict fluid via a “fluid score”, prevent fluid progression, and provide personalized, serial monitoring, in the context of predictive, preventive, and personalized medicine (PPPM) for patients at risk of retinal fluid complications.

Methods

The app comprises a convolutional neural network–Vision Transformer (CNN-ViT)–based segmentation deep learning (DL) network, trained on a small dataset of 100 training images (augmented to 992 images) from the Singapore Epidemiology of Eye Diseases (SEED) study, together with a CNN-based classification network trained on 8497 images, that can detect fluid vs. non-fluid optical coherence tomography (OCT) scans. Both networks are validated on external datasets.

Results

Internal testing for our segmentation network produced an IoU score of 83.0% (95% CI = 76.7–89.3%) and a DICE score of 90.4% (86.3–94.4%); for external testing, we obtained an IoU score of 66.7% (63.5–70.0%) and a DICE score of 78.7% (76.0–81.4%). Internal testing of our classification network produced an area under the receiver operating characteristics curve (AUC) of 99.18%, and a Youden index threshold of 0.3806; for external testing, we obtained an AUC of 94.55%, and an accuracy of 94.98% and an F1 score of 85.73% with Youden index.

Conclusion

We have developed an AI-based app with an alternative transformer-based segmentation algorithm that could potentially be applied in the clinic with a PPPM approach for serial monitoring, and could allow for the generation of retrospective data to research into the varied use of treatments for AMD and DR. The modular system of our app can be scaled to add more iterative features based on user feedback for more efficient monitoring. Further study and scaling up of the algorithm dataset could potentially boost its usability in a real-world clinical setting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Data availability

Not applicable.

Code Availability

Not applicable.

References

  1. Mitchell P, et al. Age-related macular degeneration. Lancet. 2018;392(10153):1147–59.

    Article  Google Scholar 

  2. Cheung N, Mitchell P, Wong TY. Diabetic retinopathy. Lancet. 2010;376(9735):124–36.

    Article  Google Scholar 

  3. Daruich A, et al. Central serous chorioretinopathy: Recent findings and new physiopathology hypothesis. Prog Retin Eye Res. 2015;48:82–118.

    Article  CAS  Google Scholar 

  4. Hayreh SS, Zimmerman MB. Branch retinal vein occlusion: Natural history of visual outcome. JAMA Ophthalmol. 2014;132(1):13–22.

    Article  Google Scholar 

  5. Pichi, F. and P. Neri, Complications in uveitis. p. X, 288 p. 89 illus., 73 illus. in color. online resource.

  6. Huang D, et al. Optical coherence tomography. Science. 1991;254(5035):1178–81.

    Article  CAS  Google Scholar 

  7. Schmidt-Erfurth U, et al. AI-based monitoring of retinal fluid in disease activity and under therapy. Prog Retin Eye Res. 2022;86:100972.

    Article  Google Scholar 

  8. Virgili, G., et al., Optical coherence tomography (OCT) for detection of macular oedema in patients with diabetic retinopathy. Cochrane Database Syst Rev, 2015. 1: p. CD008081.

  9. Schmidt-Erfurth U, et al. Guidelines for the management of neovascular age-related macular degeneration by the European Society of Retina Specialists (EURETINA). Br J Ophthalmol. 2014;98(9):1144–67.

    Article  Google Scholar 

  10. Group CR, et al. Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med. 2011;364(20):1897–908.

    Article  Google Scholar 

  11. Silva R, et al. Treat-and-extend versus monthly regimen in neovascular age-related macular degeneration: Results with ranibizumab from the TREND study. Ophthalmology. 2018;125(1):57–65.

    Article  Google Scholar 

  12. Flaxel CJ, et al. Age-related macular degeneration preferred practice pattern(R). Ophthalmology. 2020;127(1):P1–65.

    Article  Google Scholar 

  13. Keenan TDL, et al. Retinal specialist versus artificial intelligence detection of retinal fluid from OCT: Age-related eye disease study 2: 10-Year follow-on study. Ophthalmology. 2021;128(1):100–9.

    Article  Google Scholar 

  14. Sarhan MH, et al. Machine learning techniques for ophthalmic data processing: A review. IEEE J Biomed Health Inform. 2020;24(12):3338–50.

    Article  Google Scholar 

  15. Bogunovic H, et al. RETOUCH: The retinal OCT fluid detection and segmentation benchmark and challenge. IEEE Trans Med Imaging. 2019;38(8):1858–74.

    Article  Google Scholar 

  16. De Fauw J, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med. 2018;24(9):1342–50.

    Article  Google Scholar 

  17. Lin, M., et al., Recent advanced deep learning architectures for retinal fluid segmentation on optical coherence tomography images. Sensors (Basel), 2022. 22(8).

  18. Rim TH, et al. Computer-aided detection and abnormality score for the outer retinal layer in optical coherence tomography. Br J Ophthalmol. 2022;106(9):1301–7.

    Article  Google Scholar 

  19. Rim TH, et al. Detection of features associated with neovascular age-related macular degeneration in ethnically distinct data sets by an optical coherence tomography: Trained deep learning algorithm. Br J Ophthalmol. 2021;105(8):1133–9.

    Article  Google Scholar 

  20. Golubnitschaja O, et al. Medicine in the early twenty-first century: Paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7:23.

    Article  Google Scholar 

  21. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.

    Article  Google Scholar 

  22. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.

    Article  CAS  Google Scholar 

  23. Vaswani, A., et al., Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, Curran Associates Inc.: Long Beach, California, USA. p. 6000–6010.

  24. Dosovitskiy, A., et al., An image is worth 16x16 words: Transformers for image recognition at scale. 2021.

  25. Han, K., et al., A survey on visual transformer. ArXiv, 2020. abs/2012.12556.

  26. Raghu, M., et al. Do vision transformers see like convolutional neural networks? 2021. arXiv:2108.08810.

  27. Karimi, D., S. Vasylechko, and A. Gholipour Convolution-free medical image segmentation using transformers. 2021. arXiv:2102.13645.

  28. Chen, J., et al. TransUNet: Transformers make strong encoders for medical image segmentation. 2021. arXiv:2102.04306.

  29. Luo, X., et al. Semi-supervised medical image segmentation via cross teaching between CNN and transformer. 2021. arXiv:2112.04894.

  30. Kermany DS, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018;172(5):1122-1131 e9.

    Article  CAS  Google Scholar 

  31. Fuchs P, et al. Artificial intelligence in the management of anti-VEGF treatment: The Vienna fluid monitor in clinical practice. Ophthalmologe. 2022;119(5):520–4.

    Article  CAS  Google Scholar 

  32. Yu HJ, et al. Home monitoring of age-related macular degeneration: Utility of the ForeseeHome device for detection of neovascularization. Ophthalmol Retina. 2021;5(4):348–56.

    Article  Google Scholar 

  33. Liu Y, Holekamp NM, Heier JS. Prospective, longitudinal study: Daily self-imaging with home OCT for neovascular age-related macular degeneration. Ophthalmol Retina. 2022;6(7):575–85.

    Article  Google Scholar 

  34. Skelly, A., et al., Treat and extend treatment interval patterns with anti-VEGF therapy in nAMD patients. Vision (Basel), 2019. 3(3).

  35. Hasler PW, Flammer J. Predictive, preventive and personalised medicine for age-related macular degeneration. EPMA J. 2010;1(2):245–51.

    Article  Google Scholar 

  36. Golubnitschaja O. Time for new guidelines in advanced diabetes care: Paradigm change from delayed interventional approach to predictive, preventive & personalized medicine. EPMA J. 2010;1(1):3–12.

    Article  Google Scholar 

  37. Majithia S, et al. Cohort profile: The Singapore epidemiology of eye diseases study (SEED). Int J Epidemiol. 2021;50(1):41–52.

    Article  Google Scholar 

  38. Hassan B, et al. Deep learning based joint segmentation and characterization of multi-class retinal fluid lesions on OCT scans for clinical use in anti-VEGF therapy. Comput Biol Med. 2021;136:104727.

    Article  Google Scholar 

  39. Mantel I, et al. Automated quantification of pathological fluids in neovascular age-related macular degeneration, and its repeatability using deep learning. Translational Vision Science & Technology. 2021;10(4):17–17.

    Article  Google Scholar 

  40. Wilson M, et al. Validation and clinical applicability of whole-volume automated segmentation of optical coherence tomography in retinal disease using deep learning. JAMA Ophthalmol. 2021;139(9):964–73.

    Article  Google Scholar 

  41. Ranftl, R., A. Bochkovskiy, and V. Koltun Vision transformers for dense prediction. 2021. arXiv:2103.13413.

  42. Hu, J., L. Shen, and G. Sun. Squeeze-and-excitation networks. in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018.

  43. Lee, Y. and J. Park CenterMask : Real-time anchor-free instance segmentation. 2019. arXiv:1911.06667.

  44. Wang, S., et al., U-Net using stacked dilated convolutions for medical image segmentation. ArXiv, 2020. abs/2004.03466.

  45. Oktay, O., et al. Attention U-net: Learning where to look for the pancreas. 2018. arXiv:1804.03999.

  46. Chakravarthy U, et al. Automated identification of lesion activity in neovascular age-related macular degeneration. Ophthalmology. 2016;123(8):1731–6.

    Article  Google Scholar 

  47. Kwon JM, et al. Validation of deep-learning-based triage and acuity score using a large national dataset. PLoS ONE. 2018;13(10):e0205836.

    Article  Google Scholar 

  48. Alryalat, S.A., et al., Deep learning prediction of response to anti-VEGF among diabetic macular edema patients: Treatment response analyzer system (TRAS). Diagnostics (Basel), 2022. 12(2).

  49. Lanzetta P, Loewenstein A, Vision Academy Steering. Fundamental principles of an anti-VEGF treatment regimen: Optimal application of intravitreal anti-vascular endothelial growth factor therapy of macular diseases. Graefes Arch Clin Exp Ophthalmol. 2017;255(7):1259–73.

    Article  CAS  Google Scholar 

  50. Granstam E, et al. Switching anti-VEGF agent for wet AMD: evaluation of impact on visual acuity, treatment frequency and retinal morphology in a real-world clinical setting. Graefes Arch Clin Exp Ophthalmol. 2021;259(8):2085–93.

    Article  CAS  Google Scholar 

  51. Ran AR, et al. Three-dimensional multi-task deep learning model to detect glaucomatous optic neuropathy and myopic features from optical coherence tomography scans: A retrospective multi-centre study. Front Med (Lausanne). 2022;9:860574.

    Article  Google Scholar 

Download references

Funding

This study was funded by SingHealth Duke-NUS AM (AM-NHIC/JMT010/2020/SRDUKAMR20M0).

Author information

Author notes

  1. Ten Cheer Quek and Kengo Takahashi contributed equally to this work.

Authors and Affiliations

  1. Singapore Eye Research Institute, Singapore National Eye Centre, The Academia, 20 College Rd, Level 6 Discovery Tower, Singapore, 169856, Singapore

    Ten Cheer Quek, Sahil Thakur, Mihir Deshmukh, Rachel Marjorie Wei Wen Tseng, Yih-Chung Tham, Tyler Hyungtaek Rim, Yasuo Yanagi & Ching-Yu Cheng

  2. Asahikawa Medical University Hospital, Hokkaido, Japan

    Kengo Takahashi

  3. Department of Ophthalmology, Severance Eye Hospital, Yonsei University College of Medicine, Seoul, South Korea

    Hyun Goo Kang & Sung Soo Kim

  4. Department of Ophthalmology, Centre for Vision Research, Westmead Institute for Medical Research, University of Sydney, Sydney, Australia

    Helen Nguyen & Gerald Liew

  5. School of Optometry and Vision Science, Faculty of Science, The University of New South Wales, Sydney, NSW, Australia

    Helen Nguyen

  6. Ophthalmology and Visual Sciences Academic Clinical Program (Eye ACP), Duke-NUS Medical School, Singapore, Singapore

    Yih-Chung Tham, Tyler Hyungtaek Rim, Yasuo Yanagi & Ching-Yu Cheng

  7. Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore, Singapore

    Yih-Chung Tham & Ching-Yu Cheng

  8. Medi Whale Inc, Seoul, South Korea

    Tyler Hyungtaek Rim

  9. Department of Ophthalmology and Microtechnology, Yokohama City University, Yokohama, Japan

    Yasuo Yanagi

Authors
  1. Ten Cheer Quek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kengo Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hyun Goo Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sahil Thakur
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mihir Deshmukh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rachel Marjorie Wei Wen Tseng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Helen Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yih-Chung Tham
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Tyler Hyungtaek Rim
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sung Soo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Yasuo Yanagi
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Gerald Liew
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Ching-Yu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.H.R. conceptualized the study. T.C.Q. developed the algorithms and application, pre-processed images with associated data, and edited all manuscript versions. K.T. and T.H.R. contributed equally to the annotation/labelling of ground truth and sorted image data into their classes for training. H.G.K., S.S.K., H.N., and G.L. provided external validation data for the validation of the deep learning algorithm. S.T. provided comments on the potential operational implementation of the algorithm and app in clinical practice, in the context of PPPM. M.D., R.T., K.T., and T.Y.C. provided comments to improve the overall manuscript.

Corresponding author

Correspondence to Tyler Hyungtaek Rim.

Ethics declarations

Ethics approval

This retrospective study was deemed exempt from institutional review board (IRB) review by the SingHealth Centralised Institutional Review Board (CIRB). This study was conducted in adherence to the tenets of the Declaration of Helsinki.

Consent to participate

Written informed consent was obtained from the participants of the original studies involved.

Consent for publication

This article has been approved for publication by the principal investigator (PI) of this study, T.H.R.

Competing interests

T.H.R. was a former scientific adviser and owns stock of Medi Whale. All other authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Gerald Liew and Ching-Yu Cheng are co-last authors.

Supplementary information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 782 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quek, T.C., Takahashi, K., Kang, H.G. et al. Predictive, preventive, and personalized management of retinal fluid via computer-aided detection app for optical coherence tomography scans. EPMA Journal 13, 547–560 (2022). https://doi.org/10.1007/s13167-022-00301-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13167-022-00301-5

Keywords

search

Navigation