Skip to main content
SpringerLink
Log in

Google and DeepMind: Deep Learning Systems in Ophthalmology

  • Chapter
  • First Online:
Artificial Intelligence in Ophthalmology

  • 879 Accesses

Abstract

Deep learning has a profound potential to improve patient outcomes. To achieve this, a holistic, patient-centered approach is crucial. In ophthalmology, artificial intelligence studies have spanned a diverse spectrum including algorithm development, human computer interaction, clinical validation, and novel biomarker discovery. In this chapter we highlight the work of Google and DeepMind in these areas, as a set of end-to-end case studies for developing and implementing artificial intelligence in clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bibliography

  1. Bernardes R, et al. Digital ocular fundus imaging: a review. Ophthalmologica. 2011;226(4):161–81. https://www.karger.com/Article/Abstract/329597

  2. Drexler W, Fujimoto JG. State-of-the-art retinal optical coherence tomography. Prog Retin Eye Res. 2008;27(1):45–88. https://www.sciencedirect.com/science/article/pii/S1350946207000444

  3. Gulshan V, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–10. jamanetwork, https://jamanetwork.com/journals/jama/fullarticle/2588763

  4. De Fauw J, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med. 2018;24(9):1342–50. https://www.nature.com/articles/s41591-018-0107-6

  5. Lee CS, et al. Deep learning is effective for classifying normal versus age-related macular degeneration OCT images. Ophthalmol Retina. 2017;1(4):322–7. https://www.sciencedirect.com/science/article/pii/S2468653016301749

  6. Ting DSW, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103:167–75. https://bjo.bmj.com/content/103/2/167.abstract

  7. Chen X, et al. Automatic feature learning for glaucoma detection based on deep learning. In: International conference on medical image computing and computer-assisted intervention, 2015a. p. 669–77. https://link.springer.com/chapter/10.1007%2F978-3-319-24574-4_80

  8. Chen X, et al. Glaucoma detection based on deep convolutional neural network. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), 2015b, p. 715–8. https://ieeexplore.ieee.org/document/7318462

  9. Long E, et al. An artificial intelligence platform for the multihospital collaborative management of congenital cataracts. Nat Biomed Eng. 2017;1(2):1–8. https://www.nature.com/articles/s41551-016-0024

  10. Abràmoff MD, et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. Npj Digital Med. 2018;1:39. https://www.nature.com/articles/s41746-018-0040-6

  11. 11. Microvascular Complications and Foot Care: Standards of Medical Care in Diabetes. Diabetes Care, edited by American Diabetes Association, vol. 43, no. 5, 2020. p. S135–51. https://care.diabetesjournals.org/content/43/Supplement_1/S135

  12. Cheung N, et al. Diabetic retinopathy. Lancet. 2010;376(9735):124–36. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(09)62124-3

  13. Ting DSW, et al. Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review. Clin Exp Ophthalmol. 2016;44:260–77. https://doi.org/10.1111/ceo.12696

  14. Fong DS, et al. Retinopathy in diabetes. Diabetes Care. 2004;27(Suppl 1):S84–7. https://care.diabetesjournals.org/content/27/suppl_1/s84

  15. Guariguata L, et al. Global estimates of diabetes prevalence for 2013 and projections for 2035. Diabetes Res Clin Pract. 2014;103(2):137–49. https://www.sciencedirect.com/science/article/pii/S0168822713003859

  16. Wilkinson CP, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology. 2003;110(9):1677–82. https://www.ncbi.nlm.nih.gov/pubmed/13129861

  17. Krause J, et al. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology. 2018;125(8):1264–72. https://www.aaojournal.org/article/S0161-6420(17)32698-2/abstract

  18. Gulshan V, et al. Performance of a deep-learning algorithm vs manual grading for detecting diabetic retinopathy in India. JAMA Ophthalmol. 2019;137(9):987–93. https://jamanetwork.com/journals/jamaophthalmology/fullarticle/2734990. Accessed Apr 2020.

  19. Ruamviboonsuk P, et al. Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. npj Digital Med. 2019;2. https://www.nature.com/articles/s41746-019-0099-8

  20. Detecting Center-Involved Diabetic Macular Edema from Analysis of Retina Images Using Deep Learning. 2018. http://www.clinicaltrials.in.th/index.php?tp=regtrials&menu=trialsearch&smenu=fulltext&task=search&task2=view1&id=3819

  21. Quigley HA, Broman AT. The number of people with glaucoma worldwide in 2010 and 2020. Br J Ophthalmol. 2006;90:262–7. https://bjo.bmj.com/content/90/3/262

  22. Tham Y-C, et al. Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis. Ophthalmology. 2014;121(11):2081–90. https://www.sciencedirect.com/science/article/abs/pii/S0161642014004333

  23. Phene S, et al. Deep learning and glaucoma specialists: the relative importance of optic disc features to predict glaucoma referral in fundus photographs. Ophthalmology. 2019;126.12:1627–1639. https://www.sciencedirect.com/science/article/pii/S0161642019318755

  24. Schaekermann M, et al. Remote tool-based adjudication for grading diabetic retinopathy. Transl Vis Sci Technol. 2019;8(40). http://tvst.arvojournals.org/article.aspx?articleid=2757836

  25. Fidalgo BR, et al. Role of advanced technology in the detection of sight-threatening eye disease in a UK community setting. BMJ Open Ophthalmol. 2019;4(1). https://bmjophth.bmj.com/content/4/1/e000347

  26. Buchan JC, et al. How to defuse a demographic time bomb: the way forward? Eye. 2017;31:1519–22. https://www.nature.com/articles/eye2017114

  27. Whited JD, et al. A modeled economic analysis of a digital teleophthalmology system as used by three federal healthcare agencies for detecting proliferative diabetic retinopathy. Telemed e-Health. 2005;11:641–51. https://doi.org/10.1089/tmj.2005.11.641

  28. Bourne RRA, et al. Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. Lancet Global Health. 2017;5:e888–97. https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(17)30293-0/fulltext

  29. Sayres R, et al. Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology. 2019;126(4):552–64. https://www.aaojournal.org/article/S0161-6420(18)31575-6/fulltext

  30. Mitchell P, et al. Cost-effectiveness of ranibizumab in treatment of diabetic macular oedema (DME) causing visual impairment: evidence from the RESTORE trial. Br J Ophthalmol. 2012;96:688–93. https://bjo.bmj.com/content/96/5/688

  31. Romero-Aroca P. Managing diabetic macular edema: the leading cause of diabetes blindness. World J Diabetes. 2011;2(6):98–104. https://www.ncbi.nlm.nih.gov/pubmed/21860693

  32. Mackenzie S, et al. SDOCT imaging to identify macular pathology in patients diagnosed with diabetic maculopathy by a digital photographic retinal screening programme. PLoS One. 2011;6(5):e14811. https://doi.org/10.1371/journal.pone.0014811

  33. Wong RL, et al. Are we making good use of our public resources? The false-positive rate of screening by fundus photography for diabetic macular oedema. Hong Kong Med J. 2017;23(4):356–64. https://pubmed.ncbi.nlm.nih.gov/28684650/

  34. Varadarajan AV, et al. Predicting optical coherence tomography-derived diabetic macular edema grades from fundus photographs using deep learning. Nat Commun. 2020;11(130). https://www.nature.com/articles/s41467-019-13922-8

  35. D’Agostino RB, et al. General cardiovascular risk profile for use in primary care: the Framingham heart study. Circulation. 2008;117(6):743–53. https://www.ncbi.nlm.nih.gov/pubmed/18212285

  36. Tomašev N, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature. 2019;572:116–9. https://www.nature.com/articles/s41586-019-1390-1

  37. Wong WL, et al. Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysis. Lancet Glob Health. 2014;2(2):e106–16.

    Article  Google Scholar 

  38. Lim JH, et al. Delay to treatment and visual outcomes in patients treated with anti-vascular endothelial growth factor for age-related macular degeneration. Am J Ophthalmol. 2012;153(4):678–86. https://www.ajo.com/article/S0002-9394(11)00721-5

  39. Action on AMD. Optimising patient management: act now to ensure current and continual delivery of best possible patient care. Eye. 2020;26(S1). https://www.nature.com/articles/eye2011342

  40. Heier JS. IAI versus Sham as prophylaxis against conversion to neovascular AMD (PRO-CON). clinicaltrials.gov. https://clinicaltrials.gov/ct2/show/NCT02462889

  41. Southern California Desert Retina Consultants, MC. Prophylactic Ranibizumab for Exudative Age-related Macular Degeneration (PREVENT). clinicaltrials.gov. https://clinicaltrials.gov/ct2/show/NCT02140151

  42. Babenko B, et al. Predicting progression of age-related macular degeneration from fundus images using deep learning. arXiv, Apr 2019. https://arxiv.org/pdf/1904.05478.pdf

  43. Yim J, et al. Predicting conversion to wet age related macular degeneration using deep learning. Nat Med. 2020. https://www.nature.com/articles/s41591-020-0867-7

  44. International Council of Ophthalmology. ICO Guidelines for Diabetic Eye Care. 2017. http://www.icoph.org/downloads/ICOGuidelinesforDiabeticEyeCare.pdf

  45. AAO PPP Retina/Vitreous Committee, Hoskins Center for Quality Eye Care. Diabetic Retinopathy PPP 2019. 2019. https://www.aao.org/preferred-practice-pattern/diabetic-retinopathy-ppp

  46. Solomon SD. Diabetic retinopathy: a position statement by the American Diabetes Association. Diabetes Care. 2017;40(3):412–8. https://care.diabetesjournals.org/content/40/3/412

  47. Dornhorst A, Merrin PK. Primary, secondary and tertiary prevention of non-insulin-dependent diabetes. Postgrad Med J. 1994;70(826):529–35. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2397691

  48. Bora A, et al. Deep learning for predicting the progression of diabetic retinopathy using fundus images. ARVO Abstract, 2020.

    Google Scholar 

  49. Goff DC, et al. ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S49–73.

    Google Scholar 

  50. WHO The Top 10 Causes of Death. 2017. 2018. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death

  51. Wong TY, Mitchell P. Hypertensive retinopathy. N Engl J Med. 2004;22(351):2310–7.

    Article  Google Scholar 

  52. Sudlow C, et al. UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12(3):e1001779.

    Article  Google Scholar 

  53. Poplin R, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2:158–64. https://www.nature.com/articles/s41551-018-0195-0

  54. Ting DSW, Wong TY. Eyeing cardiovascular risk factors. Nat Biomed Eng. 2018;2:140–1. https://www.nature.com/articles/s41551-018-0210-5

  55. Xu K, et al. Show, attend and tell: neural image caption generation with visual attention. 2015. https://arxiv.org/abs/1502.03044

  56. McLean E, et al. Worldwide prevalence of anaemia, WHO vitamin and mineral nutrition information system, 1993–2005. Public Health Nutr. 2009;12(4):444–54. https://pubmed.ncbi.nlm.nih.gov/18498676/

  57. Mitani A, et al. Detection of anaemia from retinal fundus images via deep learning. Nat Biomed Eng. 2020;4:18–27. https://www.nature.com/articles/s41551-019-0487-z.

  58. Selvaraju RR, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vis. 2019;128:336–59.

    Article  Google Scholar 

  59. Smilkov D, et al. SmoothGrad: removing noise by adding noise. 2017. https://arxiv.org/abs/1706.03825

  60. Sundararajan M., et al. Axiomatic attribution for deep networks. In: Proceedings of the 34th international conference on machine learning, 2017, p. 3319–28.

    Google Scholar 

  61. Springenberg TJ, et al. Striving for simplicity: the all convolutional net. 2014. https://arxiv.org/abs/1412.6806.

  62. Jaimes A, et al. Human-centered computing: toward a human revolution. Computer. 2007;40(5):30–4.

    Google Scholar 

  63. Beede E, et al. A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In: Proceedings of the 2020 CHI conference on human factors in computing systems, 2020, p. 1–12. https://doi.org/10.1145/3313831.3376718

  64. Kelly CJ, et al. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17:195. https://doi.org/10.1186/s12916-019-1426-2

  65. Abadi M, et al. TensorFlow: a system for large-scale machine learning. In: OSDI’16: Proceedings of the 12th USENIX conference on operating systems design and implementation, 2016. p. 265–83. https://doi.org/10.5555/3026877.3026899

  66. Carneiro T, et al. Performance analysis of Google Colaboratory as a tool for accelerating deep learning applications. IEEE Access. 2018;6:61677–85. https://ieeexplore.ieee.org/abstract/document/8485684

  67. Google Health. Model architecture for predicting conversion to wet age related macular degeneration using deep learning. https://github.com/google-health/imaging-research/wet-amd-prediction

  68. Szegedy C, et al. Rethinking the inception architecture for computer vision. Comput Vis Pattern Recognit. 2016. https://www.researchgate.net/publication/306281834_Rethinking_the_Inception_Architecture_for_Computer_Vision

  69. InceptionV3. https://www.tensorflow.org/api_docs/python/tf/keras/applications/InceptionV3

  70. Ronneberger O, et al. U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, 2015, p. 234–41. https://doi.org/10.1007/978-3-319-24574-4_28

  71. van der Walt S, et al. The NumPy array: a structure for efficient numerical computation. Comput Sci Eng. 2011;13(2):22–30. https://www.researchgate.net/publication/224223550_The_NumPy_Array_A_Structure_for_Efficient_Numerical_Computation

  72. Bouskill KE, et al. Blind spots in telemedicine: a qualitative study of staff workarounds to resolving gaps in chronic disease care. BMC Health Services Res. 2018;18:617. https://research.google/pubs/pub47345/

  73. Google. AI for social good in Asia Pacific. The Keyword, Dec 2018. https://www.blog.google/around-the-globe/google-asia/ai-social-good-asia-pacific

  74. Google Research. TensorFlow: large-scale machine learning on heterogeneous distributed systems. 2015. https://www.tensorflow.org/about/bib

  75. Hutson M, et al. Quality control challenges in crowdsourcing medical labeling. 2019. https://research.google/pubs/pub48327/

  76. Schaekermann M, et al. Expert discussions improve comprehension of difficult cases in medical image assessment. CHI Conference on Human Factors in Computing Systems (CHI ‘20), April 25–30, 2020, Honolulu, HI. ACM, New York, 2020. https://doi.org/10.1145/3313831.3376290.

  77. Shlens J. Train your own image classifier with Inception in TensorFlow. https://ai.googleblog.com/, Google, 9 3 2016, https://ai.googleblog.com/2016/03/train-your-own-image-classifier-with.html. Accessed 6.5.2020

  78. Smith-Morris C, et al. Diabetic retinopathy and the cascade into vision loss. Med Anthropol. 2020;39(2):109–22. https://pubmed.ncbi.nlm.nih.gov/29338335/

  79. Verily. Launching a powerful new screening tool for diabetic eye disease in India. Verily Blog; 2019. https://blog.verily.com/2019/02/launching-powerful-new-screening-tool.html. Accessed Apr 2020.

Download references

Acknowledgements

We would like to thank Y. Liu, D. Webster, O.Ronneberger and P. Kohli for their guidance and feedback.

Author information

Author notes

  1. Xinle Liu and Akinori Mitani contributed equally.

Authors and Affiliations

  1. Google Health, Palo Alto, CA, USA

    Xinle Liu, Akinori Mitani & Derek J. Wu

  2. Google Health, London, UK

    Terry Spitz

  3. DeepMind, London, UK

    Joseph R. Ledsam

Authors
  1. Xinle Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Akinori Mitani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terry Spitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Derek J. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joseph R. Ledsam
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Ophthalmology, University of Warmia and Mazury, Olsztyn, Poland

    Andrzej Grzybowski

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Liu, X., Mitani, A., Spitz, T., Wu, D.J., Ledsam, J.R. (2021). Google and DeepMind: Deep Learning Systems in Ophthalmology. In: Grzybowski, A. (eds) Artificial Intelligence in Ophthalmology. Springer, Cham. https://doi.org/10.1007/978-3-030-78601-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78601-4_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78600-7

  • Online ISBN: 978-3-030-78601-4

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation