Abstract
Implanted visual prostheses generate visual percepts by electrically stimulating the human visual pathway using an array of electrodes. The resulting bionic vision consists of a spatial-temporal pattern of bright dots called phosphenes. This patient-specific phosphene pattern has low resolution, limited dynamic range and is spatially irregular. This paper presents a computer vision system designed to deal with these limitations, especially spatial irregularity. The system uses a new mapping called the Camera Map to decouple the flexible spatial layout of image processing from the inflexible layout of phosphenes experienced by a patient. Detailed simulations of a cortical prosthesis currently in preclinical testing were performed to create phosphene patterns for testing. The system was tested on a wearable prototype of the cortical prosthesis. Despite having limited computational resources, the system operated in real time, taking only a few milliseconds to perform image processing and visualisations of simulated prosthetic vision.
Chapter PDF
Similar content being viewed by others
Keywords
References
Barnes, N.: An overview of vision processing in implantable prosthetic vision. In: IEEE International Conference on Image Processing, pp. 1532–1535, September 2013
Brindley, G.S., Lewin, W.S.: The sensations produced by electrical stimulation of the visual cortex. The Journal of Physiology 196, 479–493 (1968)
Buffoni, L.X., Coulombe, J., Sawan, M.: Image processing strategies dedicated to visual cortical stimulators: a survey. Artificial Organs 29(8), 658–64 (2005). http://www.ncbi.nlm.nih.gov/pubmed/16048483
Cha, K., Horch, K., Normann, R.A.: Simulation of a phosphene-based visual field: visual acuity in a pixelized vision system. Annals of Biomedical Engineering 20(4), 439–449 (1992)
Chen, S.C., Suaning, G.J., Morley, J.W., Lovell, N.H.: Simulating prosthetic vision: I. Visual models of phosphenes. Vision Research 49(12), 1493–1506 (2009)
Collette, M., Horace, J., Lindsay, K., Wai Ho, L.: The Impact of Luminance Threshold Modes and Phosphene Dropout Rates in Psychophysics Testing for the Monash Vision Group’s Cortical Vision Prosthesis Gennaris. Frontiers in Human Neuroscience 7 (2013)
Crow, F.: Summed-area tables for texture mapping. In: ACM SIGGRAPH Computer Graphics. 18, pp. 207–212, July 1984. http://dl.acm.org/citation.cfm?id=808600
Dagnelie, G.: Visual Prosthetics. Springer, US, Boston, MA (2011)
Dobelle, W.H.: Artificial vision for the blind by connecting a television camera to the visual cortex. ASAIO Journal (American Society for Artificial Internal Organs: 1992) 46(1), 3–9 (2000). http://www.ncbi.nlm.nih.gov/pubmed/10667705
Greenwald, S.H., Horsager, A., Humayun, M.S., Greenberg, R.J., McMahon, M.J., Fine, I.: Brightness as a function of current amplitude in human retinal electrical stimulation. Investigative Ophthalmology & Visual Science 50(11), 5017–5025 (2009)
Humayun, M.S., Dorn, J.D., da Cruz, L., Dagnelie, G., Sahel, J.A., Stanga, P.E., Cideciyan, A.V., Duncan, J.L., Eliott, D., Filley, E., Ho, A.C., Santos, A., Safran, A.B., Arditi, A., Del Priore, L.V., Greenberg, R.J.: Interim results from the international trial of Second Sight’s visual prosthesis. Ophthalmology 119(4), 779–788 (2012)
Josh, H., Mann, C., Kleeman, L., Lui, W.L.D.: Psychophysics testing of bionic vision image processing algorithms using an FPGA Hatpack. In: 2013 IEEE International Conference on Image Processing, pp. 1550–1554. IEEE, September 2013, http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6738319
Josh, H., Yong, B., Kleeman, L.: A Real-time FPGA-based Vision System for a Bionic Eye. In: ARAA (ed.) Proceedings of Australasian Conference on Robotics and Automation. p. Online. ARAA, Melbourne, Australia (2011)
Kiral-Kornek, F.I., Savage, C.O., O’Sullivan-Greene, E., Burkitt, A.N., Grayden, D.B.: Embracing the irregular: A patient-specific image processing strategy for visual prostheses. In: International Conference of the IEEE Engineering in Medicine and Biology Society 2013, 3563–3566, January 2013. http://www.ncbi.nlm.nih.gov/pubmed/24110499
LeRoy, C.: Ou l’on rend compte de quelques tentatives que l’on a faites pour guerir plusieurs maladies par l’electricite. In: (Paris), H.A.R.S. (ed.) Memoires Math. Phys., pp. 87–89 (1755)
Li, W.H.: Wearable Computer Vision Systems for a Cortical Visual Prosthesis. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 428–435. IEEE, December 2013
Lowery, A.J.: Introducing the Monash vision group’s cortical prosthesis. In: 2013 IEEE International Conference on Image Processing, pp. 1536–1539. IEEE, September 2013
Lui, W.L.D., Browne, D., Kleeman, L., Drummond, T., Li, W.H.: Transformative reality: Augmented reality for visual prostheses. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 253–254. IEEE, October 2011. http://doi.ieeecomputersociety.org/10.1109/ISMAR.2011.6092402 http://youtu.be/J30uYYkDApY http://youtu.be/iK5ddJqNuxY
Nanduri, D., Humayun, M.S., Greenberg, R.J., McMahon, M.J., Weiland, J.D.: Retinal prosthesis phosphene shape analysis. In: International Conference of the IEEE Engineering in Medicine and Biology Society 2008, pp. 1785–1788, January 2008. http://www.ncbi.nlm.nih.gov/pubmed/19163027
Nanduri, D., Humayun, M.S., Weiland, J.D., Dorn, J., Greenberg, R.J., Fine, I.: Encoding of size and brightness of percepts in a visual prosthesis, September 2013. http://www.google.com/patents/US8527056
Naumann, J.: Search for Paradise: A Patient’s Account of the Artificial Vision Experiment. Xlibris (2012). http://www.amazon.com/Search-Paradise-Artificial-Experiment-ebook/dp/B009A86X9K
Otsu, N.: A Threshold Selection Method from Gray-Level Histograms. Ieee Transactions On Systems Man And Cybernetics 9, 62–66 (1979)
Parikh, N., Itti, L., Weiland, J.: Saliency-based image processing for retinal prostheses. Journal of neural engineering 7(1), 16006 (2010). http://www.ncbi.nlm.nih.gov/pubmed/20075505
Polimeni, J.R., Balasubramanian, M., Schwartz, E.L.: Multi-area visuotopic map complexes in macaque striate and extra-striate cortex. Vision Research 46(20), 3336–3359 (2006)
van Rheede, J.J., Kennard, C., Hicks, S.L.: Simulating prosthetic vision: Optimizing the information content of a limited visual display. Journal of Vision 10(14), 1–14 (2010)
Schiller, P., Tehovnik, E.: Visual Prosthesis. Perception 37, 1529–1559 (2008)
Srivastava, N.R.: Simulations of Cortical Prosthetic Vision. In: Dagnelie, G. (ed.) Visual Prosthetics, pp. 355–365. Springer (2011)
Stingl, K.K.T., Bartz-Schmidt, K.U., Besch, D., Braun, A., Bruckmann, A., Gekeler, F., Greppmaier, U., Hipp, S., Hörtdörfer, G., Kernstock, C., Koitschev, A., Kusnyerik, A., Sachs, H., Schatz, A., Peters, T., Wilhelm, B., Zrenner, E.: Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS. Proceedings. Biological sciences / The Royal Society 280(1757), 20130077 (2013). http://rspb.royalsocietypublishing.org/content/280/1757/20130077.abstract
Tsai, D., Morley, J.W., Suaning, G.J., Lovell, N.H.: A wearable real-time image processor for a vision prosthesis. Computer methods and programs in biomedicine 95(3), 258–69 (2009). http://www.ncbi.nlm.nih.gov/pubmed/19394713
Veraart, C., Wanet-Defalque, M.C., Gerard, B., Vanlierde, A., Delbeke, J.: Pattern Recognition with the Optic Nerve Visual Prosthesis. Artificial Organs 27(11), 996–1004 (2003)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, pp. 511–518. IEEE (2001). http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=990517
Wilke, R., Gabel, V.P., Sachs, H., Bartz Schmidt, K.U., Gekeler, F., Besch, D., Szurman, P., Stett, A., Wilhelm, B., Peters, T., Harscher, A., Greppmaier, U., Kibbel, S., Benav, H., Bruckmann, A., Stingl, K., Kusnyerik, A., Zrenner, E.: Spatial resolution and perception of patterns mediated by a subretinal 16-electrode array in patients blinded by hereditary retinal dystrophies. Investigative Ophthalmology & Visual Science 52(8), 5995–6003 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Li, W.H. (2015). A Fast and Flexible Computer Vision System for Implanted Visual Prostheses. In: Agapito, L., Bronstein, M., Rother, C. (eds) Computer Vision - ECCV 2014 Workshops. ECCV 2014. Lecture Notes in Computer Science(), vol 8927. Springer, Cham. https://doi.org/10.1007/978-3-319-16199-0_48
Download citation
DOI: https://doi.org/10.1007/978-3-319-16199-0_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16198-3
Online ISBN: 978-3-319-16199-0
eBook Packages: Computer ScienceComputer Science (R0)