Neurolight Alpha: Interfacing Computational Neural Models for Stimulus Modulation in Cortical Visual Neuroprostheses
- 1 Citations
- 729 Downloads
Abstract
Visual neuroprostheses that provide electrical stimulation along several sites of the human visual system constitute a potential tool for vision restoring for the blind. In the context of a NIH approved human clinical trials project (CORTIVIS), we now face the challenge of developing not only computationally powerful, but also flexible tools that allow us to generate useful knowledge in an efficient way. In this work, we address the development and implementation of computational models of different types of visual neurons and design a tool -Neurolight alpha- that allows interfacing these models with a visual neural prosthesis in order to create more naturalistic electrical stimulation patterns. We implement the complete pipeline, from obtaining a video stream to developing and deploying predictive models of retinal ganglion cell’s encoding of visual inputs into the control of a cortical microstimulation device which will send electrical train pulses through an Utah Array to the neural tissue.
Keywords
Visual neuroprostheses Neural encoding Computational models Artificial visionNotes
Acknowledgments
This work is supported by the Programa de Ayudas a Grupos de Excelencia de la Región de Murcia, Fundación Séneca, Agencia de Ciencia y Tecnología de la Región de Murcia.
References
- 1.Wolpaw, J.R., et al.: Brain-computer interface technology: a review of the first international meeting (2000)Google Scholar
- 2.Davis, T.S., et al.: Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves. J. Neural Eng. 13(3), 36001 (2016)CrossRefGoogle Scholar
- 3.Nuyujukian, P., et al.: Cortical control of a tablet computer by people with paralysis. PLoS ONE 13(11), e0204566 (2018)CrossRefGoogle Scholar
- 4.Fattahi, P., Yang, G., Kim, G., Abidian, M.R.: A review of organic and inorganic biomaterials for neural interfaces. Adv. Mater. 26(12), 1846–85 (2014)CrossRefGoogle Scholar
- 5.House, W.F.: Cochlear implants. Ann. Otol. Rhinol. Laryngol. 85(Suppl. 3), 3 (1976)CrossRefGoogle Scholar
- 6.Weiland, J.D., Liu, W., Humayun, M.S.: Retinal prosthesis. Annu. Rev. Biomed. Eng. 7(1), 361–401 (2005)CrossRefGoogle Scholar
- 7.Mayberg, H.S., et al.: Deep brain stimulation for treatment-resistant depression. Neuron 45(5), 651–660 (2005)CrossRefGoogle Scholar
- 8.Sengupta, A., et al.: Red-shifted channelrhodopsin stimulation restores light responses in blind mice, macaque retina, and human retina. EMBO Mol. Med. 8(11), 1248–1264 (2016)CrossRefGoogle Scholar
- 9.Vassanelli, S., Mahmud, M.: Trends and challenges in neuroengineering: toward intelligent neuroprostheses through brain inspired systems; communication. Front. Neurosci. 10, 438 (2016)CrossRefGoogle Scholar
- 10.da Cruz, L., et al.: Five-year safety and performance results from the argus II retinal prosthesis system clinical trial. Ophthalmology 123(10), 2248–2254 (2016)CrossRefGoogle Scholar
- 11.Hornig, R., et al.: Pixium vision: first clinical results and innovative developments. In: Gabel, V. (ed.) Artificial Vision, pp. 99–113. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-41876-6_8CrossRefGoogle Scholar
- 12.Fernandez, E.: Development of visual neuroprostheses: trends and challenges. Bioelectron. Med. 4(1), 12 (2018)CrossRefGoogle Scholar
- 13.Normann, R.A., Greger, B.A., House, P., Romero, S.F., Pelayo, F., Fernandez, E.: Toward the development of a cortically based visual neuroprosthesis. J. Neural Eng. 6(3), 35001 (2009)CrossRefGoogle Scholar
- 14.Dobelle, W.H.: Artificial vision for the blind by connecting a television camera to the visual cortex. ASAIO J. 46(1), 3–9 (2000)CrossRefGoogle Scholar
- 15.Troyk, P., et al.: A model for intracortical visual prosthesis research. Artif. Organs 27(11), 1005–1015 (2003)CrossRefGoogle Scholar
- 16.Lowery, A.J.: Introducing the Monash vision group’s cortical prosthesis. In: IEEE International Conference on Image Processing 2013, pp. 1536–1539 (2013)Google Scholar
- 17.Development of a Cortical Visual Neuroprosthesis for the Blind (CORTIVIS). ClinicalTrials.gov. Identifier: NCT02983370
- 18.Early Feasibility Study of the Orion Visual Cortical Prosthesis System. ClinicalTrials.gov. Identifier: NCT03344848
- 19.Shannon, R.V.: A model of threshold for pulsatile electrical stimulation of cochlear implants. Hear. Res. 40(3), 197–204 (1989). https://doi.org/10.1016/0378-5955(89)90160-3CrossRefGoogle Scholar
- 20.Golden, J.R., et al.: Simulation of visual perception and learning with a retinal prosthesis. J. Neural Eng. 16, 025003 (2019)CrossRefGoogle Scholar
- 21.Jepson, L.H., Hottowy, P., Weiner, G.A., Dabrowski, W., Litke, A.M., Chichilnisky, E.J.: High-fidelity reproduction of spatiotemporal visual signals for retinal prosthesis. Neuron 83(1), 87–92 (2014)CrossRefGoogle Scholar
- 22.Shah, N.P., Madugula, S., Chichilnisky, E.J., Shlens, J., Singer, Y.: Learning a neural response metric for retinal prosthesis (2018)Google Scholar
- 23.Beyeler, M., Boynton, G., Fine, I., Rokem, A.: pulse2percept: A Python-based simulation framework for bionic vision. In: Proceedings of the 16th Python in Science Conference, pp. 81–88 (2017)Google Scholar
- 24.Lozano, A., Soto-Sánchez, C., Garrigós, J., Martínez, J.J., Ferrández, J.M., Fernández, E.: A 3D convolutional neural network to model retinal ganglion cell’s responses to light patterns in mice. Int. J. Neural Syst. 28(10), 1850043 (2018)CrossRefGoogle Scholar
- 25.Crespo-Cano, R., Martínez-Álvarez, A., Díaz-Tahoces, A., Cuenca-Asensi, S., Ferrández, J.M., Fernández, E.: On the automatic tuning of a retina model by using a multi-objective optimization. In: Artificial Computation in Biology and Medicine, Elche, Spain, pp. 108–118 (2015)Google Scholar
- 26.Mcintosh, L., Maheswaranathan, N., Nayebi, A., Ganguli, S., Baccus, S.: Deep learning models of the retinal response to natural scenes. In: Advances in Neural Information Processing Systems, Barcelona, Spain, vol. 29, pp. 1369–1377 (2016)Google Scholar
- 27.Yan, Q., et al.: Revealing fine structures of the retinal receptive field by deep learning networks (2018). (Lateral geniculate nucleus, V1, V4...). In our work, we focus on the first stage of visual processing: the retinaGoogle Scholar
- 28.Bradski, G.: The openCV library. Dr. Dobb’s J. Softw. Tools 25, 120–125 (2000)Google Scholar
- 29.Jones, E., Oliphant, T.E., Peterson, P., et al.: SciPy: open source scientific tools for Python (2001)Google Scholar
- 30.Travis E, Oliphant. A Guide to NumPy. Trelgol Publishing, USA (2006)Google Scholar
- 31.Maynard, E.M., Nordhausen, C.T., Normann, R.A.: The utah intracortical electrode array: a recording structure for potential brain-computer interfaces. Electroencephalogr. Clin. Neurophysiol. 102(3), 228–239 (1997). https://doi.org/10.1016/s0013-4694(96)95176-0CrossRefGoogle Scholar
- 32.Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. In: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Learning (2016)Google Scholar
- 33.Chollet, F.: Keras (2015). https://github.com/fchollet/keras
- 34.Intel’s Neural Compute Stick. https://movidius.github.io/ncsdk/ncs.html
- 35.Deng, J., et al.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)Google Scholar
- 36.Baccus, S.A., Meister, M.: Fast and slow contrast adaptation in retinal circuitry. Neuron 36(5), 909–919 (2002)CrossRefGoogle Scholar
- 37.Kingma, D.P., Ba, J.L.: ADAM: a method for stochastic optimizationGoogle Scholar
- 38.Dobelle, W.H., Mladejovsky, M.G.: Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind. J. Physiol. 243(2), 553–576 (1974)CrossRefGoogle Scholar
- 39.Schmidt, E.M., Bak, M.J., Hambrecht, F.T., Kufta, C.V., O’Rourke, D.K., Vallabhanath, P.: Feasibility of a visual prosthesis for the blind based on intracortical micro stimulation of the visual cortex. Brain 119(2), 507–522 (1996)CrossRefGoogle Scholar
- 40.Davis, T.S., et al.: Spatial and temporal characteristics of V1 microstimulation during chronic implantation of a microelectrode array in a behaving macaque. J. Neural Eng. 9(6), 65003 (2012)CrossRefGoogle Scholar
- 41.Foroushani, A.N., Pack, C.C., Sawan, M.: Cortical visual prostheses: from microstimulation to functional percept. J. Neural Eng. 15(2), 21005 (2018)CrossRefGoogle Scholar
- 42.Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation (2015)Google Scholar
- 43.Benjamin Naecker, N.M.: pyret: retinal data analysis in Python - pyret 0.6.0 documentationGoogle Scholar