Advertisement

Neurolight Alpha: Interfacing Computational Neural Models for Stimulus Modulation in Cortical Visual Neuroprostheses

Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11486)

Abstract

Visual neuroprostheses that provide electrical stimulation along several sites of the human visual system constitute a potential tool for vision restoring for the blind. In the context of a NIH approved human clinical trials project (CORTIVIS), we now face the challenge of developing not only computationally powerful, but also flexible tools that allow us to generate useful knowledge in an efficient way. In this work, we address the development and implementation of computational models of different types of visual neurons and design a tool -Neurolight alpha- that allows interfacing these models with a visual neural prosthesis in order to create more naturalistic electrical stimulation patterns. We implement the complete pipeline, from obtaining a video stream to developing and deploying predictive models of retinal ganglion cell’s encoding of visual inputs into the control of a cortical microstimulation device which will send electrical train pulses through an Utah Array to the neural tissue.

Keywords

Visual neuroprostheses Neural encoding Computational models Artificial vision 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work is supported by the Programa de Ayudas a Grupos de Excelencia de la Región de Murcia, Fundación Séneca, Agencia de Ciencia y Tecnología de la Región de Murcia.

References

  1. 1.
    Wolpaw, J.R., et al.: Brain-computer interface technology: a review of the first international meeting (2000)Google Scholar
  2. 2.
    Davis, T.S., et al.: Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves. J. Neural Eng. 13(3), 36001 (2016)CrossRefGoogle Scholar
  3. 3.
    Nuyujukian, P., et al.: Cortical control of a tablet computer by people with paralysis. PLoS ONE 13(11), e0204566 (2018)CrossRefGoogle Scholar
  4. 4.
    Fattahi, P., Yang, G., Kim, G., Abidian, M.R.: A review of organic and inorganic biomaterials for neural interfaces. Adv. Mater. 26(12), 1846–85 (2014)CrossRefGoogle Scholar
  5. 5.
    House, W.F.: Cochlear implants. Ann. Otol. Rhinol. Laryngol. 85(Suppl. 3), 3 (1976)CrossRefGoogle Scholar
  6. 6.
    Weiland, J.D., Liu, W., Humayun, M.S.: Retinal prosthesis. Annu. Rev. Biomed. Eng. 7(1), 361–401 (2005)CrossRefGoogle Scholar
  7. 7.
    Mayberg, H.S., et al.: Deep brain stimulation for treatment-resistant depression. Neuron 45(5), 651–660 (2005)CrossRefGoogle Scholar
  8. 8.
    Sengupta, A., et al.: Red-shifted channelrhodopsin stimulation restores light responses in blind mice, macaque retina, and human retina. EMBO Mol. Med. 8(11), 1248–1264 (2016)CrossRefGoogle Scholar
  9. 9.
    Vassanelli, S., Mahmud, M.: Trends and challenges in neuroengineering: toward intelligent neuroprostheses through brain inspired systems; communication. Front. Neurosci. 10, 438 (2016)CrossRefGoogle Scholar
  10. 10.
    da Cruz, L., et al.: Five-year safety and performance results from the argus II retinal prosthesis system clinical trial. Ophthalmology 123(10), 2248–2254 (2016)CrossRefGoogle Scholar
  11. 11.
    Hornig, R., et al.: Pixium vision: first clinical results and innovative developments. In: Gabel, V. (ed.) Artificial Vision, pp. 99–113. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-41876-6_8CrossRefGoogle Scholar
  12. 12.
    Fernandez, E.: Development of visual neuroprostheses: trends and challenges. Bioelectron. Med. 4(1), 12 (2018)CrossRefGoogle Scholar
  13. 13.
    Normann, R.A., Greger, B.A., House, P., Romero, S.F., Pelayo, F., Fernandez, E.: Toward the development of a cortically based visual neuroprosthesis. J. Neural Eng. 6(3), 35001 (2009)CrossRefGoogle Scholar
  14. 14.
    Dobelle, W.H.: Artificial vision for the blind by connecting a television camera to the visual cortex. ASAIO J. 46(1), 3–9 (2000)CrossRefGoogle Scholar
  15. 15.
    Troyk, P., et al.: A model for intracortical visual prosthesis research. Artif. Organs 27(11), 1005–1015 (2003)CrossRefGoogle Scholar
  16. 16.
    Lowery, A.J.: Introducing the Monash vision group’s cortical prosthesis. In: IEEE International Conference on Image Processing 2013, pp. 1536–1539 (2013)Google Scholar
  17. 17.
    Development of a Cortical Visual Neuroprosthesis for the Blind (CORTIVIS). ClinicalTrials.gov. Identifier: NCT02983370
  18. 18.
    Early Feasibility Study of the Orion Visual Cortical Prosthesis System. ClinicalTrials.gov. Identifier: NCT03344848
  19. 19.
    Shannon, R.V.: A model of threshold for pulsatile electrical stimulation of cochlear implants. Hear. Res. 40(3), 197–204 (1989).  https://doi.org/10.1016/0378-5955(89)90160-3CrossRefGoogle Scholar
  20. 20.
    Golden, J.R., et al.: Simulation of visual perception and learning with a retinal prosthesis. J. Neural Eng. 16, 025003 (2019)CrossRefGoogle Scholar
  21. 21.
    Jepson, L.H., Hottowy, P., Weiner, G.A., Dabrowski, W., Litke, A.M., Chichilnisky, E.J.: High-fidelity reproduction of spatiotemporal visual signals for retinal prosthesis. Neuron 83(1), 87–92 (2014)CrossRefGoogle Scholar
  22. 22.
    Shah, N.P., Madugula, S., Chichilnisky, E.J., Shlens, J., Singer, Y.: Learning a neural response metric for retinal prosthesis (2018)Google Scholar
  23. 23.
    Beyeler, M., Boynton, G., Fine, I., Rokem, A.: pulse2percept: A Python-based simulation framework for bionic vision. In: Proceedings of the 16th Python in Science Conference, pp. 81–88 (2017)Google Scholar
  24. 24.
    Lozano, A., Soto-Sánchez, C., Garrigós, J., Martínez, J.J., Ferrández, J.M., Fernández, E.: A 3D convolutional neural network to model retinal ganglion cell’s responses to light patterns in mice. Int. J. Neural Syst. 28(10), 1850043 (2018)CrossRefGoogle Scholar
  25. 25.
    Crespo-Cano, R., Martínez-Álvarez, A., Díaz-Tahoces, A., Cuenca-Asensi, S., Ferrández, J.M., Fernández, E.: On the automatic tuning of a retina model by using a multi-objective optimization. In: Artificial Computation in Biology and Medicine, Elche, Spain, pp. 108–118 (2015)Google Scholar
  26. 26.
    Mcintosh, L., Maheswaranathan, N., Nayebi, A., Ganguli, S., Baccus, S.: Deep learning models of the retinal response to natural scenes. In: Advances in Neural Information Processing Systems, Barcelona, Spain, vol. 29, pp. 1369–1377 (2016)Google Scholar
  27. 27.
    Yan, Q., et al.: Revealing fine structures of the retinal receptive field by deep learning networks (2018). (Lateral geniculate nucleus, V1, V4...). In our work, we focus on the first stage of visual processing: the retinaGoogle Scholar
  28. 28.
    Bradski, G.: The openCV library. Dr. Dobb’s J. Softw. Tools 25, 120–125 (2000)Google Scholar
  29. 29.
    Jones, E., Oliphant, T.E., Peterson, P., et al.: SciPy: open source scientific tools for Python (2001)Google Scholar
  30. 30.
    Travis E, Oliphant. A Guide to NumPy. Trelgol Publishing, USA (2006)Google Scholar
  31. 31.
    Maynard, E.M., Nordhausen, C.T., Normann, R.A.: The utah intracortical electrode array: a recording structure for potential brain-computer interfaces. Electroencephalogr. Clin. Neurophysiol. 102(3), 228–239 (1997).  https://doi.org/10.1016/s0013-4694(96)95176-0CrossRefGoogle Scholar
  32. 32.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. In: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Learning (2016)Google Scholar
  33. 33.
    Chollet, F.: Keras (2015). https://github.com/fchollet/keras
  34. 34.
    Intel’s Neural Compute Stick. https://movidius.github.io/ncsdk/ncs.html
  35. 35.
    Deng, J., et al.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)Google Scholar
  36. 36.
    Baccus, S.A., Meister, M.: Fast and slow contrast adaptation in retinal circuitry. Neuron 36(5), 909–919 (2002)CrossRefGoogle Scholar
  37. 37.
    Kingma, D.P., Ba, J.L.: ADAM: a method for stochastic optimizationGoogle Scholar
  38. 38.
    Dobelle, W.H., Mladejovsky, M.G.: Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind. J. Physiol. 243(2), 553–576 (1974)CrossRefGoogle Scholar
  39. 39.
    Schmidt, E.M., Bak, M.J., Hambrecht, F.T., Kufta, C.V., O’Rourke, D.K., Vallabhanath, P.: Feasibility of a visual prosthesis for the blind based on intracortical micro stimulation of the visual cortex. Brain 119(2), 507–522 (1996)CrossRefGoogle Scholar
  40. 40.
    Davis, T.S., et al.: Spatial and temporal characteristics of V1 microstimulation during chronic implantation of a microelectrode array in a behaving macaque. J. Neural Eng. 9(6), 65003 (2012)CrossRefGoogle Scholar
  41. 41.
    Foroushani, A.N., Pack, C.C., Sawan, M.: Cortical visual prostheses: from microstimulation to functional percept. J. Neural Eng. 15(2), 21005 (2018)CrossRefGoogle Scholar
  42. 42.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation (2015)Google Scholar
  43. 43.
    Benjamin Naecker, N.M.: pyret: retinal data analysis in Python - pyret 0.6.0 documentationGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Dpto. Electrónica, Tecnología de Computadoras y ProyectosUniversidad Politécnica de CartagenaCartagenaSpain
  2. 2.Instituto de BioingenieríaUniversidad Miguel HernándezAlicanteSpain
  3. 3.CIBER-BBNMadridSpain

Personalised recommendations

Cite paper

Buy options