Advertisement

Neuroinformatics

, Volume 12, Issue 3, pp 435–454 | Cite as

Efficient Spiking Neural Network Model of Pattern Motion Selectivity in Visual Cortex

  • Michael Beyeler
  • Micah Richert
  • Nikil D. Dutt
  • Jeffrey L. Krichmar
Original Article

Abstract

Simulating large-scale models of biological motion perception is challenging, due to the required memory to store the network structure and the computational power needed to quickly solve the neuronal dynamics. A low-cost yet high-performance approach to simulating large-scale neural network models in real-time is to leverage the parallel processing capability of graphics processing units (GPUs). Based on this approach, we present a two-stage model of visual area MT that we believe to be the first large-scale spiking network to demonstrate pattern direction selectivity. In this model, component-direction-selective (CDS) cells in MT linearly combine inputs from V1 cells that have spatiotemporal receptive fields according to the motion energy model of Simoncelli and Heeger. Pattern-direction-selective (PDS) cells in MT are constructed by pooling over MT CDS cells with a wide range of preferred directions. Responses of our model neurons are comparable to electrophysiological results for grating and plaid stimuli as well as speed tuning. The behavioral response of the network in a motion discrimination task is in agreement with psychophysical data. Moreover, our implementation outperforms a previous implementation of the motion energy model by orders of magnitude in terms of computational speed and memory usage. The full network, which comprises 153,216 neurons and approximately 40 million synapses, processes 20 frames per second of a 40 × 40 input video in real-time using a single off-the-shelf GPU. To promote the use of this algorithm among neuroscientists and computer vision researchers, the source code for the simulator, the network, and analysis scripts are publicly available.

Keywords

Pattern motion selectivity Spiking neural network MT GPU Real-time CARLsim 

Notes

Acknowledgments

This work was supported by the Defense Advanced Research Projects Agency (DARPA) subcontract 801888-BS. We thank Jayram M. Nageswaran for his work developing the custom spiking neural network simulator. We also thank Michael Avery, Kris Carlson, and Steve Grossberg for valuable feedback and discussion on this project.

Conflict of Interest

The authors have no conflicts of interest with this manuscript.

References

  1. Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2(2), 284–299.CrossRefGoogle Scholar
  2. Adelson, E. H., & Movshon, J. A. (1982). Phenomenal coherence of moving visual patterns. Nature, 300(5892), 523–525.PubMedCrossRefGoogle Scholar
  3. Bradley, D. C., & Goyal, M. S. (2008). Velocity computation in the primate visual system. Nature Reviews Neuroscience, 9(9), 686–695. doi: 10.1038/Nrn2472.PubMedCrossRefGoogle Scholar
  4. Browning, N. A., Grossberg, S., & Mingolla, E. (2009a). Cortical dynamics of navigation and steering in natural scenes: motion-based object segmentation, heading, and obstacle avoidance. Neural Networks, 22(10), 1383–1398. doi: 10.1016/j.neunet.2009.05.007.CrossRefGoogle Scholar
  5. Browning, N. A., Grossberg, S., & Mingolla, E. (2009b). A neural model of how the brain computes heading from optic flow in realistic scenes. Cognitive Psychology, 59(4), 320–356. doi: 10.1016/j.cogpsych.2009.07.002.PubMedCrossRefGoogle Scholar
  6. Burke, D., & Wenderoth, P. (1993). The effect of interactions between one-dimensional component gratings on 2-dimensional motion perception. Vision Research, 33(3), 343–350. doi: 10.1016/0042-6989(93)90090-J.PubMedCrossRefGoogle Scholar
  7. Chey, J., Grossberg, S., & Mingolla, E. (1997). Neural dynamics of motion grouping: from aperture ambiguity to object speed and direction. Journal of the Optical Society of America a-Optics Image Science and Vision, 14(10), 2570–2594. doi: 10.1364/Josaa.14.002570.CrossRefGoogle Scholar
  8. Chubb, C., & Sperling, G. (1988). Drift-balanced random stimuli—a general basis for studying non-fourier motion perception. Journal of the Optical Society of America a-Optics Image Science and Vision, 5(11), 1986–2007. doi: 10.1364/Josaa.5.001986.CrossRefGoogle Scholar
  9. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience: Computational and mathematical modeling of neural systems (Computational neuroscience). Cambridge: Massachusetts Institute of Technology Press.Google Scholar
  10. DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1993). Spatiotemporal organization of simple-cell receptive fields in the cat's striate cortex. II. Linearity of temporal and spatial summation. Journal of Neurophysiology, 69(4), 1118–1135.PubMedGoogle Scholar
  11. Ferrera, V. P., & Wilson, H. R. (1990). Perceived direction of moving two-dimensional patterns. Vision Research, 30(2), 273–287.PubMedCrossRefGoogle Scholar
  12. Fidjeland, A. K., & Shanahan, M. P. (2010). Accelerated simulation of spiking neural networks using GPUs. In Neural Networks (IJCNN), The 2010 International Joint Conference on, 18–23 July 2010 (pp. 1–8). doi: 10.1109/IJCNN.2010.5596678.
  13. Freeman, W. T., & Adelson, E. H. (1991). The design and use of steerable filters. In IEEE Pattern Analysis and Machine Intelligence (Vol. 13, pp. 891–906).Google Scholar
  14. Grossberg, S., & Pilly, P. K. (2008). Temporal dynamics of decision-making during motion perception in the visual cortex. Vision Research, 48(12), 1345–1373. doi: 10.1016/j.visres.2008.02.019.PubMedCrossRefGoogle Scholar
  15. Hohl, S. S., Chaisanguanthum, K. S., & Lisberger, S. G. (2013). Sensory population decoding for visually guided movements. Neuron, 79(1), 167–179. doi: 10.1016/j.neuron.2013.05.026.PubMedCentralPubMedCrossRefGoogle Scholar
  16. Indiveri, G., Chicca, E., & Douglas, R. (2006). A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Transactions on Neural Networks, 17(1), 211–221. doi: 10.1109/Tnn.2005.860850.PubMedCrossRefGoogle Scholar
  17. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), 1569–1572. doi: 10.1109/Tnn.2003.820440.PubMedCrossRefGoogle Scholar
  18. Izhikevich, E. M. (2004). Which model to use for cortical spiking neurons? IEEE Transactions on Neural Networks, 15(5), 1063–1070. doi: 10.1109/Tnn.2004.832719.PubMedCrossRefGoogle Scholar
  19. Izhikevich, E. M. (2007). Dynamical systems in neuroscience: The geometry of excitability and bursting (Computational neuroscience). Cambridge: MIT Press.Google Scholar
  20. Izhikevich, E. M., Gally, J. A., & Edelman, G. M. (2004). Spike-timing dynamics of neuronal groups. Cerebral Cortex, 14(8), 933–944. doi: 10.1093/cercor/bhh053.PubMedCrossRefGoogle Scholar
  21. Khan, M., Lester, D., Plana, L., Rast, A., Jin, X., & Painkras, E. SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor. In IEEE International Joint Conference on Neural Networks, 2008 (pp. 2849–2856).Google Scholar
  22. Koch, C. (1999). Biophysics of computation: Information processing in single neurons (Computational neuroscience). New York: Oxford University Press.Google Scholar
  23. Layton, O. W., Mingolla, E., & Browning, N. A. (2012). A motion pooling model of visually guided navigation explains human behavior in the presence of independently moving objects. Journal of Vision, 12(1), doi: 10.1167/12.1.20.
  24. Livingstone, M. S., & Conway, B. R. (2007). Contrast affects speed tuning, space-time slant, and receptive-field organization of simple cells in macaque V1. Journal of Neurophysiology, 97(1), 849–857. doi: 10.1152/jn.00762.2006.PubMedCentralPubMedCrossRefGoogle Scholar
  25. Lu, Z. L., & Sperling, G. (1995). Attention-generated apparent motion. Nature, 377(6546), 237–239. doi: 10.1038/377237a0.PubMedCrossRefGoogle Scholar
  26. Majaj, N. J., Carandini, M., & Movshon, J. A. (2007). Motion integration by neurons in macaque MT is local, not global. Journal of Neuroscience, 27(2), 366–370. doi: 10.1523/JNEUROSCI.3183-06.2007.PubMedCentralPubMedCrossRefGoogle Scholar
  27. Merolla, P. A., Arthur, J. V., Shi, B. E., & Boahen, K. A. (2007). Expandable networks for neuromorphic chips. IEEE Transactions on Circuits and Systems I-Regular Papers, 54(2), 301–311. doi: 10.1109/Tcsi.2006.887474.CrossRefGoogle Scholar
  28. Movshon, J. A., & Newsome, W. T. (1996). Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. Journal of Neuroscience, 16(23), 7733–7741.PubMedGoogle Scholar
  29. Movshon, J. A., Adelson, E. H., Gizzi, M. S., & Newsome, W. T. (1985). The analysis of moving visual patterns (Pattern recognition mechanisms). New York: Springer.Google Scholar
  30. Nageswaran, J. M., Dutt, N., Krichmar, J. L., Nicolau, A., & Veidenbaum, A. V. (2009). A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Networks, 22(5–6), 791–800. doi: 10.1016/j.neunet.2009.06.028.PubMedCrossRefGoogle Scholar
  31. Nishida, S. (2011). Advancement of motion psychophysics: review 2001–2010. Journal of Vision, 11(5), Artn 11. doi: 10.1167/11.5.11.CrossRefGoogle Scholar
  32. Pack, C. C., Berezovskii, V. K., & Born, R. T. (2001). Dynamic properties of neurons in cortical area MT in alert and anaesthetized macaque monkeys. Nature, 414(6866), 905–908. doi: 10.1038/414905a.PubMedCrossRefGoogle Scholar
  33. Perrone, J. A. (2012). A neural-based code for computing image velocity from small sets of middle temporal (MT/V5) neuron inputs. Journal of Vision, 12(8), doi: 10.1167/12.8.1.
  34. Perrone, J. A., & Thiele, A. (2001). Speed skills: measuring the visual speed analyzing properties of primate MT neurons. Nature Neuroscience, 4(5), 526–532.PubMedGoogle Scholar
  35. Perrone, J. A., & Thiele, A. (2002). A model of speed tuning in MT neurons. Vision Research, 42(8), 1035–1051.PubMedCrossRefGoogle Scholar
  36. Priebe, N. J., Cassanello, C. R., & Lisberger, S. G. (2003). The neural representation of speed in macaque area MT/V5. Journal of Neuroscience, 23(13), 5650–5661.PubMedCentralPubMedGoogle Scholar
  37. Priebe, N. J., Lisberger, S. G., & Movshon, J. A. (2006). Tuning for spatiotemporal frequency and speed in directionally selective neurons of macaque striate cortex. Journal of Neuroscience, 26(11), 2941–2950. doi: 10.1523/JNEUROSCI.3936-05.2006.PubMedCentralPubMedCrossRefGoogle Scholar
  38. Raudies, F., Mingolla, E., & Neumann, H. (2011). A model of motion transparency processing with local center-surround interactions and feedback. Neural Computation, 23(11), 2868–2914. doi: 10.1162/NECO_a_00193.PubMedCrossRefGoogle Scholar
  39. Resulaj, A., Kiani, R., Wolpert, D. M., & Shadlen, M. N. (2009). Changes of mind in decision-making. Nature, 461(7261), 263–U141. doi: 10.1038/Nature08275.PubMedCentralPubMedCrossRefGoogle Scholar
  40. Richert, M., Nageswaran, J. M., Dutt, N., & Krichmar, J. L. (2011). An efficient simulation environment for modeling large-scale cortical processing. Frontiers Neuroinformatics, 5, 19. doi: 10.3389/fninf.2011.00019.CrossRefGoogle Scholar
  41. Rodman, H. R., & Albright, T. D. (1987). Coding of visual stimulus velocity in area Mt of the Macaque. Vision Research, 27(12), 2035–2048. doi: 10.1016/0042-6989(87)90118-0.PubMedCrossRefGoogle Scholar
  42. Rodman, H. R., & Albright, T. D. (1989). Single-unit analysis of pattern-motion selective properties in the middle temporal visual area (MT). Experimental Brain Research, 75(1), 53–64.PubMedCrossRefGoogle Scholar
  43. Roitman, J. D., & Shadlen, M. N. (2002). Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. Journal of Neuroscience, 22(21), 9475–9489.PubMedGoogle Scholar
  44. Rust, N. C., Mante, V., Simoncelli, E. P., & Movshon, J. A. (2006). How MT cells analyze the motion of visual patterns. Nature Neuroscience, 9(11), 1421–1431. doi: 10.1038/Nn1786.PubMedCrossRefGoogle Scholar
  45. Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology, 86(4), 1916–1936.PubMedGoogle Scholar
  46. Simoncelli, E. P., & Heeger, D. J. (1998). A model of neuronal responses in visual area MT. Vision Research, 38(5), 743–761. doi: 10.1016/S0042-6989(97)00183-1.PubMedCrossRefGoogle Scholar
  47. Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends in Neurosciences, 27(3), 161–168. doi: 10.1016/j.tins.2004.01.006.PubMedCrossRefGoogle Scholar
  48. Smith, M. A., Majaj, N. J., & Movshon, J. A. (2005). Dynamics of motion signaling by neurons in macaque area MT. Nature Neuroscience, 8(2), 220–228. doi: 10.1038/Nn1382.PubMedCrossRefGoogle Scholar
  49. Srinivasa, N., & Cruz-Albrecht, J. M. (2012). Neuromorphic adaptive plastic scalable electronics analog learning systems. IEEE Pulse, 3(1), 51–56. doi: 10.1109/Mpul.2011.2175639.PubMedCrossRefGoogle Scholar
  50. Thiele, A., Dobkins, K. R., & Albright, T. D. (2001). Neural correlates of chromatic motion perception. Neuron, 32(2), 351–358.PubMedCrossRefGoogle Scholar
  51. van Santen, J. P. H., & Sperling, G. (1985). Elaborated Reichardt detectors. Journal of the Optical Society of America a-Optics Image Science and Vision, 2(2), 300–321.CrossRefGoogle Scholar
  52. Vogelstein, R. J., Mallik, U., Culurciello, E., Cauwenberghs, G., & Etienne-Cummings, R. (2007). A multichip neuromorphic system for spike-based visual information processing. Neural Computation, 19(9), 2281–2300. doi: 10.1162/neco.2007.19.9.2281.PubMedCrossRefGoogle Scholar
  53. Wilson, H. R., Ferrera, V. P., & Yo, C. (1992). A psychophysically motivated model for 2-dimensional motion perception. Visual Neuroscience, 9(1), 79–97.PubMedCrossRefGoogle Scholar
  54. Yudanov, D., Shaaban, M., Melton, R., & Reznik, L. (2010). GPU-based simulation of spiking neural networks with real-time performance & high accuracy. In Neural Networks (IJCNN), The 2010 International Joint Conference on, 18–23 July 2010 (pp. 1–8). doi: 10.1109/IJCNN.2010.5596334.

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Michael Beyeler
    • 1
  • Micah Richert
    • 2
    • 3
  • Nikil D. Dutt
    • 1
  • Jeffrey L. Krichmar
    • 1
    • 2
  1. 1.Department of Computer ScienceUniversity of California, IrvineIrvineUSA
  2. 2.Department of Cognitive SciencesUniversity of California IrvineIrvineUSA
  3. 3.Brain CorporationSan DiegoUSA

Personalised recommendations