Advertisement

Neuromorphic Engineering

  • Giacomo Indiveri
Part of the Springer Handbooks book series (SHB)

Abstract

Neuromorphic engineering is a relatively young field that attempts to build physical realizations of biologically realistic models of neural systems using electronic circuits implemented in very large scale integration technology. While originally focusing on models of the sensory periphery implemented using mainly analog circuits, the field has grown and expanded to include the modeling of neural processing systems that incorporate the computational role of the body, that model learning and cognitive processes, and that implement large distributed spiking neural networks using a variety of design techniques and technologies. This emerging field is characterized by its multidisciplinary nature and its focus on the physics of computation, driving innovations in theoretical neuroscience, device physics, electrical engineering, and computer science.

Keywords

Very Large Scale Integration Large Scale Integration Cortical Circuit Spike Neural Network Metal Oxide Semiconductor Field Effect 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
AER

address event representation

STDP

spike-timing dependent plasticity

VLSI

very large scale integration

References

  1. [38.1]
    W.S. McCulloch, W. Pitts: A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys. 5, 115–133 (1943)MathSciNetCrossRefzbMATHGoogle Scholar
  2. [38.2]
    J. von Neumann: The Computer and the Brain (Yale Univ. Press, New Haven 1958)zbMATHGoogle Scholar
  3. [38.3]
    F. Rosenblatt: The perceptron: A probabilistic model for information storage and organization in the brain, Psychol. Rev. 65(6), 386–408 (1958)MathSciNetCrossRefGoogle Scholar
  4. [38.4]
    M.L. Minsky: Computation: Finite and Infinite Machines (Prentice-Hall, Upper Saddle River 1967)zbMATHGoogle Scholar
  5. [38.5]
    J.J. Hopfield: Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA 79(8), 2554–2558 (1982)MathSciNetCrossRefGoogle Scholar
  6. [38.6]
    D.E. Rumelhart, J.L. McClelland: Foundations, parallel distributed processing. In: Explorations in the Microstructure of Cognition, ed. by D.E. Rumelhart, J.L. McClelland (MIT, Cambridge 1986)Google Scholar
  7. [38.7]
    T. Kohonen: Self-Organization and Associative Memory, Springer Series in Information Sciences, 2nd edn. (Springer, Berlin Heidelberg 1988)CrossRefzbMATHGoogle Scholar
  8. [38.8]
    J. Hertz, A. Krogh, R.G. Palmer: Introduction to the Theory of Neural Computation (Addison-Wesley, Reading 1991)Google Scholar
  9. [38.9]
    K. Fukushima, Y. Yamaguchi, M. Yasuda, S. Nagata: An electronic model of the retina, Proc. IEEE 58(12), 1950–1951 (1970)CrossRefGoogle Scholar
  10. [38.10]
    T. Hey: Richard Feynman and computation, Contemp. Phys. 40(4), 257–265 (1999)CrossRefGoogle Scholar
  11. [38.11]
    C.A. Mead: Analog VLSI and Neural Systems (Addison-Wesley, Reading 1989)CrossRefzbMATHGoogle Scholar
  12. [38.12]
    C. Mead: Neuromorphic electronic systems, Proc. IEEE 78(10), 1629–1636 (1990)CrossRefGoogle Scholar
  13. [38.13]
    M. Mahowald, R.J. Douglas: A silicon neuron, Nature 354, 515–518 (1991)CrossRefGoogle Scholar
  14. [38.14]
    M. Mahowald: The silicon retina, Sci. Am. 264, 76–82 (1991)CrossRefGoogle Scholar
  15. [38.15]
    R. Sarpeshkar: Brain power – borrowing from biology makes for low power computing – bionic ear, IEEE Spectrum 43(5), 24–29 (2006)CrossRefGoogle Scholar
  16. [38.16]
    R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jimenez, A. Linares-Barranco, G. Jiménez-Moreno, A. Civit-Balcells, B. Linares-Barranco: Spike events processing for vision systems, Int. Symp. Circuits Syst. (ISCAS, Piscataway) (2007)Google Scholar
  17. [38.17]
    G. Indiveri, T.K. Horiuchi: Frontiers in neuromorphic engineering, Front. Neurosci. 5(118), 1–2 (2011)Google Scholar
  18. [38.18]
    Telluride neuromorphic cognition engineering workshop, http://ine-web.org/workshops/workshops-overview
  19. [38.19]
    The Capo Caccia Workshops toward Cognitive Neuromorphic Engineering. http://capocaccia.ethz.ch.
  20. [38.20]
    K.A. Boahen: Neuromorphic microchips, Sci. Am. 292(5), 56–63 (2005)CrossRefGoogle Scholar
  21. [38.21]
    R.J. Douglas, M.A. Mahowald, C. Mead: Neuromorphic analogue VLSI, Annu. Rev. Neurosci. 18, 255–281 (1995)CrossRefGoogle Scholar
  22. [38.22]
    W. Maass, E.D. Sontag: Neural systems as nonlinear filters, Neural Comput. 12(8), 1743–1772 (2000)CrossRefGoogle Scholar
  23. [38.23]
    A. Belatreche, L.P. Maguire, M. McGinnity: Advances in design and application of spiking neural networks, Soft Comput. 11(3), 239–248 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  24. [38.24]
    R. Brette, M. Rudolph, T. Carnevale, M. Hines, D. Beeman, J.M. Bower, M. Diesmann, A. Morrison, P.H. Harris Jr., F.C. Goodman, M. Zirpe, T. Natschläger, D. Pecevski, B. Ermentrout, M. Djurfeldt, A. Lansner, O. Rochel, T. Vieville, E. Muller, A.P. Davison, S. El Boustani, A. Destexhe: Simulation of networks of spiking neurons: A review of tools and strategies, J. Comput. Neurosci. 23(3), 349–398 (2007)MathSciNetCrossRefGoogle Scholar
  25. [38.25]
    J. Brader, W. Senn, S. Fusi: Learning real world stimuli in a neural network with spike-driven synaptic dynamics, Neural Comput. 19, 2881–2912 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  26. [38.26]
    P. Rowcliffe, J. Feng: Training spiking neuronal networks with applications in engineering tasks, IEEE Trans. Neural Netw. 19(9), 1626–1640 (2008)CrossRefGoogle Scholar
  27. [38.27]
    The Blue Brain Project. EPFL website. (2005) http://bluebrain.epfl.ch/
  28. [38.28]
    E. Izhikevich, G. Edelman: Large-scale model of mammalian thalamocortical systems, Proc. Natl. Acad. Sci. USA 105, 3593–3598 (2008)CrossRefGoogle Scholar
  29. [38.29]
    Brain-Inspired Multiscale Computation in Neuromorphic Hybrid Systems (BrainScaleS). FP7 269921 EU Grant 2011–2015 Google Scholar
  30. [38.30]
    Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE). US Darpa Initiative (http://www.darpa.mil/dso/solicitations/baa08-28.html) (2009)
  31. [38.31]
    R. Freidman: Reverse engineering the brain, Biomed. Comput. Rev. 5(2), 10–17 (2009)Google Scholar
  32. [38.32]
    B.V. Benjamin, P. Gao, E. McQuinn, S. Choudhary, A.R. Chandrasekaran, J.M. Bussat, R. Alvarez-Icaza, J.V. Arthur, P.A. Merolla, K. Boahen: Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations, Proc. IEEE 102(5), 699–716 (2014)CrossRefGoogle Scholar
  33. [38.33]
    R.J. Douglas, K. Martin: Recurrent neuronal circuits in the neocortex, Curr. Biol. 17(13), R496–R500 (2007)CrossRefGoogle Scholar
  34. [38.34]
    R.J. Douglas, K.A.C. Martin: Neural circuits of the neocortex, Annu. Rev. Neurosci. 27, 419–451 (2004)CrossRefGoogle Scholar
  35. [38.35]
    C.D. Gilbert, T.N. Wiesel: Clustered intrinsic connections in cat visual cortex, J. Neurosci. 3, 1116–1133 (1983)Google Scholar
  36. [38.36]
    G.F. Cooper: The computational complexity of probabilistic inference using bayesian belief networks, Artif. Intell. 42(2/3), 393–405 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  37. [38.37]
    D.J.C. MacKay: Information Theory, Inference and Learning Algorithms (Cambridge Univ. Press, Cambridge 2003)zbMATHGoogle Scholar
  38. [38.38]
    A. Steimer, W. Maass, R. Douglas: Belief propagation in networks of spiking neurons, Neural Comput. 21, 2502–2523 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  39. [38.39]
    W. Maass: On the computational power of winner-take-all, Neural Comput. 12(11), 2519–2535 (2000)MathSciNetCrossRefGoogle Scholar
  40. [38.40]
    W. Maass, P. Joshi, E.D. Sontag: Computational aspects of feedback in neural circuits, PLOS Comput. Biol. 3(1), 1–20 (2007)MathSciNetCrossRefGoogle Scholar
  41. [38.41]
    L.F. Abbott, W.G. Regehr: Synaptic computation, Nature 431, 796–803 (2004)CrossRefGoogle Scholar
  42. [38.42]
    R. Gütig, H. Sompolinsky: The tempotron: A neuron that learns spike timing–based decisions, Nat. Neurosci. 9, 420–428 (2006)CrossRefGoogle Scholar
  43. [38.43]
    T. Wennekers, N. Ay: Finite state automata resulting from temporal information maximization and a temporal learning rule, Neural Comput. 10(17), 2258–2290 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  44. [38.44]
    U. Rutishauser, R. Douglas: State-dependent computation using coupled recurrent networks, Neural Comput. 21, 478–509 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  45. [38.45]
    P. Dayan, L.F. Abbott: Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT, Cambridge 2001)zbMATHGoogle Scholar
  46. [38.46]
    M. Arbib (Ed.): The Handbook of Brain Theory and Neural Networks, 2nd edn. (MIT, Cambridge 2002)Google Scholar
  47. [38.47]
    G. Rachmuth, H.Z. Shouval, M.F. Bear, C.-S. Poon: A biophysically-based neuromorphic model of spike rate- and timing-dependent plasticity, Proc. Natl. Acad. Sci. USA 108(49), E1266–E1274 (2011)CrossRefGoogle Scholar
  48. [38.48]
    J. Schemmel, D. Brüderle, K. Meier, B. Ostendorf: Modeling synaptic plasticity within networks of highly accelerated I & F neurons, Int. Symp. Circuits Syst. (ISCAS, Piscataway) (2007) pp. 3367–3370Google Scholar
  49. [38.49]
    J.H.B. Wijekoon, P. Dudek: Compact silicon neuron circuit with spiking and bursting behaviour, Neural Netw. 21(2/3), 524–534 (2008)CrossRefGoogle Scholar
  50. [38.50]
    D. Brüderle, M.A. Petrovici, B. Vogginger, M. Ehrlich, T. Pfeil, S. Millner, A. Grübl, K. Wendt, E. Müller, M.-O. Schwartz, D.H. de Oliveira, S. Jeltsch, J. Fieres, M. Schilling, P. Müller, O. Breitwieser, V. Petkov, L. Muller, A.P. Davison, P. Krishnamurthy, J. Kremkow, M. Lundqvist, E. Muller, J. Partzsch, S. Scholze, L. Zühl, C. Mayr, A. Destexhe, M. Diesmann, T.C. Potjans, A. Lansner, R. Schüffny, J. Schemmel, K. Meier: A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems, Biol. Cybern. 104(4), 263–296 (2011)CrossRefGoogle Scholar
  51. [38.51]
    C. Tomazou, F.J. Lidgey, D.G. Haigh (Eds.): Analogue IC Design: The Current-Mode Approach (Peregrinus, Stevenage, Herts., UK 1990)Google Scholar
  52. [38.52]
    S.-C. Liu, J. Kramer, G. Indiveri, T. Delbruck, R.J. Douglas: Analog VLSI: Circuits and Principles (MIT Press, Cambridge 2002)Google Scholar
  53. [38.53]
    C. Bartolozzi, G. Indiveri: Synaptic dynamics in analog VLSI, Neural Comput. 19(10), 2581–2603 (2007)CrossRefzbMATHGoogle Scholar
  54. [38.54]
    E.M. Drakakis, A.J. Payne, C. Toumazou: Log-domain state-space: A systematic transistor-level approach for log-domain filtering, IEEE Trans. Circuits Syst. II 46(3), 290–305 (1999)CrossRefGoogle Scholar
  55. [38.55]
    D.R. Frey: Log-domain filtering: An approach to current-mode filtering, IEE Proc G 140(6), 406–416 (1993)Google Scholar
  56. [38.56]
    S.-C. Liu, T. Delbruck: Neuromorphic sensory systems, Curr. Opin. Neurobiol. 20(3), 288–295 (2010)CrossRefGoogle Scholar
  57. [38.57]
    A. Destexhe, Z.F. Mainen, T.J. Sejnowski: Kinetic models of synaptic transmission. In: Methods in Neuronal Modelling, from Ions to Networks, ed. by C. Koch, I. Segev (MIT Press, Cambridge 1998) pp. 1–25Google Scholar
  58. [38.58]
    G. Indiveri, B. Linares-Barranco, T.J. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Häfliger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. Saighi, T. Serrano-Gotarredona, J. Wijekoon, Y. Wang, K. Boahen: Neuromorphic silicon neuron circuits, Front. Neurosci. 5, 1–23 (2011)Google Scholar
  59. [38.59]
    P. Livi, G. Indiveri: A current-mode conductance-based silicon neuron for address-event neuromorphic systems, Int. Symp. Circuits Syst. (ISCAS) (2009) pp. 2898–2901Google Scholar
  60. [38.60]
    L.F. Abbott, S.B. Nelson: Synaptic plasticity: Taming the beast, Nat. Neurosci. 3, 1178–1183 (2000)CrossRefGoogle Scholar
  61. [38.61]
    R.A. Legenstein, C. Näger, W. Maass: What can a neuron learn with spike-timing-dependent plasticity?, Neural Comput. 17(11), 2337–2382 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  62. [38.62]
    S.A. Bamford, A.F. Murray, D.J. Willshaw: Spike-timing-dependent plasticity with weight dependence evoked from physical constraints, IEEE Trans, Biomed. Circuits Syst. 6(4), 385–398 (2012)CrossRefGoogle Scholar
  63. [38.63]
    S. Mitra, S. Fusi, G. Indiveri: Real-time classification of complex patterns using spike-based learning in neuromorphic VLSI, IEEE Trans. Biomed. Circuits Syst. 3(1), 32–42 (2009)CrossRefGoogle Scholar
  64. [38.64]
    G. Indiveri, E. Chicca, R.J. Douglas: A VLSI array of low-power spiking neurons and bistable synapses with spike–timing dependent plasticity, IEEE Trans. Neural Netw. 17(1), 211–221 (2006)CrossRefGoogle Scholar
  65. [38.65]
    A. Bofill, I. Petit, A.F. Murray: Synchrony detection and amplification by silicon neurons with STDP synapses, IEEE Trans. Neural Netw. 15(5), 1296–1304 (2004)CrossRefGoogle Scholar
  66. [38.66]
    S. Fusi, M. Annunziato, D. Badoni, A. Salamon, D.J. Amit: Spike–driven synaptic plasticity: Theory, simulation, VLSI implementation, Neural Comput. 12, 2227–2258 (2000)CrossRefGoogle Scholar
  67. [38.67]
    P. Häfliger, M. Mahowald: Weight vector normalization in an analog VLSI artificial neuron using a backpropagating action potential. In: Neuromorphic Systems: Engineering Silicon from Neurobiology, ed. by L.S. Smith, A. Hamilton (World Scientific, London 1998) pp. 191–196CrossRefGoogle Scholar
  68. [38.68]
    P.A. Merolla, J.V. Arthur, R. Alvarez-Icaza, A. Cassidy, J. Sawada, F. Akopyan, B.L. Jackson, N. Imam, A. Chandra, C. Guo, Y. Nakamura, B. Brezzo, I. Vo, S.K. Esser, R. Appuswamy, B. Taba, A. Amir, M.D. Flickner, W.P. Risk, R. Manohar, D.S. Modha: A million spiking-neuron integrated circuit with a scalable communication network and interface, Science 345(6197), 668–673 (2014)CrossRefGoogle Scholar
  69. [38.69]
    R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodriguez, L. Camunas-Mesa, R. Berner, M. Rivas-Perez, T. Delbruck, S.-C. Liu, R. Douglas, P. Häfliger, G. Jimenez-Moreno, A. Civit-Ballcels, T. Serrano-Gotarredona, A.J. Acosta-Jiménez, B. Linares-Barranco: CAVIAR: A 45k neuron, 5M synapse, 12G connects/s aer hardware sensory–processing–learning–actuating system for high-speed visual object recognition and tracking, IEEE Trans. Neural Netw. 20(9), 1417–1438 (2009)CrossRefGoogle Scholar
  70. [38.70]
    E. Chicca, A.M. Whatley, P. Lichtsteiner, V. Dante, T. Delbruck, P. Del Giudice, R.J. Douglas, G. Indiveri: A multi-chip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity, IEEE Trans. Circuits Syst. I 5(54), 981–993 (2007)CrossRefGoogle Scholar
  71. [38.71]
    T.Y.W. Choi, P.A. Merolla, J.V. Arthur, K.A. Boahen, B.E. Shi: Neuromorphic implementation of orientation hypercolumns, IEEE Trans. Circuits Syst. I 52(6), 1049–1060 (2005)MathSciNetCrossRefGoogle Scholar
  72. [38.72]
    M. Mahowald: An Analog VLSI System for Stereoscopic Vision (Kluwer, Boston 1994)CrossRefGoogle Scholar
  73. [38.73]
    K.A. Boahen: Point-to-point connectivity between neuromorphic chips using address-events, IEEE Trans. Circuits Syst. II 47(5), 416–434 (2000)CrossRefzbMATHGoogle Scholar
  74. [38.74]
    A.J. Martin, M. Nystrom: Asynchronous techniques for system-on-chip design, Proc. IEEE 94, 1089–1120 (2006)CrossRefGoogle Scholar
  75. [38.75]
    G. Schoner: Dynamical systems approaches to cognition. In: Cambridge Handbook of Computational Cognitive Modeling, ed. by R. Sun (Cambridge Univ. Press, Cambridge 2008) pp. 101–126Google Scholar
  76. [38.76]
    G. Indiveri, E. Chicca, R.J. Douglas: Artificial cognitive systems: From VLSI networks of spiking neurons to neuromorphic cognition, Cogn. Comput. 1, 119–127 (2009)CrossRefGoogle Scholar
  77. [38.77]
    M. Giulioni, P. Camilleri, M. Mattia, V. Dante, J. Braun, P. Del Giudice: Robust working memory in an asynchronously spiking neural network realized in neuromorphic VLSI, Front. Neurosci. 5, 1–16 (2011)Google Scholar
  78. [38.78]
    E. Neftci, J. Binas, U. Rutishauser, E. Chicca, G. Indiveri, R. Douglas: Synthesizing Cognition in neuromorphic electronic Systems, Proc. Natl. Acad. Sci. USA 110(37), E3468–E3476 (2013)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Inst. NeuroinformaticsUniversity of Zurich and ETH ZurichZurichSwitzerland

Personalised recommendations