Advertisement

Cognitive Computation

, Volume 1, Issue 2, pp 119–127 | Cite as

Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition

  • Giacomo IndiveriEmail author
  • Elisabetta Chicca
  • Rodney J. Douglas
Article

Abstract

Neuromorphic engineering (NE) is an emerging research field that has been attempting to identify neural types of computational principles, by implementing biophysically realistic models of neural systems in Very Large Scale Integration (VLSI) technology. Remarkable progress has been made recently, and complex artificial neural sensory-motor systems can be built using this technology. Today, however, NE stands before a large conceptual challenge that must be met before there will be significant progress toward an age of genuinely intelligent neuromorphic machines. The challenge is to bridge the gap from reactive systems to ones that are cognitive in quality. In this paper, we describe recent advancements in NE, and present examples of neuromorphic circuits that can be used as tools to address this challenge. Specifically, we show how VLSI networks of spiking neurons with spike-based plasticity mechanisms and soft winner-take-all architectures represent important building blocks useful for implementing artificial neural systems able to exhibit basic cognitive abilities.

Keywords

Neuromorphic engineering Cognition Spike-based learning Winner-take-all Soft WTA VLSI 

Notes

Acknowledgments

This work was supported by the DAISY (FP6-2005-015803) EU Grant, by the Swiss National Science Foundation under Grant PMPD2-110298/1, and by the Swiss Federal Institute of Technology Zurich Grant TH02017404.

References

  1. 1.
    Abbott L, Nelson S. Synaptic plasticity: taming the beast. Nat Neurosci 2000;3:1178–83.PubMedCrossRefGoogle Scholar
  2. 2.
    Abrahamsen J, Hafliger P, Lande T. A time domain winner-take-all network of integrate-and-fire neurons. In: 2004 IEEE international symposium on circuits and systems, vol. 5. 2004. p. V-361–4.Google Scholar
  3. 3.
    Amari S, Arbib MA. Competition and cooperation in neural nets. In: Metzler J, editor. Systems neuroscience. Academic Press; 1977. p. 119–65Google Scholar
  4. 4.
    Amit DJ, Fusi S. Dynamic learning in neural networks with material synapses. Neural Comput. 1994;6:957.CrossRefGoogle Scholar
  5. 5.
    Arthur J, Boahen K. Learning in silicon: timing is everything. In: Weiss Y, Schölkopf B, Platt J, editors. Advances in neural information processing systems, vol. 18. Cambridge, MA: MIT Press; 2006. p. 1–8.Google Scholar
  6. 6.
    Badoni D, Giulioni M, Dante V, Del Giudice P. An aVLSI recurrent network of spiking neurons with reconfigurable and plastic synapses. In: Proceedings of the IEEE international symposium on circuits and systems. IEEE; 2006. p. 1227–30.Google Scholar
  7. 7.
    Bartolozzi C, Indiveri G. Synaptic dynamics in analog VLSI. Neural Comput. 2007;19(10):2581–603.PubMedCrossRefGoogle Scholar
  8. 8.
    Ben-Yishai R, Lev Bar-Or R, Sompolinsky H. Theory of orientation tuning in visual cortex. Proc Natl Acad Sci USA. 1995;92(9):3844–8.PubMedCrossRefGoogle Scholar
  9. 9.
    Bennett A. Large competitive networks. Network. 1990;1:449–62.CrossRefGoogle Scholar
  10. 10.
    Binzegger T, Douglas RJ, Martin K. A quantitative map of the circuit of cat primary visual cortex. J Neurosci. 2004;24(39):8441–53.PubMedCrossRefGoogle Scholar
  11. 11.
    Brader J, Senn W, Fusi S. Learning real world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 2007;19:2881–912.PubMedCrossRefGoogle Scholar
  12. 12.
    The Capo Caccia, workshops toward cognitive neuromorphic engineering. http://cne.ini.uzh.ch/capocaccia08. April 2008.
  13. 13.
    Chan V, Liu SC, van Schaik A. AER EAR: a matched silicon cochlea pair with address event representation interface. IEEE Trans Circuit Syst I. 2006;54(1):48–59, Special issue on sensors.Google Scholar
  14. 14.
    Chicca E, Fusi S. Stochastic synaptic plasticity in deterministic a VLSI networks of spiking neurons. In: Rattay F, editor. Proceedings of the world congress on neuroinformatics, ARGESIM reports, 2001. Vienna: ARGESIM/ASIM Verlag; 2001. p. 468–77.Google Scholar
  15. 15.
    Chicca E, Indiveri G, Douglas R. An event based VLSI network of integrate-and-fire neurons. In: Proceedings of IEEE international symposium on circuits and systems. IEEE; 2004. p. V-357–60.Google Scholar
  16. 16.
    Chicca E, Indiveri G, Douglas R. Context dependent amplification of both rate and event-correlation in a VLSI network of spiking neurons. In: Schölkopf B, Platt J, Hofmann T, editors. Advances in neural information processing systems, vol. 19. Neural Information Processing Systems Foundation. Cambridge, MA: MIT Press; 2007, in press.Google Scholar
  17. 17.
    Chicca E, Whatley AM, Dante V, Lichtsteiner P, Delbrück T, Del Giudice P, et al. A multi-chip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans Circuit Syst I Regular Paper. 2007;5(54):981–93.CrossRefGoogle Scholar
  18. 18.
    Choi TYW, Merolla PA, Arthur JV, Boahen KA, Shi BE. Neuromorphic implementation of orientation hypercolumns. IEEE Trans Circuit Syst I. 2005;52(6):1049–60.CrossRefGoogle Scholar
  19. 19.
    Culurciello E, Etienne-Cummings R, Boahen K. Arbitrated address-event representation digital image sensor. Electron Lett. 2001;37(24):1443–5.CrossRefGoogle Scholar
  20. 20.
    Dayan P, Abbott L. Theoretical neuroscience: computational and mathematical modeling of neural systems. Cambridge, MA: MIT Press; 2001.Google Scholar
  21. 21.
    Deiss S, Douglas R, Whatley A. A pulse-coded communications infrastructure for neuromorphic systems. In: Maass W, Bishop CM. editors. Pulsed neural networks, chapter 6. Cambridge, MA: MIT Press; 1998. p. 157–78.Google Scholar
  22. 22.
    Deneve S. Bayesian spiking neurons 1: inference. Neural Comput. 2007;20(1):91–117.CrossRefGoogle Scholar
  23. 23.
    Deneve S, Latham P, Pouget A. Efficient computation and cue integration with noisy population codes. Nat Neurosci. 2001;4(8):826–31.PubMedCrossRefGoogle Scholar
  24. 24.
    DeYong MR, Findley RL, Fields C. The design, fabrication, and test of a new VLSI hybrid analog-digital neural processing element. IEEE Trans Neural Netw. 1992;3(3):363–74.PubMedCrossRefGoogle Scholar
  25. 25.
    Douglas R, Mahowald M. Silicon neurons. In: M. Arbib, editor. The handbook of brain theory and neural networks. Boston, MA: MIT Press; 1995. p. 282–9.Google Scholar
  26. 26.
    Douglas R, Mahowald M, Martin K. Hybrid analog-digital architectures for neuromorphic systems. In: Proceedings of IEEE world congress on computational intelligence, vol. 3. IEEE; 1994. p. 1848–53.Google Scholar
  27. 27.
    Douglas R, Mahowald M, Mead C. Neuromorphic analogue VLSI. Annu Rev Neurosci. 1995;18:255–81.PubMedCrossRefGoogle Scholar
  28. 28.
    Douglas R, Martin K. Neural circuits of the neocortex. Annl Rev Neurosci. 2004;27:419–51.CrossRefGoogle Scholar
  29. 29.
    Fusi S. Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol Cybernet. 2002;87:459–70.CrossRefGoogle Scholar
  30. 30.
    Fusi S, Abbott LF. Limits on the memory storage capacity of bounded synapses. Nat Neurosci. 2007;10:485–93.PubMedGoogle Scholar
  31. 31.
    Fusi S, Annunziato M, Badoni D, Salamon A, Amit DJ. Spike–driven synaptic plasticity: theory, simulation, VLSI implementation. Neural Comput. 2000;12:2227–58.PubMedCrossRefGoogle Scholar
  32. 32.
    Giulioni M, Camilleri P, Dante V, Badoni D, Indiveri G, Braun J, et al. A VLSI network of spiking neurons with plastic fully configurable “stop-learning” synapses. In: Proceedings of IEEE international conference on electronics, circuits, and systems, ICECS 2008. IEEE; 2008. p. 678–81.Google Scholar
  33. 33.
    Gütig R, Sompolinsky H. The tempotron: a neuron that learns spike timing-based decisions. Nat Neurosci. 2006;9:420–28. doi: 10.1038/nn1643.
  34. 34.
    Häfliger P. Adaptive wta with an analog vlsi neuromorphic learning chip. IEEE Trans Neural Netw. 2007;18(2):551–72.PubMedCrossRefGoogle Scholar
  35. 35.
    Hahnloser R, Sarpeshkar R, Mahowald M, Douglas R, Seung S. Digital selection and analog amplification co-exist in an electronic circuit inspired by neocortex. Nature. 2000;405(6789):947–51.PubMedCrossRefGoogle Scholar
  36. 36.
    Hansel D, Sompolinsky H. Methods in neuronal modeling, chap. Modeling feature selectivity in local cortical circuits. Cambridge, MA: MIT Press; 1998. p. 499–567.Google Scholar
  37. 37.
    Hylander P, Meador J, Frie E. VLSI implementation of pulse coded winner take all networks. In: Proceedings of the 36th Midwest symposium on circuits and systems, vol. 1. 1993. p. 758–61.Google Scholar
  38. 38.
    Indiveri G, Chicca E, Douglas R. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans Neural Netw. 2006;17(1):211–21.PubMedCrossRefGoogle Scholar
  39. 39.
    Indiveri G, Fusi S. Spike-based learning in VLSI networks of integrate-and-fire neurons. In: Proceedings on IEEE international symposium on circuits and systems, ISCAS 2007. 2007. p. 3371–4.Google Scholar
  40. 40.
    Indiveri G, Mürer R, Kramer J. Active vision using an analog VLSI model of selective attention. IEEE Trans Circuit Syst II. 2001;48(5):492–500.CrossRefGoogle Scholar
  41. 41.
    Kandel ER, Schwartz J, Jessell TM. Principles of neural science. McGraw Hill; 2000.Google Scholar
  42. 42.
    Lazzaro J, Wawrzynek J, Mahowald M, Sivilotti M, Gillespie D. Silicon auditory processors as computer peripherals. IEEE Trans Neural Netw. 1993;4:523–8.PubMedCrossRefGoogle Scholar
  43. 43.
    Lichtsteiner P, Posch C, Delbruck T. An 128 × 128 120 dB 15μs-latency temporal contrast vision sensor. IEEE J Solid State Circuit. 43(2):566–76.CrossRefGoogle Scholar
  44. 44.
    Ma W, Beck J, Latham P, Pouget A. Bayesian inference with probabilistic population codes. Nat Neurosci. 2006;9(11):1432–8.PubMedCrossRefGoogle Scholar
  45. 45.
    Mallik U, Vogelstein R, Culurciello E, Etienne-Cummings R, Cauwenberghs G. A real-time spike-domain sensory information processing system. In: Proceedings of IEEE international symposium on circuits and systems, vol. 3. 2005. p. 1919–22.Google Scholar
  46. 46.
    Markram H, Lübke J, Frotscher M, Sakmann B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science. 1997;275:213–5.PubMedCrossRefGoogle Scholar
  47. 47.
    Mead C. Analog VLSI and neural systems. Reading, MA: Addison-Wesley; 1989.Google Scholar
  48. 48.
    Mead C. Neuromorphic electronic systems. Proc IEEE. 1990;78(10):1629–36.CrossRefGoogle Scholar
  49. 49.
    Merolla PA, Arthur JV, Shi BE, Boahen KA. Expandable networks for neuromorphic chips. IEEE Trans Circuit System I Fundam Theory Appl 2007;54(2):301–11.CrossRefGoogle Scholar
  50. 50.
    Mitra S, Fusi S, Indiveri G. Real-time classification of complex patterns using spike-based learning in neuromorphic VLSI. IEEE Trans Biomed Circuit Syst 2009;3(1), accepted Sept 2008, in press.Google Scholar
  51. 51.
    Oster M, Liu SC. A winner-take-all spiking network with spiking inputs. In: 11th IEEE international conference on electronics, circuits and systems (ICECS 2004). 2004.Google Scholar
  52. 52.
    Ott T, Stoop R. The neurodynamics of belief-propagation on binary markov random fields. In: Saul LK, Weiss Y, Bottou L, editors. Advances in neural information processing systems, vol. 18. Cambridge, MA: MIT Press; 2006. p. 1057–64.Google Scholar
  53. 53.
    Bofill-i Petit A, Murray A. Learning temporal correlations in biologically-inspired aVLSI. In: Proceedings of IEEE international symposium on circuits and systems, vol. V. IEEE; 2003. p. 817–20.Google Scholar
  54. 54.
    Bofill-i Petit A, Murray AF. Synchrony detection and amplification by silicon neurons with STDP synapses. IEEE Trans Neural Netw 2004;15(5):1296–304.PubMedCrossRefGoogle Scholar
  55. 55.
    Rao P. Bayesian computation in recurrent neural circuits. Neural Comput. 2004;16:1–38.PubMedCrossRefGoogle Scholar
  56. 56.
    Riis H, Hafliger P. Spike based learning with weak multi-level static memory. In: Proceedings of IEEE international symposium on circuits and systems. IEEE; 2004. p. 393–6.Google Scholar
  57. 57.
    Rutishauser U, Douglas R. State-dependent computation using coupled recurrent networks. Neural Comput. 2008; in press.Google Scholar
  58. 58.
    van Schaik A. Building blocks for electronic spiking neural networks. Neural Netw. 2001;14(6–7):617–28.PubMedCrossRefGoogle Scholar
  59. 59.
    Schemmel J, Fieres J, Meier K. Wafer-scale integration of analog neural networks. In: Proceedings of the IEEE international joint conference on neural networks. 2008, in press.Google Scholar
  60. 60.
    Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, Gómez-Rodríguez F, et al. AER building blocks for multi-layer multi-chip neuromorphic vision systems. In: Becker S, Thrun S, Obermayer K, editors. Advances in neural information processing systems, vol. 15. Cambridge, MA: MIT Press; 2005.Google Scholar
  61. 61.
    Somers DC, Nelson SB, Sur M. An emergent model of orientation selectivity in cat visual cortical simple cells. J Neurosci. 1995;15:5448–65.PubMedGoogle Scholar
  62. 62.
    Steimer A, Maass W, Douglas R. Belief-propagation in networks of spiking neurons. Neural Comput. 2009, submitted.Google Scholar
  63. 63.
    Systems of neuromorphic adaptive plastic scalable electronics (SyNAPSE). http://www.darpa.mil/dso/solicitations/baa08-28.htm. 2008.
  64. 64.
    Telluride neuromorphic cognition engineering workshop. http://ine-web.org/workshops/workshops-overview/.
  65. 65.
    van Vreeswijk C, Sompolinsky H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science. 1996;274(5293):1724–6.PubMedCrossRefGoogle Scholar
  66. 66.
    Wijekoon J, Dudek P. Compact silicon neuron circuit with spiking and bursting behaviour. Neural Netw. 2008;21(2–3):524–34.PubMedGoogle Scholar
  67. 67.
    Wilimzig C, Schneider S, Schöner G. The time course of saccadic decision making: dynamic field theory. Neural Netw. 2006;19:1059–74.PubMedCrossRefGoogle Scholar
  68. 68.
    Yang Z, Murray A, Worgotter F, Cameron K, Boonsobhak V. A neuromorphic depth-from-motion vision model with stdp adaptation. IEEE Trans Neural Netw. 2006;17(2):482–95.PubMedCrossRefGoogle Scholar
  69. 69.
    Yu A, Dayan P. Inference,attention, and decision in a Bayesian neural architecture. In: Saul LK, Weiss Y, Bottou L, editors. Advances in neural information processing systems, vol. 17. Cambridge, MA: MIT Press; 2005. p. 1577–84.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Giacomo Indiveri
    • 1
    • 2
    Email author
  • Elisabetta Chicca
    • 1
    • 2
  • Rodney J. Douglas
    • 1
    • 2
  1. 1.Institute of NeuroinformaticsUniversity of Zurich, ETH ZurichZurichSwitzerland
  2. 2.University of Zurich, ETH ZurichZurichSwitzerland

Personalised recommendations