Skip to main content
SpringerLink
Log in

Event-based silicon retinas and cochleas

  • Chapter
Frontiers in Sensing

Abstract

This chapter reviews neuromorphic silicon retinas and cochleas that are based on the structure and operation of their biological counterparts. These devices are built using conventional chip fabrication technologies, using transistor circuits that emulate neural computations from biology. In first generation sensors, the analog outputs of every cell were read out serially at fixed sample rates. The new generation of sensors reports only the outputs of active cells through digital events (spikes) that are communicated asynchronously. Such sensors respond more quickly with reduced power consumption. Their digital “address-event” outputs rapidly convey precise timing information about the scene that is only attained from conventional sensors if they are continuously sampled at high rates. The sparseness, low latency, and spatio-temporal structure of this new form of sensor output data can benefit subsequent post-processing algorithms. Tradeoffs in the design of neuromorphic visual and auditory sensors are discussed. Examples are given of vision algorithms that process the address-events, using their spatio-temporal coherence, for low-level feature extraction and for object tracking.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Abdalla H, Horiuchi T (2005) An ultrasonic filterbank with spiking neurons, IEEE Intl. Symp. on Circuits and Systems (ISCAS 2005), pp. 4201–4204

    Google Scholar 

  • Barbaro M, Burgi PY, Mortara A, Nussbaum P, Heitger F (2002) A 100 × 100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding. IEEE J. of Solid-State Circuits 37: 160–172

    Article  Google Scholar 

  • Bauer D, Belbachir AN, Donath N, Gritsch G, Kohn B, et al. (2007) Embedded vehicle speed estimation system using an asynchronous temporal contrast vision sensor. EURASIP J Embedded Syst 2007 (art. ID 82 174): 1–12

    Google Scholar 

  • Belbachir AN, Litzenberger M, Posch C, Schon P (2007) Real-time vision using a smart sensor system. IEEE Intl Symp Industrial Electronics 2007. ISIE 2007, pp. 1968–1973.

    Google Scholar 

  • Berner R, Lichtsteiner P, Delbruck T (2008) Self-timed vertacolor dichromatic vision sensor for low power face detection. IEEE Intl Symp Circuits and Systems (ISCAS 2008), pp. 1032–1035

    Google Scholar 

  • Boerlin M, Delbruck T, Eng K (2009) Getting to know your neighbors: Unsupervised learning of topography from real-world, event-based input. Neural Computation 21: 216–238

    Article  PubMed  Google Scholar 

  • Chan V, Liu S-C, van Schaik A (2007) AER EAR: A matched silicon cochlea pair with address event representation interface. IEEE Trans Circuits and Systems I: Regular Papers 54: 48–59

    Article  Google Scholar 

  • Chicca E, Whatley AM, Lichtsteiner P, Dante V, Delbruck T, et al. (2006) A multi-chip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans Circuits and Systems I: Regular Papers 54: 981–993

    Article  Google Scholar 

  • Choi TYW, Merolla PA, Arthur JV, Boahen KA, Shi BE (2005) Neuromorphic implementation of orientation hypercolumns. IEEE Trans Circuits and Systems I: Regular Papers 52: 1049–1060

    Article  Google Scholar 

  • Conradt J, Berner C. M., Delbruck T (2009) An embedded AER dynamic vision sensor for low-latency pole balancing. 5th IEEE Workshop on Embedded Computer Vision (in conjunction with ICCV 2009), Kyoto, Japan

    Google Scholar 

  • Delbruck T, Lichtsteiner P (2007) Fast sensory motor control based on event-based hybrid neuromorphic-procedural system. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 845–848

    Google Scholar 

  • Delbruck T (2008) Frame-free dynamic digital vision. Proc Intl Symp Secure-Life Electronics, Advanced Electronics for Quality Life and Society, pp. 21–26. Tokyo: University of Tokyo

    Google Scholar 

  • Douglas R, Mahowald M, Mead C (1995) Neuromorphic Analog VLSI. Ann Rev Neurosci 18: 255–281

    Article  PubMed  CAS  Google Scholar 

  • Fasnacht D, Delbruck T (2007) Dichromatic spectral measurement circuit in vanilla CMOS. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 3091–3094

    Google Scholar 

  • Fossum ER (1997) CMOS image sensors: electronic camera-on-a-chip. IEEE Trans Electron Devices 44: 1689–1698

    Article  Google Scholar 

  • Fragniere E (2005) A 100-channel analog CMOS auditory filter bank for speech recognition. IEEE ISSCC Dig of Tech Papers, pp. 140–589

    Google Scholar 

  • Fu Z, Delbruck T, Lichtsteiner P, Culurciello E (2008) An address-event fall detector for assisted living applications. IEEE Trans Biomed Circuits and Systems 2: 88–96.

    Article  Google Scholar 

  • Georgiou J, Toumazou C (2005) A 126-uW cochlear chip for a totally implantable system. IEEE J. Solid-State Circuits 40: 430–443

    Article  Google Scholar 

  • Gold B, Morgan N (2000) Speech and audio signal processing: John Wiley and Sons, Inc. New York, NY

    Google Scholar 

  • Gritsch G, Litzenberger M, Donath N, Kohn B (2008) Real-time vehicle classification using a smart embedded device with a’ silicon retina’ optical sensor. ITSC08, pp. 534–538. Bejing, China

    Google Scholar 

  • Hamilton T, Tapson J, Jin CT, van Schaik A (2008) An active 2-D silicon cochlea. IEEE Trans Biomed Circuits and Systems 2: 30–43

    Article  Google Scholar 

  • Indiveri G, Liu S-C, Delbruck T, Douglas R (2009) Neuromorphic systems. In: L Squire (ed) Encyclopedia of neuroscience, pp. 521–528: Academic Press

    Google Scholar 

  • jAER (2007) jAER Real time sensory-motor processing for spike based address-event representation (AER) sensors and systems available: http://jaer.wiki.sourceforge.net

    Google Scholar 

  • Katsiamis A, Drakakis E, Lyon R (2009) A biomimetic, 4.5uW, 120+ dB, log-domain cochlea channel with AGC. IEEE J Solid-State Circuits 44: 1006–1022

    Article  Google Scholar 

  • Kramer J (2002) An ON/OFFtransient imager with event-driven, asynchronous read-out. IEEE Intl Symp Circuits and Systems (ISCAS 2002), pp. 165–168

    Google Scholar 

  • Lazzaro J, Wawrzynek J, Mahowald M, Sivilotti M, Gillespie D (1993) Silicon auditory processors as computer peripherals. IEEE Trans Neural Networks 4: 523–528

    Article  CAS  Google Scholar 

  • Lennie P (2003) The cost of cortical computation. Current Biology 13: 493–497

    Article  PubMed  CAS  Google Scholar 

  • Lenoro-Bardallo JA, Serrano-Gotarredona T, Linares-Barranco B (2010) A spatial calibrated contrast AER vision sensor with adjustable contrast threshold. IEEE Intl Symp Circuits and Systems (ISCAS 2010), pp. 2426–2429

    Google Scholar 

  • Lichtsteiner P, Posch C, Delbruck T (2006) A 128×128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change. ISSCC Dig Tech. Papers, pp. 508–509 (27.9). San Francisco

    Google Scholar 

  • Lichtsteiner P, Posch C, Delbruck T (2008) A 128×128 120 dB 15us latency asynchronous temporal contrast vision sensor. IEEE J Solid State Circuits 43: 566–576

    Article  Google Scholar 

  • Linares-Barranco A, Gómez-Rodríguez F, Jiménez A, Delbruck T, Lichtsteiner P (2007) Using FPGA for visuo-motor control with a silicon retina and a humanoid robot. IEEE Intl Symp Circuits and Systems (ISCAS 2007), pp. 1192–1195

    Google Scholar 

  • Litzenberger M, Posch C, Bauer D, Schön P, Kohn B, et al. (2006) Embedded vision system for real-time object tracking using an asynchronous transient vision sensor. IEEE Digital Signal Proc Workshop 2006, pp. 173–178. Grand Teton, Wyoming

    Google Scholar 

  • Liu SC, Kramer J, Indiveri G, Delbruck T, Douglas R (2002) Analog VLSI: circuits and principles: MIT Press, Cambridge, MA

    Google Scholar 

  • Liu SC and Delbruck T (2010) Neuromorphic sensory systems. Curr. Opin. in Neurobiol 20: 288–295

    Article  Google Scholar 

  • Liu SC, van Schaik A, Minch BA, Delbruck T (2010a) Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms. IEEE Intl Symp Circuits and Systems 2010 (ISCAS 2010), pp. 2027–2030

    Google Scholar 

  • Liu SC, Mesgarani N, Harris, J, Hermansky, H (2010b) The use of spike-based representations for hardware audition systems. IEEE Intl Symp Circuits and Systems 2010 (ISCAS 2010), pp. 505–508

    Google Scholar 

  • Lyon RF, Mead C (1988) An analog electronic cochlea. IEEE Trans Acoustics Speech and Signal Processing 36: 1119–1134

    Article  Google Scholar 

  • Mahowald MA (1992) VLSI analogs of neuronal visual processing: a synthesis of form and function. Computation and neural systems, Caltech, Pasadena, California

    Google Scholar 

  • Mahowald MA (1994) An analog VLSI system for stereoscopic vision: Kluwer, Boston, MA

    Google Scholar 

  • Mahowald MA, Mead C (1991) The silicon retina. Sci Am 264: 76–82

    Article  PubMed  CAS  Google Scholar 

  • Mallik U, Clapp M, Choi E, Cauwenberghs G, Etienne-Cummings R (2005) Temporal change threshold detection imager. IEEE ISSCC Dig Tech. Papers, pp. 362–363

    Google Scholar 

  • Martignoli S, van der Vyver J-J, Kern A, Uwate Y, Stoop R (2007) Analog electronic cochlea with mammalian hearing characteristics. Applied Physics Letters 91 (064 108)

    Google Scholar 

  • Massari N, Gottardi M, Jawed S (2008) A 100uW 64 × 128-pixel contrast-based asynchronous bin ary vision sensor for wireless sensor networks. IEEE ISSCC Dig Tech Papers, pp. 588–638

    Google Scholar 

  • Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78: 1629–1636

    Article  Google Scholar 

  • Olsson JAM, Hafliger P (2009) Live demonstration of an asynchronous integrate-and-fire pixel-event vision sensor. IEEE Intl Symp Circuits and Systems (ISCAS 2009), pp. 774 774

    Google Scholar 

  • Pelgrom M, Tuinhout H, Vertregt M (1998) Transistor matching in analog CMOS applications. IEDM Tech Dig: 915–918

    Google Scholar 

  • Posch C, Hofstatter M, Matolin D, Vanstraelen G, Schon P, et al. (2007) A dual-line optical transient sensor with on-chip precision time-stamp generation. IEEE ISSCC Dig Tech Papers, pp. 500–618

    Google Scholar 

  • Posch C, Matolin D, Wohlgenannt R (2010) A QVGA 143 dB DR asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression. IEEE ISSCC Dig Tech. Papers, pp. 400–401

    Google Scholar 

  • Posch C, Matolin D, Wohlgenannt R, Maier T, Litzenberger M (2009) A microbolometer asynchronous dynamic vision sensor for LWIR. IEEE Sensors Journal 9: 654–664

    Article  Google Scholar 

  • Rodieck R (1998) The first steps in seeing: Sinauer Associates, Sunderland, MA

    Google Scholar 

  • Ruedi PF, Heim P, Gyger S, Kaess F, Arm C, et al. (2009) An SoC combining a 132 dB QVGA pixel array and a 32 b DSP/MCU processor for vision applications. IEEE ISSCC Dig Tech Papers, pp. 46–47

    Google Scholar 

  • Ruedi PF, Heim P, Kaess F, Grenet E, Heitger F, et al. (2003) A 128 × 128, pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction. IEEE J. Solid-State Circuits 38: 2325–2333

    Article  Google Scholar 

  • Sarpeshkar R (1998) Analog versus digital: Extrapolating from electronics to neurobiology. Neural Computation 10: 1601–38

    Article  PubMed  CAS  Google Scholar 

  • Sarpeshkar R, Baker MS C, Sit JJ, Turicchia L, Zhak S (2005) An analog bionic ear processor with zero-crossing detection. IEEE ISSCC Dig Tech Papers, pp. 78–79

    Google Scholar 

  • Sarpeshkar R, Lyon RF (1998) A low-power wide-dynamic-range analog VLSI cochlea. Analog Integrated Circuits and Signal Processing 16: 245–274

    Article  Google Scholar 

  • Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, et al. (2009) CAVIAR: A 45 k neuron, 5 M synapse, 12 G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Trans Neur al Networks 20: 1417–1438

    Article  Google Scholar 

  • Uysal I, Sathyendra H, Harris JG (2006) A biologically plausible system approach for noise robust vowel recognition. IEEE Proc Midwest Symp Circuits and Systems, pp. 245–249

    Google Scholar 

  • van Schaik A, Shamma S (2004) A neuromorphic sound localizer. Analog Integrated Circuits and Signal Processing 39: 267–273

    Article  Google Scholar 

  • Watts L, Kerns DA, Lyon RF, Mead CA (1992) Improved implementation of the silicon cochlea. IEEE J. of Solid State Circuits 27: 692–700

    Article  Google Scholar 

  • Wen B, Boahen K (2006) A 360-channel speech preprocessor that emulates the cochlear amplifier. IEEE ISSCC Dig Tech Papers, pp. 556–557

    Google Scholar 

  • Zaghloul KA, Boahen K (2004) Optic nerve signals in a neuromorphic chip II: Testing and results. IEEE Trans Biomed Engineering 51: 667–675

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Neuroinformatics, University of Zürich and ETH Zürich, Winterthurerstr. 190, 8057, Zürich, Switzerland

    Tobi Delbruck

Authors
  1. Tobi Delbruck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shih-Chii Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag/Wien

About this chapter

Cite this chapter

Delbruck, T., Liu, SC. (2012). Event-based silicon retinas and cochleas. In: Frontiers in Sensing. Springer, Vienna. https://doi.org/10.1007/978-3-211-99749-9_6

Download citation

Publish with us

Policies and ethics

Search

Navigation