Encyclopedia of Computational Neuroscience

Living Edition
| Editors: Dieter Jaeger, Ranu Jung

Neuromorphic Sensors, Cochlea

  • Shih-Chii LiuEmail author
  • André van Schaik
Living reference work entry
DOI: https://doi.org/10.1007/978-1-4614-7320-6_118-1

Keywords

Outer Hair Cell Basilar Membrane Automatic Gain Control Tectorial Membrane Spiral Ganglion Cell 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Synonyms

Definition

The biological cochlea is a bony, fluid-filled structure in the inner ear. It performs the transduction between the pressure signal representing the acoustic input and the neural signals that carry information to the brain. The cochlea is spiraled from the base to the apex, containing approximately 2.5 turns. The internal part of the cochlea is divided into three chambers (scalae): scala vestibuli, scala media, and scala tympani; Reissner’s membrane separates the first chamber from the second chamber, and the basilar membrane (BM) separates the second and third chambers. A specialized structure, the organ of Corti, sits atop the basilar membrane. It contains both the inner and outer hair cells (IHCs and OHCs, respectively). The tips of these cells have hairlike structures, called stereocilia. Deflections in the stereocilia of the IHCs generate neural signals that travel to the brain. Neural signals from the brain can alter the length and width of the OHC cell bodies, affecting the mechanical properties of the BM, and thus its response to sound vibrations.

Airborne sound enters the ear via the ear canal and vibrates the eardrum, which in turn causes vibration of the three middle ear bones, one of which causes the cochlear fluid to vibrate. These small bones perform impedance transformation through a levering action, so that airborne sound can be transmitted into the fluid-filled cochlea.

At the base of the cochlea, the physical characteristics of the BM (it is narrower and stiffer) are such that it responds with greater movement to high frequency stimuli, whereas at the apex, the BM is wider and more flexible, so that it responds better to low frequency stimuli. Each position along the BM can thus be assigned a characteristic frequency that produces the greatest deflection at that place.

When the BM moves, the organ of Corti is displaced. The stereocilia of the IHCs are then displaced by the viscous drag of the cochlear fluid, making the displacement of the IHC’s stereocilia proportional to the velocity of the basilar membrane motion. This movement of the stereocilia causes a change in membrane potential within the IHC, which in turn causes the release of neurotransmitter from the IHC. This release causes spiking activity in neurons, the spiral ganglion cells, whose axons make up the auditory nerve.

The stereocilia of the OHCs are attached to another structure, the tectorial membrane, and they are thus displaced due to the shear between the rigid upper surface of the organ of Corti and the tectorial membrane. The OHCs change their length as a function of their membrane potential and provide active undamping of the mechanical structure. This allows the selectivity of the cochlear filters to adapt as a function of the intensity of the input sound.

Detailed Description

Biological cochleas use a space-to-rate encoding in which the input sound is encoded as trains of pulses created from the outputs of a set of broadly frequency-selective channels (Shamma 1985). The pulses are phase locked for low frequencies, and this phase locking disappears for frequencies above around 3 kHz (Palmer and Russell 1986). Encoding the information this way allows sparser sampling of frequency information according to the active frequency channels rather than the maximal sampling rate required to capture all information from a single audio source.

Silicon cochleas have been developed for over 20 years starting with the work of Lyon and Mead (1988). These designs model the biophysics of the BM as a large number of coupled filter stages. The architecture of the silicon cochlea varies from the cascaded architectures (Watts et al. 1992; Sarpeshkar et al. 1998; van Schaik et al. 1996) first introduced by Lyon and Mead (1988), modeling the phenomenological output of the cochlea, to a resistively coupled bank of band-pass filters (Watts et al. 1992; Wen and Boahen 2006; Fragniere 2005; Hamilton et al. 2008) modeling the role of the BM and the cochlear fluid more explicitly (Fig. 1). The cascaded architecture is preferred over the coupled band-pass architecture for better matching, ease of implementation, and a sharp high frequency roll-off of the filters.

The IHCs have been modeled as half-wave rectifiers (HWRs) along with possible models of automatic gain control of outer hair cells (Hamilton et al. 2008; Wen and Boahen 2009; Katsiamis et al. 2009). Recent cochleas also include circuits that model the role of spiral ganglion cells by asynchronous output quantization in the time domain with transmitted data encoded using the address-event representation (AER) (Wen and Boahen 2009; Lazzaro et al. 1993; Kumar et al. 1998; Sarpeshkar et al. 2005; Georgiou and Toumazou 2005; Abdalla and Horiuchi 2005; Chan et al. 2007; Liu and Delbruck 2010; Liu et al. 2010a).
Fig. 1

Architectures of cochleas. (a) Cascaded and (b) resistive-coupled stages

Filter Channel Circuits

Each stage of the filter bank usually consists of a second order section (SOS) filter in the case of the cascaded architecture. The characteristic frequencies of the filters along the cascade vary logarithmically with position similar to the approximate logarithmic dependence of the preferred frequency selectivity with position along the basilar membrane of the biological cochlea.

Each SOS consists of two forward filters, made from a transconductance amplifier and a capacitor, and one feedback transconductance amplifier. The difference of the outputs of the forward filters within a single SOS is the input to the subsequent IHC circuit. This difference readout adds a desirable zero to the transfer function without introducing undesirable gain proportional to frequency, as would occur with a temporal high-pass filter (van Schaik et al. 1996). It sharpens the filter response to approximate closer a band-pass response and reduces the phase accumulation across the cascade.

A half-wave rectification circuit implementing the IHC is a simplified model of the response of the IHCs of the biological cochlea. The circuit output drives spiral ganglion cell (SGC) circuits, each implemented as an integrate-and-fire neuron model. The number of events created in a cycle is set by the amplitude and frequency of the input and the amount of charge needed to generate an event.

Event Data from Cochlea

Figure 2 shows the SGC outputs (AER events) and the analog input sound waveform from the latest binaural cochlear system with 64 channels, called the AEREAR2. The outputs are in response to a spoken sentence at a distance of 1 m from the system at normal speech volume (65 dB LAF SPL as measured with a Bruel & Kjaer 2250).
Fig. 2

Response to spoken sentence “The shadow vanished.” (a) Events or SGC outputs from the 64 channels of one ear of the binaural AEREAR2. (b) Sampled audio from right microphone preamplifier output. Low channels correspond to high frequencies and high channels to low frequencies. Best characteristic frequencies range from 100 Hz to 20 kHz

The AEREAR2 system implements a model of the BM, the IHC, and multiple SGCs with different thresholds driven by each IHC. It is the first fully integrated system that combines features of previous silicon cochlea designs that are robust to mismatch, along with novel features for easier programmability of the architecture and operating parameters. The chip includes integrated microphone preamplifiers (Baker and Sarpeshkar 2003), local gain adjustment, and on-chip digitally controlled biases (Delbruck and Lichtsteiner 2006). The individual basilar membrane and SGC circuit signals can also be digitized and read into the computer through the USB port. It has open-sourced host software APIs and algorithms which enable rapid development of application scenarios (Delbruck 2007). A bus-powered USB board enables easy interfacing of the AEREAR2 to standard PCs for control and processing.

The time-stamped AER events with a 1 ms resolution are sent to a PC where they are processed for applications. For natural sounds, off-chip microphone preamplifiers (MAX9814) with 20 dB range of automatic gain control or on-chip microphone preamplifiers with an 18 dB range of digitally controllable gain can be used. Input can also be applied from a PC sound card directly to the filter cascade (i.e., bypassing the preamplifiers).

In Fig. 2a, the dots represent events recorded from the AEREAR2 in response to a spoken sentence. Some channels are more excitable than others and have background activity leading to a sustained background event rate of about 4 keps. The average event rate is 80 keps and the peak event rate is around 325 keps. The sampled microphone output recorded using the onboard ADC is displayed in Fig. 2b.

Applications

Various cochlea implementations including the binaural AEREAR2 have been used for various auditory tasks including spatial audition (Chan et al. 2007; Abdalla and Horiuchi 2008; Chan et al. 2010; Finger and Liu 2011). Processing is done on the asynchronous spike events rather than on spectrograms generated from sampled raw audio inputs. Post-processing can be cheaper because only a sparse stream of events needs to be processed. This event-based system is being tested for other auditory tasks such as speaker identification (Chakrabartty and Liu 2010; Liu et al. 2010b; Abdollahi and Liu 2011; Li et al. 2012) and multi-sensor fusion.

References

  1. Abdalla H, Horiuchi TK (2005) An ultrasonic filterbank with spiking neurons. IEEE Int Symp Circuits Syst 5:4201–4204Google Scholar
  2. Abdalla H, Horiuchi TK (2008) Binaural spectral cues for ultrasonic localization. IEEE Int Symp Circuits Syst 2110–2113Google Scholar
  3. Abdollahi M, Liu S-C (2011) Speaker-independent isolated digit recognition using an AER silicon cochlea. Biomed Circuits Syst Conf 269–272Google Scholar
  4. Baker MW, Sarpeshkar R (2003) A low-power high-PSRR current-mode microphone preamplifier. IEEE J Solid-State Circuits 38:1671–1678CrossRefGoogle Scholar
  5. Chakrabartty S, Liu S-C (2010) Exploiting spike-based dynamics in a silicon cochlea for speaker identification. IEEE Int Symp Circuits Syst 513–516Google Scholar
  6. Chan V, Liu S-C, van Schaik A (2007) AER EAR: a matched silicon cochlea pair with address event representation interface. IEEE Trans Circuits Syst I Regul Pap 54:48–59CrossRefGoogle Scholar
  7. Chan V, Jin C, van Schaik A (2010) Adaptive sound localization with a silicon cochlea pair. Front Neurosci 4(196):1–11, doi:10.3389/fnins.2010.00196Google Scholar
  8. Delbruck T (2007) jAER open source project. Available http://jaerproject.net
  9. Delbruck T, Lichtsteiner P (2006) Fully programmable bias current generator with 24 bit resolution per bias. IEEE Int Symp Circuits Syst:2849–2852Google Scholar
  10. Finger H, Liu S-C (2011) Estimating the location of a sound source with a spike-timing localization algorithm. IEEE Int Symp Circuits Syst 2461–2464Google Scholar
  11. Fragniere E (2005) A 100-channel analog CMOS auditory filter bank for speech recognition. ISSCC Dig Tech Pap 140–589Google Scholar
  12. Georgiou J, Toumazou C (2005) A 126-uW cochlear chip for a totally implantable system. IEEE J Solid-State Circuits 40:430–443CrossRefGoogle Scholar
  13. Hamilton T, Jin C, van Schaik A, Tapson J (2008) An active 2-D silicon cochlea. IEEE Trans Biomed Circuits Syst 2:30–43PubMedCrossRefGoogle Scholar
  14. Katsiamis A, Drakakis E, Lyon R (2009) A 4.5uW, 120 + dB, log-domain cochlea channel with AGC. IEEE J Solid-State Circuits 44:1006–1022CrossRefGoogle Scholar
  15. Kumar N, Himmelbauer W, Cauwenberghs G, Andreou A (1998) An analog VLSI chip with asynchronous interface for auditory feature extraction. IEEE Trans Circuits Syst II 45:600–606CrossRefGoogle Scholar
  16. Lazzaro J, Wawrzynek J, Mahowald M, Sivilotti S, Gillespie D (1993) Silicon auditory processors as computer peripherals. IEEE Trans Neural Netw 4:523–528PubMedCrossRefGoogle Scholar
  17. Li C-H, Delbruck T, Liu S-C (2012) Real-time speaker identification using the AEREAR2 event-based silicon cochlea. IEEE Int Symp Circuits Syst 1159–1162Google Scholar
  18. Liu S-C, Delbruck T (2010) Neuromorphic sensory systems. Curr Opin Neurobiol 20:1–8CrossRefGoogle Scholar
  19. Liu S-C, van Schaik A, Minch B, Delbrück T (2010) Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms. IEEE Int Symp Circuits Syst 2027–2030Google Scholar
  20. Liu S-C, Mesgarani N, Harris J, Hermansky H (2010) The use of spike-based representations for hardware audition systems. IEEE Int Symp Circuits Syst 505–508Google Scholar
  21. Lyon RF, Mead C (1988) An analog electronic cochlea. IEEE Trans Acoust Speech Signal Process 36:1119–1134CrossRefGoogle Scholar
  22. Palmer AR, Russell IJ (1986) Phase-locking in the cochlear nerve of the guinea-pig and its relation to the receptor potential of inner hair-cells. Hear Res 24:1–15PubMedCrossRefGoogle Scholar
  23. Sarpeshkar R, Lyon RF, Mead C (1998) A low-power wide-dynamic-range analog VLSI cochlea. Analog Integr Circuits Signal Process 16:245–274CrossRefGoogle Scholar
  24. Sarpeshkar R, Salthouse C, Sit JJ, Baker M, Zhak S, Lu T, Turicchia L, Balster S (2005) An ultra-low-power programmable analog bionic ear processor. IEEE Trans Biomed Eng 2:711–727CrossRefGoogle Scholar
  25. Shamma S (1985) Speech processing in the auditory system I: the representation of speech sounds in the responses of the auditory nerve. J Acoust Soc Am 78:1612–1621PubMedCrossRefGoogle Scholar
  26. van Schaik A, Fragnière E, Vittoz E (1996) Improved silicon cochlea using compatible lateral bipolar transistors. Adv Neural Inf Process Syst 8:671–677Google Scholar
  27. Watts L, Kerns D, Lyon R, Mead C (1992) Improved implementation of the silicon cochlea. IEEE J Solid State Circuits 27:692–700CrossRefGoogle Scholar
  28. Wen B, Boahen K (2006) A 360-channel speech preprocessor that emulates the cochlear amplifier. ISSCC Dig Tech Pap 556–557Google Scholar
  29. Wen B, Boahen K (2009) A silicon cochlea with active coupling. IEEE Trans Biomed Circuits Syst 3:444–455CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.University of Zurich and ETH ZurichZurichSwitzerland
  2. 2.University of Western SydneyPenrithAustralia