Abstract
Neurobiological systems have evolved over a billion years and serve as a good template for some text engineers to mimic when designing intelligent sensors and systems. For instance, neurobiological systems exploit noise and system non-linearity as a computational aid to push the limits of performance and energy efficiency. In contrast, in man-made technologies, these artifacts are generally considered to be a nuisance. This chapter’s focus is on the neuromorphic concept of “sensing-to-learn” and “learning-to-sense,” which are grounded in key neuromorphic adaptation principles based on noise exploitation and non-linear sensory processing techniques. “Noise shaping” and “jump-resonance” are two techniques that can extract salient sensing cues by exploring the synergy between noise and system non-linearity. We illustrate these concepts in the context of auditory and olfaction pathways, and we argue how these principles can be used to design the next generation of neuromorphic sensory interfaces.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Abbreviations
- DVS:
-
Dynamic vision sensor
- ATIS:
-
Asynchronous time imaging sensor
- BM:
-
Basilar membrane
- IHC:
-
Inner hair cells
- CF:
-
Characteristic frequency
- CAR:
-
Cascaded asymmetric resonator
- CAR-FAC:
-
Cascade of asymmetric resonators with fast-acting compression
- STRF:
-
Spectro-temporal receptive field
- OHC:
-
Outer hair cells
- DIHC:
-
Digital inner hair cell
- DOH:
-
Digital outer hair cell
- AGC:
-
Automatic gain control
- EER:
-
Equal error rate
- PD:
-
Probability of detection
- SVM:
-
Support vector machine
- SNR:
-
Signal-to-noise ratio
- AC:
-
Association connection
References
Arisoy, F.D., et al.: Bioinspired photocatalytic shark-skin surfaces with antibacterial and antifouling activity via nanoimprint lithography. ACS Appl. Mater. Interfaces (2018). https://doi.org/10.1021/acsami.8b05066
Barth, F.G.: Spider strain detection. In: Barth, F.G., Humphrey, J.A.C., Srinivasan, M.V. (eds.) Frontiers in Sensing: From Biology to Engineering, pp. 251–273. Springer, New York/Wien (2012). ISBN: 9783211997482
Miles, R.N., Hoy, R.R.: The development of a biologically-inspired directional microphone for hearing aids. Audiol. Neuro-Otol. 11(2):86–94 (2006)
Sarpeshkar, R.: Neuromorphic and biomorphic engineering systems. McGraw-Hill Yearbook of Science & Technology 2009, pp. 250–252. McGraw-Hill, New York (2009)
Barth, F.G.: Spider mechanoreceptors. Curr. Opin. Neurobiol. 14(4):415–422 (2004). https://doi.org/10.1016/j.conb.2004.07.005
Lichtsteiner, P., Delbruck, T.: 64 × 64 event-driven logarithmic temporal derivative silicon retina. In: IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 157–160 (2005)
Lichtsteiner, P., Posch, C., Delbruck, T.: A 128 × 128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change. In: IEEE International Solid-State Circuits Conference (ISSCC), pp. 2060–2069 (2006)
Lichtsteiner, P.: An AER temporal contrast vision sensor. Ph. D. Thesis, ETH Zurich, Department of Physics (D-PHYS), Zurich (2006))
Mar, D.J., Chow, C.C., Gerstner, W., Adams, R.W., Collins, J.J.: Noise shaping in populations of coupled model neurons. Proc. Natl. Acad. Sci. U. S. A. 96, 10450–10455 (1999)
Linden, J.F., Liu, R.C., Sahani, M., Schreiner, C.E., Merzenich, M.M.: Spectrotemporal structure of receptive fields in areas AI and AAF of mouse auditory cortex. J. Neurophysiol. 90(4):2660–2675 (2003)
Aono, K., Shaga, R., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for extracting speaker discriminative formant trajectories. IEEE Trans. Biomed. Circuits Syst. 7(4):389–400 (2013)
Altoé, A., Pulkki, V.: Transmission line cochlear models: improved accuracy and efficiency. J. Acoust. Soc. Am. 136:302–308 (2014). https://doi.org/10.1121/1.4896416
Glasberg, B.R., Moore, B.C., Nimmo-smith, I.: Comparison of auditory filter shapes derived with three different maskers maskers. J. Acoust. Soc. Am. 75:536 (1984). https://doi.org/10.1121/1.390487
Patterson, R.D., Unoki, M., Irino, T.: Extending the domain of center frequencies for the compressive gammachirp auditory filter. J. Acoust. Soc. Am. 114, 1529–1542 (2003). https://doi.org/10.1121/1.1600720
Lyon, R.F., Katsiamis, A.G., Drakakis, E.M.: History and future of auditory filter models. In: IEEE International Symposium on Circuits and Systems (ISCAS) (Paris), pp. 3809–3812 (2010). https://doi.org/10.1109/ISCAS.2010.5537724
Wang, S., Koickal, T.J., Hamilton, A., Cheung, R., Smith, L.S.: A bio-realistic analog CMOS cochlea filter with high tunability and ultra steep roll-off. IEEE Trans. Biomed. Circ. Syst. 9, 297–311 (2015). https://doi.org/10.1109/TBCAS.2014.2328321
Lyon, R.F.: Human and Machine Hearing -Extracting Meaning from Sound. Cambridge University Press, Mountain View (2017)
Xu, Y., Thakur, C.S., Singh, R.K., Hamilton, T.J., Wang, R.M., van Schaik, A.: A FPGA implementation of the CAR-FAC cochlear model. Front. Neurosci. 12:198 (2018)
Lyon Richard, F., Moore, Brian CJ.: Human and Machine Hearing: Extracting Meaning from SoundHuman and Machine Learning: Extracting Meaning from Sound, 567pp. Cambridge University Press, Cambridge (2017). Price:£ 54.99. ISBN: 978-1-107-007536. Acoust. Soc. Am. J. 144:567 (2018)
Xu Y., Thakur, C.S., Hamilton, T.J., Wang, R., van Schaik, A.: An FPGA implementation of the CAR-FAC cochlear model. Front. Neurosci. 12:198 (2018)
Thakur, C.S., et al.: FPGA implementation of the CAR model of cochlea. In: IEEE International Symposium on Circuits and Systems (ISCAS) (2014)
Lyon, R.F.: Human and Machine Hearing -Extracting Meaning from Sound. Cambridge University Press, Mountain View (2017)
Thakur, C.S., Hamilton, T.J., Tapson, J., van Schaik, A., Lyon, R.F.: FPGA implementation of the CAR model of the cochlea. In: 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1853–1856 (2014)
Thakur, C.S., Hamilton, T.J., Tapson, J., van Schaik, A., Lyon, R.F.: Live demonstration: FPGA implementation of the CAR model of the cochlea. In: 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1853–1856 (2014)
Xu, Y., Thakur, C.S., et al.: Electronic cochlea: car-FAC model on FPGA. In: IEEE Biomedical Circuits and Systems Conference (BioCAS 2016) (2016)
Fukuma, A., Matsubara, M.: Jump resonance criteria of nonlinear control systems. IEEE Trans. Autom. Control 11(4):699–706 (1966)
Martin, P., Mehta, A.D., Hudspeth, A.J.: Negative hair-bundle stiffness betrays a mechanism for mechanical amplification by the hair cell. Proc. Natl. Acad. Sci. 97(22):12026–12031 (2000)
Aono, K., Shaga, R.K., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon auditory front-ends for extracting speaker discriminative formant trajectories. Biomed. Circuits Syst. IEEE Trans. PP(99):1 (2013)
Aono, K., Shaga, R.K., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for formant trajectory encoding. In: 2012 IEEE 55th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 85–88, 5–8 Aug 2012
Moni, R.S., Rao, K.R.: Jump-phenomenon in active-RC filters. IEEE Trans. Circuits Syst. 29(1):54–55 (1982)
Aono, K., Shaga, R., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for extracting speaker discriminative formant trajectories. IEEE Trans. Biomed. Circuits Syst. 7(4):389–400 (2013)
Chakrabartty, S., Cauwenberghs, G.: Gini-support vector machine: quadratic entropy based multi-class probability regression. J. Mach. Learn. Res. 8:813–839 (2007)
Fazel, A., Chakrabartty, S.: An overview of statistical pattern recognition techniques for speaker verification. IEEE Circuits Syst. Mag. 11(2):62–81 (2011)
Aono, K., Shaga, R., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for extracting speaker discriminative formant trajectories. IEEE Trans. Biomed. Circuits Syst. 7(4):389–400 (2013)
Chi, T., Ru, P., Shamma, S.A.: Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118(2):887–906 (2005). https://doi.org/10.1121/1.1945807
Patil, K., et al.: Music in our ears: the biological bases of musical timbre perception. PLoS Comput. Biol. 8(11):e1002759 (2012)
Amari, S., Cichocki, A., Yang, H.H.: A new learning algorithm for blind signal separation. In: Advanced Neural Information Processing Systems (NIPS), vol. 8, pp. 757–763. MIT Press, Cambridge, MA (1996)
Chakrabartty, S., Deng, Y., Cauwenberghs, G.: Robust speech feature extraction by growth transformation in reproducing Kernel Hilbert space. IEEE Trans. Speech Lang. Acoust. 15(6):1842–1849 (2007)
Patterson, R., Moore, B.: Auditory filters and excitation patterns as representations of frequency resolution. Freq. Sel. Hearing 363:123–177 (1986)
Smith, E.C., Lewicki, M.S.: Efficient auditory coding. Nature 439:978–982 (2006)
Chi, T., Ru, P., Shamma, S.: Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118:887–906 (2005)
Chi, T., Ru, P., Shamma, S.: Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118:887–906 (2005)
Wessinger, C.M., VanMeter, J., Tian, B., Lare, J.V., Pekar, J., Rauschecker, J.P.: Hierarchical organization of the human auditory cortex revealed by functional magnetic resonance imaging. J. Cogn. Neurosci. 13:1–7 (2001)
Okada, K., Rong, F., Venezia, J., Matchin, W., Hsieh, I.-H., Saberi, K., Serences, J.T., Hickok, G.: Hierarchical organization of human auditory cortex: evidence from acoustic invariance in the response to intelligible speech. Cereb. Cortex 20:2486–2495 (2010)
Boemio, A., Fromm, S., Braun, A., Poeppel, D.: Hierarchical and asymmetric temporal sensitivity in human auditory cortices. Nat. Neurosci. 8:389–395 (2005)
Fazel, A., Chakrabartty, S.: Sparse auditory reproducing kernel (SPARK) features for noise-robust speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(4) (2012). https://doi.org/10.1109/TASL.2011.2179294
Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nature Neurosci. 2:1019–1025 (1999)
Wahba, G.: Splines Models for Observational Data. Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7:219–269 (1995)
Fazel, A., Chakrabartty, S.: Sparse auditory reproducing kernel (SPARK) features for noise-robust speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(4) (2012). https://doi.org/10.1109/TASL.2011.2179294
Hirsch, H.G., Pearce, D.: The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In: Proceedings of ASR, pp. 181–188 (2000)
Mar, D.J., Chow, C.C., Gerstner, W., Adams, R.W., Collins, J.J.: Noise shaping in populations of coupled model neurons. Proc. Natl. Acad. Sci. U. S. A. 96:10450–10455 (1999)
Gore, A., Chakrbartty, S.: Large Margin Analog-to-digital converters with applications in Neural Prosthetics. In: Advances in Neural Information Processing Systems (NIPS) (2006)
Pearce, T.C.: Computational parallels between the biological olfactory pathway and its analogue ‘the electronic nose’: Part I. Biological olfaction. Biosystems 41:43–67 (1997)
Pearce, T.C.: Computational parallels between the biological olfactory pathway and its analogue ‘the electronic nose’: Part II. Sensor-based machine olfaction. Biosystems 41:69–90 (1997)
Raman, B., Stopfer, M., Semancik, S.: Mimicking biological design and computing principles in artificial olfaction. ACS Chem. Neurosci. 2:487–499 (2011)
Persaud, K., Dodd, G.: Analysis of discrimination mechanisms in the mammalian olfactory system using a model nose. Nature 299:352–355 (1982)
Nagle, T., Schiffman, S.S., Gutierrez-Osuna, R.: The how and why of electronic noses. IEEE Spectr. 35(9):22–34 (1998)
Pearce, T.C., et al.: Handbook of Machine Olfaction: Electronic Nose Technology. Wiley, Wiley-VCH Verlag GmbH & Co. KGaA (2006)
Raman, B.: Sensor-based machine olfaction with neuromorphic models of the olfactory system. Ph. D. Dissertation, Texas A&M University, College Station (2005)
Ache, B.W., Young, J.M.: Olfaction: diverse species, conserved principles. Neuron 48:417–430 (2005)
Hallem, E.A., Carlson, J.R.: Coding of odors by a receptor repertoire. Cell 125:143–160 (2006)
Uchida, N., Poo, C., Haddad, R.: Coding and transformationsin the olfactory system. Ann. Rev. Neurosci. 37:363–385 (2014)
Farivar, S.: Cytoarchitecture of the locust olfactory system. In: Biology. Califonia Institute of Technology (2005). https://doi.org/10.7907/4Y60-KH68
Belluscio, L., et al.: A map of pheromone receptor activation in the mammalian brain. Cell 97(2):209–220 (1999)
Laurent, G.: A systems perspective on olfactory coding. Science 286:723–728 (1999)
Pearce, T.C., et al.: Robust stimulus encoding in olfactory processing: hyperacuity and efficient signal transmission. In: Emergent Neural Computational Architectures Based on Neuroscience, Springer, Berlin, Heidelberg, pp. 461–479 (2001)
Carandini, M., Heeger, D.J.: Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13:51–62
Olsen, S.R., Bhandawat, V., Wilson, R.I.: Divisive normalization in olfactory population codes. Neuron 66:287–299 (2010)
Yaksi, E., Wilson, R.: Electrical coupling between olfactory glomeruli. Neuron 67:1034–1047 (2010)
Zhu, P., Frank, T., Friedrich, R.W.: Equalization of odor representations by a network of electrically coupled inhibitory interneurons. Nat. Neurosci. 16(11):1678–1686 (2013)
Friedrich, R.W., Laurent, G.: Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity. Science 291:889–894 (2001)
Wilson, C.D., et al.: A primacy code for odor identity. Nat. Commun. 8:1477 (2017)
Nizampatnam, S., et al.: Dynamic contrast enhancement and flexible odor codes. Nat. Commun. 9:3062 (2018)
Saha, D., et al.: A spatiotemporal coding mechanism for background-invariant odor recognition. Nat. Neurosci. 16:1830–1839 (2013)
Rokni, D., et al.: An olfactory cocktail party: figure-ground segregation of odorants in rodents. Nat. Neurosci. 17:1225–1232 (2014)
Khan, A.G., Thattai, M., Bhalla, U.S.: Odor representations in the rat olfactory bulb change smoothly with morphing stimuli. Neuron 57(4):571–585 (2008)
Niessing, J., Friedrich, R.W.: Olfactory pattern classification by discrete neuronal network states. Nature 465:47–52 (2010)
Liang, D.G.: Perception of odor mixtures. In: Doty, R.L., (ed.) The Handbook of Olfaction and Gustation, pp. 283–298. Marcel Dekker, New York (1995)
Jortner, R.A., Farivar, S., Laurent, G.: A simple connectivity scheme for sparse coding in an olfactory system. J. Neurosci. 14(7):1659–1669 (2007)
Gerstner, W.: Population dynamics of spiking neurons: fast transients, asynchronous states, and locking. Neural Comput. 12(1):43–89 (2000)
Chacron, M.J., Lindner, B., Longtin, A.: Noise shaping by interval correlations increases information transfer. Phys. Rev. Lett. 92:080601 (2004)
Wiesenfeld, K., Satija, I.: Noise tolerance of frequency-locked dynamics. Phys. Rev. B 36:2483–2492 (1987)
Gore, A., Chakrabartty, S.: Min-max optimization framework for designing sigma-delta learners: theory and hardware. IEEE Trans. Circuits Syst. I 57(3):604–617 (2010). https://doi.org/10.1109/TCSI.2009.2025002
Chakrabartty, S.: Multiple-input multiple-output analog-to-digital converter US Patent no: 7,479,911, Issued 20 Jan 2009
Perera, A., et al.: A dimensionality-reduction technique inspired by receptor convergence in the olfactory system. Sens. Actuat B: Chem. 116:17–22 (2006)
Raman, B., et al.: Processing of chemical sensor array with a biologically inspired model of olfactory coding. IEEE Trans. Neural Netw. 17:1015–1024 (2006)
Raman, B., Gutierrez-Osuna, R.: Concentration normalization with a model of gain control in the olfactory bulb. Sens. Actuat. B: Chem. 116:36–42 (2006)
Raman, B., Yamanaka, T., Gutierrez-Osuna, R.: Contrast enhancement of gas sensor array patterns with a neurodynamics model of the olfactory bulb. Sens. Actuat. B: Chem. 119:547–555 (2006)
Stopfer, M., et al.: Odor identity vs. intensity coding in an olfactory system. Neuron 39(6):991–1004 (2003)
Raman, B., Gutierrez-Osuna, R.: Chemosensory processing in a spiking model of the olfactory bulb: chemotopic convergence and center surround inhibition. In: Proceedings of Advances in Neural Information Processing Systems (NIPS 2004), Vancouver, vol. 17 (2004)
Raman, B., et al.: Sensor-based machine olfaction with a neurodynamics model of the olfactory bulb. In: IEEE Intelligence Robots and Systems. IEEE, Sendai (2004)
Raman, B., Gutierrez-Osuna, R.: Mixture segmentation and background suppression in chemosensor arrays with a model of olfactory bulb-cortex interaction. In: International Joint Conference on Neural Networks. IEEE, Montreal (2005)
Raman, B., et al.: A bioinspired methodology for artificial olfaction. Anal. Chem. 80:8364–8371 (2008)
Saha, D., et al.: Engaging and disengaging recurrent inhibition coincides with sensing and unsensing of a sensory stimulus. Nat. Commun. 8:15413 (2017)
Persaud, K., Dodd, G.: Analysis of discrimination mechanisms in the mammalian olfactory system using a model nose. Nature 299:352–355 (1982)
Ache, B.W., Young, J.M.: Olfaction: diverse species, conserved principles. Neuron 48:417–430 (2005)
Saha, D., et al.: Explosive sensing with insect-based biorobots. Biosens. Bioelectron. X 6:100050 (2020)
Acknowledgements
The authors would like to thank the Scheme for Promotion of Academic and Research Collaboration (SPARC), MHRD, Govt. of India, for funding (SPARC/2018-2019/P606/SL) this work. The authors acknowledge the Brain, Computation and Learning workshop at the Indian Institute of Science, India, where much of this work has been done.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Singapore Pte Ltd.
About this entry
Cite this entry
Chakrabartty, S., Raman, B., Thakur, C.S. (2023). Sensing-to-Learn and Learning-to-Sense: Principles for Designing Neuromorphic Sensors. In: Thakor, N.V. (eds) Handbook of Neuroengineering. Springer, Singapore. https://doi.org/10.1007/978-981-16-5540-1_122
Download citation
DOI: https://doi.org/10.1007/978-981-16-5540-1_122
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5539-5
Online ISBN: 978-981-16-5540-1
eBook Packages: EngineeringReference Module Computer Science and Engineering