Skip to main content

Sensing-to-Learn and Learning-to-Sense: Principles for Designing Neuromorphic Sensors

  • Living reference work entry
  • First Online:
Handbook of Neuroengineering

Abstract

Neurobiological systems have evolved over a billion years and serve as a good template for some text engineers to mimic when designing intelligent sensors and systems. For instance, neurobiological systems exploit noise and system non-linearity as a computational aid to push the limits of performance and energy efficiency. In contrast, in man-made technologies, these artifacts are generally considered to be a nuisance. This chapter’s focus is on the neuromorphic concept of “sensing-to-learn” and “learning-to-sense,” which are grounded in key neuromorphic adaptation principles based on noise exploitation and non-linear sensory processing techniques. “Noise shaping” and “jump-resonance” are two techniques that can extract salient sensing cues by exploring the synergy between noise and system non-linearity. We illustrate these concepts in the context of auditory and olfaction pathways, and we argue how these principles can be used to design the next generation of neuromorphic sensory interfaces.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

References

  1. Arisoy, F.D., et al.: Bioinspired photocatalytic shark-skin surfaces with antibacterial and antifouling activity via nanoimprint lithography. ACS Appl. Mater. Interfaces (2018). https://doi.org/10.1021/acsami.8b05066

    Google Scholar 

  2. Barth, F.G.: Spider strain detection. In: Barth, F.G., Humphrey, J.A.C., Srinivasan, M.V. (eds.) Frontiers in Sensing: From Biology to Engineering, pp. 251–273. Springer, New York/Wien (2012). ISBN: 9783211997482

    Google Scholar 

  3. Miles, R.N., Hoy, R.R.: The development of a biologically-inspired directional microphone for hearing aids. Audiol. Neuro-Otol. 11(2):86–94 (2006)

    Article  Google Scholar 

  4. Sarpeshkar, R.: Neuromorphic and biomorphic engineering systems. McGraw-Hill Yearbook of Science & Technology 2009, pp. 250–252. McGraw-Hill, New York (2009)

    Google Scholar 

  5. Barth, F.G.: Spider mechanoreceptors. Curr. Opin. Neurobiol. 14(4):415–422 (2004). https://doi.org/10.1016/j.conb.2004.07.005

    Article  Google Scholar 

  6. Lichtsteiner, P., Delbruck, T.: 64 × 64 event-driven logarithmic temporal derivative silicon retina. In: IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 157–160 (2005)

    Google Scholar 

  7. Lichtsteiner, P., Posch, C., Delbruck, T.: A 128 × 128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change. In: IEEE International Solid-State Circuits Conference (ISSCC), pp. 2060–2069 (2006)

    Google Scholar 

  8. Lichtsteiner, P.: An AER temporal contrast vision sensor. Ph. D. Thesis, ETH Zurich, Department of Physics (D-PHYS), Zurich (2006))

    Google Scholar 

  9. Mar, D.J., Chow, C.C., Gerstner, W., Adams, R.W., Collins, J.J.: Noise shaping in populations of coupled model neurons. Proc. Natl. Acad. Sci. U. S. A. 96, 10450–10455 (1999)

    Article  Google Scholar 

  10. Linden, J.F., Liu, R.C., Sahani, M., Schreiner, C.E., Merzenich, M.M.: Spectrotemporal structure of receptive fields in areas AI and AAF of mouse auditory cortex. J. Neurophysiol. 90(4):2660–2675 (2003)

    Article  Google Scholar 

  11. Aono, K., Shaga, R., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for extracting speaker discriminative formant trajectories. IEEE Trans. Biomed. Circuits Syst. 7(4):389–400 (2013)

    Article  Google Scholar 

  12. Altoé, A., Pulkki, V.: Transmission line cochlear models: improved accuracy and efficiency. J. Acoust. Soc. Am. 136:302–308 (2014). https://doi.org/10.1121/1.4896416

    Article  Google Scholar 

  13. Glasberg, B.R., Moore, B.C., Nimmo-smith, I.: Comparison of auditory filter shapes derived with three different maskers maskers. J. Acoust. Soc. Am. 75:536 (1984). https://doi.org/10.1121/1.390487

    Article  Google Scholar 

  14. Patterson, R.D., Unoki, M., Irino, T.: Extending the domain of center frequencies for the compressive gammachirp auditory filter. J. Acoust. Soc. Am. 114, 1529–1542 (2003). https://doi.org/10.1121/1.1600720

    Article  Google Scholar 

  15. Lyon, R.F., Katsiamis, A.G., Drakakis, E.M.: History and future of auditory filter models. In: IEEE International Symposium on Circuits and Systems (ISCAS) (Paris), pp. 3809–3812 (2010). https://doi.org/10.1109/ISCAS.2010.5537724

  16. Wang, S., Koickal, T.J., Hamilton, A., Cheung, R., Smith, L.S.: A bio-realistic analog CMOS cochlea filter with high tunability and ultra steep roll-off. IEEE Trans. Biomed. Circ. Syst. 9, 297–311 (2015). https://doi.org/10.1109/TBCAS.2014.2328321

    Article  Google Scholar 

  17. Lyon, R.F.: Human and Machine Hearing -Extracting Meaning from Sound. Cambridge University Press, Mountain View (2017)

    Book  Google Scholar 

  18. Xu, Y., Thakur, C.S., Singh, R.K., Hamilton, T.J., Wang, R.M., van Schaik, A.: A FPGA implementation of the CAR-FAC cochlear model. Front. Neurosci. 12:198 (2018)

    Article  Google Scholar 

  19. Lyon Richard, F., Moore, Brian CJ.: Human and Machine Hearing: Extracting Meaning from SoundHuman and Machine Learning: Extracting Meaning from Sound, 567pp. Cambridge University Press, Cambridge (2017). Price: 54.99. ISBN: 978-1-107-007536. Acoust. Soc. Am. J. 144:567 (2018)

    Google Scholar 

  20. Xu Y., Thakur, C.S., Hamilton, T.J., Wang, R., van Schaik, A.: An FPGA implementation of the CAR-FAC cochlear model. Front. Neurosci. 12:198 (2018)

    Article  Google Scholar 

  21. Thakur, C.S., et al.: FPGA implementation of the CAR model of cochlea. In: IEEE International Symposium on Circuits and Systems (ISCAS) (2014)

    Google Scholar 

  22. Lyon, R.F.: Human and Machine Hearing -Extracting Meaning from Sound. Cambridge University Press, Mountain View (2017)

    Book  Google Scholar 

  23. Thakur, C.S., Hamilton, T.J., Tapson, J., van Schaik, A., Lyon, R.F.: FPGA implementation of the CAR model of the cochlea. In: 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1853–1856 (2014)

    Google Scholar 

  24. Thakur, C.S., Hamilton, T.J., Tapson, J., van Schaik, A., Lyon, R.F.: Live demonstration: FPGA implementation of the CAR model of the cochlea. In: 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1853–1856 (2014)

    Google Scholar 

  25. Xu, Y., Thakur, C.S., et al.: Electronic cochlea: car-FAC model on FPGA. In: IEEE Biomedical Circuits and Systems Conference (BioCAS 2016) (2016)

    Google Scholar 

  26. Fukuma, A., Matsubara, M.: Jump resonance criteria of nonlinear control systems. IEEE Trans. Autom. Control 11(4):699–706 (1966)

    Article  Google Scholar 

  27. Martin, P., Mehta, A.D., Hudspeth, A.J.: Negative hair-bundle stiffness betrays a mechanism for mechanical amplification by the hair cell. Proc. Natl. Acad. Sci. 97(22):12026–12031 (2000)

    Article  Google Scholar 

  28. Aono, K., Shaga, R.K., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon auditory front-ends for extracting speaker discriminative formant trajectories. Biomed. Circuits Syst. IEEE Trans. PP(99):1 (2013)

    Google Scholar 

  29. Aono, K., Shaga, R.K., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for formant trajectory encoding. In: 2012 IEEE 55th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 85–88, 5–8 Aug 2012

    Google Scholar 

  30. Moni, R.S., Rao, K.R.: Jump-phenomenon in active-RC filters. IEEE Trans. Circuits Syst. 29(1):54–55 (1982)

    Article  Google Scholar 

  31. Aono, K., Shaga, R., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for extracting speaker discriminative formant trajectories. IEEE Trans. Biomed. Circuits Syst. 7(4):389–400 (2013)

    Article  Google Scholar 

  32. Chakrabartty, S., Cauwenberghs, G.: Gini-support vector machine: quadratic entropy based multi-class probability regression. J. Mach. Learn. Res. 8:813–839 (2007)

    MATH  Google Scholar 

  33. Fazel, A., Chakrabartty, S.: An overview of statistical pattern recognition techniques for speaker verification. IEEE Circuits Syst. Mag. 11(2):62–81 (2011)

    Article  Google Scholar 

  34. Aono, K., Shaga, R., Chakrabartty, S.: Exploiting jump-resonance hysteresis in silicon cochlea for extracting speaker discriminative formant trajectories. IEEE Trans. Biomed. Circuits Syst. 7(4):389–400 (2013)

    Article  Google Scholar 

  35. Chi, T., Ru, P., Shamma, S.A.: Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118(2):887–906 (2005). https://doi.org/10.1121/1.1945807

    Article  Google Scholar 

  36. Patil, K., et al.: Music in our ears: the biological bases of musical timbre perception. PLoS Comput. Biol. 8(11):e1002759 (2012)

    Article  Google Scholar 

  37. Amari, S., Cichocki, A., Yang, H.H.: A new learning algorithm for blind signal separation. In: Advanced Neural Information Processing Systems (NIPS), vol. 8, pp. 757–763. MIT Press, Cambridge, MA (1996)

    Google Scholar 

  38. Chakrabartty, S., Deng, Y., Cauwenberghs, G.: Robust speech feature extraction by growth transformation in reproducing Kernel Hilbert space. IEEE Trans. Speech Lang. Acoust. 15(6):1842–1849 (2007)

    Article  Google Scholar 

  39. Patterson, R., Moore, B.: Auditory filters and excitation patterns as representations of frequency resolution. Freq. Sel. Hearing 363:123–177 (1986)

    Google Scholar 

  40. Smith, E.C., Lewicki, M.S.: Efficient auditory coding. Nature 439:978–982 (2006)

    Article  Google Scholar 

  41. Chi, T., Ru, P., Shamma, S.: Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118:887–906 (2005)

    Article  Google Scholar 

  42. Chi, T., Ru, P., Shamma, S.: Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118:887–906 (2005)

    Article  Google Scholar 

  43. Wessinger, C.M., VanMeter, J., Tian, B., Lare, J.V., Pekar, J., Rauschecker, J.P.: Hierarchical organization of the human auditory cortex revealed by functional magnetic resonance imaging. J. Cogn. Neurosci. 13:1–7 (2001)

    Article  Google Scholar 

  44. Okada, K., Rong, F., Venezia, J., Matchin, W., Hsieh, I.-H., Saberi, K., Serences, J.T., Hickok, G.: Hierarchical organization of human auditory cortex: evidence from acoustic invariance in the response to intelligible speech. Cereb. Cortex 20:2486–2495 (2010)

    Article  Google Scholar 

  45. Boemio, A., Fromm, S., Braun, A., Poeppel, D.: Hierarchical and asymmetric temporal sensitivity in human auditory cortices. Nat. Neurosci. 8:389–395 (2005)

    Article  Google Scholar 

  46. Fazel, A., Chakrabartty, S.: Sparse auditory reproducing kernel (SPARK) features for noise-robust speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(4) (2012). https://doi.org/10.1109/TASL.2011.2179294

  47. Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nature Neurosci. 2:1019–1025 (1999)

    Article  Google Scholar 

  48. Wahba, G.: Splines Models for Observational Data. Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990)

    Google Scholar 

  49. Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7:219–269 (1995)

    Article  Google Scholar 

  50. Fazel, A., Chakrabartty, S.: Sparse auditory reproducing kernel (SPARK) features for noise-robust speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(4) (2012). https://doi.org/10.1109/TASL.2011.2179294

  51. Hirsch, H.G., Pearce, D.: The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In: Proceedings of ASR, pp. 181–188 (2000)

    Google Scholar 

  52. Mar, D.J., Chow, C.C., Gerstner, W., Adams, R.W., Collins, J.J.: Noise shaping in populations of coupled model neurons. Proc. Natl. Acad. Sci. U. S. A. 96:10450–10455 (1999)

    Article  Google Scholar 

  53. Gore, A., Chakrbartty, S.: Large Margin Analog-to-digital converters with applications in Neural Prosthetics. In: Advances in Neural Information Processing Systems (NIPS) (2006)

    Google Scholar 

  54. Pearce, T.C.: Computational parallels between the biological olfactory pathway and its analogue ‘the electronic nose’: Part I. Biological olfaction. Biosystems 41:43–67 (1997)

    Article  Google Scholar 

  55. Pearce, T.C.: Computational parallels between the biological olfactory pathway and its analogue ‘the electronic nose’: Part II. Sensor-based machine olfaction. Biosystems 41:69–90 (1997)

    Google Scholar 

  56. Raman, B., Stopfer, M., Semancik, S.: Mimicking biological design and computing principles in artificial olfaction. ACS Chem. Neurosci. 2:487–499 (2011)

    Article  Google Scholar 

  57. Persaud, K., Dodd, G.: Analysis of discrimination mechanisms in the mammalian olfactory system using a model nose. Nature 299:352–355 (1982)

    Article  Google Scholar 

  58. Nagle, T., Schiffman, S.S., Gutierrez-Osuna, R.: The how and why of electronic noses. IEEE Spectr. 35(9):22–34 (1998)

    Article  Google Scholar 

  59. Pearce, T.C., et al.: Handbook of Machine Olfaction: Electronic Nose Technology. Wiley, Wiley-VCH Verlag GmbH & Co. KGaA (2006)

    Google Scholar 

  60. Raman, B.: Sensor-based machine olfaction with neuromorphic models of the olfactory system. Ph. D. Dissertation, Texas A&M University, College Station (2005)

    Google Scholar 

  61. Ache, B.W., Young, J.M.: Olfaction: diverse species, conserved principles. Neuron 48:417–430 (2005)

    Article  Google Scholar 

  62. Hallem, E.A., Carlson, J.R.: Coding of odors by a receptor repertoire. Cell 125:143–160 (2006)

    Article  Google Scholar 

  63. Uchida, N., Poo, C., Haddad, R.: Coding and transformationsin the olfactory system. Ann. Rev. Neurosci. 37:363–385 (2014)

    Article  Google Scholar 

  64. Farivar, S.: Cytoarchitecture of the locust olfactory system. In: Biology. Califonia Institute of Technology (2005). https://doi.org/10.7907/4Y60-KH68

  65. Belluscio, L., et al.: A map of pheromone receptor activation in the mammalian brain. Cell 97(2):209–220 (1999)

    Article  Google Scholar 

  66. Laurent, G.: A systems perspective on olfactory coding. Science 286:723–728 (1999)

    Article  Google Scholar 

  67. Pearce, T.C., et al.: Robust stimulus encoding in olfactory processing: hyperacuity and efficient signal transmission. In: Emergent Neural Computational Architectures Based on Neuroscience, Springer, Berlin, Heidelberg, pp. 461–479 (2001)

    Google Scholar 

  68. Carandini, M., Heeger, D.J.: Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13:51–62

    Google Scholar 

  69. Olsen, S.R., Bhandawat, V., Wilson, R.I.: Divisive normalization in olfactory population codes. Neuron 66:287–299 (2010)

    Article  Google Scholar 

  70. Yaksi, E., Wilson, R.: Electrical coupling between olfactory glomeruli. Neuron 67:1034–1047 (2010)

    Article  Google Scholar 

  71. Zhu, P., Frank, T., Friedrich, R.W.: Equalization of odor representations by a network of electrically coupled inhibitory interneurons. Nat. Neurosci. 16(11):1678–1686 (2013)

    Article  Google Scholar 

  72. Friedrich, R.W., Laurent, G.: Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity. Science 291:889–894 (2001)

    Article  Google Scholar 

  73. Wilson, C.D., et al.: A primacy code for odor identity. Nat. Commun. 8:1477 (2017)

    Article  Google Scholar 

  74. Nizampatnam, S., et al.: Dynamic contrast enhancement and flexible odor codes. Nat. Commun. 9:3062 (2018)

    Article  Google Scholar 

  75. Saha, D., et al.: A spatiotemporal coding mechanism for background-invariant odor recognition. Nat. Neurosci. 16:1830–1839 (2013)

    Article  Google Scholar 

  76. Rokni, D., et al.: An olfactory cocktail party: figure-ground segregation of odorants in rodents. Nat. Neurosci. 17:1225–1232 (2014)

    Article  Google Scholar 

  77. Khan, A.G., Thattai, M., Bhalla, U.S.: Odor representations in the rat olfactory bulb change smoothly with morphing stimuli. Neuron 57(4):571–585 (2008)

    Article  Google Scholar 

  78. Niessing, J., Friedrich, R.W.: Olfactory pattern classification by discrete neuronal network states. Nature 465:47–52 (2010)

    Article  Google Scholar 

  79. Liang, D.G.: Perception of odor mixtures. In: Doty, R.L., (ed.) The Handbook of Olfaction and Gustation, pp. 283–298. Marcel Dekker, New York (1995)

    Google Scholar 

  80. Jortner, R.A., Farivar, S., Laurent, G.: A simple connectivity scheme for sparse coding in an olfactory system. J. Neurosci. 14(7):1659–1669 (2007)

    Article  Google Scholar 

  81. Gerstner, W.: Population dynamics of spiking neurons: fast transients, asynchronous states, and locking. Neural Comput. 12(1):43–89 (2000)

    Article  Google Scholar 

  82. Chacron, M.J., Lindner, B., Longtin, A.: Noise shaping by interval correlations increases information transfer. Phys. Rev. Lett. 92:080601 (2004)

    Article  Google Scholar 

  83. Wiesenfeld, K., Satija, I.: Noise tolerance of frequency-locked dynamics. Phys. Rev. B 36:2483–2492 (1987)

    Article  Google Scholar 

  84. Gore, A., Chakrabartty, S.: Min-max optimization framework for designing sigma-delta learners: theory and hardware. IEEE Trans. Circuits Syst. I 57(3):604–617 (2010). https://doi.org/10.1109/TCSI.2009.2025002

    Article  MathSciNet  Google Scholar 

  85. Chakrabartty, S.: Multiple-input multiple-output analog-to-digital converter US Patent no: 7,479,911, Issued 20 Jan 2009

    Google Scholar 

  86. Perera, A., et al.: A dimensionality-reduction technique inspired by receptor convergence in the olfactory system. Sens. Actuat B: Chem. 116:17–22 (2006)

    Article  Google Scholar 

  87. Raman, B., et al.: Processing of chemical sensor array with a biologically inspired model of olfactory coding. IEEE Trans. Neural Netw. 17:1015–1024 (2006)

    Article  Google Scholar 

  88. Raman, B., Gutierrez-Osuna, R.: Concentration normalization with a model of gain control in the olfactory bulb. Sens. Actuat. B: Chem. 116:36–42 (2006)

    Article  Google Scholar 

  89. Raman, B., Yamanaka, T., Gutierrez-Osuna, R.: Contrast enhancement of gas sensor array patterns with a neurodynamics model of the olfactory bulb. Sens. Actuat. B: Chem. 119:547–555 (2006)

    Article  Google Scholar 

  90. Stopfer, M., et al.: Odor identity vs. intensity coding in an olfactory system. Neuron 39(6):991–1004 (2003)

    Google Scholar 

  91. Raman, B., Gutierrez-Osuna, R.: Chemosensory processing in a spiking model of the olfactory bulb: chemotopic convergence and center surround inhibition. In: Proceedings of Advances in Neural Information Processing Systems (NIPS 2004), Vancouver, vol. 17 (2004)

    Google Scholar 

  92. Raman, B., et al.: Sensor-based machine olfaction with a neurodynamics model of the olfactory bulb. In: IEEE Intelligence Robots and Systems. IEEE, Sendai (2004)

    Google Scholar 

  93. Raman, B., Gutierrez-Osuna, R.: Mixture segmentation and background suppression in chemosensor arrays with a model of olfactory bulb-cortex interaction. In: International Joint Conference on Neural Networks. IEEE, Montreal (2005)

    Google Scholar 

  94. Raman, B., et al.: A bioinspired methodology for artificial olfaction. Anal. Chem. 80:8364–8371 (2008)

    Article  Google Scholar 

  95. Saha, D., et al.: Engaging and disengaging recurrent inhibition coincides with sensing and unsensing of a sensory stimulus. Nat. Commun. 8:15413 (2017)

    Article  Google Scholar 

  96. Persaud, K., Dodd, G.: Analysis of discrimination mechanisms in the mammalian olfactory system using a model nose. Nature 299:352–355 (1982)

    Article  Google Scholar 

  97. Ache, B.W., Young, J.M.: Olfaction: diverse species, conserved principles. Neuron 48:417–430 (2005)

    Article  Google Scholar 

  98. Saha, D., et al.: Explosive sensing with insect-based biorobots. Biosens. Bioelectron. X 6:100050 (2020)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Scheme for Promotion of Academic and Research Collaboration (SPARC), MHRD, Govt. of India, for funding (SPARC/2018-2019/P606/SL) this work. The authors acknowledge the Brain, Computation and Learning workshop at the Indian Institute of Science, India, where much of this work has been done.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chetan Singh Thakur .

Editor information

Editors and Affiliations

Section Editor information

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Singapore Pte Ltd.

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Chakrabartty, S., Raman, B., Thakur, C.S. (2022). Sensing-to-Learn and Learning-to-Sense: Principles for Designing Neuromorphic Sensors. In: Thakor, N.V. (eds) Handbook of Neuroengineering. Springer, Singapore. https://doi.org/10.1007/978-981-15-2848-4_122-1

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-2848-4_122-1

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-2848-4

  • Online ISBN: 978-981-15-2848-4

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics