Abstract
The biologically inspired spiking neurons used in neuromorphic computing are nonlinear filters with dynamic state variables, which is distinct from the stateless neuron models used in deep learning. The new version of Intel’s neuromorphic research processor, Loihi 2, supports an extended range of stateful spiking neuron models with programmable dynamics. Here, we showcase advanced neuron models that can be used to efficiently process streaming data in simulation experiments on emulated Loihi 2 hardware. In one example, Resonate-and-Fire (RF) neurons are used to compute the Short Time Fourier Transform (STFT) with similar computational complexity but 47x less output bandwidth than the conventional STFT. In another example, we describe an algorithm for optical flow estimation using spatiotemporal RF neurons that requires over 90x fewer operations than a conventional DNN-based solution. We also demonstrate backpropagation methods to train non-linear spiking RF neurons for audio classification tasks, suitable for efficient execution on Loihi 2. We conclude with another application of nonlinear filtering showing a cascade of Hopf resonators exhibiting computational properties seen in the cochlea, such as self-normalization. Taken together, this work presents new techniques for an efficient spike-based spectrogram encoder that can be used for signal processing applications.
Similar content being viewed by others
Notes
https://lava-nc.org. The lava-dl deep learning library is available at https://github.com/lava-nc/lava-dl
References
Davies, M., Srinivasa, N., Lin, T., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y., Wild, A., Yang, Y., & Wang, H. (2018). Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1), 82–99.
Davies, M., Wild, A., Orchard, G., Sandamirskaya, Y., Guerra, G. A. F., Joshi, P., Plank, P., & Risbud, S. R. (2021). Advancing neuromorphic computing with Loihi: A survey of results and outlook. Proceedings of the IEEE, 109(5), 911–934.
Arthur, J. V., & Boahen, K. A. (2010). Silicon-neuron design: A dynamical systems approach. IEEE Transactions on Circuits and Systems I, 58(5), 1034–1043.
Frady, E. P., & Sommer, F. T. (2019). Robust computation with rhythmic spike patterns. PNAS, 116(36), 18050–18059.
Polikar, R. (1996). Fundamental concepts & an overview of the wavelet theory.
Gallego, G., Delbruck, T., Orchard, G. M., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A., Conradt, J., Daniilidis, K., & Scaramuzza, D. (2020). Event-based vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of Optical Society of America, 2(2), 284–299.
Zihao Zhu, A., Thakur, D., Özaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018). The multivehicle stereo event camera dataset: An event camera dataset for 3D perception. IEEE Robotics and Automation Letters, 3(3), 2032–2039.
Shrestha, S. B., & Orchard, G. (2018). SLAYER: Spike layer error reassignment in time. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 1412–1421).
Neftci, E. O., Mostafa, H., & Zenke, F. (2019). Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine, 36, 61–63.
Izhikevich, E. M. (2001). Resonate-and-fire neurons. Neural Networks, 14(6), 883–894.
Anumula, J., Neil, D., Delbruck, T., & Liu, S.-C. (2018). Feature representations for neuromorphic audio spike streams. Frontiers in Neuroscience, 12.
Warden, P. (2018). Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209
Zilany, M. S., Bruce, I. C., & Carney, L. H. (2014). Updated parameters and expanded simulation options for a model of the auditory periphery. The Journal of the Acoustical Society of America, 135(1), 283–286.
Yin, B., Corradi, F., & Bohté, S. M. (2021). Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. preprint arXiv:2103.12593
Rybakov, O., Kononenko, N., Subrahmanya, N., Visontai, M., & Laurenzo, S. (2020). Streaming keyword spotting on mobile devices. In Interspeech.
Zhang, Y., Suda, N., Lai, L., & Chandra, V. (2017). Hello edge: Keyword spotting on microcontrollers. arXiv preprint arXiv:1711.07128
Goldstein, J. (1967). Auditory nonlinearity. Journal of the Acoustical Society of America, 41, 676–689.
Tapson, J., Hamilton, T. J., Jin, C., & van Schaik (2008). A. Self-tuned regenerative amplification and the HOPF bifurcation. In IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1768–1771).
Eguíluz, V. M., Ospeck, M., Choe, Y., Hudspeth, A., & Magnasco, M. O. (2000). Essential nonlinearities in hearing. Physical Review Letters, 84(22), 5232–5235.
Kern, A., & Stoop, R. (2003). Essential role of couplings between hearing nonlinearities. Physical Review Letters, 91(12), 128101.
Liu, K. J. R. (1993). Novel Parallel architectures for short-time fourier transform. IEEE TCS II, 40(12), 786–790.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interest
The authors are employees of Intel Labs. Sophia Sanborn contributed to this research while at Intel, and is currently at UC Berkeley. On behalf of Intel, Daniel Ben Dayan Rubin filed a patent application 17/643,652, AD7779-US named “Peak self-normalization gain control based on Hopf resonators cascade signal spectral decomposition”. The authors declare no additional competing interests. E. Paxon Frady, Sophia Sanborn, Sumit Bam Shrestha, Daniel Ben Dayan Rubin, performed experiments and analysis. E. Paxon Frady, Sumit Bam Shreshtha, Daniel Ben Dayan Rubin, Garrick Orchard, Friedrich T. Sommer, and Mike Davies wrote the paper.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Frady, E.P., Sanborn, S., Shrestha, S.B. et al. Efficient Neuromorphic Signal Processing with Resonator Neurons. J Sign Process Syst 94, 917–927 (2022). https://doi.org/10.1007/s11265-022-01772-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-022-01772-5