Skip to main content
SpringerLink
Log in

Embedded Neuromorphic Using Intel’s Loihi Processor

  • Chapter
  • First Online:
Embedded Machine Learning for Cyber-Physical, IoT, and Edge Computing

  • 420 Accesses

Abstract

Recently, spiking neural networks (SNNs) have demonstrated great success due to their high-performance and low-energy consumption, which makes them suitable for being implemented on embedded devices, such as neuromorphic chips. This chapter presents an overview of event-based SNNs on neuromorphic hardware and their applications. It provides outlooks on the neuromorphic computing platforms, with a special focus on the Intel Loihi research chip. Afterward, a case study on a “car vs. background” classifier implemented on Loihi is discussed in detail.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amir, A., Taba, B., Berg, D.J., Melano, T., McKinstry, J.L., di Nolfo, C., Nayak, T.K., Andreopoulos, A., Garreau, G., Mendoza, M., Kusnitz, J., DeBole, M., Esser, S.K., Delbrück, T., Flickner, M., Modha, D.S.: A low power, fully event-based gesture recognition system. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017, pp. 7388–7397. IEEE Computer Society (2017). https://doi.org/10.1109/CVPR.2017.781

  2. Bekolay, T., Bergstra, J., Hunsberger, E., DeWolf, T., Stewart, T.C., Rasmussen, D., Choo, F., Voelker, A., Eliasmith, C.: Nengo: a python tool for building large-scale functional brain models. Frontiers Neuroinformatics 7, 48 (2013). https://doi.org/10.3389/fninf.2013.00048

    Google Scholar 

  3. Bellec, G., Salaj, D., Subramoney, A., Legenstein, R., Maass, W.: Long short-term memory and learning-to-learn in networks of spiking neurons. In: Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada, pp. 795–805 (2018). https://proceedings.neurips.cc/paper/2018/hash/c203d8a151612acf12457e4d67635a95-Abstract.html

  4. Benjamin, B.V., Gao, P., McQuinn, E., Choudhary, S., Chandrasekaran, A., Bussat, J., Alvarez-Icaza, R., Arthur, J.V., Merolla, P., Boahen, K.: Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102(5), 699–716 (2014). https://doi.org/10.1109/JPROC.2014.2313565

    Article  Google Scholar 

  5. Binas, J., Neil, D., Liu, S., Delbrück, T.: DDD17: end-to-end DAVIS driving dataset. CoRR abs/1711.01458 (2017). http://arxiv.org/abs/1711.01458

  6. Blouw, P., Choo, X., Hunsberger, E., Eliasmith, C.: Benchmarking keyword spotting efficiency on neuromorphic hardware. CoRR abs/1812.01739 (2018). http://arxiv.org/abs/1812.01739

  7. Bovik, A.C., Clark, M., Geisler, W.S.: Multichannel texture analysis using localized spatial filters. IEEE Trans. Pattern Anal. Mach. Intell. 12(1), 55–73 (1990). https://doi.org/10.1109/34.41384

    Article  Google Scholar 

  8. Brandli, C., Berner, R., Yang, M., Liu, S., Delbrück, T.: A 240 × 180 130 db 3 μs latency global shutter spatiotemporal vision sensor. IEEE J. Solid State Circuits 49(10), 2333–2341 (2014). https://doi.org/10.1109/JSSC.2014.2342715

    Article  Google Scholar 

  9. Buettner, K., George, A.D.: Heartbeat classification with spiking neural networks on the Loihi neuromorphic processor. In: IEEE Computer Society Annual Symposium on VLSI, ISVLSI 2021, Tampa, FL, USA, July 7-*9, 2021, pp. 138–143. IEEE (2021). https://doi.org/10.1109/ISVLSI51109.2021.00035

  10. Capra, M., Bussolino, B., Marchisio, A., Masera, G., Martina, M., Shafique, M.: Hardware and software optimizations for accelerating deep neural networks: Survey of current trends, challenges, and the road ahead. IEEE Access 8, 225134–225180 (2020). https://doi.org/10.1109/ACCESS.2020.3039858

    Article  Google Scholar 

  11. Ceolini, E., Frenkel, C., Shrestha, S.B., Taverni, G., Khacef, L., Payvand, M., Donati, E.: Hand-gesture recognition based on EMG and event-based camera sensor fusion: A benchmark in neuromorphic computing. Front. Neurosci. 14 (2020). https://doi.org/10.3389/fnins.2020.00637. https://www.frontiersin.org/article/10.3389/fnins.2020.00637

  12. Chen, S., Guo, M.: Live demonstration: CeleX-V: A 1M pixel multi-mode event-based sensor. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16–20, 2019, pp. 1682–1683. Computer Vision Foundation / IEEE (2019). https://doi.org/10.1109/CVPRW.2019.00214. http://openaccess.thecvf.com/content_CVPRW_2019/html/EventVision/Chen_Live_Demonstration_CeleX-V_A_1M_Pixel_Multi-Mode_Event-Based_Sensor_CVPRW_2019_paper.html

  13. Cheng, W., Luo, H., Yang, W., Yu, L., Chen, S., Li, W.: DET: A high-resolution DVS dataset for lane extraction. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16–20, 2019, pp. 1666–1675. Computer Vision Foundation/IEEE (2019). https://doi.org/10.1109/CVPRW.2019.00210. http://openaccess.thecvf.com/content_CVPRW_2019/html/EventVision/Cheng_DET_A_High-Resolution_DVS_Dataset_for_Lane_Extraction_CVPRW_2019_paper.html

  14. Corporation, I.: Lava: A software framework for neuromorphic computing. https://lava-nc.org/

  15. Davies, M., Srinivasa, N., Lin, T., Chinya, G.N., Cao, Y., Choday, S.H., Dimou, G.D., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y., Wild, A., Yang, Y., Wang, H.: Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1), 82–99 (2018). https://doi.org/10.1109/MM.2018.112130359

    Article  Google Scholar 

  16. Davies, M., Wild, A., Orchard, G., Sandamirskaya, Y., Guerra, G.A.F., Joshi, P., Plank, P., Risbud, S.R.: Advancing neuromorphic computing with Loihi: A survey of results and outlook. Proc. IEEE 109(5), 911–934 (2021). https://doi.org/10.1109/JPROC.2021.3067593

    Article  Google Scholar 

  17. Davison, A.P., Brüderle, D., Eppler, J.M., Kremkow, J., Müller, E.B., Pecevski, D., Perrinet, L.U., Yger, P.: PyNN: a common interface for neuronal network simulators. Frontiers Neuroinformatics 2, 11 (2008). https://doi.org/10.3389/neuro.11.011.2008

    Article  Google Scholar 

  18. Delmerico, J.A., Cieslewski, T., Rebecq, H., Faessler, M., Scaramuzza, D.: Are we ready for autonomous drone racing? the UZH-FPV drone racing dataset. In: International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, May 20–24, 2019, pp. 6713–6719. IEEE (2019). https://doi.org/10.1109/ICRA.2019.8793887

  19. DeWolf, T., Jaworski, P., Eliasmith, C.: Nengo and low-power AI hardware for robust, embedded neurorobotics. Frontiers Neurorobotics 14, 568359 (2020). https://doi.org/10.3389/fnbot.2020.568359

    Article  Google Scholar 

  20. Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Comput. Vis. Image Underst. 106(1), 59–70 (2007). https://doi.org/10.1016/j.cviu.2005.09.012

    Article  Google Scholar 

  21. Finateu, T., Niwa, A., Matolin, D., Tsuchimoto, K., Mascheroni, A., Reynaud, E., Mostafalu, P., Brady, F.T., Chotard, L., LeGoff, F., Takahashi, H., Wakabayashi, H., Oike, Y., Posch, C.: 5.10 A 1280×720 back-illuminated stacked temporal contrast event-based vision sensor with 4.86μm pixels, 1.066geps readout, programmable event-rate controller and compressive data-formatting pipeline. In: 2020 IEEE International Solid- State Circuits Conference, ISSCC 2020, San Francisco, CA, USA, February 16–20, 2020, pp. 112–114. IEEE (2020). https://doi.org/10.1109/ISSCC19947.2020.9063149

  22. Fischer, T., Milford, M.: Event-based visual place recognition with ensembles of temporal windows. IEEE Robotics Autom. Lett. 5(4), 6924–6931 (2020). https://doi.org/10.1109/LRA.2020.3025505

    Article  Google Scholar 

  23. Frenkel, C., Lefebvre, M., Legat, J., Bol, D.: A 0.086-mm2 12.7-pJ/SOP 64k-synapse 256-neuron online-learning digital spiking neuromorphic processor in 28-nm CMOS. IEEE Trans. Biomed. Circuits Syst. 13(1), 145–158 (2019). https://doi.org/10.1109/TBCAS.2018.2880425

  24. Furber, S.B., Galluppi, F., Temple, S., Plana, L.A.: The SpiNNaker project. Proc. IEEE 102(5), 652–665 (2014). https://doi.org/10.1109/JPROC.2014.2304638

    Article  Google Scholar 

  25. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., Amit, D.J.: Spike-driven synaptic plasticity: Theory, simulation, VLSI implementation. Neural Comput. 12(10), 2227–2258 (2000). https://doi.org/10.1162/089976600300014917

    Article  Google Scholar 

  26. Gallego, G., Delbrück, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A.J., Conradt, J., Daniilidis, K., Scaramuzza, D.: Event-based vision: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(1), 154–180 (2022). https://doi.org/10.1109/TPAMI.2020.3008413

    Article  Google Scholar 

  27. Gehrig, M., Aarents, W., Gehrig, D., Scaramuzza, D.: DSEC: A stereo event camera dataset for driving scenarios. IEEE Robotics Autom. Lett. 6(3), 4947–4954 (2021). https://doi.org/10.1109/LRA.2021.3068942

    Article  Google Scholar 

  28. Gerstner, W., Kistler, W.M.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press (2002). https://doi.org/10.1017/CBO9780511815706

  29. Ghosh, R., Gupta, A., Silva, A.N., Soares, A., Thakor, N.V.: Spatiotemporal filtering for event-based action recognition. CoRR abs/1903.07067 (2019). http://arxiv.org/abs/1903.07067

  30. Guo, M., Huang, J., Chen, S.: Live demonstration: A 768 × 640 pixels 200Meps dynamic vision sensor. In: IEEE International Symposium on Circuits and Systems, ISCAS 2017, Baltimore, MD, USA, May 28–31, 2017, p. 1. IEEE (2017). https://doi.org/10.1109/ISCAS.2017.8050397

  31. Hodgkin, A.L., Huxley, A.F.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117(4), 500–544 (1952). https://doi.org/10.1113/jphysiol.1952.sp004764. https://physoc.onlinelibrary.wiley.com/doi/abs/10.1113/jphysiol.1952.sp004764

  32. Hu, Y., Binas, J., Neil, D., Liu, S., Delbrück, T.: DDD20 end-to-end event camera driving dataset: Fusing frames and events with deep learning for improved steering prediction. In: 23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020, Rhodes, Greece, September 20–23, 2020, pp. 1–6. IEEE (2020). https://doi.org/10.1109/ITSC45102.2020.9294515

  33. Izhikevich, E.M.: Simple model of spiking neurons. IEEE Trans. Neural Networks 14(6), 1569–1572 (2003). https://doi.org/10.1109/TNN.2003.820440

    Article  MathSciNet  Google Scholar 

  34. Jouppi, N.P., Young, C., Patil, N., Patterson, D.A., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., Boyle, R., Cantin, P., Chao, C., Clark, C., Coriell, J., Daley, M., Dau, M., Dean, J., Gelb, B., Ghaemmaghami, T.V., Gottipati, R., Gulland, W., Hagmann, R., Ho, C.R., Hogberg, D., Hu, J., Hundt, R., Hurt, D., Ibarz, J., Jaffey, A., Jaworski, A., Kaplan, A., Khaitan, H., Killebrew, D., Koch, A., Kumar, N., Lacy, S., Laudon, J., Law, J., Le, D., Leary, C., Liu, Z., Lucke, K., Lundin, A., MacKean, G., Maggiore, A., Mahony, M., Miller, K., Nagarajan, R., Narayanaswami, R., Ni, R., Nix, K., Norrie, T., Omernick, M., Penukonda, N., Phelps, A., Ross, J., Ross, M., Salek, A., Samadiani, E., Severn, C., Sizikov, G., Snelham, M., Souter, J., Steinberg, D., Swing, A., Tan, M., Thorson, G., Tian, B., Toma, H., Tuttle, E., Vasudevan, V., Walter, R., Wang, W., Wilcox, E., Yoon, D.H.: In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, ISCA 2017, Toronto, ON, Canada, June 24–28, 2017, pp. 1–12. ACM (2017). https://doi.org/10.1145/3079856.3080246

  35. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980

  36. Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images. University of Toronto (2012)

    Google Scholar 

  37. Lagorce, X., Orchard, G., Galluppi, F., Shi, B.E., Benosman, R.: HOTS: A hierarchy of event-based time-surfaces for pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1346–1359 (2017). https://doi.org/10.1109/TPAMI.2016.2574707

    Article  Google Scholar 

  38. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  39. Li, H., Liu, H., Ji, X., Li, G., Shi, L.: Cifar10-dvs: An event-stream dataset for object classification. Front. Neurosci. 11 (2017). https://doi.org/10.3389/fnins.2017.00309. https://www.frontiersin.org/article/10.3389/fnins.2017.00309

  40. Lichtsteiner, P., Posch, C., Delbrück, T.: A 128×128 120 db 15 μs latency asynchronous temporal contrast vision sensor. IEEE J. Solid State Circuits 43(2), 566–576 (2008). https://doi.org/10.1109/JSSC.2007.914337

    Article  Google Scholar 

  41. Lin, C., Wild, A., Chinya, G.N., Cao, Y., Davies, M., Lavery, D.M., Wang, H.: Programming spiking neural networks on Intel’s Loihi. Computer 51(3), 52–61 (2018). https://doi.org/10.1109/MC.2018.157113521

    Article  Google Scholar 

  42. Lin, C., Wild, A., Chinya, G.N., Lin, T., Davies, M., Wang, H.: Mapping spiking neural networks onto a manycore neuromorphic architecture. In: Foster, J.S., Grossman, D. (eds.) Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2018, Philadelphia, PA, USA, June 18–22, 2018, pp. 78–89. ACM (2018). https://doi.org/10.1145/3192366.3192371

  43. Lin, Y., Ding, W., Qiang, S., Deng, L., Li, G.: ES-ImageNet: A million event-stream classification dataset for spiking neural networks. CoRR abs/2110.12211 (2021). https://arxiv.org/abs/2110.12211

  44. Liu, C., Bellec, G., Vogginger, B., Kappel, D., Partzsch, J., Neumärker, F., Höppner, S., Maass, W., Furber, S.B., Legenstein, R., Mayr, C.G.: Memory-efficient deep learning on a SpiNNaker 2 prototype. Front. Neurosci. 12 (2018). https://doi.org/10.3389/fnins.2018.00840. https://www.frontiersin.org/article/10.3389/fnins.2018.00840

  45. Liu, Q., Furber, S.: Real-time recognition of dynamic hand postures on a neuromorphic system. Int. J. Electr. Comput. Eng. 9(5), 507–514 (2015). https://publications.waset.org/vol/101

    Google Scholar 

  46. Liu, T., Mahjoubfar, A., Prusinski, D., Stevens, L.: Neuromorphic computing for content-based image retrieval. CoRR abs/2008.01380 (2020). https://arxiv.org/abs/2008.01380

  47. Maass, W.: Networks of spiking neurons: The third generation of neural network models. Neural Networks 10(9), 1659–1671 (1997). https://doi.org/10.1016/S0893-6080(97)00011-7

    Article  Google Scholar 

  48. Massa, R., Marchisio, A., Martina, M., Shafique, M.: An efficient spiking neural network for recognizing gestures with a DVS camera on the Loihi neuromorphic processor. In: 2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19–24, 2020, pp. 1–9. IEEE (2020). https://doi.org/10.1109/IJCNN48605.2020.9207109

  49. McCulloch, W.S., Pitts, W.H.: A logical calculus of the ideas immanent in nervous activity. In: Boden, M.A. (ed.) The Philosophy of Artificial Intelligence, Oxford Readings in Philosophy, pp. 22–39. Oxford University Press (1990)

    Google Scholar 

  50. Merolla, P.A., Arthur, J.V., Alvarez-Icaza, R., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y., Brezzo, B., Vo, I., Esser, S.K., Appuswamy, R., Taba, B., Amir, A., Flickner, M.D., Risk, W.P., Manohar, R., Modha, D.S.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014). https://doi.org/10.1126/science.1254642. https://www.science.org/doi/abs/10.1126/science.1254642

  51. Michaelis, C., Lehr, A.B., Oed, W., Tetzlaff, C.: Brian2Loihi: An emulator for the neuromorphic chip Loihi using the spiking neural network simulator Brian. CoRR abs/2109.12308 (2021). https://arxiv.org/abs/2109.12308

  52. Moradi, S., Ning, Q., Stefanini, F., Indiveri, G.: A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs). IEEE Trans. Biomed. Circuits Syst. 12(1), 106–122 (2018). https://doi.org/10.1109/TBCAS.2017.2759700

    Article  Google Scholar 

  53. Mueggler, E., Rebecq, H., Gallego, G., Delbrück, T., Scaramuzza, D.: The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM. Int. J. Robotics Res. 36(2), 142–149 (2017). https://doi.org/10.1177/0278364917691115

    Article  Google Scholar 

  54. Neftci, E.O., Mostafa, H., Zenke, F.: Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Process. Mag. 36(6), 51–63 (2019). https://doi.org/10.1109/MSP.2019.2931595

    Article  Google Scholar 

  55. Orchard, G., Frady, E.P., Rubin, D.B.D., Sanborn, S., Shrestha, S.B., Sommer, F.T., Davies, M.: Efficient neuromorphic signal processing with Loihi 2. In: IEEE Workshop on Signal Processing Systems, SiPS 2021, Coimbra, Portugal, October 19–21, 2021, pp. 254–259. IEEE (2021). https://doi.org/10.1109/SiPS52927.2021.00053

  56. Orchard, G., Jayawant, A., Cohen, G., Thakor, N.V.: Converting static image datasets to spiking neuromorphic datasets using saccades. CoRR abs/1507.07629 (2015). http://arxiv.org/abs/1507.07629

  57. Pan, Z., Wu, J., Zhang, M., Li, H., Chua, Y.: Neural population coding for effective temporal classification. In: International Joint Conference on Neural Networks, IJCNN 2019 Budapest, Hungary, July 14–19, 2019, pp. 1–8. IEEE (2019). https://doi.org/10.1109/IJCNN.2019.8851858

  58. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E.Z., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: PyTorch: An imperative style, high-performance deep learning library. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8–14, 2019, Vancouver, BC, Canada, pp. 8024–8035 (2019). https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html

  59. Patel, K., Hunsberger, E., Batir, S., Eliasmith, C.: A spiking neural network for image segmentation. CoRR abs/2106.08921 (2021). https://arxiv.org/abs/2106.08921

  60. Paugam-Moisy, H., Bohté, S.M.: Computing with spiking neuron networks. In: Rozenberg, G., Bäck, T., Kok, J.N. (eds.) Handbook of Natural Computing, pp. 335–376. Springer (2012). https://doi.org/10.1007/978-3-540-92910-9_10

  61. Ponulak, F., Kasinski, A.J.: Introduction to spiking neural networks: Information processing, learning and applications. Acta Neurobiol. Exp. 71(4), 409–33 (2011)

    MATH  Google Scholar 

  62. Posch, C., Matolin, D., Wohlgenannt, R.: A QVGA 143 db dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid State Circuits 46(1), 259–275 (2011). https://doi.org/10.1109/JSSC.2010.2085952

    Article  Google Scholar 

  63. Rasmussen, D.: NengoDL: Combining deep learning and neuromorphic modelling methods. CoRR abs/1805.11144 (2018). http://arxiv.org/abs/1805.11144

  64. Rathi, N., Srinivasan, G., Panda, P., Roy, K.: Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net (2020). https://openreview.net/forum?id=B1xSperKvH

  65. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017, pp. 6517–6525. IEEE Computer Society (2017). https://doi.org/10.1109/CVPR.2017.690

  66. Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  67. Renner, A., Evanusa, M., Orchard, G., Sandamirskaya, Y.: Event-based attention and tracking on neuromorphic hardware. In: 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2020, Genova, Italy, August 31 - September 2, 2020, p. 132. IEEE (2020). https://doi.org/10.1109/AICAS48895.2020.9073789

  68. Renner, A., Sheldon, F., Zlotnik, A., Tao, L., Sornborger, A.T.: The backpropagation algorithm implemented on spiking neuromorphic hardware. CoRR abs/2106.07030 (2021). https://arxiv.org/abs/2106.07030

  69. Rocki, K., Essendelft, D.V., Sharapov, I., Schreiber, R., Morrison, M., Kibardin, V., Portnoy, A., Dietiker, J., Syamlal, M., James, M.: Fast stencil-code computation on a wafer-scale processor. In: Cuicchi, C., Qualters, I., Kramer, W.T. (eds.) Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020, Virtual Event / Atlanta, Georgia, USA, November 9–19, 2020, p. 58. IEEE/ACM (2020). https://doi.org/10.1109/SC41405.2020.00062

  70. Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 65(6), 386–408 (1958)

    Article  Google Scholar 

  71. Rückauer, B., Känzig, N., Liu, S., Delbrück, T., Sandamirskaya, Y.: Closing the accuracy gap in an event-based visual recognition task. CoRR abs/1906.08859 (2019). http://arxiv.org/abs/1906.08859

  72. Rueckauer, B., Bybee, C., Goettsche, R., Singh, Y., Mishra, J., Wild, A.: NxTF: An API and compiler for deep spiking neural networks on Intel Loihi. CoRR abs/2101.04261 (2021). https://arxiv.org/abs/2101.04261

  73. Rueckauer, B., Lungu, I.A., Hu, Y., Pfeiffer, M., Liu, S.C.: Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front. Neurosci. 11 (2017). https://doi.org/10.3389/fnins.2017.00682. https://www.frontiersin.org/article/10.3389/fnins.2017.00682

  74. Ruf, B., Schmitt, M.: Hebbian learning in networks of spiking neurons using temporal coding. In: Mira, J., Moreno-Díaz, R., Cabestany, J. (eds.) Biological and Artificial Computation: From Neuroscience to Technology, International Work-Conference on Artificial and Natural Neural Networks, IWANN ’97, Lanzarote, Canary Islands, Spain, June 4–6, 1997, Proceedings, Lecture Notes in Computer Science, vol. 1240, pp. 380–389. Springer (1997). https://doi.org/10.1007/BFb0032496

  75. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  76. Schmitt, S., Klähn, J., Bellec, G., Grübl, A., Güttler, M., Hartel, A., Hartmann, S., de Oliveira, D.H., Husmann, K., Jeltsch, S., Karasenko, V., Kleider, M., Koke, C., Kononov, A., Mauch, C., Müller, E., Müller, P., Partzsch, J., Petrovici, M.A., Schiefer, S., Scholze, S., Thanasoulis, V.N., Vogginger, B., Legenstein, R., Maass, W., Mayr, C., Schüffny, R., Schemmel, J., Meier, K.: Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system. In: 2017 International Joint Conference on Neural Networks, IJCNN 2017, Anchorage, AK, USA, May 14–19, 2017, pp. 2227–2234. IEEE (2017). https://doi.org/10.1109/IJCNN.2017.7966125

  77. See, H., Lim, B., Li, S., Yao, H., Cheng, W., Soh, H., Tee, B.C.K.: ST-MNIST - the spiking tactile MNIST neuromorphic dataset. CoRR abs/2005.04319 (2020). https://arxiv.org/abs/2005.04319

  78. Shrestha, S.B., Orchard, G.: SLAYER: spike layer error reassignment in time. In: Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada, pp. 1419–1428 (2018). https://proceedings.neurips.cc/paper/2018/hash/82f2b308c3b01637c607ce05f52a2fed-Abstract.html

    Google Scholar 

  79. Sironi, A., Brambilla, M., Bourdis, N., Lagorce, X., Benosman, R.: HATS: histograms of averaged time surfaces for robust event-based object classification. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18–22, 2018, pp. 1731–1740. Computer Vision Foundation / IEEE Computer Society (2018). https://doi.org/10.1109/CVPR.2018.00186. http://openaccess.thecvf.com/content_cvpr_2018/html/Sironi_HATS_Histograms_of_CVPR_2018_paper.html

  80. Son, B., Suh, Y., Kim, S., Jung, H., Kim, J., Shin, C., Park, K., Lee, K., Park, J.M., Woo, J., Roh, Y., Lee, H., Wang, Y.M., Ovsiannikov, I.A., Ryu, H.: 4.1 A 640×480 dynamic vision sensor with a 9μm pixel and 300Meps address-event representation. In: 2017 IEEE International Solid-State Circuits Conference, ISSCC 2017, San Francisco, CA, USA, February 5–9, 2017, pp. 66–67. IEEE (2017). https://doi.org/10.1109/ISSCC.2017.7870263

  81. Srinivasan, G., Panda, P., Roy, K.: STDP-based unsupervised feature learning using convolution-over-time in spiking neural networks for energy-efficient neuromorphic computing. ACM J. Emerg. Technol. Comput. Syst. 14(4), 44:1–44:12 (2018). https://doi.org/10.1145/3266229

  82. Stewart, K., Orchard, G., Shrestha, S.B., Neftci, E.: Online few-shot gesture learning on a neuromorphic processor. IEEE J. Emerg. Sel. Topics Circuits Syst. 10(4), 512–521 (2020). https://doi.org/10.1109/JETCAS.2020.3032058

    Article  Google Scholar 

  83. Stimberg, M., Brette, R., Goodman, D.F.: Brian 2, an intuitive and efficient neural simulator. eLife 8, e47314 (2019). https://doi.org/10.7554/eLife.47314

    Article  Google Scholar 

  84. Stuijt, J., Sifalakis, M., Yousefzadeh, A., Corradi, F.: µBrain: An event-driven and fully synthesizable architecture for spiking neural networks. Front. Neurosci. 15 (2021). https://doi.org/10.3389/fnins.2021.664208. https://www.frontiersin.org/article/10.3389/fnins.2021.664208

  85. Suh, Y., Choi, S., Ito, M., Kim, J., Lee, Y., Seo, J., Jung, H., Yeo, D., Namgung, S., Bong, J., Yoo, S., Shin, S., Kwon, D., Kang, P., Kim, S., Na, H., Hwang, K., Shin, C., Kim, J., Park, P.K.J., Kim, J., Ryu, H., Park, Y.: A 1280×960 dynamic vision sensor with a 4.95-μm pixel pitch and motion artifact minimization. In: IEEE International Symposium on Circuits and Systems, ISCAS 2020, Seville, Spain, October 10–21, 2020, pp. 1–5. IEEE (2020). https://doi.org/10.1109/ISCAS45731.2020.9180436

  86. Tang, G., Kumar, N., Michmizos, K.P.: Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020, Las Vegas, NV, USA, October 24, 2020 - January 24, 2021, pp. 6090–6097. IEEE (2020). https://doi.org/10.1109/IROS45743.2020.9340948

  87. Taunyazov, T., Sng, W., Lim, B., See, H., Kuan, J., Ansari, A.F., Tee, B.C.K., Soh, H.: Event-driven visual-tactile sensing and learning for robots. In: Toussaint, M., Bicchi, A., Hermans, T. (eds.) Robotics: Science and Systems XVI, Virtual Event / Corvallis, Oregon, USA, July 12–16, 2020 (2020). https://doi.org/10.15607/RSS.2020.XVI.020

  88. Tayarani-Najaran, M.H., Schmuker, M.: Event-based sensing and signal processing in the visual, auditory, and olfactory domain: A review. Front. Neural Circ. 15 (2021). https://doi.org/10.3389/fncir.2021.610446. https://www.frontiersin.org/article/10.3389/fncir.2021.610446

  89. de Tournemire, P., Nitti, D., Perot, E., Migliore, D., Sironi, A.: A large scale event-based detection dataset for automotive. CoRR abs/2001.08499 (2020). https://arxiv.org/abs/2001.08499

  90. Viale, A., Marchisio, A., Martina, M., Masera, G., Shafique, M.: CarSNN: An efficient spiking neural network for event-based autonomous cars on the Loihi neuromorphic research processor. In: International Joint Conference on Neural Networks, IJCNN 2021, Shenzhen, China, July 18–22, 2021, pp. 1–10. IEEE (2021). https://doi.org/10.1109/IJCNN52387.2021.9533738

  91. Viale, A., Marchisio, A., Martina, M., Masera, G., Shafique, M.: LaneSNNs: Spiking neural networks for lane detection on the Loihi neuromorphic processor. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022. IEEE (2022)

    Google Scholar 

  92. Vitale, A., Renner, A., Nauer, C., Scaramuzza, D., Sandamirskaya, Y.: Event-driven vision and control for UAVs on a neuromorphic chip. In: IEEE International Conference on Robotics and Automation, ICRA 2021, Xi’an, China, May 30 - June 5, 2021, pp. 103–109. IEEE (2021). https://doi.org/10.1109/ICRA48506.2021.9560881

  93. Wang, Q., Zhang, Y., Yuan, J., Lu, Y.: Space-time event clouds for gesture recognition: From RGB cameras to event cameras. In: IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Waikoloa Village, HI, USA, January 7–11, 2019, pp. 1826–1835. IEEE (2019). https://doi.org/10.1109/WACV.2019.00199

  94. Wang, Z., Guo, L., Adjouadi, M.: A biological plausible generalized leaky integrate-and-fire neuron model. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014, Chicago, IL, USA, August 26–30, 2014, pp. 6810–6813. IEEE (2014). https://doi.org/10.1109/EMBC.2014.6945192

  95. Wu, Y., Deng, L., Li, G., Zhu, J., Shi, L.: Spatio-temporal backpropagation for training high-performance spiking neural networks. Front. Neurosci. 12 (2018). https://doi.org/10.3389/fnins.2018.00331. https://www.frontiersin.org/article/10.3389/fnins.2018.00331

  96. Zhu, A.Z., Thakur, D., Özaslan, T., Pfrommer, B., Kumar, V., Daniilidis, K.: The multivehicle stereo event camera dataset: An event camera dataset for 3D perception. IEEE Robotics Autom. Lett. 3(3), 2032–2039 (2018). https://doi.org/10.1109/LRA.2018.2800793

    Article  Google Scholar 

Download references

Acknowledgements

This work has been supported in part by the Doctoral College Resilient Embedded Systems, which is run jointly by the TU Wien’s Faculty of Informatics and the UAS Technikum Wien. This work was also supported in part by the NYUAD Center for Interacting Urban Networks (CITIES), funded by Tamkeen under the NYUAD Research Institute Award CG001, Center for Cybersecurity (CCS), funded by Tamkeen under the NYUAD Research Institute Award G1104, and Center for Artificial Intelligence and Robotics (CAIR), funded by Tamkeen under the NYUAD Research Institute Award CG010.

Author information

Authors and Affiliations

  1. Technische Universität Wien (TU Wien), Vienna, Austria

    Alberto Marchisio

  2. eBrain Lab, Division of Engineering, New York University Abu Dhabi, Abu Dhabi, UAE

    Muhammad Shafique

Authors
  1. Alberto Marchisio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Shafique
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Marchisio .

Editor information

Editors and Affiliations

  1. Colorado State University, Fort Collins, CO, USA

    Sudeep Pasricha

  2. New York University Abu Dhabi, Abu Dhabi, Abu Dhabi, United Arab Emirates

    Muhammad Shafique

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Marchisio, A., Shafique, M. (2024). Embedded Neuromorphic Using Intel’s Loihi Processor. In: Pasricha, S., Shafique, M. (eds) Embedded Machine Learning for Cyber-Physical, IoT, and Edge Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-39932-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-39932-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-39931-2

  • Online ISBN: 978-3-031-39932-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation