Advertisement

Very Large-Scale Neuromorphic Systems for Biological Signal Processing

  • Francky CatthoorEmail author
  • Srinjoy Mitra
  • Anup Das
  • Siebren Schaafsma
Chapter

Abstract

This chapter is a white paper describing a platform for scaled-up neuromorphic systems to ‘human brain size’ complexity. Such a system will be necessary for massive search and analysis tasks while interacting with biological data. This system would consist of similar number of neurons and synapses as in an adult human brain. One of the largest bottlenecks is the huge synaptic complexity that would result from connecting billions of neurons. The purpose of this chapter is to describe a feasible architecture that could handle the enormous communication bandwidth necessary for such a large-scale neuromorphic system. The proposed approach is grounded in the assumption that we would only be able to appreciate the utility of a neuromorphic system when it is somewhat similar to the human brain in terms of energy consumption and size. Inspired by the recent advancements in SoC architecture, a novel scalable intercluster communication network is proposed here. A particularly useful instantiation of this occurs for the global synaptic communication, interconnecting the local clusters of synapse arrays. The core of the proposed solution is a novel switching architecture in the CMOS back end of line (BEOL) that is expected to be extremely power efficient. In contrast to a fixed predefined bus that is shared over all connected local clusters, the proposed solution will allow a multitude of dedicated point-to-point connections that can be switched dynamically.

Keywords

Neuromorphic computing Global synapse network Brain-scale complexity Biology algorithms/applications 

Notes

Acknowledgement

The authors would like to acknowledge the interesting discussions with their colleagues Rudy Lauwereins, Diederik Verkest, Soeren Steudel, Marc Van Bladel and Aneta Markova during the preparation of the material in this paper and also the results produced by the MSc students Francesco Dell’Anna, Ahmed Ammar, Ahmed Abdelmoneem, Thibaut Marty and Gagandeep Singh. Many of them have also contributed to some quantitative data in this material. We also acknowledge the support of the Horizon 2020 NeuRAM3 EC project.

References

  1. 1.
    C. Mead, Analog VLSI and Neural Systems (Addison-Wesley, Reading, 1989)zbMATHGoogle Scholar
  2. 2.
    M. Mahowald, VLSI Analogs of Neuronal Visual Processing: A Synthesis of Form and Function (California Institute of Technology, Pasadena, 1992)Google Scholar
  3. 3.
    R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gomez-Rodriguez, L. Camunas-Mesa, R. Berner, M. Rivas-Perez, T. Delbruck, S.C. Liu, R. Douglas, P. Hafliger, G. Jimenez-Moreno, A. Civit Ballcels, T. Serrano-Gotarredona, A.J. Acosta-Jimenez, B. Linares-Barranco, CAVIAR: A 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Trans. Neural Netw. 20(9), 1417–1438 (2009)CrossRefGoogle Scholar
  4. 4.
    S. Mitra, S. Fusi, G. Indiveri, Real-time classification of complex patterns using spike-based learning in neuromorphic VLSI. IEEE Trans. Biomed. Circuits Syst. 3(1), 32–42 (2009)CrossRefGoogle Scholar
  5. 5.
    S.C. Liu, A. Van Schaik, B.A. Minch, T. Delbruck, Asynchronous binaural spatial audition sensor with 2??64??4 channel output. IEEE Trans. Biomed. Circuits Syst. 8(4), 453–464 (2014)CrossRefGoogle Scholar
  6. 6.
    J. Hasler, B. Marr, Finding a roadmap to achieve large neuromorphic hardware systems. Front. Neurosci. 7(7), 1–29 (2013)Google Scholar
  7. 7.
    S. Furber, Large-scale neuromorphic computing systems. J. Neural Eng. 13(5), 51001 (2016)CrossRefGoogle Scholar
  8. 8.
    B. Pakkenberg, D. Pelvig, L. Marner, M.J. Bundgaard, H.J.G. Gundersen, J.R. Nyengaard, L. Regeur, Aging and the human neocortex. Exp. Gerontol. 38(1–2), 95–99 (2003)CrossRefGoogle Scholar
  9. 9.
    J. Hsu, IBM’s new brain. IEEE Spectr. 51(10), 17–19 (2014)CrossRefGoogle Scholar
  10. 10.
    IBM, Lawrence Livermore National Laboratory and IBM Collaborate to Build Brain-Inspired Supercomputer, (2016), Available http://www-03.ibm.com/press/us/en/pressrelease/49424.wss. Accessed 25 Aug 2016
  11. 11.
    S. Scholze, H. Eisenreich, S. Hoppner, G. Ellguth, S. Henker, M. Ander, S. Hanzsche, J. Partzsch, C. Mayr, R. Schuffny, A 32 GBit/s communication SoC for a waferscale neuromorphic system. Integr. VLSI J. 45(1), 61–75 (2012)CrossRefGoogle Scholar
  12. 12.
    B.V. Benjamin, P. Gao, E. McQuinn, S. Choudhary, A.R. Chandrasekaran, J.M. Bussat, R. Alvarez-Icaza, J.V. Arthur, P.A. Merolla, K. Boahen, Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102(5), 699–716 (2014)CrossRefGoogle Scholar
  13. 13.
    S.B. Furber, F. Galluppi, S. Temple, L.A. Plana, The SpiNNaker project. Proc. IEEE 102(5), 652–665 (2014)CrossRefGoogle Scholar
  14. 14.
    C. Eliasmith, T.C. Stewart, X. Choo, T. Bekolay, T. Dewolf, Y. Tang, D. Rasmussen, A large-scale model of the functioning brain. Science (80-. ) 338, 1202–1205 (2012)CrossRefGoogle Scholar
  15. 15.
    T.M. Wong, R. Preissl, P. Datta, M.D. Flickner, R. Singh, S.K. Esser, E. McQuinn, R. Appuswamy, W.P. Risk, H.D. Simon, D.S. Modha, IBM internal Research Report 10 14. 10502, 1–3 (2012)Google Scholar
  16. 16.
    P.a. Merolla, J.V. Arthur, R. Alvarez-Icaza, A.S. Cassidy, J. Sawada, F. Akopyan, B.L. Jackson, N. Imam, C. Guo, Y. Nakamura, B. Brezzo, I. Vo, S.K. Esser, R. Appuswamy, B. Taba, A. Amir, M.D. Flickner, W.P. Risk, R. Manohar, D.S. Modha, A million spiking-neuron integrated circuit with a scalable communication network and interface. Science (80-. ). 345(6197), 668–673 (2014)CrossRefGoogle Scholar
  17. 17.
    S. Moradi, N. Imam, R. Manohar, G. Indiveri, A memory-efficient routing method for large-scale spiking neural networks. Eur. Conf. Circuit Theory Des. 2013, 1–4 (2013)Google Scholar
  18. 18.
    G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    T. Serrano-gotarredona, T. Prodromakis, B. Linares-Barranco, A proposal for hybrid memristor-CMOS spiking neuromorphic learning systems. IEEE Circ. Syst. Magaz. 74–88, 2nd quarter (2013)Google Scholar
  20. 20.
    S.H. Jo, T. Chang, I. Ebong, B.B. Bhadviya, P. Mazumder, W. Lu, Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10(4), 1297–1301 (2010)CrossRefGoogle Scholar
  21. 21.
    K.A. Boahen, Point-to-point connectivity between neuromorphic chips using address events. IEEE Trans. Circuits Syst. II Analog Digit. Signal Process. 47(5), 416–434 (2000)CrossRefzbMATHGoogle Scholar
  22. 22.
    C. Gamrat, O. Bichler, D. Roclin, Memristive based device arrays combined with spike based coding can enable efficient implementations of embedded neuromorphic circuits. Tech. Dig. Int. Electron Devices Meet. IEDM 2016, 4.5.1–4.5.7 (2016)Google Scholar
  23. 23.
    G. Piccolboni, G. Molas, J.M. Portal, R. Coquand, M. Bocquet, D. Garbin, E. Vianello, C. Carabasse, V. Delaye, C. Pellissier, T. Magis, C. Cagli, M. Gely, O. Cueto, D. Deleruyelle, G. Ghibaudo, B. De Salvo, L. Perniola, Investigation of the potentialities of vertical resistive RAM (VRRAM) for neuromorphic applications. Tech. Dig. Int. Electron Devices Meet. IEDM 2016, 17.2.1–17.2.4 (2016)Google Scholar
  24. 24.
    G.W. Burr, P. Narayanan, R.M. Shelby, S. Sidler, I. Boybat, C. Di Nolfo, Y. Leblebici, Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: Comparative performance analysis (accuracy, speed, and power). Tech. Dig. Int. Electron Devices Meet. IEDM 2016(408), 4.4.1–4.4.4 (2016)Google Scholar
  25. 25.
    S. Kim, M. Ishii, S. Lewis, T. Perri, M. Brightsky, W. Kim, R. Jordan, G.W. Burr, N. Sosa, A. Ray, J. Han, C. Miller, K. Hosokawa, C. Lam, NVM Neuromorphic Core with 64k–cell (256-by-256) Phase Change Memory Synaptic Array with On-Chip Neuron Circuits for Continuous In-Situ Learning. (2015), pp. 443–446Google Scholar
  26. 26.
    D. Lee, J. Park, K. Moon, J. Jang, S. Park, M. Chu, J. Kim, J. Noh, M. Jeon, B.H. Lee, B. Lee, B.G. Lee, H. Hwang, Oxide based nanoscale analog synapse device for neural signal recognition system. Tech. Dig. - Int. Electron Devices Meet. IEDM 2016, 4.7.1–4.7.4 (2016)Google Scholar
  27. 27.
    S.B. Eryilmaz, D. Kuzum, S. Yu, H.S.P. Wong, Device and system level design considerations for analog-non-volatile-memory based neuromorphic architectures. Tech. Dig. Int. Electron Devices Meet. IEDM 2016, 4.1.1–4.1.4 (2016)Google Scholar
  28. 28.
    S. Yu, P.Y. Chen, Y. Cao, L. Xia, Y. Wang, H. Wu, Scaling-up resistive synaptic arrays for neuro-inspired architecture: Challenges and prospect. Tech. Dig. Int. Electron Devices Meet. IEDM 2016, 17.3.1–17.3.4 (2016)Google Scholar
  29. 29.
    Y. Tang, J.R. Nyengaard, D.M.G. De Groot, H.J.G. Gundersen, Total regional and global number of synapses in the human brain neocortex. Synapse 41(3), 258–273 (2001)CrossRefGoogle Scholar
  30. 30.
    G. Indiveri, F. Corradi, N. Qiao, Neuromorphic Architectures for Spiking Deep Neural Networks (IEEE IEDM intnl. conf., Washington DC, 2015), pp. 68–71Google Scholar
  31. 31.
    D. Vainbrand, R. Ginosar, Scalable network-on-chip architecture for configurable neural networks. Microprocess. Microsyst. 35(2), 152–166 (2011)CrossRefGoogle Scholar
  32. 32.
    K.K. Hidetomo Kobayashi, T. Ohmaru, S. Yoneda, Processor with 4.9-us Break-even Time in Power Gating Using Crystalline In-Ga-Zn-Oxide Transistor, in Cool Chips Conference (2013)Google Scholar
  33. 33.
    R. Perin, T.K. Berger, H. Markram, A synaptic organizing principle for cortical neuronal groups. Proc. Natl. Acad. Sci. U. S. A. 108(13), 5419–5424 (2011)CrossRefGoogle Scholar
  34. 34.
    R.B. Levy, A.D. Reyes, Spatial profile of excitatory and inhibitory synaptic connectivity in mouse primary auditory cortex. J. Neurosci. 32(16), 5609–5619 (2012)CrossRefGoogle Scholar
  35. 35.
    M. Beyeler, K.D. Carlson, T.S. Chou, N. Dutt, J.L. Krichmar, CARLsim 3: A user-friendly and highly optimized library for the creation of neurobiologically detailed spiking neural networks. Proc. Int. Jt. Conf. Neural Netw. 2015 (2015)Google Scholar
  36. 36.
    ARM, AMBA-lite. Available https://www.arm.com/products/amba-open-specifications.php. Accessed 13 Sept 2016
  37. 37.
    A. Leroy, D. Milojevic, D. Verkest, F. Robert, F. Catthoor, Concepts and implementation of spatial division multiplexing for guaranteed throughput in networks-on-chip. IEEE Trans. Comput. 57(9), 1182–1195 (2008)MathSciNetCrossRefGoogle Scholar
  38. 38.
    K. Heyrman, A. Papanikolaou, F. Gatthoor, P. Veelaert, W. Philips, Control for power gating of wires. IEEE Trans. Very Large Scale Integr. Syst. 18(9), 1287–1300 (2010)CrossRefGoogle Scholar
  39. 39.
    F. Catthoor, P. Raghavan, A. Lambrechts, M. Jayapala, A. Kritikakou, J. Absar, Ultra-low Energy Domain-Specific Instruction-set Processors (Springer, Dordrecht, 2010)CrossRefGoogle Scholar
  40. 40.
    S.V. Gheorghita, F. Vandeputte, K. De Bosschere, M. Palkovic, J. Hamers, A. Vandecappelle, S. Mamagkakis, T. Basten, L. Eeckhout, H. Corporaal, F. Catthoor, System-scenario-based design of dynamic embedded systems. ACM Trans. Des. Autom. Electron. Syst. 14(1), 1–45 (2009)CrossRefGoogle Scholar
  41. 41.
    M. Jayapala, F. Barat, T.V. der Aa, F. Catthoor, G. Deconinck, H. Corporaal, Clustered L0 buffer organisation for low energy embedded processors. IEEE Trans. Comput. 54(6) (2005)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Francky Catthoor
    • 1
    Email author
  • Srinjoy Mitra
    • 2
  • Anup Das
    • 1
  • Siebren Schaafsma
    • 1
  1. 1.IMECLeuvenBelgium
  2. 2.School of EngineeringUniversity of GlasgowGlasgowUK

Personalised recommendations