Advertisement

Neural Processing Letters

, Volume 38, Issue 2, pp 131–153 | Cite as

Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network

  • Sandeep PandeEmail author
  • Fearghal Morgan
  • Seamus Cawley
  • Tom Bruintjes
  • Gerard Smit
  • Brian McGinley
  • Snaider Carrillo
  • Jim Harkin
  • Liam McDaid
Article

Abstract

Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 % of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 % for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications

Keywords

Modular neural networks (MNN) Spiking neural networks (SNN)  Synaptic connectivity Network on chip (NoC) EMBRACE 

Notes

Acknowledgments

This research is supported by International Centre for Graduate Education in Micro and Nano-Engineering (ICGEE), Irish Research Council for Science, Engineering and Technology (IRCSET), Science Foundation, Ireland (Grant No. 07/SRC/I1169) and Xilinx University Programme.

References

  1. 1.
    Haykin SS (1999) Neural networks: a comprehensive foundation, vol 13. Prentice Hall, New JerseyzbMATHGoogle Scholar
  2. 2.
    Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671CrossRefGoogle Scholar
  3. 3.
    Wulfram G, Werner MK (2002) Spiking neuron models. Cambridge University Press, CambridgezbMATHGoogle Scholar
  4. 4.
    Maguire LP, McGinnity TM, Glackin B, Ghani A, Belatreche A, Harkin J (2007) Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing 71(1–3):13–29CrossRefGoogle Scholar
  5. 5.
    Jim H, Fearghal M, Liam M, Steve H, Brian M, Seamus C (2009) A reconfigurable and biologically inspired paradigm for computation using network-on-chip and spiking neural networks. Int J Reconfig Comput 2009:1–13Google Scholar
  6. 6.
    Seamus C, Fearghal M, Brian M, Sandeep P, Liam M, Snaider C, Jim H (2011) Hardware spiking neural network prototyping and application. Genet Program Evolvable Mach 12:257–280CrossRefGoogle Scholar
  7. 7.
    Dmitri V, Ran G (2011) Scalable network-on-chip architecture for configurable neural networks. Microprocess Microsyst 35(2):152–166 (special issue on network-on-chip architectures and design methodologies)Google Scholar
  8. 8.
    Morgan F, Cawley S, McGinley B, Pande S, McDaid LJ, Glackin B, Maher J, Harkin J (2009) Exploring the evolution of NoC-based spiking neural networks on FPGAs. In: International conference on field-programmable technology, 2009. IEEE, Sydney, pp 300–303Google Scholar
  9. 9.
    Daniel NO, Scott W, Michael S (1993) Modular learning. MIT Press, Cambridge, pp 369–377Google Scholar
  10. 10.
    Sandeep P, Fearghal M, Seamus C, McGinley B, Jim H, Snaider C, McDaid L (2011) Addressing the hardware resource requirements of network-on-chip based neural architectures. In: International conference on neural computation theory and applications. NCTA, ParisGoogle Scholar
  11. 11.
    Nate K, Risto M (2008) Evolving neural networks for fractured domains. In: Proceedings of the 10th annual conference on genetic and evolutionary computation, GECCO ’08. ACM, New York, pp 1405–1412Google Scholar
  12. 12.
    Harkin J, Morgan F, Hall S, Dudek P, Dowrick T, McDaid L (2008) Reconfigurable platforms and the challenges for large-scale implementations of spiking neural networks. In: International conference on field programmable logic and applications, 2008. IEEE, Heidelberg, pp 483–486Google Scholar
  13. 13.
    Yajie C, Hall S, McDaid L, Buiu O, Kelly P (2006) A solid state neuron for the realisation of highly scaleable third generation neural networks. In: 8th International conference on solid-state and integrated circuit technology, 2006. ICSICT, Beijing, pp 1071–1073Google Scholar
  14. 14.
    Furber S, Brown A (2009) Biologically-inspired massively-parallel architectures-computing beyond a million processors. In: Ninth international conference on application of concurrency to system design, 2009. ACSD, Augsburg, pp 3–12Google Scholar
  15. 15.
    Andres U, Ca Pea-Reyes, Sanchez E (2005) An FPGA platform for on-line topology exploration of spiking neural networks. Microprocess Microsyst 29(5):211–223CrossRefGoogle Scholar
  16. 16.
    Pearson MJ, Pipe AG, Mitchinson B, Gurney K, Melhuish C, Gilhespy I, Nibouche M (2007) Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach. IEEE Trans Neural Netw 18(5):1472–1487CrossRefGoogle Scholar
  17. 17.
    Ros E, Ortigosa EM, Agis R, Carrillo R, Arnold M (2006) Real-time computing platform for spiking neurons (RT-spike). IEEE Trans Neural Netw 17(4):1050–1063CrossRefGoogle Scholar
  18. 18.
    Vogelstein RJ, Mallik U, Vogelstein JT, Cauwenberghs G (2007) Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. IEEE Trans Neural Netw 18(1):253–265CrossRefGoogle Scholar
  19. 19.
    Ehrlich M, Mayr C, Eisenreich H, Henker S, Srowig A, Grubl A, Schemmel J, Schuffny R (2007) Wafer-scale VLSI implementations of pulse coupled neural networks. Proceedings of the international conference on sensors, circuits and instrumentation systems. CEA, AcapulcoGoogle Scholar
  20. 20.
    Glackin B, McGinnity TM, Maguire LP, Wu QX, Belatreche A (2005) A novel approach for the implementation of large scale spiking neural networks on FPGA hardware. In: Computational intelligence and bioinspired systems. CIBS, Barcelona, pp 552–563Google Scholar
  21. 21.
    Schemmel J, Fieres J, Meier K (2008) Wafer-scale integration of analog neural networks. In: IEEE international joint conference on neural networks (2008) IEEE world congress on computational intelligence. IEEE, Hong Kong, pp 431–438Google Scholar
  22. 22.
    Chen Y, Hall S, McDaid L, Buiu O, Kelly P (2006) On the design of a low power compact spiking neuron cell based on charge-coupled synapses. In: International joint conference on neural networks, 2006. IJCNN, Brisbane, pp 1511–1517Google Scholar
  23. 23.
    Chen Y, McDaid L, Hall S, Kelly P (2008) A programmable facilitating synapse device. In: IEEE international joint conference on neural networks (2008) IEEE world congress on computational intelligence. IJCNN, Barcelona, pp 1615–1620Google Scholar
  24. 24.
    Furber S, Temple S, Brown A (2006) On-chip and inter-chip networks for modeling large-scale neural systems. In: Proceedings of 2006 IEEE international symposium on circuits and Systems, 2006. ISCAS, Kos, p 4Google Scholar
  25. 25.
    Emery R, Yakovlev A, Chester G (2009) Connection-centric network for spiking neural networks. In: 3rd ACM/IEEE international symposium on networks-on-chip (2009) NoCS 2009. IEEE, San Diego, pp 144–152Google Scholar
  26. 26.
    Pande S, Morgan F, Cawley S, McGinley B, Carrillo S, Harkin J, McDaid L (2010) EMBRACE-SysC for analysis of NoC-based spiking neural network architectures. In: System on chip international symposium on (SoC), 2010. SoC-2010, Tampere, Finland, pp 139–145Google Scholar
  27. 27.
    Johannes S, Wieringa BM, Matzke M, Mnte TF (1996) Hierarchical visual stimuli: electrophysiological evidence for separate left hemispheric global and local processing mechanisms in humans. Neurosci Lett 210(2):111–114CrossRefGoogle Scholar
  28. 28.
    Van Essen DC, Anderson CH, Felleman DJ (1992) Information processing in the primate visual system: an integrated systems perspective. Science 255(5043):419–423CrossRefGoogle Scholar
  29. 29.
    Binzegger T, Douglas RJ, Martin KAC (2007) Stereotypical bouton clustering of individual neurons in cat primary visual cortex. J Neurosci 27(45):12242–12254CrossRefGoogle Scholar
  30. 30.
    Happel BLM, Murre JMJ (1994) Design and evolution of modular neural network architectures. Neural Netw 7(6–7):985–1004CrossRefGoogle Scholar
  31. 31.
    Auda G, Kamel MS (1999) Modular neural networks a survey. Int J Neural Syst 9(2):129–151CrossRefGoogle Scholar
  32. 32.
    Ronco GP (1995) Modular neural networks: a state of the art. Rapport technique CSC95026, vol 1. Center of System and Control, University of Glasgow, Lanarkshire, pp 1–22Google Scholar
  33. 33.
    Sheng-uei G, Shanchun L, Tan SK (2004) Neural network task decomposition based on output partitioning. J Instit Eng Singap 44:78–89Google Scholar
  34. 34.
    Bao-Liang L, Ito M (1999) Task decomposition and module combination based on class relations: a modular neural network for pattern classification. IEEE Trans Neural Netw 10(5):1244–1256Google Scholar
  35. 35.
    Thangavelautham J, D’Eleuterio GMT (2004) A neuroevolutionary approach to emergent task decomposition. Proceedings of 8th parallel problem solving from nature. Springer, Heidelberg, pp 991–1000Google Scholar
  36. 36.
    Khare VR, Yao Xin, Sendhoff B, Jin Yaochu, Wersing H, (2005) Co-evolutionary modular neural networks for automatic problem decomposition. In: The (2005) IEEE congress on evolutionary computation, vol 3. CES, Edinburgh, pp 2691–2698Google Scholar
  37. 37.
    Santos JM, Alexandre LA, de Sa JM (2006) Modular neural network task decomposition via entropic clustering. In: Sixth international conference on intelligent systems design and applications (2006) vol 1. IEEE Computer Society Press, Jinan, pp 62–67Google Scholar
  38. 38.
    Pizzuti C (2012) A multiobjective genetic algorithm to find communities in complex networks. IEEE Trans Evolut Comput 16(3):418–430CrossRefGoogle Scholar
  39. 39.
    Brooks R (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom 2(1):14–23CrossRefGoogle Scholar
  40. 40.
    Cawley S, Pande S, McDaid L, McGinley B, Morgan F (2009) Memory efficient storage of reconfigurable topology information in network-on-chip based spiking neural networks, internal report. National University of Ireland, Galway, Bio-Inspired Electronics and Reconfigurable SystemsGoogle Scholar
  41. 41.
    Maher J, McGinley B, Rocke P, Morgan F (2006) Intrinsic hardware evolution of neural networks in reconfigurable analogue and digital devices. In: 14th Annual IEEE symposium on field-programmable custom computing machines. FCCM, Seattle, pp 321–322Google Scholar
  42. 42.
    Vaughan R (2008) Massively multi-robot simulation in stage. Swarm Intell 2:189–208CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Sandeep Pande
    • 1
    Email author
  • Fearghal Morgan
    • 1
  • Seamus Cawley
    • 1
  • Tom Bruintjes
    • 2
  • Gerard Smit
    • 2
  • Brian McGinley
    • 1
  • Snaider Carrillo
    • 3
  • Jim Harkin
    • 3
  • Liam McDaid
    • 3
  1. 1.Bio-Inspired Electronics and Reconfigurable ComputingNational University of IrelandGalwayIreland
  2. 2.Computer Architecture for Embedded SystemsUniversity of TwenteEnschedeThe Netherlands
  3. 3.Intelligent Systems Research CentreUniversity of UlsterDerryNorthern Ireland, UK

Personalised recommendations