Skip to main content
SpringerLink
Log in

Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 % of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 % for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. EMulating Biologically-inspiRed ArChitectures in hardwarE.

References

  1. Haykin SS (1999) Neural networks: a comprehensive foundation, vol 13. Prentice Hall, New Jersey

    MATH  Google Scholar 

  2. Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671

    Article  Google Scholar 

  3. Wulfram G, Werner MK (2002) Spiking neuron models. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  4. Maguire LP, McGinnity TM, Glackin B, Ghani A, Belatreche A, Harkin J (2007) Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing 71(1–3):13–29

    Article  Google Scholar 

  5. Jim H, Fearghal M, Liam M, Steve H, Brian M, Seamus C (2009) A reconfigurable and biologically inspired paradigm for computation using network-on-chip and spiking neural networks. Int J Reconfig Comput 2009:1–13

    Google Scholar 

  6. Seamus C, Fearghal M, Brian M, Sandeep P, Liam M, Snaider C, Jim H (2011) Hardware spiking neural network prototyping and application. Genet Program Evolvable Mach 12:257–280

    Article  Google Scholar 

  7. Dmitri V, Ran G (2011) Scalable network-on-chip architecture for configurable neural networks. Microprocess Microsyst 35(2):152–166 (special issue on network-on-chip architectures and design methodologies)

    Google Scholar 

  8. Morgan F, Cawley S, McGinley B, Pande S, McDaid LJ, Glackin B, Maher J, Harkin J (2009) Exploring the evolution of NoC-based spiking neural networks on FPGAs. In: International conference on field-programmable technology, 2009. IEEE, Sydney, pp 300–303

  9. Daniel NO, Scott W, Michael S (1993) Modular learning. MIT Press, Cambridge, pp 369–377

    Google Scholar 

  10. Sandeep P, Fearghal M, Seamus C, McGinley B, Jim H, Snaider C, McDaid L (2011) Addressing the hardware resource requirements of network-on-chip based neural architectures. In: International conference on neural computation theory and applications. NCTA, Paris

  11. Nate K, Risto M (2008) Evolving neural networks for fractured domains. In: Proceedings of the 10th annual conference on genetic and evolutionary computation, GECCO ’08. ACM, New York, pp 1405–1412

  12. Harkin J, Morgan F, Hall S, Dudek P, Dowrick T, McDaid L (2008) Reconfigurable platforms and the challenges for large-scale implementations of spiking neural networks. In: International conference on field programmable logic and applications, 2008. IEEE, Heidelberg, pp 483–486

  13. Yajie C, Hall S, McDaid L, Buiu O, Kelly P (2006) A solid state neuron for the realisation of highly scaleable third generation neural networks. In: 8th International conference on solid-state and integrated circuit technology, 2006. ICSICT, Beijing, pp 1071–1073

  14. Furber S, Brown A (2009) Biologically-inspired massively-parallel architectures-computing beyond a million processors. In: Ninth international conference on application of concurrency to system design, 2009. ACSD, Augsburg, pp 3–12

  15. Andres U, Ca Pea-Reyes, Sanchez E (2005) An FPGA platform for on-line topology exploration of spiking neural networks. Microprocess Microsyst 29(5):211–223

    Article  Google Scholar 

  16. Pearson MJ, Pipe AG, Mitchinson B, Gurney K, Melhuish C, Gilhespy I, Nibouche M (2007) Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach. IEEE Trans Neural Netw 18(5):1472–1487

    Article  Google Scholar 

  17. Ros E, Ortigosa EM, Agis R, Carrillo R, Arnold M (2006) Real-time computing platform for spiking neurons (RT-spike). IEEE Trans Neural Netw 17(4):1050–1063

    Article  Google Scholar 

  18. Vogelstein RJ, Mallik U, Vogelstein JT, Cauwenberghs G (2007) Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. IEEE Trans Neural Netw 18(1):253–265

    Article  Google Scholar 

  19. Ehrlich M, Mayr C, Eisenreich H, Henker S, Srowig A, Grubl A, Schemmel J, Schuffny R (2007) Wafer-scale VLSI implementations of pulse coupled neural networks. Proceedings of the international conference on sensors, circuits and instrumentation systems. CEA, Acapulco

  20. Glackin B, McGinnity TM, Maguire LP, Wu QX, Belatreche A (2005) A novel approach for the implementation of large scale spiking neural networks on FPGA hardware. In: Computational intelligence and bioinspired systems. CIBS, Barcelona, pp 552–563

  21. Schemmel J, Fieres J, Meier K (2008) Wafer-scale integration of analog neural networks. In: IEEE international joint conference on neural networks (2008) IEEE world congress on computational intelligence. IEEE, Hong Kong, pp 431–438

  22. Chen Y, Hall S, McDaid L, Buiu O, Kelly P (2006) On the design of a low power compact spiking neuron cell based on charge-coupled synapses. In: International joint conference on neural networks, 2006. IJCNN, Brisbane, pp 1511–1517

  23. Chen Y, McDaid L, Hall S, Kelly P (2008) A programmable facilitating synapse device. In: IEEE international joint conference on neural networks (2008) IEEE world congress on computational intelligence. IJCNN, Barcelona, pp 1615–1620

  24. Furber S, Temple S, Brown A (2006) On-chip and inter-chip networks for modeling large-scale neural systems. In: Proceedings of 2006 IEEE international symposium on circuits and Systems, 2006. ISCAS, Kos, p 4

  25. Emery R, Yakovlev A, Chester G (2009) Connection-centric network for spiking neural networks. In: 3rd ACM/IEEE international symposium on networks-on-chip (2009) NoCS 2009. IEEE, San Diego, pp 144–152

  26. Pande S, Morgan F, Cawley S, McGinley B, Carrillo S, Harkin J, McDaid L (2010) EMBRACE-SysC for analysis of NoC-based spiking neural network architectures. In: System on chip international symposium on (SoC), 2010. SoC-2010, Tampere, Finland, pp 139–145

  27. Johannes S, Wieringa BM, Matzke M, Mnte TF (1996) Hierarchical visual stimuli: electrophysiological evidence for separate left hemispheric global and local processing mechanisms in humans. Neurosci Lett 210(2):111–114

    Article  Google Scholar 

  28. Van Essen DC, Anderson CH, Felleman DJ (1992) Information processing in the primate visual system: an integrated systems perspective. Science 255(5043):419–423

    Article  Google Scholar 

  29. Binzegger T, Douglas RJ, Martin KAC (2007) Stereotypical bouton clustering of individual neurons in cat primary visual cortex. J Neurosci 27(45):12242–12254

    Article  Google Scholar 

  30. Happel BLM, Murre JMJ (1994) Design and evolution of modular neural network architectures. Neural Netw 7(6–7):985–1004

    Article  Google Scholar 

  31. Auda G, Kamel MS (1999) Modular neural networks a survey. Int J Neural Syst 9(2):129–151

    Article  Google Scholar 

  32. Ronco GP (1995) Modular neural networks: a state of the art. Rapport technique CSC95026, vol 1. Center of System and Control, University of Glasgow, Lanarkshire, pp 1–22

  33. Sheng-uei G, Shanchun L, Tan SK (2004) Neural network task decomposition based on output partitioning. J Instit Eng Singap 44:78–89

    Google Scholar 

  34. Bao-Liang L, Ito M (1999) Task decomposition and module combination based on class relations: a modular neural network for pattern classification. IEEE Trans Neural Netw 10(5):1244–1256

    Google Scholar 

  35. Thangavelautham J, D’Eleuterio GMT (2004) A neuroevolutionary approach to emergent task decomposition. Proceedings of 8th parallel problem solving from nature. Springer, Heidelberg, pp 991–1000

  36. Khare VR, Yao Xin, Sendhoff B, Jin Yaochu, Wersing H, (2005) Co-evolutionary modular neural networks for automatic problem decomposition. In: The (2005) IEEE congress on evolutionary computation, vol 3. CES, Edinburgh, pp 2691–2698

  37. Santos JM, Alexandre LA, de Sa JM (2006) Modular neural network task decomposition via entropic clustering. In: Sixth international conference on intelligent systems design and applications (2006) vol 1. IEEE Computer Society Press, Jinan, pp 62–67

  38. Pizzuti C (2012) A multiobjective genetic algorithm to find communities in complex networks. IEEE Trans Evolut Comput 16(3):418–430

    Article  Google Scholar 

  39. Brooks R (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom 2(1):14–23

    Article  Google Scholar 

  40. Cawley S, Pande S, McDaid L, McGinley B, Morgan F (2009) Memory efficient storage of reconfigurable topology information in network-on-chip based spiking neural networks, internal report. National University of Ireland, Galway, Bio-Inspired Electronics and Reconfigurable Systems

  41. Maher J, McGinley B, Rocke P, Morgan F (2006) Intrinsic hardware evolution of neural networks in reconfigurable analogue and digital devices. In: 14th Annual IEEE symposium on field-programmable custom computing machines. FCCM, Seattle, pp 321–322

  42. Vaughan R (2008) Massively multi-robot simulation in stage. Swarm Intell 2:189–208

    Article  Google Scholar 

Download references

Acknowledgments

This research is supported by International Centre for Graduate Education in Micro and Nano-Engineering (ICGEE), Irish Research Council for Science, Engineering and Technology (IRCSET), Science Foundation, Ireland (Grant No. 07/SRC/I1169) and Xilinx University Programme.

Author information

Authors and Affiliations

  1. Bio-Inspired Electronics and Reconfigurable Computing, National University of Ireland, Galway, Ireland

    Sandeep Pande, Fearghal Morgan, Seamus Cawley & Brian McGinley

  2. Computer Architecture for Embedded Systems, University of Twente, Enschede, The Netherlands

    Tom Bruintjes & Gerard Smit

  3. Intelligent Systems Research Centre, University of Ulster, Derry, Northern Ireland, UK

    Snaider Carrillo, Jim Harkin & Liam McDaid

Authors
  1. Sandeep Pande
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fearghal Morgan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seamus Cawley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tom Bruintjes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gerard Smit
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brian McGinley
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Snaider Carrillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jim Harkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Liam McDaid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sandeep Pande.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pande, S., Morgan, F., Cawley, S. et al. Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network. Neural Process Lett 38, 131–153 (2013). https://doi.org/10.1007/s11063-012-9274-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9274-5

Keywords

Search

Navigation