Science China Technological Sciences

, Volume 56, Issue 2, pp 509–517 | Cite as

Parallel architecture and optimization for discrete-event simulation of spike neural networks

  • YuHua Tang
  • BaiDa Zhang
  • JunJie Wu
  • TianJiang Hu
  • Jing Zhou
  • FuDong Liu
Article

Abstract

Spike neural networks are inspired by animal brains, and outperform traditional neural networks on complicated tasks. However, spike neural networks are usually used on a large scale, and they cannot be computed on commercial, off-the-shelf computers. A parallel architecture is proposed and developed for discrete-event simulations of spike neural networks. Furthermore, mechanisms for both parallelism degree estimation and dynamic load balance are emphasized with theoretical and computational analysis. Simulation results show the effectiveness of the proposed parallelized spike neural network system and its corresponding support components.

Keywords

spike neural network discrete event simulation intelligent parallelization framework 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Goddard N H, Hood G. Parallel Genesis for Large-scale Modeling. New York: Plenum Press, 1997Google Scholar
  2. 2.
    Bower J M, Beeman D, Wylde A M. The book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System. New York: Springer-Verlag, 1998Google Scholar
  3. 3.
    Migliore M. Parallel network simulations with NEURON. J Comput Neurosci, 2006, 21(2): 119–129MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Carnevale N T, Hines M L. The NEURON Book. Cambridge: Cambridge University Press, 2006CrossRefGoogle Scholar
  5. 5.
    Delorme A. SpikeNET: A simulator for modeling large networks of integrate and fire neurons. Neurocomputing, 1999, 26: 989–996CrossRefGoogle Scholar
  6. 6.
    Davison A P. PyNN: a common interface for neuronal network simulators. Front Neuroinf, 2008, 2: 1858–1876Google Scholar
  7. 7.
    Goodman D, Brette R. Brian: a simulator for spiking neural networks in Python. Front Neuroinf, 2008, 2: 138–147CrossRefGoogle Scholar
  8. 8.
    Goodman D F, Brette R. The brian simulator. Front Neurosci, 2009, 3(2): 192–201CrossRefGoogle Scholar
  9. 9.
    Gewaltig M O, Diesmann M. NEST (neural simulation tool). Scholarpedia, 2007, 2(4): 1430–1437CrossRefGoogle Scholar
  10. 10.
    Eppler J M. PyNEST: a convenient interface to the NEST simulator. Frontiers Neuroinf, 2008. 2: 372–388Google Scholar
  11. 11.
    Alamdari A S. Biological Neural Networks (BNNs) Toolbox for MATLAB: User Guide. General Public License, 2004Google Scholar
  12. 12.
    Markram H. The blue brain project. Nature Rev Neurosci, 2006, 7(2): 153–160MathSciNetCrossRefGoogle Scholar
  13. 13.
    Meier K. The FACETS Project. Wikipedia, 2005, 3(3): 202–215Google Scholar
  14. 14.
    Brette R. Simulation of networks of spiking neurons: A review of tools and strategies. J Comput Neurosci, 2007, 23(3): 349–398MathSciNetCrossRefGoogle Scholar
  15. 15.
    Buyya, R. High Performance Cluster Computing: Architectures and Systems, vol 1. Upper SaddleRiver, NJ: Prentice Hall, 1999Google Scholar
  16. 16.
    Strey A. A comparison of OpenMP and MPI for neural network simulations on a SunFire 6800. Adv Parallel Comput, 2004. 13: 201–208CrossRefGoogle Scholar
  17. 17.
    Morrison A. Advancing the boundaries of high-connectivity network simulation with distributed computing. Neural Comput, 2005, 17(8): 1776–1801MATHCrossRefGoogle Scholar
  18. 18.
    Nageswaran J M, Dutt N. A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Networks, 2009, 22(5): 791–800CrossRefGoogle Scholar
  19. 19.
    Cheung K, Schultz S R, Leong P H. A parallel spiking neural network simulator. International Conference on Field-Programmable Technology. Sydney: ACM, 2009. 78–91Google Scholar
  20. 20.
    Hennessy J L, Patterson D A, Goldberg D. Computer Architecture: A Quantitative Approach. San Francisco: Morgan Kaufmann. 2003Google Scholar
  21. 21.
    Pacheco P, Camperi M, Uchino T. Parallel neurosys: A system for the simulation of very large networks of biologically accurate neurons on parallel computers. Neurocomputing, 2000, 32: 1095–1102CrossRefGoogle Scholar
  22. 22.
    Khan M M. SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor. IEEE International Joint Conference on Neural Networks. Hong Kong: IEEE, 2008. 2849–2855Google Scholar
  23. 23.
    Pecevski D, Natschl T, Schuch K. PCSIM: a parallel simulation environment for neural circuits fully integrated with Python. Front Neuroinf, 2009, 3: 113–124Google Scholar
  24. 24.
    Plesser H. Efficient parallel simulation of large-scale neuronal networks on clusters of multiprocessor computers. European Conference on Parallel and Distributed Computing. Rennes: IRISA, 2007. 672–681Google Scholar
  25. 25.
    Wilson E C, Goodman P H, Harris F C. Implementation of a biologically realistic parallel neocortical-neural network simulator. Proceedings of the 10th SIAM Conference on Parallel Process for Scientific Computing. Philadelphia: SIAM, 2001. 12–23Google Scholar
  26. 26.
    Nageswaran J M. Efficient simulation of large-scale spiking neural networks using CUDA graphics processors. International Joint Conference on Neural Networks. Atlanta: IEEE, 2009. 287–295Google Scholar
  27. 27.
    Hines M L, Eichner H, Schurmann F, Neuron splitting in compute-bound parallel network simulations enables runtime scaling with twice as many processors. J Comput Neurosci, 2008, 25(1): 203–210MathSciNetCrossRefGoogle Scholar
  28. 28.
    Lobb C J. Parallel event-driven neural network simulations using the Hodgkin-Huxley neuron model. Workshop on Principles of Advanced and Distributed Simulation. Monterey: IEEE, 2005. 102–108Google Scholar
  29. 29.
    Mohraz K, Schott U, Pauly M. Parallel simulation of pulse-coded neural networks. IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics. Berlin: IEEE, 1997. 23–28Google Scholar
  30. 30.
    Izhikevich E M. Simple model of spiking neurons. IEEE T Neural Network, 2004, 14(6): 1569–1572MathSciNetCrossRefGoogle Scholar
  31. 31.
    Vreeken J. Spiking neural networks, an introduction. Institute for Information and Computing Sciences, Utrecht University Technical Report UU-CS-2003-008, 2002Google Scholar
  32. 32.
    Gerstner W, Kistler W M. Spiking neuron models. vol 15. Cambridge, UK: Cambridge University Press, 2002CrossRefGoogle Scholar
  33. 33.
    Yang X J. The TianHe-1A supercomputer: its hardware and software. J Comput Sci Technol, 2011. 26(3): 344–351CrossRefGoogle Scholar
  34. 34.
    Brette R. Exact simulation of integrate-and-fire models with exponential currents. Neural Comput, 2007, 19(10): 2604–2609MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Science China Press and Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • YuHua Tang
    • 1
    • 2
  • BaiDa Zhang
    • 1
    • 2
  • JunJie Wu
    • 1
    • 2
  • TianJiang Hu
    • 3
  • Jing Zhou
    • 1
    • 2
  • FuDong Liu
    • 1
    • 2
  1. 1.Department of Computer Science and Technology, School of ComputerNational University of Defense TechnologyChangshaChina
  2. 2.State Key Laboratory of High Performance ComputingNational University of Defense TechnologyChangshaChina
  3. 3.College of Mechatronic Engineering and AutomationNational University of Defense TechnologyChangshaChina

Personalised recommendations