Abstract
Spike neural networks are inspired by animal brains, and outperform traditional neural networks on complicated tasks. However, spike neural networks are usually used on a large scale, and they cannot be computed on commercial, off-the-shelf computers. A parallel architecture is proposed and developed for discrete-event simulations of spike neural networks. Furthermore, mechanisms for both parallelism degree estimation and dynamic load balance are emphasized with theoretical and computational analysis. Simulation results show the effectiveness of the proposed parallelized spike neural network system and its corresponding support components.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Goddard N H, Hood G. Parallel Genesis for Large-scale Modeling. New York: Plenum Press, 1997
Bower J M, Beeman D, Wylde A M. The book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System. New York: Springer-Verlag, 1998
Migliore M. Parallel network simulations with NEURON. J Comput Neurosci, 2006, 21(2): 119–129
Carnevale N T, Hines M L. The NEURON Book. Cambridge: Cambridge University Press, 2006
Delorme A. SpikeNET: A simulator for modeling large networks of integrate and fire neurons. Neurocomputing, 1999, 26: 989–996
Davison A P. PyNN: a common interface for neuronal network simulators. Front Neuroinf, 2008, 2: 1858–1876
Goodman D, Brette R. Brian: a simulator for spiking neural networks in Python. Front Neuroinf, 2008, 2: 138–147
Goodman D F, Brette R. The brian simulator. Front Neurosci, 2009, 3(2): 192–201
Gewaltig M O, Diesmann M. NEST (neural simulation tool). Scholarpedia, 2007, 2(4): 1430–1437
Eppler J M. PyNEST: a convenient interface to the NEST simulator. Frontiers Neuroinf, 2008. 2: 372–388
Alamdari A S. Biological Neural Networks (BNNs) Toolbox for MATLAB: User Guide. General Public License, 2004
Markram H. The blue brain project. Nature Rev Neurosci, 2006, 7(2): 153–160
Meier K. The FACETS Project. Wikipedia, 2005, 3(3): 202–215
Brette R. Simulation of networks of spiking neurons: A review of tools and strategies. J Comput Neurosci, 2007, 23(3): 349–398
Buyya, R. High Performance Cluster Computing: Architectures and Systems, vol 1. Upper SaddleRiver, NJ: Prentice Hall, 1999
Strey A. A comparison of OpenMP and MPI for neural network simulations on a SunFire 6800. Adv Parallel Comput, 2004. 13: 201–208
Morrison A. Advancing the boundaries of high-connectivity network simulation with distributed computing. Neural Comput, 2005, 17(8): 1776–1801
Nageswaran J M, Dutt N. A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Networks, 2009, 22(5): 791–800
Cheung K, Schultz S R, Leong P H. A parallel spiking neural network simulator. International Conference on Field-Programmable Technology. Sydney: ACM, 2009. 78–91
Hennessy J L, Patterson D A, Goldberg D. Computer Architecture: A Quantitative Approach. San Francisco: Morgan Kaufmann. 2003
Pacheco P, Camperi M, Uchino T. Parallel neurosys: A system for the simulation of very large networks of biologically accurate neurons on parallel computers. Neurocomputing, 2000, 32: 1095–1102
Khan M M. SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor. IEEE International Joint Conference on Neural Networks. Hong Kong: IEEE, 2008. 2849–2855
Pecevski D, Natschl T, Schuch K. PCSIM: a parallel simulation environment for neural circuits fully integrated with Python. Front Neuroinf, 2009, 3: 113–124
Plesser H. Efficient parallel simulation of large-scale neuronal networks on clusters of multiprocessor computers. European Conference on Parallel and Distributed Computing. Rennes: IRISA, 2007. 672–681
Wilson E C, Goodman P H, Harris F C. Implementation of a biologically realistic parallel neocortical-neural network simulator. Proceedings of the 10th SIAM Conference on Parallel Process for Scientific Computing. Philadelphia: SIAM, 2001. 12–23
Nageswaran J M. Efficient simulation of large-scale spiking neural networks using CUDA graphics processors. International Joint Conference on Neural Networks. Atlanta: IEEE, 2009. 287–295
Hines M L, Eichner H, Schurmann F, Neuron splitting in compute-bound parallel network simulations enables runtime scaling with twice as many processors. J Comput Neurosci, 2008, 25(1): 203–210
Lobb C J. Parallel event-driven neural network simulations using the Hodgkin-Huxley neuron model. Workshop on Principles of Advanced and Distributed Simulation. Monterey: IEEE, 2005. 102–108
Mohraz K, Schott U, Pauly M. Parallel simulation of pulse-coded neural networks. IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics. Berlin: IEEE, 1997. 23–28
Izhikevich E M. Simple model of spiking neurons. IEEE T Neural Network, 2004, 14(6): 1569–1572
Vreeken J. Spiking neural networks, an introduction. Institute for Information and Computing Sciences, Utrecht University Technical Report UU-CS-2003-008, 2002
Gerstner W, Kistler W M. Spiking neuron models. vol 15. Cambridge, UK: Cambridge University Press, 2002
Yang X J. The TianHe-1A supercomputer: its hardware and software. J Comput Sci Technol, 2011. 26(3): 344–351
Brette R. Exact simulation of integrate-and-fire models with exponential currents. Neural Comput, 2007, 19(10): 2604–2609
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tang, Y., Zhang, B., Wu, J. et al. Parallel architecture and optimization for discrete-event simulation of spike neural networks. Sci. China Technol. Sci. 56, 509–517 (2013). https://doi.org/10.1007/s11431-012-5084-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11431-012-5084-2