Skip to main content

Parallel Brain Simulator: A Multi-scale and Parallel Brain-Inspired Neural Network Modeling and Simulation Platform

Abstract

The brain is naturally a parallel and distributed system. Reverse engineering a cognitive brain is considered to be a grand challenge. In this paper, we present the parallel brain simulator (PBS), a parallel and distributed platform for modeling the cognitive brain at multiple scales. Inspired by large-scale graph computation, PBS can be considered as a universal parallel execution engine, which is aimed at reducing the complexity of distributed programming and providing an easy to use programmable platform for computational neuroscientists and artificial intelligence researchers for modeling and simulation of large-scale neural networks. As illustrative examples and validations, three brain-inspired neural networks which are built on PBS are introduced, including the 1:1 human hippocampus network, the 1:1 mouse whole-brain network and the CASIA brain simulator built for cognitive robotics. We deploy PBS on both commodity clusters and supercomputers, and a scalable performance is achieved. In addition, we provide evaluations on the scalability and performance of both lumped synapse-based simulation and non-lumped synapse-based simulation with different data-graph distribution methods to show the effectiveness and usability of the PBS platform.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Notes

  1. https://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators.

  2. http://ilab.usc.edu/toolkit/home.shtml.

  3. https://grey.colorado.edu/emergent/.

  4. http://bii.ia.ac.cn/brainbo.html.

  5. http://hadoop.apache.org/zookeeper.

  6. http://www.rabbitmq.com.

  7. http://bii.ia.ac.cn/CASIA-Brain.htm.

  8. BrainBo: http://bii.ia.ac.cn/brainbo.html.

  9. http://sourceware.org/binutils/docs/gprof/.

References

  1. Aleksander I, Morton H. An introduction to neural computing. London: Chapman and Hall; 1990.

    Google Scholar 

  2. Ananthanarayanan R, Esser S, Simon H, Modha D. The cat is out of the bag: cortical simulations with 109 neurons and 1013 synapses. In: Proceedings of the SC conference on high performance networking and computing, 2009. 2009;1: p. 1–12 IEEE.

  3. Azevedo F, Carvalho L, Grinberg L, Farfel JM, Ferretti R, Leite R, Jilho WJ, Lent R, Herculano-Houzel S. Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain. J Comp Neurol. 2009;513(5):532–41.

    Article  PubMed  Google Scholar 

  4. Beeman D, Wang Z, Edwards M, Bhalla U, Cornelis H, Bower JM. The GENESIS 3.0 project: a universal graphical user interface and database for research, collaboration, and education in computational neuroscience. BMC Neurosci. 2007;8(suppl 2):4.

    Article  Google Scholar 

  5. Boss BD, Peterson GM, Cowan WM. On the number of neurons in the dentate gyrus of the rat. Brain Res. 1985;338(1):144–50.

    CAS  Article  PubMed  Google Scholar 

  6. Brette R, Rudolph M, Carnevale T, Hines M, Beeman D, Bower JM, Diesmann M, Morrison A, Goodman PH, Zirpe M, Natschlger T, Pecevski D, Ermentrout B, Djurfeldt M, Lansner A, Rochel O, Vieville T, Muller E, Davison AP, Boustani SE, Destexh A. Simulation of networks of spiking neurons: a review of tools and strategies. J Comp Neurosci. 2007;23(3):349–98.

    Article  Google Scholar 

  7. Carnevale N, Hines M. The NEURON book. Cambridge: Cambridge University Press; 2006.

    Book  Google Scholar 

  8. Cutsuridis V, Graham B, Cobb S, Vida I. Hippocampal microcircuits. New York: Springer; 2010.

    Book  Google Scholar 

  9. Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters. In: Proceedings of the USENIX symposium on operating systems design and implementation, 2004. OSDI, 2004. 2004; IEEE.

  10. Eliasmith C. How to build a brain: a neural architecture for biological cognition. Oxford: Oxford University Press; 2013.

    Book  Google Scholar 

  11. Eliasmith C, Anderson C. Neural engineering: computation, representation, and dynamics in neurobiological systems. Cambridge: MIT press; 2004.

    Google Scholar 

  12. Eliasmith C, Stewart T, Choo X, Bekolay T, Dewolf T, Tang Y, Rasmussen D. A large-scale model of the functioning brain. Science. 2012;338:1202–5.

    CAS  Article  PubMed  Google Scholar 

  13. Eliasmith C, Trujilo O. The use and abuse of large-scale brain models. Curr Option Neurobiol. 2014;25:1–6.

    CAS  Article  Google Scholar 

  14. Gewaltig MO, Morrison A, Plesser HE. NEST by example: an introduction to the Neural Simulation Tool NEST. In: Le Novère N, editor. Computational systems neurobiology. Springer; 2012. p. 533–558.

  15. Gonzalez J, Low Y, Gu H, Bickson D. Powergraph: distributed graph-parallel computation on natural graphs. In: Proceedings of the USENIX symposium on operating systems design and implementation, 2012. OSDI, 2012. 2012; IEEE.

  16. Goodman D, Brette R. Brian: a simulator for spiking neural networks in python. Front Neuroinform. 2008;2:5.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Hammarlund P, Ekeberg O. Large neural network simulations on multiple hardware platforms. J Comput Neurosci. 1998;5(4):443–59.

    CAS  Article  PubMed  Google Scholar 

  18. Hodgkin AL, Huxley AF. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol. 1952;117(4):500–544.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  19. Izhikevich E. Simple model of spiking neurons. IEEE Trans Neural Netw. 2003;14:1569–72.

    CAS  Article  PubMed  Google Scholar 

  20. Jaeger H. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. In: Technical report, German National Research Center for Information Technology. 2001.

  21. Kunkel S, Schmidt M, Eppler JM, Plesser HE, Masumoto G, Igarashi J, Ishii S, Fukai T, Morrison A, Diesmann M, Helias M. Spiking network simulation code for petascale computers. Front Neuroinform. 2014;10(8):78.

    Google Scholar 

  22. Lansner A, Diesmann M. Virtues, pitfalls, and methodology of neuronal network modeling and simulations on supercomputers. In: Proceedings of the Chapter 10 in Nicolas Le Novre computational systems biology, Springer 2012.

  23. Le NN. Computational systems neurobiology. New York: Springer; 2012.

    Google Scholar 

  24. Low Y, Gonzalez J, Kyrola A, Bickson D, Guestrin C, Hellerstein JM. Distributed graphlab: a framework for machine learning and data mining in the cloud. In: Proceedings of the very large database endowment, 2012. VLDB, 2012. 2012;5(8), IEEE.

  25. Malewicz G, Austern MH, Bik AJC, Dehnert JC, Horn I, Leiser N, Czajkowski G. Pregel: a system for large-scale graph processing. In: ACM special interest group on management of data, 2010. SIGMOD, 2010. 2010; IEEE.

  26. Miller G. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol Rev. 1956;63(2):81–97.

    CAS  Article  PubMed  Google Scholar 

  27. Morrison A, Aertsen A, Diesmann M. Spike-timing dependent plasticity in balanced random networks. Neural Comput. 2007;19:1437–67.

    Article  PubMed  Google Scholar 

  28. Oh SW, Harris JA, Ng L, Winslow B, Cain N, Mihalas S, Wang Q, Lau C, Kuan L, Henry AM, Mortrud MT, Ouellette B, Nguyen TN, Sorensen SA, Slaughterbeck CR, Wakeman W, Li Y, Feng D, Ho A, Nicholas E, Hirokawa KE, Bohn P, Joines KM, Peng H, Hawrylycz MJ, Phillips JW, Hohmann JG, Wohnoutka P, Gerfen CR, Koch C, Bernard A, Dang C, Jones AR, Zeng H. A mesoscale connectome of the mouse brain. Nature. 2014;508:207–14.

    CAS  Article  PubMed  Google Scholar 

  29. Pecevski D, Natschlger T, Schuch K. Pcsim: a parallel simulation environment for neural circuits fully integrated with python. Front Neuroinform. 2009;3:11.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Rotter S, Diesman M. Exact digital simulation of time-invariant linear systems with applications to neuronal modeling. Biol Cybern. 1999;81(5/6):381–402.

    CAS  Article  PubMed  Google Scholar 

  31. Seress L. Interspecies comparison of the hippocampal formation shows increased emphasis on the region superior in the ammon’s horn of the human brain. J Hirnforsch. 1988;29:335–40.

    CAS  PubMed  Google Scholar 

  32. Song S, Miller KD, Abbott LF. Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci. 2000;3:919–26.

    CAS  Article  PubMed  Google Scholar 

  33. Yonezawa A, Watanabe T, Yokokawa M, Sato M, Hirao K. Advanced institute for computational science (aics): Japanese national high-performance computing research institute and its 10-petaflops supercomputer. In: Proceedings of the SC conference on high performance networking and computing, 2011 2011.13, pp. 1–8 IEEE.

  34. Zhang T, Zeng Y, Xu B. A computational effort towards the microscale mouse brain connectome from the mesoscale. Manuscript, 2015.

Download references

Funding

This study was funded by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB02060007) and Beijing Municipal Science and Technology Commission (Z151100000915070).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yi Zeng.

Ethics declarations

Conflict of Interest

Xin Liu, Yi Zeng, Tielin Zhang, and Bo Xu declare that they have no conflict of interest.

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Additional informed consent was obtained from all patients for which identifying information is included in this article.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by the any of the authors.

Additional information

Xin Liu and Yi Zeng contributed equally to this work and serve as co-first authors.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Zeng, Y., Zhang, T. et al. Parallel Brain Simulator: A Multi-scale and Parallel Brain-Inspired Neural Network Modeling and Simulation Platform. Cogn Comput 8, 967–981 (2016). https://doi.org/10.1007/s12559-016-9411-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-016-9411-y

Keywords

  • Neural network simulator
  • Graph computation
  • Parallel simulation
  • Spiking neural network
  • PBS