# Evaluation of the computational efficacy in GPU-accelerated simulations of spiking neurons

## Abstract

To understand the mechanism of information processing by a biological neural network, computer simulation of a large-scale spiking neural network is an important method. However, because of a high computation cost of the simulation of a large-scale spiking neural network, the simulation requires high performance computing implemented by a supercomputer or a computer cluster. Recently, hardware for parallel computing such as a multi-core CPU and a graphics card with a graphics processing unit (GPU) is built in a gaming computer and a workstation. Thus, parallel computing using this hardware is becoming widespread, allowing us to obtain powerful computing power for simulation of a large-scale spiking neural network. However, it is not clear how much increased performance the parallel computing method using a new GPU yields in the simulation of a large-scale spiking neural network. In this study, we compared computation time between the computing methods with CPUs and GPUs in a simulation of neuronal models. We developed computer programs of neuronal simulations for the computing systems that consist of a gaming graphics card with new architecture (the NVIDIA GTX 1080) and an accelerator board using a GPU (the NVIDIA Tesla K20C). Our results show that the computing systems can perform a simulation of a large number of neurons faster than CPU-based systems. Furthermore, we investigated the accuracy of a simulation using single precision floating point. We show that the simulation results of single precision were accurate enough compared with those of double precision, but chaotic neuronal response calculated by a GPU using single precision is prominently different from that calculated by a CPU using double precision. Furthermore, the difference in chaotic dynamics appeared even if we used double precision of a GPU. In conclusion, the GPU-based computing system exhibits a higher computing performance than the CPU-based system, even if the GPU system includes data transfer from a graphics card to host memory.

## Keywords

GPGPU CUDA Acceleration Spiking neural network Neuron model## Mathematics Subject Classification

Simulation and numerical modeling Numerical chaos## Notes

### Acknowledgements

This work was supported by JSPS KAKENHI Grant No. 15K07146.

## References

- 1.Aihara K, Matsumoto G, Ikegaya Y (1984) Periodic and non-periodic responses of a periodically forced Hodgkin–Huxley oscillator. J Theor Biol 109(2):249–269CrossRefGoogle Scholar
- 2.Ananthanarayanan R, Esser SK, Simon HD, Modha DS (2009) The cat is out of the bag: cortical simulations with \(10^9\) neurons, \(10^{13}\) synapses. In: IEEE computer society, pp. 1–12Google Scholar
- 3.Baladron J, Fasoli D, Faugeras O (2012) Three applications of GPU computing in neuroscience. Comput Sci Eng 14(3):40–47CrossRefGoogle Scholar
- 4.Bernhard F, Keriven R (2006) Spiking neurons on GPUs. In computational science ICCS 2006. Springer, Berlin, pp 236–243CrossRefGoogle Scholar
- 5.Beyeler M, Dutt ND, Krichmar JL (2013) Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule. Neural Netw 48:109–124CrossRefGoogle Scholar
- 6.Beyeler M, Oros N, Dutt N, Krichmar JL (2015) A GPU-accelerated cortical neural network model for visually guided robot navigation. Neural Netw 72:75–87CrossRefGoogle Scholar
- 7.Beyeler M, Richert M, Dutt ND, Krichmar JL (2014) Efficient spiking neural network model of pattern motion selectivity in visual cortex. Neuroinform 12(3):435–454CrossRefGoogle Scholar
- 8.Bray LCJ, Anumandla SR, Thibeault CM, Hoang RV, Goodman PH, Dascalu SM, Bryant BD, Harris FC (2012) Real-time human–robot interaction underlying neurorobotic trust and intent recognition. Neural Netw 32:130–137CrossRefGoogle Scholar
- 9.Brette R (2015) Philosophy of the spike: rate-based vs. spike-based theories of the brain. Front Syst Neurosci 9:151CrossRefGoogle Scholar
- 10.Carlson KD, Nageswaran JM, Dutt N, Krichmar JL (2014) An efficient automated parameter tuning framework for spiking neural networks. Front Neurosci 8:10CrossRefGoogle Scholar
- 11.Cheng J, Grossman M, McKercher T (2014) Professional CUDA C programming. Wrox, BirminghamGoogle Scholar
- 12.de Camargo RY, Rozante L, Song SW (2011) A multi-GPU algorithm for large-scale neuronal networks. Concurr Comput Pract Exp 23(6):556–572CrossRefGoogle Scholar
- 13.Dinkelbach HU, Vitay J, Beuth F, Hamker FH (2012) Comparison of GPU- and CPU-implementations of mean-firing rate neural networks on parallel hardware. Network 23(4):212–236CrossRefGoogle Scholar
- 14.Fidjeland AK, Shanahan MP (2010) Accelerated simulation of spiking neural networks using GPUs. In 2010 International joint conference on Neural networks (IJCNN), pp 1–8Google Scholar
- 15.Gangal H, Dar G (2014) Mode locking, chaos and bifurcations in Hodgkin–Huxley neuron forced by sinusoidal current. Chaot Simul Model 3:287–294Google Scholar
- 16.Gerstner W, Kistler WM (2002) Spiking neuron model, chapter noise in spiking neuron models. Cambridge University Press, New York, pp 157–209CrossRefzbMATHGoogle Scholar
- 17.Goodman DFM (2010) Code generation: a strategy for neural network simulators. Neuroinformatics 8(3):183–196CrossRefGoogle Scholar
- 18.Hoang RV, Tanna D, Bray JCL, Dascalu SM, Harris FCJ (2013) A novel CPU/GPU simulation environment for large-scale biologically realistic neural modeling. Front Neuroinform 7:19CrossRefGoogle Scholar
- 19.Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117(4):500–544CrossRefGoogle Scholar
- 20.Igarashi J, Shouno O, Fukai T, Tsujino H (2011) Real-time simulation of a spiking neural network model of the basal ganglia circuitry using general purpose computing on graphics processing units. Neural Netw 24(9):950–960CrossRefGoogle Scholar
- 21.Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Netw 14(6):1569–1572MathSciNetCrossRefGoogle Scholar
- 22.Izhikevich EM (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw 15(5):1063–1070CrossRefGoogle Scholar
- 23.Izhikevich EM, Edelman GM (2008) Large-scale model of mammalian thalamocortical systems. Proc Nat Acad Sci USA 105(9):3593–3598CrossRefGoogle Scholar
- 24.Liao S, Wang P (2014) On the mathematically reliable long-term simulation of chaotic solutions of Lorenz equation in the interval [0 10000]. Sci China Phys Mech Astron 57:330–335CrossRefGoogle Scholar
- 25.Morris C, Lecar H (1981) Voltage oscillations in the barnacle giant muscle fiber. Biophys J 35:193–213CrossRefGoogle Scholar
- 26.Nageswaran JM, Dutt N, Krichmar JL, Nicolau A, Veidenbaum A (2009) Efficient simulation of large-scale spiking neural networks using CUDA graphics processors. In: 2009 Proceedings of international joint conference on neural networks, pp 2145–2152Google Scholar
- 27.Nageswaran JM, Dutt N, Krichmar JL, Nicolau A, Veidenbaum AV (2009) A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Netw 22(5–6):791–800CrossRefGoogle Scholar
- 28.Pallipuram VK, Bhuiyan MA, Smith MC (2011) Evaluation of GPU architectures using spiking neural networks. In: 2011 Symposium application accelerators in high-performance computing, pp. 93–102Google Scholar
- 29.Pallipuram VK, Bhuiyan M, Smith MC (2012) A comparative study of GPU programming models and architectures using neural networks. J Supercomput 61(3):673–718CrossRefGoogle Scholar
- 30.Pallipuram VK, Smith MC, Sarma N, Anand R, Weill E, Sapra K (2015) Subjective versus objective: classifying analytical models for productive heterogeneous performance prediction. J Supercomput 71:162–201CrossRefGoogle Scholar
- 31.Richert M, Nageswaran JM, Dutt N, Krichmar JL (2011) An efficient simulation environment for modeling large-scale cortical processing. Front Neuroinform 5:19CrossRefGoogle Scholar
- 32.Trappenberg T (2010) Fundamentals of computational neuroscience. OUP, OxfordzbMATHGoogle Scholar
- 33.Wang F (2015) Simulation tool for asynchronous cortical streams (STACS): interfacing with spiking neural networks. Proc Comput Sci 61:322–327CrossRefGoogle Scholar
- 34.Yamazaki T, Igarashi J (2013) Realtime cerebellum: a large-scale spiking network model of the cerebellum that runs in realtime using a graphics processing unit. Neural Netw 47:103–111CrossRefGoogle Scholar
- 35.Yavuz E, Turner J, Nowotny T (2016) GeNN: a code generation framework for accelerated brain simulations. Sci Rep 6:18854CrossRefGoogle Scholar