Virtues, Pitfalls, and Methodology of Neuronal Network Modeling and Simulations on Supercomputers

Chapter

Abstract

The number of neurons and synapses in biological brains is very large, on the order of millions and billions respectively even in small animals like insects and mice. By comparison most neuronal network models developed and simulated up to now have been tiny, comprising many orders of magnitude less neurons than their real counterpart, with an even more dramatic difference when it comes to the number of synapses. In this chapter we discuss why and when it may be important to work with large-scale, if not full-scale, neuronal network and brain models and to run simulations on supercomputers. We describe the state-of-the-art in large-scale neural simulation technology and methodology as well as ways to analyze and visualize output from such simulations. Finally we discuss the challenges and future trends in this field.

Bibliography

  1. Amit DJ, Brunel N (1997) Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb Cortex 7:237–252PubMedCrossRefGoogle Scholar
  2. Ananthanarayanan R, Esser SK, Simon HD, Modha DS (2009) The cat is out of the bag: cortical simulations with 109neurons, 1013synapses. In: Proceedings of the conference on high performance networking and computing (SC09), Portland, Oregon, USA:Article no. 63. ACM New York, NY, USAGoogle Scholar
  3. Arbib MA, Grethe JS (2001) Computing the brain: a guide to neuroinformatics. Academic, San DiegoGoogle Scholar
  4. Babajani-Feremi A, Soltanian-Zadeh H (2010) Multi-area neural mass modeling of EEG and MEG signals. Neuroimage 52(3):793–811PubMedCrossRefGoogle Scholar
  5. Bailey J, Hammerstrom D (1988) Why VLSI implementations of associative VLCNs require connection multiplexing. International Conference on Neural Networks, San Diego, USA 2, pp 173–180Google Scholar
  6. Beaulieu C, Colonnier M (1983) The number of neurons in the different laminae of the binocular and monocular regions of area 17 in the cat. J Comp Neurology 217:337–344CrossRefGoogle Scholar
  7. Binzegger T, Douglas RJ, Martin KAC (2004) A quantitative map of the circuit of cat primary visual cortex. J Neurosci 39(24):8441–8453CrossRefGoogle Scholar
  8. Boahen KA (2000) Point-to-point connectivity between neuromorphic chips using address events. IEEE Trans Circuits Syst II: Analog Digit Signal Process 47:416–434CrossRefGoogle Scholar
  9. Bojak I, Oostendorp TF, Reid AT, Kötter R (2010) Connecting mean field models of neural activity to EEG and fMRI data. Brain Topogr 23:139–149PubMedCrossRefGoogle Scholar
  10. Brette R (2007) Exact simulation of integrate-and-fire models with exponential currents. Neural Comput 19(10):2604–2609PubMedCrossRefGoogle Scholar
  11. Brette R, Gerstner W (2005) Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J Neurophysiol 94(5):3637–3642PubMedCrossRefGoogle Scholar
  12. Brette R, Rudolph M, Carnevale T, Hines M, Beeman D, Bower JM, Diesmann M, Goodman PH, Harris FC Jr, Zirpe M, Natschläger T, Pecevski D, Ermentrout B, Djurfeldt M, Lansner A, Rochel O, Vieville T, Muller E, Davison A, El Boustani S, Destexhe A (2007) Simulation of networks of spiking neurons: a review of tools and strategies. J Comput Neurosci 23:349–398PubMedCrossRefGoogle Scholar
  13. Brette R, Rudolph M, Carnevale T, Hines M, Beeman D, Bower JM, Diesmann M, Morrison A, Goodman PH, Harris FC Jr, Zirpe M, Natschläger T, Pecevski D, Ermentrout B, Djurfeldt M, Lansner A, Rochel O, Vieville T, Muller E, Davison AP, El Boustani S, Destexhe A (2007b) Simulation of networks of spiking neurons: a review of tools and strategies. J Comput Neurosci 23(3):349–398PubMedCrossRefGoogle Scholar
  14. Brown A, Furber S, Reeve J, Wilson P, Zwolinski M, Chad J, Plana LA, Lester D (2010) A communication infrastructure for a million processor machine. In: Proceedings of ACM international conference on computing frontiers, Bertinoro, Italy, pp 75–76Google Scholar
  15. Brunel N (2000) Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J Comput Neurosci 8(3):183–208PubMedCrossRefGoogle Scholar
  16. Brunel N, van Rossum MCW (2007) Lapicque’s 1907 paper: from frogs to integrate-and-fire. Biol Cybern 97:337–339PubMedCrossRefGoogle Scholar
  17. Buxton RB, Uludag K, Dubowitz DJ, Liu TT (2004) Modeling the hemodynamic response to brain activation. Neuroimage 23(Supplement 1):S220–S233PubMedCrossRefGoogle Scholar
  18. Cannon R, Gewaltig M-O (2011) Fostering sustainability of software tools in neuroscience. arXiv:1205.3025v1 [q-bio.NC]Google Scholar
  19. Chemla S, Chavane F (2010) Voltage-sensitive dye imaging: technique review and models. J Physiol Paris 104(1–2):40–50PubMedCrossRefGoogle Scholar
  20. Cheung K, Schultz SR, Leong PHW (2009) A parallel spiking neural network simulator. In: International conference on Field-Programmable Technology (FPT’09), Sydney, NSW, pp 247–254Google Scholar
  21. Davison AP, Brüderle D, Eppler J, Kremkow J, Muller E, Pecevski D, Perrinet L, Yger P (2009) PyNN: a common interface for neuronal network simulators. Front Neuroinform 2(11). doi:10.3389/neuro.11.011.2008
  22. De Schutter E (2008) Why are computational neuroscience and systems biology so separate? PLoS Comput Biol 4(5):e1000078PubMedCrossRefGoogle Scholar
  23. Dehghani N, Bédard C, Cash S, Halgren E, Destexhe A (2010) Comparative power spectral analysis of simultaneous electroencephalographic and magnetoencephalographic recordings in humans suggests non-resistive extracellular media: EEG and MEG power spectra. J Comput Neurosci 29(3):405–421PubMedCrossRefGoogle Scholar
  24. D’Haene M, Schrauwen B (2010) Fast and exact simulation methods applied on a broad range of neuron models. Neural Comput 22(6):1468–1472PubMedCrossRefGoogle Scholar
  25. Diesmann M, Gewaltig M-O (2002) NEST: An environment for neural systems simulations. Forschung und wisschenschaftliches Rechnen, Beitraege zum Heinz-Billing-Preis 2001. In: Plesser T, Macho V (eds). Göttingen, Ges fuer Wiss Datenverarbeitung: 43–70Google Scholar
  26. Djurfeldt M (2012) The connection-set algebra–a novel formalism for the representation of connectivity structure in neuronal Network models. Neuroinformatics. doi: 10.1007/s12021-012-9146-1. Online first
  27. Djurfeldt M, Lansner A (2007) Workshop report: 1st INCF workshop on large-scale modeling of the nervous system, Nature Precedings, Stockholm, SwedenGoogle Scholar
  28. Djurfeldt M, Lundqvist M, Johansson C, Rehn M, Ekeberg Ö, Lansner A (2008) Brain-scale simulation of the neocortex on the IBM Blue Gene/L supercomputer. IBM J Res Dev 52:31–41CrossRefGoogle Scholar
  29. Djurfeldt M, Hjorth J, Eppler JM, Dudani N, Helias M, Potjans TC, Bhalla US, Diesmann M, Hellgren Kotaleski J, Ekeberg Ö (2010) Run-time interoperability between neuronal network simulators based on the MUSIC framework. Neuroinformatics 8:43–60PubMedCrossRefGoogle Scholar
  30. Douglas R, Mahovald M, Mead C (1995) Neuromorphic analogue VLSI. Ann Rev Neurosci 18(255–281)Google Scholar
  31. Ermentrout GB, Kopell N (1984) Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J Appl Math 44:1133–1149CrossRefGoogle Scholar
  32. Fidjeland AK, Shanahan MP (2010) Accelerated simulation of spiking neural networks using GPUs. IEEE International Joint Conference on Neural Networks (IJCNN), Barcelona. IEEE, 345 E 47TH ST, New York, NY 10017Google Scholar
  33. Fourcaud-Trocmé N, Hansel D, van Vreeswijk C, Brunel N (2003) How spike generation mechanisms determine the neuronal response to fluctuating inputs. J Neurosci 23:11628–11640PubMedGoogle Scholar
  34. Gabbott PL, Somogyi P (1986) Quantitative distribution of GABA-immunoreactive neurons in the visual cortex (area 17) of the cat. Exp Brain Res 61(2):323–331PubMedGoogle Scholar
  35. Gleeson P, Crook S, Cannon RC, Hines ML, Billings GO, Farinella M, Morse TM, Davison A, Ray S, Bhalla US, Barnes SR, Dimitrova YD, Silver RA (2010) NeuroML: a language for describing data driven models of neurons and networks with a high degree of biological detail. PLoS Comput Biol 6:e1000815PubMedCrossRefGoogle Scholar
  36. Göbel W, Helmchen F (2007) In vivo calcium imaging of neural network function. Physiology 22:358–365PubMedCrossRefGoogle Scholar
  37. Grün S (2009) Data-driven significance estimation for precise spike correlation. J Neurophysiol 101:1126–1140PubMedCrossRefGoogle Scholar
  38. Guerrero-Rivera R, Morrison A, Diesmann M, Pearce TC (2006) Programmable logic construction kits for hyper-real-time neuronal modeling. Neural Comput 18(11):2651–2679PubMedCrossRefGoogle Scholar
  39. Hammarlund P, Ekeberg Ö (1998) Large neural network simulations on multiple hardware platforms. J Comp Neurosci 5:443–459CrossRefGoogle Scholar
  40. Hansel D, Mato G, Meunier C (1995) Synchrony in excitatory neural networks. Neural Comp 7(2):307–337CrossRefGoogle Scholar
  41. Hanuschkin A, Kunkel S, Helias M, Morrison A, Diesmann M (2010) A general and efficient method for incorporating precise spike times in globally time-driven simulations. Front Neuroinform 4:113PubMedCrossRefGoogle Scholar
  42. Hines ML, Markram H, Schürmann F (2008) Fully implicit parallel simulation of single neurons. J Comput Neurosci 25(3):439–448PubMedCrossRefGoogle Scholar
  43. Homma R, Baker B, Jin L, Garaschuk O, Konnerth A, Cohen L, Bleau C, Canepari M, Djurisic M, Zecevic D (2009) Wide-field and two-photon imaging of brain activity with voltage- and calcium-sensitive dyes. Methods Mol Biol 489:43–79PubMedCrossRefGoogle Scholar
  44. Indiveri G, Chicca E, Douglas RJ (2009) Artificial cognitive systems: from VLSI networks of spiking neurons to neuromorphic cognition. Cogn Comput 1:119–127CrossRefGoogle Scholar
  45. Izhikevich EM (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw 5(15):1063–1070CrossRefGoogle Scholar
  46. Jin X, Galluppi F, Patterson C, Rast A, Davies S, Temple S, Furber S (2010a) Algorithm and software for simulation of spiking neural networks on the multi-chip SpiNNaker system. International Joint Conference on Neural Networks (IJCNN 2010), Barcelona, 18–23 July 2010, pp 649–656Google Scholar
  47. Jin X, Lujan M, Plana LA, Davies S, Temple S, Furber S (2010b) Modeling spiking neural networks on SpiNNaker. Comput Sci Eng 12(5):91–97CrossRefGoogle Scholar
  48. Jo SH, Chang T, Ebong I, Bhadviya BB, Mazumder P, Lu W (2010) Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett 10:1297–1301PubMedCrossRefGoogle Scholar
  49. King JG, Hines M, Hill S, Goodman PH, Markram H, Schürmann F (2009) A component-based extension framework for large-scale parallel simulations in NEURON. Front Neuroinform 3:1–11CrossRefGoogle Scholar
  50. Kobayashi R, Tsubo Y, Shinomoto S (2009) Made-to-order spiking neuron model equipped with a multi-timescale adaptive threshold. Front Comput Neurosci 3(9)Google Scholar
  51. Lansner A (2009) Associative memory models – from cell assembly theory to biophysically detailed cortex simulations. Trends Neurosci 32:178–186PubMedCrossRefGoogle Scholar
  52. Lansner A, Fransén E (1992) Modeling Hebbian cell assemblies comprised of cortical neurons. Netw Comput Neural Syst 3:105–119CrossRefGoogle Scholar
  53. Likharev K, Mayr A, Muckra I, Turel O (2003) CrossNets high-performance neuromorphic architectures for CMOL circuits. Molecular electronics III. In: Reimers J, Picconatto C, Ellenbogen J, Shashidhar R (eds) New York, NY Acad Sci 1006: 146–163Google Scholar
  54. Lindén H, Pettersen KH, Einevoll GT (2010) Intrinsic dendritic filtering gives low-pass power spectra of local field potentials. J Comput Neurosci 29(3):423–444PubMedCrossRefGoogle Scholar
  55. Lundqvist M, Rehn M, Djurfeldt M, Lansner A (2006) Attractor dynamics in a modular network model of the neocortex. Netw Comput Neural Syst 17:253–276CrossRefGoogle Scholar
  56. Lundqvist M, Compte A, Lansner A (2010a) Bistable, irregular firing and population oscillations in a modular attractor memory network. PLoS Comput Biol 6(6):1–12CrossRefGoogle Scholar
  57. Lundqvist M, Herman P, Lansner A (2010b) Theta and gamma power increases and alpha/beta power decreases with memory load in an attractor network model. J Cogn Neurosci 23(10):3008–3020CrossRefGoogle Scholar
  58. MacGregor RJ (1987) Neural and brain modeling. Academic, San DiegoGoogle Scholar
  59. Maguire LP, McGinnity TM, Glackin B, Ghani A, Belatreche A, Harkin J (2007) Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing 71:13–29CrossRefGoogle Scholar
  60. Markram H (2006) The blue brain project. Nat Rev Neurosci 7:153–160PubMedCrossRefGoogle Scholar
  61. Migliore M, Cannia C, Lytton WW, Markram H, Hines ML (2006) Parallel network simulations with NEURON. J Comput Neurosci 21(2):119–223PubMedCrossRefGoogle Scholar
  62. Misra J, Saha I (2010) Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 74:239–255CrossRefGoogle Scholar
  63. Morrison A, Diesmann M (2008) Maintaining causality in discrete time neuronal network simulations. In: Beim Graben P, Zhou C, Thiel M, Kurths J (eds) Lectures in supercomputational neuroscience: dynamics in complex brain networks. Springer, Berlin, pp 267–278CrossRefGoogle Scholar
  64. Morrison A, Mehring C, Geisel T, Aertsen A, Diesmann M (2005) Advancing the boundaries of high connectivity network simulation with distributed computing. Neural Comput 17(8): 1776–1801PubMedCrossRefGoogle Scholar
  65. Morrison A, Mehring C, Geisel T, Aertsen A, Diesmann M (2005b) Advancing the boundaries of high connectivity network simulation with distributed computing. Neural Comput 17(8): 1776–1801PubMedCrossRefGoogle Scholar
  66. Morrison A, Aertsen A, Diesmann M (2007a) Spike-timing dependent plasticity in balanced random networks. Neural Comput 19:1437–1467PubMedCrossRefGoogle Scholar
  67. Morrison A, Straube S, Plesser HE, Diesmann M (2007b) Exact subthreshold integration with continuous spike times in discrete time neural network simulations. Neural Comput 19(1): 47–79PubMedCrossRefGoogle Scholar
  68. Morrison A, Diesmann M, Gerstner W (2008) Phenomenological models of synaptic plasticity based on spike-timing. Biol Cybern 98:459–478PubMedCrossRefGoogle Scholar
  69. Nageswarana JM, Dutt N, Krichmar JL, Nicolau A, Veidenbaum AV (2009) A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Netw 22:791–800CrossRefGoogle Scholar
  70. Noble D (2006) The music of life: biology beyond genes. Oxford University Press, OxfordGoogle Scholar
  71. Nordlie E, Gewaltig M-O, Plesser HE (2009) Towards reproducible descriptions of neuronal network models. PLoS Comput Biol 5(8):e1000456PubMedCrossRefGoogle Scholar
  72. Plesser HE, Diesmann M (2009) Simplicity and efficiency of integrate-and-fire neuron models. Neural Comput 21:353–359PubMedCrossRefGoogle Scholar
  73. Potjans T, Diesmann M (2011) The cell-type specific connectivity of the local cortical network explains prominent features of neuronal activity. arXiv:1106.5678 [q-bio.NC]Google Scholar
  74. Potjans W, Morrison A, Diesmann M (2010) Enabling functional neural circuit simulations with distributed computing of neuromodulated plasticity. Front Comput Neurosci 4(141). doi:10.3389/fncom.2010.00141
  75. Rasch M, Logothetis NK, Kreiman G (2009) From neurons to circuits: linear estimation of local field potentials. J Neurosci 29:13785–13796PubMedCrossRefGoogle Scholar
  76. Rotter S, Diesmann M (1999) Exact digital simulation of time-invariant linear systems with applications to neuronal modeling. Biol Cybern 81(5/6):381–402PubMedCrossRefGoogle Scholar
  77. Schemmel J, Brüderle D, Meier K, Ostendorf B (2007) Modeling synaptic plasticity within networks of highly accelerated I&F neurons. In: IEEE international symposium on circuits and systems art. no. 4253401, pp 3367–3370Google Scholar
  78. Schemmel J, Brüderle D, Grübl A, Hock M, Meier K, Millner S (2010) A wafer-scale neuromorphic hardware system for large-scale neural modeling. In: ISCAS 2010 – IEEE International Symposium on Circuits and Systems: nano-bio circuit fabrics and systems, pp 1947–1950Google Scholar
  79. Sotero RC, Trujillo-Barreto NJ (2008) Biophysical model for integrating neuronal activity, EEG, fMRI and metabolism. Neuroimage 39(1):290–309PubMedCrossRefGoogle Scholar
  80. Stettler DD, Axel R (2009) Representations of Odor in the Piriform Cortex. Neuron 63: 854–864PubMedCrossRefGoogle Scholar
  81. Thomas D, Luk W (2009) FPGA accelerated simulation of biologically plausible spiking neural networks. In: 17th IEEE symposium on field programmable custom computing machines, Napa, California, pp 45–52Google Scholar
  82. Thomson AM, West DC, Wang Y, Bannister AP (2002) Synaptic connections and small circuits involving excitatory and inhibitory neurons in layer 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. Cereb Cortex 12:936–953PubMedCrossRefGoogle Scholar
  83. Traub RD, Whittington MA, Buhl EH, Jefferys JGR, Faulkner HJ (1999) On the mechanism of the γ – β frequency shift in neuronal oscillations induced in rat hippocampal slices by tetanic stimulation. J Neurosci 19(3):1088–1105PubMedGoogle Scholar
  84. van Elburg RAJ, van Ooyen A (2009) Generalization of the event-based Carnevale-Hines integration scheme for integrate-and-fire models. Neural Comput 21(7):1913–1930. doi:10.1162/neco.2009.07-08-815 PubMedCrossRefGoogle Scholar
  85. van Vreeswijk C, Sompolinsky H (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274:1724–1726PubMedCrossRefGoogle Scholar
  86. Winograd T (1975) Breaking the complexity barrier (again). ACM SIGPLAN Notices 10:1 (Jan. 1975) (jointly issued as SIGIR FORUM 9:3 Winter 1974), pp 13–30. Reprinted. In: Barstow D, Shrobe H, Sandewall E (eds) Interactive programming environments. McGraw Hill, New York 1984 pp 3–18Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  1. 1.Department of Numerical Analysis and Computer ScienceStockholm UniversityStockholmSweden
  2. 2.Department of Computational Biology, School of Computer Science and CommunicationRoyal Institute of TechnologyStockholmSweden
  3. 3.Institute of Neuroscience and Medicine (INM-6), Computational and Systems NeuroscienceResearch Center JuelichJuelichGermany
  4. 4.Faculty of MedicineRWTH Aachen UniversityAachenGermany
  5. 5.RIKEN Brain Science InstituteWako CityJapan

Personalised recommendations