Abstract
Workflows for the acquisition and analysis of data in the natural sciences exhibit a growing degree of complexity and heterogeneity, are increasingly performed in large collaborative efforts, and often require the use of high-performance computing (HPC). Here, we explore the reasons for these new challenges and demands and discuss their impact with a focus on the scientific domain of computational neuroscience. We argue for the need of software platforms integrating HPC systems that allow scientists to construct, comprehend and execute workflows composed of diverse data generation and processing steps using different tools. As a use case we present a concrete implementation of such a complex workflow, covering diverse topics such as HPC-based simulation using the NEST software, access to the SpiNNaker neuromorphic hardware platform, complex data analysis using the Elephant library, and interactive visualization methods for facilitating further analysis. Tools are embedded into a web-based software platform under development by the Human Brain Project, called the Collaboratory. On the basis of this implementation, we discuss the state of the art and future challenges in constructing large, collaborative workflows with access to HPC resources.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
References
van Albada, S.J., Rowley, A.G., Hopkins, M., Schmidt, M., Senk, J., Stokes, A.B., Galluppi, F., Lester, D.R., Diesmann, M., Furber, S.B.: Full-scale simulation of a cortical microcircuit on SpiNNaker. In: Frontiers in Neuroinformatics Conference Abstract: Neuroinformatics 2016 (2016). http://dx.doi.org/10.3389/conf.fninf.2016.20.00029
van Albada, S.J., Helias, M., Diesmann, M.: Scalability of asynchronous networks is limited by one-to-one mapping between effective connectivity and correlations. PLoS Comput. Biol. 11(9), e1004490 (2015). http://dx.doi.org/10.1371/journal.pcbi.1004490
Appelt, W.: WWW based collaboration with the BSCW system. In: Pavelka, J., Tel, G., Bartošek, M. (eds.) SOFSEM 1999. LNCS, vol. 1725, pp. 66–78. Springer, Heidelberg (1999). doi:10.1007/3-540-47849-3_4
Brüderle, D., Petrovici, M.A., Vogginger, B., Ehrlich, M., Pfeil, T., Millner, S., Grübl, A., Wendt, K., Müller, E., Schwartz, M.O., de Oliveira, D.H., Jeltsch, S., Fieres, J., Schilling, M., Müller, P., Breitwieser, O., Petkov, V., Muller, L., Davison, A.P., Krishnamurthy, P., Kremkow, J., Lundqvist, M., Muller, E., Partzsch, J., Scholze, S., Zühl, L., Mayr, C., Destexhe, A., Diesmann, M., Potjans, T.C., Lansner, A., Schüffny, R., Schemmel, J., Meier, K.: A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems. Biol. Cybern. 104(4–5), 263–296 (2011). http://dx.doi.org/10.1007/s00422-011-0435-9
Davison, A.P., Brüderle, D., Kremkow, J., Muller, E., Pecevski, D., Perrinet, L., Yger, P.: PyNN: a common interface for neuronal network simulators. Front. Neuroinformatics 2(11), 204 (2009). http://dx.doi.org/10.3389/neuro.11.011.2008
Davison, A.P., Hines, M.L., Muller, E.: Trends in programming languages for neuroscience simulations. Front. Neurosci. 3(3), 374–380 (2009). http://dx.doi.org/10.3389/neuro.01.036.2009
Davison, A.P., Yger, P., Muller, E., Kremkow, J., Brüderle, D., Perrinet, L., Eppler, J., Pecevski, D., Debeissat, N., Djurfeldt, M., Schmuker, M., Kaplan, B., Natschlaeger, T., Ray, S., Zaytsev, Y., Gravier, A.: PyNN 0.7.5. https://pypi.python.org/pypi/PyNN/0.7.5
Davison, A.P., Yger, P., Muller, E., Kremkow, J., Brüderle, D., Perrinet, L., Eppler, J., Pecevski, D., Debeissat, N., Djurfeldt, M., Schmuker, M., Kaplan, B., Natschlaeger, T., Ray, S., Zaytsev, Y., Antolik, J., Gravier, A., Close, T., Breitwieser, O., Schücker, J., Schmidt, M.: PyNN 0.8.0 (2015). https://github.com/NeuralEnsemble/PyNN/releases/tag/0.8.0
Denker, M., Grün, S.: Designing workflows for the reproducible analysis of electrophysiological data. In: Amunts, K., Grandinetti, L., Lippert, T., Petkov, N. (eds.) BrainComp 2015. LNCS, vol. 10087, pp. 58–72. Springer, Cham (2016). doi:10.1007/978-3-319-50862-7_5
Dougiamas, M., Taylor, P.: Moodle: Using learning communities to create an open source course management system. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications (EDMEDIA) (2003). http://research.moodle.net/id/eprint/33
Eppler, J.M., Pauli, R., Peyser, A., Ippen, T., Morrison, A., Senk, J., Schenck, W., Bos, H., Helias, M., Schmidt, M., Kunkel, S., Jordan, J., Gewaltig, M.O., Bachmann, C., Schuecker, J., Albada, S., Zito, T., Deger, M., Michler, F., Hagen, E., Setareh, H., Riquelme, L., Shirvani, A., Duarte, R., Deepu, R., Plesser, H.E.: Nest 2.8.0 (2015). https://doi.org/10.5281/zenodo.32969
Furber, S.B., Lester, D.R., Plana, L.A., Garside, J.D., Painkras, E., Temple, S., Brown, A.D.: Overview of the SpiNNaker system architecture. IEEE Trans. Comput. 62(12), 2454–2467 (2013). http://dx.doi.org/10.1109/TC.2012.142
Garcia, S., Davison, A.P., Rodgers, C., Yger, P., Mahnoun, Y., Estabanez, L., Sobolev, A., Brizzi, T., Jaillet, F., Rautenberg, P., Wachtler, T., Dejean, C., Pröpper, R., Guarino, D.: Neo 0.4.1. https://github.com/NeuralEnsemble/python-neo/releases/tag/0.4.1
Garcia, S., Guarino, D., Jaillet, F., Jennings, T., Pröpper, R., Rautenberg, P.L., Rodgers, C.C., Sobolev, A., Wachtler, T., Yger, P., Davison, A.P.: Neo: an object model for handling electrophysiology data in multiple formats. Front. Neuroinformatics 8, 10 (2014). http://dx.doi.org/10.3389/fninf.2014.00010
Gewaltig, M.O., Diesmann, M.: NEST (NEural Simulation Tool). Scholarpedia 2(4) (2007). http://dx.doi.org/10.4249/scholarpedia.1430
Grewe, J., Wachtler, T., Benda, J.: A bottom-up approach to data annotation in neurophysiology. Front. Neuroinformatics 5 (2011). http://dx.doi.org/10.3389/fninf.2011.00016
Kunkel, S., Schmidt, M., Eppler, J.M., Plesser, H.E., Masumoto, G., Igarashi, J., Ishii, S., Fukai, T., Morrison, A., Diesmann, M., Helias, M.: Spiking network simulation code for petascale computers. Front. Neuroinformatics 8(78), 1 (2014). http://dx.doi.org/10.3389/fninf.2014.00078
Potjans, T.C., Diesmann, M.: The cell-type specific cortical microcircuit: relating structure and activity in a full-scale spiking network model. Cereb. Cortex 24(3), 785–806 (2014). http://dx.doi.org/10.1093/cercor/bhs358
Schmidt, M., Bakker, R., Shen, K., Bezgin, G., Hilgetag, C.C., Diesmann, M., van Albada, S.J.: Full-density multi-scale account of structure and dynamics of macaque visual cortex (2015). arXiv preprint https://arxiv.org/abs/1511.09364
Stokes, A.B., Rowley, A.G.D., Knight, J., Lester, D.R., Rast, A., Hopkins, M.W., Davidson, S., Temple, S., Plana, L., Davies, S., Sharpe, T., Patterson, C., Furber, S.B.: sPyNNaker 3.0.0 (2016). https://github.com/SpiNNakerManchester/sPyNNaker/releases/tag/3.0.0
Yegenoglu, A., Denker, M., Phan, L., Holstein, D., Chorley, P., Ito, J., Jennings, T., Meyes, R., Quaglio, P., Rostami, V., Sprenger, J., Torre, E., Davison, A., Grün, S.: Elephant - open-source tool for the analysis of electrophysiological data sets. In: Bernstein Conference 2015: Abstract Book, p. W-05 (2015). http://dx.doi.org/10.12751/nncn.bc2015.0126
Yegenoglu, A., Davison, A.P., Holstein, D., Muller, E., Torre, E., Hagen, E., Gosmann, J., Sprenger, J., Ito, J., Denker, M., Chorley, P., Yger, P., Quaglio, P., Meyes, R., Rostami, V., Ray, S., Pröpper, R., Gerkin, R.C., Telenczuk, B.: Elephant 0.3.0. https://github.com/NeuralEnsemble/elephant/releases/tag/0.3.0
Zehl, L., Jaillet, F., Stoewer, A., Grewe, J., Sobolev, A., Wachtler, T., Brochier, T.G., Riehle, A., Denker, M., Grün, S.: Handling metadata in a neurophysiology laboratory. Front. Neuroinformatics 10 (2016). http://dx.doi.org/10.3389/fninf.2016.00026
Acknowledgments
This project has received funding from the Helmholtz Portfolio Supercomputing and Modeling for the Human Brain (SMHB), the European Union’s Horizon 2020 research and innovation programme under grant agreement No 720270 (HBP SGA1), and the DFG SPP Priority Program 1665 (GR 1753/4-1 and DE 2175/1-1).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Senk, J. et al. (2017). A Collaborative Simulation-Analysis Workflow for Computational Neuroscience Using HPC. In: Di Napoli, E., Hermanns, MA., Iliev, H., Lintermann, A., Peyser, A. (eds) High-Performance Scientific Computing. JHPCS 2016. Lecture Notes in Computer Science(), vol 10164. Springer, Cham. https://doi.org/10.1007/978-3-319-53862-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-53862-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53861-7
Online ISBN: 978-3-319-53862-4
eBook Packages: Computer ScienceComputer Science (R0)