Abstract
Building upon previous experiments can be used to accomplish new goals. In computing, it is imperative to reuse computer code to continue development on specific projects. Reproducibility is a fundamental building block in science, and experimental reproducibility issues have recently been of great concern. It may be surprising that reproducibility is also of concern in computational science. In this study, we used a previously published code to investigate neural network activity and we were unable to replicate our original results. This led us to investigate the code in question, and we found that several different aspects, attributable to floating-point arithmetic, were the cause of these replicability issues. Furthermore, we uncovered other manifestations of this lack of replicability in other parts of the computation with this model. The simulated model is a standard system of ordinary differential equations, very much like those commonly used in computational neuroscience. Thus, we believe that other researchers in the field should be vigilant when using such models and avoid drawing conclusions from calculations if their qualitative results can be substantially modified through non-reproducible circumstances.
This is a preview of subscription content, log in via an institution to check access.
References
Ben-ari, Y. (2002). Excitatory actions of gaba during development: The nature of the nurture. Nature Reviews Neuroscience, 3(9), 728–739.
Ben-ari, Y., Gaiarsa, J.-L., Tyzio, R., & Khazipov, R. (2007). GABA: A pioneer transmitter that excites immature neurons and generates primitive oscillations. Physiological Reviews, 87(4), 1215–1284.
Blanco, W., Bertram, R., & Tabak, J. (2017). The effects of GABAergic polarity changes on episodic neural network activity in developing neural systems. Frontiers in Computational Neuroscience, 11, 88.
Datta, B. N. (2010) Numerical linear algebra and applications. Siam.
Drummond, C. (2009) Replicability is not reproducibility: nor is it good science.
Hansel, D., Mato, G., Meunier, C., & Neltner, L. (1998). On numerical simulations of integrate-and-fire neural networks. Neural Computation, 10(2), 467–483.
Higham, N. J. (2002) Accuracy and stability of numerical algorithms. Siam. 0898718023.
IEEE Standard for Floating-Point Arithmetic. (2019) IEEE Std 754-2019 (Revision of IEEE 754-2008), p. 1–84.
Mcdougal, R. A., Bulanova, A. S., & Lytton, W. W. (2016). Reproducibility in Computational Neuroscience Models and Simulations. IEEE transactions on bio-medical engineering, 63(10), 2021–2035, 03/08.
Morrison, A., Straube, S., Plesser, H. E., & Diesmann, M. (2007). Exact subthreshold integration with continuous spike times in discrete-time neural network simulations. Neural Computation, 19(1), 47–79.
Nordlie, E., Gewaltig, M.-O., & Plesser, H. E. (2009). Towards reproducible descriptions of neuronal network models. PLoS computational biology, 5(8), e1000456.
Schling, B. (2011) The Boost C++ Libraries. XML Press, 262 p 0982219199, 9780982219195.
Shelley, M. J., & Tao, L. (2001). Efficient and accurate time-stepping schemes for integrate-and-fire neuronal networks. Journal of Computational Neuroscience, 11(2), 111–119.
Tabak, J., Mascagni, M., & Bertram, R. (2010). Mechanism for the universal pattern of activity in developing neuronal networks. Journal of Neurophysiology, 103(4), 2208–2221.
Waltemath, D., Adams, R., Beard, D. A., Bergmann, F. T., et al. (2011). Minimum information about a simulation experiment (MIASE). PLoS computational biology, 7(4), e1001122.
Acknowledgements
This research started in 2016 when WB was hosted by the Mathematics department and the Institute of Molecular Biophysics at Florida State University. WB was also supported by a scholarship (Process #202320/2015-4) from the Brazilian National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq). PHL was supported by an undergraduate scientific research scholarship from CNPq in 2017 and from the State University of Rio Grande do Norte (UERN) in 2018.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interests.
Disclaimer
Certain commercial equipment, instruments, or materials are identified in this paper in order to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology (NIST), nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose.
Additional information
Action Editor: David Terman
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(PDF 201 kb)
Rights and permissions
About this article
Cite this article
Blanco, W., Lopes, P.H., de S. Souza, A.A. et al. Non-replicability circumstances in a neural network model with Hodgkin-Huxley-type neurons. J Comput Neurosci 48, 357–363 (2020). https://doi.org/10.1007/s10827-020-00748-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10827-020-00748-3