Reproducibility of scientific research has become an important focus of discussion in the life sciences community. There are well-known and well-publicized problems with the replicability of many experimental results. One of the many responses to this crisis of confidence in the experimental body of knowledge is an effort by the Science Exchange Network to independently replicate a selection of published data (https://validation.scienceexchange.com/#/). Reasons for the lack of reproducibility range from a simple lack of sufficient metadata about the experiments all the way to selective reporting of experimental results by the investigators. The issues unfortunately are not limited to the experimental sciences, but also apply to computational research. It is still common to find publications of mathematical models, for instance, that do not contain a complete description of the model, including a complete list of parameters, and a detailed description of the computations underlying the reported results. Even in the case of complete information, the published model may not reproduce the behavior reported in the paper. Often, necessary code to independently run the model is not provided. Model types such as agent-based models or hybrid multiscale models come with their own particular set of problems, such as model specification or details of model implementation. These shortcomings reduce the impact of the work, if it cannot be effectively used to build on by other researchers, and the lack of trust in the findings based on computational work. Here too, there are efforts underway to address the problem. For instance, the recently established Center for Reproducible Biomedical Modeling (http://reproduciblebiomodels.org/), funded by the U.S. National Institutes of Health, aims to provide the capability to assure that models are reproducible.
Scientific journals must play an essential role in the remediation of this problem, including the Bulletin of Mathematical Biology. Currently, the journal does not have any formal requirements to provide sufficient information to reproduce reported results, e.g., deposition of models in an easily usable form, such as coding of ODE models in the Systems Biology Markup Language (SBML), deposition of code for agent-based models as well as seeds for random number generators together with precise descriptions of model tools used, or scripts for the analysis of experimental data. What, if anything, should the Bulletin do to increase reproducibility of work reported in its pages? As is the case very often, here too the devil is in the details. It should be in the interest of authors to make their work as impactful as possible. On the other hand, publication should not be more onerous than absolutely necessary. In this issue, we publish five invited articles that address different aspects of this problem, with some proposed solutions. We hope that these articles can stimulate a discussion among the members of the mathematical biology community in general, and the Bulletin Editorial Board. The end result will be a revision, if needed, of the publication guidelines for Bulletin authors. We hope to receive many comments from the community and will establish an appropriate forum for discussion, if needed. In order to coordinate the input effectively, we request that you e-mail thoughts and responses directly to both Reinhard Laubenbacher (email@example.com) and Alan Hastings (firstname.lastname@example.org).