Reproducible Research Using Biomodels

Mendes, Pedro

doi:10.1007/s11538-018-0498-z

Perspectives Article
Published: 06 September 2018

Reproducible Research Using Biomodels

Pedro Mendes ORCID: orcid.org/0000-0001-6507-9168¹

Bulletin of Mathematical Biology volume 80, pages 3081–3087 (2018)Cite this article

597 Accesses
7 Citations
11 Altmetric
Metrics details

Abstract

Like other types of computational research, modeling and simulation of biological processes (biomodels) is still largely communicated without sufficient detail to allow independent reproduction of results. But reproducibility in this area of research could easily be achieved by making use of existing resources, such as supplying models in standard formats and depositing code, models, and results in public repositories.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

Baker M (2016) 1,500 scientists lift the lid on reproducibility. Nature 533(7604):452–454. https://doi.org/10.1038/533452a
Article Google Scholar
Chance B, Garfinkel D, Higgins J, Hess B (1960) Metabolic control mechanisms. V. A solution for the equations representing interaction between glycolysis and respiration in ascites tumor cells. J Biol Chem 235(8):2426–2439
Google Scholar
Chelliah V, Juty N, Ajmera I, Ali R, Dumousseau M, Glont M, Hucka M, Jalowicki G, Keating S, Knight-Schrijver V, Lloret-Villas A, Natarajan KN, Pettit JB, Rodriguez N, Schubert M, Wimalaratne SM, Zhao Y, Hermjakob H, Le Novère N, Laibe C (2015) BioModels: ten-year anniversary. Nucleic Acids Res 43:D542–548. https://doi.org/10.1093/nar/gku1181
Article Google Scholar
Claerbout JF, Karrenbach M (1992) Electronic documents give reproducible research a new meaning. Soc Explor Geophys. https://doi.org/10.1190/1.1822162
Dada JO, Spasić I, Paton NW, Mendes P (2010) SBRML: a markup language for associating systems biology data with models. Bioinformatics 26(7):932–938. https://doi.org/10.1093/bioinformatics/btq069
Article Google Scholar
Gillespie DT (1976) A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys 22:403–434
MathSciNet Article Google Scholar
Gleeson P, Crook S, Cannon RC, Hines ML, Billings GO, Farinella M, Morse TM, Davison AP, Ray S, Bhalla US, Barnes SR, Dimitrova YD, Silver RA (2010) NeuroML: a language for describing data driven models of neurons and networks with a high degree of biological detail. PLoS Comput Biol 6(6):e1000815. https://doi.org/10.1371/journal.pcbi.1000815
MathSciNet Article Google Scholar
Goodman SN, Fanelli D, Ioannidis JPA (2016) What does research reproducibility mean? Sci Transl Med 8(341):341ps12. https://doi.org/10.1126/scitranslmed.aaf5027
Article Google Scholar
Greenbaum D, Rozowsky J, Stodden V, Gerstein M (2017) Structuring supplemental materials in support of reproducibility. Genome Biol 18:64. https://doi.org/10.1186/s13059-017-1205-3
Article Google Scholar
Guerreiro M (2017) Forking software used in eLife papers to GitHub . https://elifesciences.org/inside-elife/dbcb6949/forking-software-used-in-elife-papers-to-github
Hoops S, Sahle S, Gauges R, Lee C, Pahle J, Simus N, Singhal M, Xu L, Mendes P, Kummer U (2006) COPASI: a COmplex PAthway SImulator. Bioinformatics 22(24):3067–3074. https://doi.org/10.1093/bioinformatics/btl485
Article Google Scholar
Hothorn T, Held L, Friede T (2009) Biometrical journal and reproducible research. Biom J 51(4):553–555. https://doi.org/10.1002/bimj.200900154
MathSciNet Article Google Scholar
Hothorn T, Leisch F (2011) Case studies in reproducibility. Brief Bioinform 12(3):288–300. https://doi.org/10.1093/bib/bbq084
Article Google Scholar
Hübner K, Sahle S, Kummer U (2011) Applications and trends in systems biology in biochemistry: systems biology in biochemical research. FEBS J 278(16):2767–2857. https://doi.org/10.1111/j.1742-4658.2011.08217.x
Article Google Scholar
Hucka M, Finney A, Sauro HM, Bolouri H, Doyle JC, Kitano H, Arkin AP, Bornstein BJ, Bray D, Cornish-Bowden A, Cuellar AA, Dronov S, Gilles ED, Ginkel M, Gor V, Goryanin II, Hedley WJ, Hodgman TC, Hofmeyr JH, Hunter PJ, Juty NS, Kasberger JL, Kremling A, Kummer U, Le Novère N, Loew LM, Lucio D, Mendes P, Minch E, Mjolsness ED, Nakayama Y, Nelson MR, Nielsen PF, Sakurada T, Schaff JC, Shapiro BE, Shimizu TS, Spence HD, Stelling J, Takahashi K, Tomita M, Wagner J, Wang J (2003) The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models. Bioinformatics 19(4):524–531. https://doi.org/10.1093/bioinformatics/btg015
Article Google Scholar
Le Novère N, Bornstein B, Broicher A, Courtot M, Donizelli M, Dharuri H, Li L, Sauro H, Schilstra M, Shapiro B, Snoep JL, Hucka M (2006) BioModels database: a free, centralized database of curated, published, quantitative kinetic models of biochemical and cellular systems. Nucleic Acids Res 34(Database issue):D689–91 10.1093/nar/gkj092
Article Google Scholar
Le Novère N, Finney A, Hucka M, Bhalla US, Campagne F, Collado-Vides J, Crampin EJ, Halstead M, Klipp E, Mendes P, Nielsen P, Sauro H, Shapiro B, Snoep JL, Spence HD, Wanner BL (2005) Minimum information requested in the annotation of biochemical models (MIRIAM). Natute Biotechnol 23(12):1509–15. https://doi.org/10.1038/nbt1156
Article Google Scholar
Le Novère N, Hucka M, Mi H, Moodie S, Schreiber F, Sorokin A, Demir E, Wegner K, Aladjem MI, Wimalaratne SM, Bergman FT, Gauges R, Ghazal P, Kawaji H, Li L, Matsuoka Y, Villéger A, Boyd SE, Calzone L, Courtot M, Dogrusoz U, Freeman TC, Funahashi A, Ghosh S, Jouraku A, Kim S, Kolpakov F, Luna A, Sahle S, Schmidt E, Watterson S, Wu G, Goryanin I, Kell DB, Sander C, Sauro H, Snoep JL, Kohn K, Kitano H (2009) The systems biology graphical notation. Nat Biotechnol 27(8):735–741. https://doi.org/10.1038/nbt.1558
Article Google Scholar
Lloyd CM, Halstead MD, Nielsen PF (2004) CellML: its future, present and past. Prog Biophys Mol Biol 85(2–3):433–450. https://doi.org/10.1016/j.pbiomolbio.2004.01.004
Article Google Scholar
Loew L, Beckett D, Egelman EH, Scarlata S (2015) Reproducibility of research in biophysics. Biophys J 108(7):E1. https://doi.org/10.1016/j.bpj.2015.03.002
Article Google Scholar
Maddox J, Randi J, Stewart WW (1988) High-dilution experiments a delusion. Nature 334(6180):287–290. https://doi.org/10.1038/334287a0
Article Google Scholar
Mesirov JP (2010) Accessible reproducible research. Science 327(5964):415–416. https://doi.org/10.1126/science.1179653
Article Google Scholar
Moraru I, Morgan F, Li Y, Loew L, Schaff J, Lakshminarayana A, Slepchenko B, Gao F, Blinov M (2008) Virtual cell modelling and simulation software environment. IET Syst Biol 2(5):352–362. https://doi.org/10.1049/iet-syb:20080102
Article Google Scholar
Pahle J, Challenger JD, Mendes P, McKane AJ (2012) Biochemical fluctuations, optimisation and the linear noise approximation. BMC Syst Biol 6(1):86. https://doi.org/10.1186/1752-0509-6-86
Article Google Scholar
Peng RD (2009) Reproducible research and biostatistics. Biostatistics 10(3):405–408. https://doi.org/10.1093/biostatistics/kxp014
Article Google Scholar
Peng RD (2011) Reproducible research in computational science. Science 334(6060):1226–1227. https://doi.org/10.1126/science.1213847
Article Google Scholar
Piccolo SR, Frampton MB (2016) Tools and techniques for computational reproducibility. GigaScience. https://doi.org/10.1186/s13742-016-0135-4
Article Google Scholar
Plesser HE (2018) Reproducibility vs. replicability: a brief history of a confused terminology. Front Neuroinform 11:76. https://doi.org/10.3389/fninf.2017.00076
Article Google Scholar
Popper K (1959) The logic of scientific discovery. Hutchinson, London
MATH Google Scholar
Sandve GK, Nekrutenko A, Taylor J, Hovig E (2013) Ten simple rules for reproducible computational research. PLoS Comput Biol 9(10):e1003285. https://doi.org/10.1371/journal.pcbi.1003285
Article Google Scholar
Stodden V, McNutt M, Bailey DH, Deelman E, Gil Y, Hanson B, Heroux MA, Ioannidis JPA, Taufer M (2016) Enhancing reproducibility for computational methods. Science 354(6317):1240–1241. https://doi.org/10.1126/science.aah6168
Article Google Scholar
Stodden V, Seiler J, Ma Z (2018) An empirical analysis of journal policy effectiveness for computational reproducibility. Proc Natl Acad Sci 115(11):2584–2589. https://doi.org/10.1073/pnas.1708290115
Article Google Scholar
Swat M, Moodie S, Wimalaratne S, Kristensen N, Lavielle M, Mari A, Magni P, Smith M, Bizzotto R, Pasotti L, Mezzalana E, Comets E, Sarr C, Terranova N, Blaudez E, Chan P, Chard J, Chatel K, Chenel M, Edwards D, Franklin C, Giorgino T, Glont M, Girard P, Grenon P, Harling K, Hooker A, Kaye R, Keizer R, Kloft C, Kok J, Kokash N, Laibe C, Laveille C, Lestini G, Mentré F, Munafo A, Nordgren R, Nyberg H, Parra-Guillen Z, Plan E, Ribba B, Smith G, Trocóniz I, Yvon F, Milligan P, Harnisch L, Karlsson M, Hermjakob H, Le Novère N (2015) Pharmacometrics markup language (PharmML): opening new perspectives for model exchange in drug development: PharmML - pharmacometrics markup language. CPT: Pharmacometr Syst Pharmacol 4(6):316–319. https://doi.org/10.1002/psp4.57
Article Google Scholar
Waltemath D, Adams R, Beard DA, Bergmann FT, Bhalla US, Britten R, Chelliah V, Cooling MT, Cooper J, Crampin EJ, Garny A, Hoops S, Hucka M, Hunter P, Klipp E, Laibe C, Miller AK, Moraru I, Nickerson D, Nielsen P, Nikolski M, Sahle S, Sauro HM, Schmidt H, Snoep JL, Tolle D, Wolkenhauer O, Le Novère N (2011) Minimum information about a simulation experiment (MIASE). PLoS Comput Biol 7(4):e1001122. https://doi.org/10.1371/journal.pcbi.1001122
Article Google Scholar
Waltemath D, Adams R, Bergmann FT, Hucka M, Kolpakov F, Miller AK, Moraru II, Nickerson D, Sahle S, Snoep JL, Le Novère N (2011) Reproducible computational biology experiments with SED-ML: the simulation experiment description markup language. BMC Syst Biol 5(1):198. https://doi.org/10.1186/1752-0509-5-198
Article Google Scholar

Download references

Funding

Funding was provided by National Institute of General Medical Sciences (Grant No. GM080219).

Author information

Authors and Affiliations

Center for Quantitative Medicine and Department of Cell Biology, University of Connecticut School of Medicine, Farmington, CT, 06030, USA
Pedro Mendes

Authors

Pedro Mendes
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro Mendes.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mendes, P. Reproducible Research Using Biomodels. Bull Math Biol 80, 3081–3087 (2018). https://doi.org/10.1007/s11538-018-0498-z

Download citation

Received: 04 June 2018
Accepted: 29 August 2018
Published: 06 September 2018
Issue Date: December 2018
DOI: https://doi.org/10.1007/s11538-018-0498-z

Keywords

Reproducibility
Biomodels
Standards