Reproducibility of Model-Based Results in Systems Biology

  • Dagmar Waltemath
  • Ron Henkel
  • Felix Winter
  • Olaf Wolkenhauer


Science requires that results are reproducible. This is naturally expected for wet-lab experiments and it is equally important for model-based results published in the literature. Reproducibility, in general, requires standards that provide the information necessary and tools that enable others to re-use this information. In computational biology, reproducibility requires not only a coded form of the model but also a coded form of the experimental setup to reproduce the analysis of the model. Well-established databases and repositories store and provide mathematical models. Recently, these databases started to distribute simulation setups together with the model code. These developments facilitate the reproduction of results. In this chapter, we outline the necessary steps towards reproducing model-based results in computational biology. We exemplify the workflow using a prominent example model of the Cell Cycle and state-of-the-art tools and standards.


Reproducibility Simulation experiments Standards SED-ML SBML CellML Model management 



Systems Biology Markup Language


Chemical Entities of Biological Interest


Resource Description Framework


Gene Ontology


TErminology for the Description of DYnamics


The COmputational Modeling in BIology NEtwork


Extensible Markup Language


Web Ontology Language


Minimum Information Required in the Annotation of Models


Uniform Resource Name


Physiome Model Repository


Complex Pathway SImulator


Simulation Experiment Description Markup Language


Minimum Information About a Simulation Experiment


Information Retrieval


Drug Disease Model Resources


  1. 1.
    Tyson J (1991) Modeling the cell division cycle: cdc2 and cyclin interactions. Proc Natl Acad Sci 88(16):7328PubMedCentralPubMedCrossRefGoogle Scholar
  2. 2.
    Novak B, Tyson J (1997) Modeling the control of DNA replication in fission yeast. Proc Natl Acad Sci 94(17):9147PubMedCentralPubMedCrossRefGoogle Scholar
  3. 3.
    Gardner TS, Dolnik M, Collins JJ (1998) A theory for controlling cell cycle dynamics using a reversibly binding inhibitor. Proc Natl Acad Sci 95(24):14190–14195PubMedCentralPubMedCrossRefGoogle Scholar
  4. 4.
    Csikasz-Nagy A, Battogtokh D, Chen KC et al (2006) Analysis of a generic model of eukaryotic cell-cycle regulation. Biophys J 90(12):4361–4379PubMedCentralPubMedCrossRefGoogle Scholar
  5. 5.
    Ferrell J Jr (2009) Q&A: systems biology. J Biol 8:2PubMedCentralPubMedCrossRefGoogle Scholar
  6. 6.
    Li C, Donizelli M, Rodriguez N et al (2010) BioModels database: an enhanced, curated and annotated resource for published quantitative kinetic models. BMC Syst Biol 4(1):92PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    Ashburner M, Ball C, Blake J et al (2000) Gene ontology: tool for the unification of biology. Nat Genet 25(1):25PubMedCentralPubMedCrossRefGoogle Scholar
  8. 8.
    de Matos P, Alcántara R, Dekker A et al (2010). Chemical entities of biological interest: an update. Nucleic acids research, 38(suppl 1):D249-D254. Google Scholar
  9. 9.
    Lassila O, Swick R et al (1998) Resource description framework (RDF) model and syntax specification. Accessed 19 Mar 2013
  10. 10.
    Herrgård MJ, Swainston N, Dobson P et al (2008) A consensus yeast metabolic network reconstruction obtained from a community approach to systems biology. Nat Biotechnol 26:1155–1160PubMedCentralPubMedCrossRefGoogle Scholar
  11. 11.
    Davenport TH, Prusak L (2000) Working knowledge: How organizations manage what they know. Harvard Business Press, BostonGoogle Scholar
  12. 12.
    Klipp E, Liebermeister W, Helbig A et al (2007) Systems biology standards—the community speaks. Nat Biotechnol 25(4):390–391PubMedCrossRefGoogle Scholar
  13. 13.
    Garfinkel D (1969) Construction of biochemical computer models. FEBS Lett 2:S9–S13PubMedCrossRefGoogle Scholar
  14. 14.
    Bellinger G, Castro D, Mills A (2004) Data, information, knowledge, and wisdom. Accessed 13 Mar 2013
  15. 15.
    Le Novère N (2006) Model storage, exchange and integration. BMC Neurosci 7:S11PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Bray T, Paoli J, Sperberg-McQueen CM et al (2010) Extensible markup language (XML) 1.0. Accessed 13 Mar 2013
  17. 17.
    McGuinness D, van Harmelen F et al (2004) OWL web ontology language overview. Accessed 13 Mar 2013
  18. 18.
    Strömbäck L, Hall D, Lambrix P (2007) A review of standards for data exchange within systems biology. Proteomics 7(6):857–867PubMedCrossRefGoogle Scholar
  19. 19.
    Hucka M, Bergmann F, Keating S et al (2010) The systems biology markup language (SBML): language specification for level 3 Version 1. Nature proceedingsGoogle Scholar
  20. 20.
    Cuellar A, Lloyd C, Nielsen P et al (2003) An overview of CellML 1.1, a biological model description language. Simulation 79(12):740–747CrossRefGoogle Scholar
  21. 21.
    Gleeson P, Crook S, Cannon R et al (2010) NeuroML: a language for describing data driven models of neurons and networks with a high degree of biological detail. PLoS Comput Biol 6(6):e1000815PubMedCentralPubMedCrossRefGoogle Scholar
  22. 22.
    Chen M, Freier A, Köhler J et al (2002) The biology Petri net markup language. In: J Desel, M Weske (ed) Proceedings of the Promise 2002, Potsdam, 2002Google Scholar
  23. 23.
    Lloyd CM, Halstead MDB, Nielsen PF (2004) CellML: its future, present and past. Prog Biophys Mol Biol 85:433–450PubMedCrossRefGoogle Scholar
  24. 24.
    Cuellar A, Nielsen P, Halstead M et al (2006) Cellml 1.1 specification. Accessed 13 Mar 2013
  25. 25.
    Le Novère N, Finney A, Hucka M et al (2005) Minimum information requested in the annotation of biochemical models (MIRIAM). Nat Biotechnol 23(12):1509–1515PubMedCrossRefGoogle Scholar
  26. 26.
    Yu T, Lloyd C, Nickerson D et al (2011) The Physiome model repository 2. Bioinformatics 27(5):743–744PubMedCrossRefGoogle Scholar
  27. 27.
    Snoep J, Olivier B (2003) JWS online cellular systems modelling and microbiology. Microbiology 149(11):3045–3047PubMedCrossRefGoogle Scholar
  28. 28.
    Le Novère N, Bornstein B, Broicher A et al (2006) BioModels database: a free, centralized database of curated, published, quantitative kinetic models of biochemical and cellular systems. Nucleic Acids Res 34(suppl 1):D689–D691PubMedCentralPubMedCrossRefGoogle Scholar
  29. 29.
    Henkel R, Endler L, Peters A, Le Novère N et al (2010) Ranked retrieval of computational biology models. BMC Bioinformatics 11(1):423PubMedCentralPubMedGoogle Scholar
  30. 30.
    Novak B, Tyson J (1993) Numerical analysis of a comprehensive model of M-phase control in Xenopus oocyte extracts and intact embryos. J Cell Sci 106(4):1153–1168PubMedGoogle Scholar
  31. 31.
    Chen K, Calzone L, Csikasz-Nagy A et al (2004) Integrative analysis of cell cycle control in budding yeast. Mol Biol Cell 15(8):3841–3862PubMedCentralPubMedCrossRefGoogle Scholar
  32. 32.
    Olivier B, Snoep J (2004) Web-based kinetic modelling using JWS online. Bioinformatics 20(13):2143–2144PubMedCrossRefGoogle Scholar
  33. 33.
    Wolstencroft K, Owen S, du Preez F et al (2011) The SEEK: a platform for sharing data and models in systems biology. Methods Enzymol 500:629–655PubMedCrossRefGoogle Scholar
  34. 34.
    Mendes P, Hoops S, Sahle S et al (2006) Computational modeling of biochemical networks using COPASI. Methods Mol Biol 500(2):17–59Google Scholar
  35. 35.
    Miller AK, Marsh J, Reeve A et al (2010) An overview of the CellML API and its implementation. BMC Bioinformatics 11:178PubMedCentralPubMedCrossRefGoogle Scholar
  36. 36.
    Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. Addison-Wesley, New YorkGoogle Scholar
  37. 37.
    Kelton W, Barton R (2003) Experimental design for simulation: experimental design for simulation. In: Proceedings of the 35th conference on Winter Simulation: driving innovation, New Orleans, 7–10 December 2003Google Scholar
  38. 38.
    Pawlikowski K, Jeong H, Lee J (2002) On credibility of simulation studies of telecommunication networks. Communications Magazine, IEEE 40(1):132–139CrossRefGoogle Scholar
  39. 39.
    Waltemath D, Adams R, Bergmann F et al (2011) Reproducible computational biology experiments with SED-ML—The simulation experiment description markup language. BMC Syst Biol 5(1):198PubMedCentralPubMedCrossRefGoogle Scholar
  40. 40.
    Waltemath D, Adams R, Beard D et al (2011) Minimum information about a simulation experiment (MIASE). PLoS Comput Biol 7(4):e1001122PubMedCentralPubMedCrossRefGoogle Scholar
  41. 41.
    Clark J, DeRose S (1999) XML path language (XPath) version 1.0. Accessed 13 Mar 2013
  42. 42.
    Courtot M, Juty N, Knüpfer C et al (2011) Controlled vocabularies and semantics in systems biology. Mol Syst Biol 7:543PubMedCentralPubMedCrossRefGoogle Scholar
  43. 43.
    Adams R (2012) SED-ED, a workflow editor for computational biology experiments written in SED-ML. Oxf Bioinform 28(8):1180–1181CrossRefGoogle Scholar
  44. 44.
    Henkel R, Le Novère N, Wolkenhauer O et al (2012) Considerations of graph-based concepts to manage computational biology models and associated simulations. In: Goltz U (ed) Proceedings of the INFORMATIK2012. Brunswig, Germany, pp 16–21Google Scholar
  45. 45.
    Nature editors (2005) In pursuit of systems. Nature 435(7038):1Google Scholar
  46. 46.
    Guyton A, Abernathy B, Langston J et al (1959) Relative importance of venous and arterial resistances in controlling venous return and cardiac output. Am J Physiol-Legacy Content 196(5):1008–1014Google Scholar
  47. 47.
    Montani JP, Mizelle HL, Adair TH et al (1989) Regulation of cardiac output during aldosterone-induced hypertension. J Hypertens 7(6):S206CrossRefGoogle Scholar
  48. 48.
    Bergmann F, Sauro H (2006) SBW-a modular framework for systems biology. In: Proceedings of the 38th conference on Winter Simulation, Monterey, 3–6 Dec 2006Google Scholar
  49. 49.
    Nordlie E, Gewaltig M, Plesser H (2009) Towards reproducible descriptions of neuronal network models. PLoS Comput Biol 5(8):e1000456PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Dagmar Waltemath
    • 1
  • Ron Henkel
    • 1
  • Felix Winter
    • 1
  • Olaf Wolkenhauer
    • 1
    • 2
  1. 1.Department of Systems Biology and BioinformaticsRostock UniversityRostockGermany
  2. 2.Stellenbosch Institute for Advanced Study (STIAS)Wallenberg Research Centre at Stellenbosch UniversityStellenboschSouth Africa

Personalised recommendations