# Experimental design and model reduction in systems biology

- 23 Downloads

## Abstract

### Background

In systems biology, the dynamics of biological networks are often modeled with ordinary differential equations (ODEs) that encode interacting components in the systems, resulting in highly complex models. In contrast, the amount of experimentally available data is almost always limited, and insufficient to constrain the parameters. In this situation, parameter estimation is a very challenging problem. To address this challenge, two intuitive approaches are to perform experimental design to generate more data, and to perform model reduction to simplify the model. Experimental design and model reduction have been traditionally viewed as two distinct areas, and an extensive literature and excellent reviews exist on each of the two areas. Intriguingly, however, the intrinsic connections between the two areas have not been recognized.

### Results

Experimental design and model reduction are deeply related, and can be considered as one unified framework. There are two recent methods that can tackle both areas, one based on model manifold and the other based on profile likelihood. We use a simple sum-of-two-exponentials example to discuss the concepts and algorithmic details of both methods, and provide Matlab-based code and implementation which are useful resources for the dissemination and adoption of experimental design and model reduction in the biology community.

### Conclusions

From a geometric perspective, we consider the experimental data as a point in a high-dimensional data space and the mathematical model as a manifold living in this space. Parameter estimation can be viewed as a projection of the data point onto the manifold. By examining the singularity around the projected point on the manifold, we can perform both experimental design and model reduction. Experimental design identifies new experiments that expand the manifold and remove the singularity, whereas model reduction identifies the nearest boundary, which is the nearest singularity that suggests an appropriate form of a reduced model. This geometric interpretation represents one step toward the convergence of experimental design and model reduction as a unified framework.

## Keywords

experimental design model reduction model manifold profile likelihood## Notes

### Acknowledgments

This work is supported by funding from the National Science Foundation (CCF1552784). Peng Qiu is an ISAC Marylou Ingram Scholar.

## References

- 1.Lander, A. D. (2004) A calculus of purpose. PLoS Biol., 2, e164CrossRefGoogle Scholar
- 2.Sobie, E. A., Lee, Y. S., Jenkins, S. L. and Iyengar, R. (2011) Systems biology—biomedical modeling. Sci. Signal., 4, tr2CrossRefGoogle Scholar
- 3.Fages, F., Gay, S. and Soliman, S. (2015) Inferring reaction systems from ordinary differential equations. Theor. Comput. Sci., 599, 64–78CrossRefGoogle Scholar
- 4.Jha, S. K. and Langmead, C. J. (2012) Exploring behaviors of stochastic differential equation models of biological systems using change of measures. BMC Bioinformatics, 13, S8CrossRefGoogle Scholar
- 5.Kauffman, S. A. (1969) Metabolic stability and epigenesis in randomly constructed genetic nets. J. Theor. Biol., 22, 437–467CrossRefGoogle Scholar
- 6.Sachs, K., Gifford, D., Jaakkola, T., Sorger, P. and Lauffenburger, D. A. (2002) Bayesian network approach to cell signaling pathway modeling. Sci. STKE, 2002, pe38Google Scholar
- 7.Koch, I. (2015) Petri nets in systems biology. Soft. Syst. Model., 14, 703–710CrossRefGoogle Scholar
- 8.Materi, W. and Wishart, D. S. (2007) Computational systems biology in drug discovery and development: methods and applications. Drug Discov. Today, 12, 295–303CrossRefGoogle Scholar
- 9.Machado, D., Costa, R. S., Rocha, M., Ferreira, E. C., Tidor, B. and Rocha, I. (2011) Modeling formalisms in systems biology. AMB Express, 1, 45CrossRefGoogle Scholar
- 10.Bartocci, E. and Lió, P. (2016) Computational modeling, formal analysis, and tools for systems biology. PLoS Comput. Biol., 12, e1004591CrossRefGoogle Scholar
- 11.Kitano, H. (2002) Computational systems biology. Nature, 420, 206–210CrossRefGoogle Scholar
- 12.Aldridge, B. B., Burke, J. M., Lauffenburger, D. A. and Sorger, P. K. (2006) Physicochemical modelling of cell signalling pathways. Nat. Cell Biol., 8, 1195–1203CrossRefGoogle Scholar
- 13.Anderson, J., Chang, Y. C. and Papachristodoulou, A. (2011) Model decomposition and reduction tools for large-scale networks in systems biology. Automatica, 47, 1165–1174CrossRefGoogle Scholar
- 14.Quaiser, T., Dittrich, A., Schaper, F. and Mönnigmann, M. (2011) A simple work flow for biologically inspired model reduction—application to early JAK-STAT signaling. BMC Syst. Biol., 5, 30CrossRefGoogle Scholar
- 15.Villaverde, A. F., Henriques, D., Smallbone, K., Bongard, S., Schmid, J., Cicin-Sain, D., Crombach, A., Saez-Rodriguez, J., Mauch, K., Balsa-Canto, E., et al. (2015) BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. BMC Syst. Biol., 9, 8CrossRefGoogle Scholar
- 16.Machta, B. B., Chachra, R., Transtrum, M. K. and Sethna, J. P. (2013) Parameter space compression underlies emergent theories and predictive models. Science, 342, 604–607CrossRefGoogle Scholar
- 17.Boyd, S. and Vandenberghe, L. (2004) Convex Optimization. New York: Cambridge University PressCrossRefGoogle Scholar
- 18.Moles, C. G., Mendes, P. and Banga, J. R. (2003) Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res., 13, 2467–2474CrossRefGoogle Scholar
- 19.Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007) Parameter estimation for differential equations: a generalized smoothing approach. J. R. Stat. Soc. Series B Stat. Methodol., 69, 741–796.CrossRefGoogle Scholar
- 20.Zenker, S., Rubin, J. and Clermont, G. (2007) From inverse problems in mathematical physiology to quantitative differential diagnoses. PLoS Comput. Biol., 3, e204CrossRefGoogle Scholar
- 21.Campbell, D. A. and Chkrebtii, O. (2013) Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates. Math. Biosci., 246, 283–292CrossRefGoogle Scholar
- 22.Banga, J. R. and Balsa-Canto, E. (2008) Parameter estimation and optimal experimental design. Essays Biochem., 45, 195–210CrossRefGoogle Scholar
- 23.Kreutz, C. and Timmer, J. (2009) Systems biology: experimental design. FEBS J., 276, 923–942CrossRefGoogle Scholar
- 24.Meyer, P., Cokelaer, T., Chandran, D., Kim, K. H., Loh, P. R., Tucker, G., Lipson, M., Berger, B., Kreutz, C., Raue, A. (2014) Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach. BMC Syst. Biol., 8, 13CrossRefGoogle Scholar
- 25.Apri, M., de Gee, M. and Molenaar, J. (2012) Complexity reduction preserving dynamical behavior of biochemical networks. J. Theor. Biol., 304, 16–26CrossRefGoogle Scholar
- 26.Danø, S., Madsen, M. F., Schmidt, H. and Cedersund, G. (2006) Reduction of a biochemical model with preservation of its basic dynamic properties. FEBS J., 273, 4862–4877CrossRefGoogle Scholar
- 27.Kourdis, P. D., Palasantza, A. G. and Goussis, D. A. (2013) Algorithmic asymptotic analysis of the NF-κB signaling system. Comput. Math. Appl., 65, 1516–1534CrossRefGoogle Scholar
- 28.Radulescu, O., Gorban, A. N., Zinovyev, A. and Noel, V. (2012) Reduction of dynamical biochemical reactions networks in computational biology. Front. Genet., 3, 131CrossRefGoogle Scholar
- 29.Vanlier, J., Tiemann, C. A., Hilbers, P. A. J. and van Riel, N. A. W. (2012) An integrated strategy for prediction uncertainty analysis. Bioinformatics, 28, 1130–1135CrossRefGoogle Scholar
- 30.Vanlier, J., Tiemann, C. A., Hilbers, P. A. J. and van Riel, N. A. W. (2012) A Bayesian approach to targeted experiment design. Bioinformatics, 28, 1136–1142CrossRefGoogle Scholar
- 31.Huan, X. and Marzouk, Y. M. (2013) Simulation-based optimal Bayesian experimental design for nonlinear systems. J. Comput. Phys., 232, 288–317CrossRefGoogle Scholar
- 32.Pauwels, E., Lajaunie, C. and Vert, J. P. (2014) A Bayesian active learning strategy for sequential experimental design in systems biology. BMC Syst. Biol., 8, 102CrossRefGoogle Scholar
- 33.Liepe, J., Filippi, S., Komorowski, M. and Stumpf, M. P. H. (2013) Maximizing the information content of experiments in systems biology. PLoS Comput. Biol., 9, e1002888CrossRefGoogle Scholar
- 34.Busetto, A. G., Hauser, A., Krummenacher, G., Sunnåker, M., Dimopoulos, S., Ong, C. S., Stelling, J. and Buhmann, J. M. (2013) Near-optimal experimental design for model selection in systems biology. Bioinformatics, 29, 2625–2632CrossRefGoogle Scholar
- 35.Faller, D., Klingmüller, U. and Timmer, J. (2003) Simulation methods for optimal experimental design in systems biology. Simulation, 79, 717–725CrossRefGoogle Scholar
- 36.Casey, F. P., Baird, D., Feng, Q., Gutenkunst, R. N., Waterfall, J. J., Myers, C. R., Brown, K. S., Cerione, R. A. and Sethna, J. P. (2007) Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model. IET Syst. Biol., 1, 190–202CrossRefGoogle Scholar
- 37.Krüger, R. and Heinrich, R. (2004) Model reduction and analysis of robustness for the
*Wnt*/β-Catenin signal transduction pathway. Genome Inform., 15, 138–148Google Scholar - 38.Gerdtzen, Z. P., Daoutidis, P. and Hu, W. S. (2004) Non-linear reduction for kinetic models of metabolic reaction networks. Metab. Eng., 6, 140–154CrossRefGoogle Scholar
- 39.Vora, N. and Daoutidis, P. (2001) Nonlinear model reduction of chemical reaction systems. Aiche J., 47, 2320–2332CrossRefGoogle Scholar
- 40.Lam, S. H. (2013) Model reductions with special CSP data. Combust. Flame, 160, 2707–2711CrossRefGoogle Scholar
- 41.Kuo, J. C. W. and Wei, J. (1969) Lumping analysis in monomolecular reaction systems. analysis of approximately lumpable system. Ind. Eng. Chem. Fundam., 8, 124–133CrossRefGoogle Scholar
- 42.Liao, J. C. and Lightfoot, E. N. Jr. (1988) Lumping analysis of biochemical reaction systems with time scale separation. Biotechnol. Bioeng., 31, 869–879CrossRefGoogle Scholar
- 43.Brochot, C., Tóth, J. and Bois, F. Y. (2005) Lumping in pharmacokinetics. J. Pharmacokinet. Pharmacodyn., 32, 719–736CrossRefGoogle Scholar
- 44.Dokoumetzidis A, Aarons L (2009) Proper lumping in systems biology models. IET Syst. Biol., 3, 40–51CrossRefGoogle Scholar
- 45.Seigneur, C., Stephanopoulos, G. and Carr Jr., R. W. (1982) Dynamic sensitivity analysis of chemical reaction systems: a variational method. Chem. Eng. Sci., 37, 845–853CrossRefGoogle Scholar
- 46.Turányi, T., Bérces, T. and Vajda, S. (1989) Reaction rate analysis of complex kinetic systems. Int. J. Chem. Kinet., 21, 83–99CrossRefGoogle Scholar
- 47.Petzold, L. and Zhu, W. (1999) Model reduction for chemical kinetics: an optimization approach. Aiche J., 45, 869–886CrossRefGoogle Scholar
- 48.Liu, G., Swihart, M. T. and Neelamegham, S. (2005) Sensitivity, principal component and flux analysis applied to signal transduction: the case of epidermal growth factor mediated signaling. Bioinformatics, 21, 1194–1202CrossRefGoogle Scholar
- 49.Schmidt, H., Madsen, M. F., Danø, S. and Cedersund, G. (2008) Complexity reduction of biochemical rate expressions. Bioinformatics, 24, 848–854CrossRefGoogle Scholar
- 50.Steiert, B., Raue, A., Timmer, J. and Kreutz, C. (2012) Experimental design for parameter estimation of gene regulatory networks. PLoS One, 7, e40052CrossRefGoogle Scholar
- 51.Maiwald, T., Hass, H., Steiert, B., Vanlier, J., Engesser, R., Raue, A., Kipkeew, F., Bock, H. H., Kaschek, D., Kreutz, C., et al. (2016) Driving the model to its limit: profile likelihood based model reduction. PLoS One, 11, e0162366CrossRefGoogle Scholar
- 52.Transtrum, M. K. and Qiu, P. (2012) Optimal experiment selection for parameter estimation in biological differential equation models. BMC Bioinformatics, 13, 181CrossRefGoogle Scholar
- 53.Transtrum, M. K. and Qiu, P. (2014) Model reduction by manifold boundaries. Phys. Rev. Lett., 113, 098701CrossRefGoogle Scholar
- 54.Transtrum, M. K. and Qiu, P. (2016) Bridging mechanistic and phenomenological models of complex biological systems. PLoS Comput. Biol., 12, e1004915CrossRefGoogle Scholar
- 55.Kutalik, Z., Cho, K. H. and Wolkenhauer, O. (2004) Optimal sampling time selection for parameter estimation in dynamic pathway modeling. Biosystems, 75, 43–55CrossRefGoogle Scholar
- 56.Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G. and Meyer, T. (2009) Optimal experimental design for parameter estimation of a cell signaling model. PLoS Comput. Biol., 5, e1000558CrossRefGoogle Scholar
- 57.Hagen, D. R., White, J. K. and Tidor, B. (2013) Convergence in parameters and predictions using computational experimental design. Interface Focus, 3, 20130008CrossRefGoogle Scholar
- 58.Toni, T., Welch, D., Strelkowa, N., Ipsen, A. and Stumpf, M. P. (2009) Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface, 6, 187–202CrossRefGoogle Scholar
- 59.Frieden, B.R. (2000) Physics from fisher information: a unification. Am. J. Phys., 68, 1064–1065CrossRefGoogle Scholar
- 60.Transtrum, M. K., Machta, B. B. and Sethna, J. P. (2011) Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys. Rev. E Stat. Nonlin. Soft Matter Phys., 83, 036701CrossRefGoogle Scholar
- 61.Leis, J. R. and Kramer, M. A. (1988) The simultaneous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Trans. Math. Softw., 14, 45–60CrossRefGoogle Scholar
- 62.Kumar, A., Christofides, P. D. and Daoutidis, P. (1998) Singular perturbation modeling of nonlinear processes with nonexplicit time-scale multiplicity. Chem. Eng. Sci., 53, 1491–1504CrossRefGoogle Scholar
- 63.Snowden, T. J., van der Graaf, P. H. and Tindall, M. J. (2017) Methods of model reduction for large-scale biological systems: a survey of current methods and trends. Bull. Math. Biol., 79, 1449–1486CrossRefGoogle Scholar
- 64.Heinrich, R. and Schuster, S. (1996) The Regulation of Cellular Systems. Springer: New YorkCrossRefGoogle Scholar
- 65.Voit, E. (2012) A First Course in Systems Biology. 1st ed., Garland Science: New YorkGoogle Scholar
- 66.Okino, M. S. and Mavrovouniotis, M. L. (1998) Simplification of mathematical models of chemical reaction systems. Chem. Rev., 98, 391–408CrossRefGoogle Scholar
- 67.Wolf, J. and Heinrich, R. (2000) Effect of cellular interaction on glycolytic oscillations in yeast: a theoretical investigation. Biochem. J., 345, 321–334CrossRefGoogle Scholar
- 68.Sauter, T., Gilles, E. D., Allgöwer, F., Saez-Rodriguez, J., Conzelmann, H. and Bullinger, E. (2004) Reduction of mathematical models of signal transduction networks: simulation-based approach applied to EGF receptor signalling. Syst. Biol. (Stevenage), 1, 159–169CrossRefGoogle Scholar
- 69.Liebermeister, W., Baur, U. and Klipp, E. (2005) Biochemical network models simplified by balanced truncation. FEBS J., 272, 4034–4043CrossRefGoogle Scholar
- 70.Maertens, J., Donckels, B., Lequeux, G. and Vanrolleghem, P. (2005) Metabolic model reduction by metabolite pooling on the basis of dynamic phase planes and metabolite correlation analysis. In Proceedings of the Conference on Modeling and Simulation in Biology, Medicine and Biomedical Engineering. Linkping, SwedenGoogle Scholar