Mathematical Programming

, Volume 141, Issue 1–2, pp 405–433 | Cite as

A robust optimization approach to experimental design for model discrimination of dynamical systems

Full Length Paper Series A
First Online:
Received:
Accepted:

Abstract

A high-ranking goal of interdisciplinary modeling approaches in science and engineering are quantitative prediction of system dynamics and model based optimization. Quantitative modeling has to be closely related to experimental investigations if the model is supposed to be used for mechanistic analysis and model predictions. Typically, before an appropriate model of an experimental system is found different hypothetical models might be reasonable and consistent with previous knowledge and available data. The parameters of the models up to an estimated confidence region are generally not known a priori. Therefore one has to incorporate possible parameter configurations of different models into a model discrimination algorithm which leads to the need for robustification. In this article we present a numerical algorithm which calculates a design of experiments allowing optimal discrimination of different hypothetic candidate models of a given dynamical system for the most inappropriate (worst case) parameter configurations within a parameter range. The design comprises initial values, system perturbations and the optimal placement of measurement time points, the number of measurements as well as the time points are subject to design. The statistical discrimination criterion is worked out rigorously for these settings, a derivation from the Kullback-Leibler divergence as optimization objective is presented for the case of discontinuous Heaviside-functions modeling the measurement decision which are replaced by continuous approximations during the optimization procedure. The resulting problem can be classified as a semi-infinite optimization problem which we solve in an outer approximations approach stabilized by a suggested homotopy strategy whose efficiency is demonstrated. We present the theoretical framework, algorithmic realization and numerical results.

Keywords

Model discrimination Experimental design Semi infinite optimization Kullback-Leibler distance 

Mathematics Subject Classification

62K05 90C34 90C90 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Albersmeyer, J.: Adjoint-based algorithms and numerical methods for sensitivity generation and optimization of large scale dynamic systems. Ph.D. thesis, University of Heidelberg, Heidelberg (2010). http://www.ub.uni-heidelberg.de/archiv/11651
  2. 2.
    Albersmeyer, J., Bock, H.G.: Sensitivity generation in an adaptive BDF-method. In: Modeling, Simulation and Optimization of Complex Processes: Proceedings of the Third International Conference on High Performance Scientific Computing. Springer, Berlin (2008)Google Scholar
  3. 3.
    Apgar J.F., Toettcher J.E., Endy D., White F.M., Tidor B.: Stimulus design for model selection and validation in cell signaling. PLoS Comput. Biol. 4(2), e30 (2008). doi: 10.1371/journal.pcbi.0040030 MathSciNetCrossRefGoogle Scholar
  4. 4.
    Arora, J.S., Elwakeil, O.A., Chahande, A.I., Hsieh, C.C.: Global optimization methods for engineering applications: A review. Struct. Multidisciplin. Optim. 9, 137–159 (1995). http://dx.doi.org/10.1007/BF01743964. doi: 10.1007/BF01743964
  5. 5.
    Atkinson A.C., Fedorov V.V.: The design of experiments for discriminating between two rival models. Biometrika 62(1), 57–70 (1975). doi: 10.1093/biomet/62.1.57 MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Balsa-Canto E., Alonso A.A., Banga J.R.: Computational procedures for optimal experimental design in biological systems. IET Syst. Biol. 2(4), 163–172 (2008). doi: 10.1049/iet-syb:20070069 CrossRefGoogle Scholar
  7. 7.
    Bauer I., Bock H.G., Körkel S., Schlöder J.P.: Numerical methods for optimum experimental design in DAE systems. J. Comput. Appl. Math. 120, 1–25 (2000)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Bell, B.M.: Automatic differentiation software cppad (2010). http://www.coin-or.org/CppAD/
  9. 9.
    Bell B.M., Burke J.V.: Algorithmic differentiation of implicit functions and optimal values. In: Bischof, C.H., Bücker, H.M., Hovland, P.D., Naumann, U., Utke, J. (eds) Advances in Automatic Differentiation, pp. 67–77. Springer, Berlin (2008)CrossRefGoogle Scholar
  10. 10.
    Bernacki J.P., Murphy R.M.: Model discrimination and mechanistic interpretation of kinetic data in protein aggregation studies. Biophys. J. 96, 2871–2887 (2009). doi: 10.1016/j.bpj.2008.12.3903 CrossRefGoogle Scholar
  11. 11.
    Biegler L.T., Cervantes A.M., Wächter A.: Advances in simultaneous strategies for dynamic process optimization. Optim. Chem. Eng. Sci. 57, 575–593 (2001)CrossRefGoogle Scholar
  12. 12.
    Billingsley P.: Probability and Measure. Wiley, New York (1986)MATHGoogle Scholar
  13. 13.
    Bock, H.G.: Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgleichungen. In: Bonner Mathematische Schriften, vol. 183. University of Bonn (1987)Google Scholar
  14. 14.
    Bock, H.G., Plitt, K.J.: A multiple shooting algorithm for direct solution of optimal control problems. In: Proceedings of the Ninth IFAC World Congress, Budapest. Pergamon, Oxford (1984)Google Scholar
  15. 15.
    Burnham K.P., Anderson D.R.: Model Selection and Multimodel inference: A practical information-theoretic approach. Springer, Berlin (2002)Google Scholar
  16. 16.
    Byrne G.D., Hindmarsh A.C.: A polyalgorithm for the numerical solution of ordinary differential equations. ACM Trans. Math. Softw. 1(1), 71–96 (1975)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Calvo M., Montijano J.I., Rández L.: On the change of step size in multistep codes. Numer. Algorithms 4, 283–304 (1993)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Chernoff, H.: Large-sample theory: parametric case. Ann. Math. Stat. 27(1), 1–22 (1956). http://www.jstor.org/stable/2236974
  19. 19.
    Cooney M.J., McDonald K.A.: Optimal dynamic experiments for bioreactor model discrimination. Appl. Microbiol. Biotechnol. 43, 826–837 (1995)CrossRefGoogle Scholar
  20. 20.
    Goldbeter A.: Biochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour. Cambridge University Press, Cambridge (1996)CrossRefMATHGoogle Scholar
  21. 21.
    Hettich, R., Kortanek, K.O.: Semi-infinite programming: theory, methods, and applications. SIAM Rev. 35(3), 380–429 (1993). http://www.jstor.org/stable/2132425
  22. 22.
    Horn R.: Statistical methods for model discrimination. applications to gating kinetics and permeation of the acetylcholine receptor channel. Biophys. J. 51, 255–263 (1987)CrossRefGoogle Scholar
  23. 23.
    HSL: A collection of fortran codes for large-scale scientific computation. See http://www.hsl.rl.ac.uk (2007)
  24. 24.
    Jain R., Knorr A.L., Bernacki J., Srivastava R.: Investigation of bacteriophage ms2 viral dynamics using model discrimination analysis and the implications for phage therapy. Biotechnol. Progress 22(6), 1650–1658 (2006). doi: 10.1021/bp060161s Google Scholar
  25. 25.
    Körkel S., Bauer I., Bock H.G., Schlöder J.P.: A sequential approach for nonlinear optimum experimental design in DAE systems. In: Keil, F., Mackens, W., Voss, H., Werther, J. (eds) Scientific Computing in Chemical Engineering II, vol. 2, Springer, Berlin (1999)Google Scholar
  26. 26.
    Kremling A., Fischer S., Gadkar K., Doyle F.J., Sauter T., Bullinger E., Allgöwer F., Gilles E.D.: A benchmark for methods in reverse engineering and model discrimination: problem formulation and solutions. Genome Res. 14(9), 1773–1785 (2004). doi: 10.1101/gr.1226004 CrossRefGoogle Scholar
  27. 27.
    Kreutz C., Timmer J.: Systems biology: experimental design. FEBS J. 276, 923–942 (2009). doi: 10.1111/j.1742-4658.2008.06843.x CrossRefGoogle Scholar
  28. 28.
    Kullback S.: Information Theory and Statistics. Dover, New York (1997)MATHGoogle Scholar
  29. 29.
    Lacey L., Dunne A.: The design of pharmacokinetic experiments for model discrimination. J. Pharmacokinet. Pharmacodyn. 12, 351–365 (1984)CrossRefGoogle Scholar
  30. 30.
    Leineweber, D.B.: Efficient reduced sqp methods for the optimization of chemical processes described by large sparse dae models. Ph.D. thesis, University of Heidelberg (1998)Google Scholar
  31. 31.
    Levchenko A., Iglesias P.: Models of eukaryotic gradient sensing: application to chemotaxis of amoebae and neutrophils. Biophys. J. 82, 50–63 (2002). doi: 10.1016/S0006-3495(02)75373-3 CrossRefGoogle Scholar
  32. 32.
    López-Fidalgo J., Tommasi C., Trandafir P.C.: An optimal experimental design criterion for discriminating between non-normal models. J. R. Stat. Soc. Ser. B 69(2), 231–242 (2007). doi: 10.1111/j.1467-9868.2007.00586.x MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Melykuti, B., August, E., Papachristodoulou, A., El-Samad, H.: Discriminating between rival biochemical network models: three approaches to optimal experiment design. BMC Syst. Biol. 4(1), 38 (2010). doi: 10.1186/1752-0509-4-38. http://www.biomedcentral.com/1752-0509/4/38
  34. 34.
    Myung J.I., Pitt M.A.: Optimal experimental design for model discrimination. Psychol. Rev. 116(3), 499–518 (2009). doi: 10.1037/a0016104 CrossRefGoogle Scholar
  35. 35.
    Polak E.: On the convergence of optimization algorithms. Rev. Française Informat. Recherche Opérationnelle 3(16), 17–34 (1969)MathSciNetMATHGoogle Scholar
  36. 36.
    Polak, E.: On the mathematical foundations of nondifferentiable optimization in engineering design. SIAM Rev. 29(1), 21–89 (1987). http://www.jstor.org/stable/2030936
  37. 37.
    Polak, E.: On the use of consistent approximations in the solution of semi-infinite optimization and optimal control problems. Math. Program. 62, 385–414 (1993). http://dx.doi.org/10.1007/BF01585175. doi: 10.1007/BF01585175
  38. 38.
    Polak E.: Optimization: Algorithms and Consistent Approximations. Springer, Berlin (1997)CrossRefMATHGoogle Scholar
  39. 39.
    Pronzato, L., Walter, E.: Robust experiment design via maximin optimization. Math. Biosci. 89(2), 161–176 (1988). doi: 10.1016/0025-5564(88)90097-1. http://www.sciencedirect.com/science/article/B6VHX-45FKF3H-4W/2/2d1d3988042e1e36ffb9cafde61c1634
  40. 40.
    Pérez, V., Renaud, J., Watson, L.: Homotopy curve tracking in approximate interior point optimization. Optim. Eng. 10, 91–108 (2009). http://dx.doi.org/10.1007/s11081-008-9042-6. doi: 10.1007/s11081-008-9042-6 Google Scholar
  41. 41.
    Salmon D.: Minimax controller design. IEEE Trans. Automat. Control 13(4), 369–376 (1968). doi: 10.1109/TAC.1968.1098941 CrossRefGoogle Scholar
  42. 42.
    Shimizu K., Aiyoshi E.: Necessary conditions for min-max problems and algorithms by a relaxation procedure. IEEE Trans. Automat. Control 25(1), 62–66 (1980)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Skanda D., Lebiedz D.: An optimal experimental design approach to model discrimination in dynamic biochemical systems. Bioinformatics 26(7), 939–945 (2010). doi: 10.1093/bioinformatics/btq074 CrossRefGoogle Scholar
  44. 44.
    Stewart W.E., Shon Y., Box G.E.P.: Discrimination and goodness of fit of multiresponse mechanistic models. AIChE J. 44(6), 1404–1412 (1998). doi: 10.1002/aic.690440618 CrossRefGoogle Scholar
  45. 45.
    Stoer J., Bulirsch R.: Introduction to Numerical Analysis, 3rd edn. No. 12 in Texts in Applied Mathematics. Springer, New York (2002)Google Scholar
  46. 46.
    Stricker C., Redman S., Daley D.: Statistical analysis of synaptic transmission: model discrimination and confidence limits. Biophys. J. R. Stat. Soc. Ser. B 67, 532–547 (1994)Google Scholar
  47. 47.
    Takors R., Wiechert W., Weuster-Botz D.: Experimental design for the identification of macrokinetic models and model discrimination. Biotechnol. Bioeng. 56(5), 564–576 (1997)CrossRefGoogle Scholar
  48. 48.
    Timmer J., Müller T.G., Swameye I., Sandra O., Klingmüller U.: Modeling the nonlinear dynamics of cellular signal transduction. Int. J. Bifurcat. Chaos 14(6), 2069–2079 (2004)CrossRefMATHGoogle Scholar
  49. 49.
    Uciński, D., Bogacka, B.: T-optimum designs for multiresponse dynamic heteroscedastic models. In: Bucchianico, A.D., Lauter, H. (eds.) Proceedings of the 7th International Workshop on Model-Oriented Design and Analysis, pp. 191–199. Physica Verlag, New York (2004)Google Scholar
  50. 50.
    Wächter, A.: An interior point algorithm for large-scale nonlinear optimization with applications in process engineering. Ph.D. thesis, Carnegie Mellon University (2002)Google Scholar
  51. 51.
    Wächter A., Biegler L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006). doi: 10.1007/s10107-004-0559-y MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer and Mathematical Optimization Society 2012

Authors and Affiliations

  1. 1.Center for Systems Biology (ZBSA)University of FreiburgFreiburgGermany
  2. 2.Institute for Numerical MathematicsUniversity of UlmUlmGermany

Personalised recommendations