Fast derivatives of likelihood functionals for ODE based models using adjoint-state method
- 103 Downloads
We consider time series data modeled by ordinary differential equations (ODEs), widespread models in physics, chemistry, biology and science in general. The sensitivity analysis of such dynamical systems usually requires calculation of various derivatives with respect to the model parameters. We employ the adjoint state method (ASM) for efficient computation of the first and the second derivatives of likelihood functionals constrained by ODEs with respect to the parameters of the underlying ODE model. Essentially, the gradient can be computed with a cost (measured by model evaluations) that is independent of the number of the ODE model parameters and the Hessian with a linear cost in the number of the parameters instead of the quadratic one. The sensitivity analysis becomes feasible even if the parametric space is high-dimensional. The main contributions are derivation and rigorous analysis of the ASM in the statistical context, when the discrete data are coupled with the continuous ODE model. Further, we present a highly optimized implementation of the results and its benchmarks on a number of problems. The results are directly applicable in (e.g.) maximum-likelihood estimation or Bayesian sampling of ODE based statistical models, allowing for faster, more stable estimation of parameters of the underlying ODE model.
KeywordsSensitivity analysis Ordinary differential equations Gradient Hessian Statistical computing Mathematical statistics Algorithm
We would like to thank Xavier Woot de Trixhe from Janssen Pharmaceutica for numerous very interesting discussions on PK/PD, virology, biological pathways modeling, NLMEMs and on life in general. They were an important source of motivation and provided a view from a different perspective. And we would like to thank the anonymous reviewers for their substantial input, enhancing the quality of the paper.
- Haber T, Melicher V, Michiels N, Kovac T, Nemeth B, Claes J (2016) DiffMEM. https://bitbucket.org/tomhaber/diffmem/branch/analysis
- Melicher V, Vrábel’ V (2013) On a continuation approach in Tikhonov regularization and its application in piecewise-constant parameter identification. Inverse Probl 29(11):115,008Google Scholar
- Moré JJ (1978) The Levenberg–Marquardt algorithm: implementation and theory. In: Watson G (ed) Numerical analysis, lecture notes in mathematics, vol 630, Springer, Berlin, pp 105–116Google Scholar
- Murray JD (2002) Mathematical biology I: an introduction, interdisciplinary applied mathematics, vol 17, 3rd edn. Springer-Verlag, New YorkGoogle Scholar
- Serban R, Hindmarsh AC (2005) CVODES: the sensitivity-enabled ODE solver in SUNDIALS. In: ASME 2005 international design engineering technical conferences and computers and information in engineering conference, American Society of Mechanical Engineers, pp 257–269Google Scholar
- Slodička M, Balážová A (2010) Decomposition method for solving multi-species reactive transport problems coupled with first-order kinetics applicable to a chain with identical reaction rates. Journal of Computational and Applied Mathematics 234(4):1069–1077, proceedings of the Thirteenth International Congress on Computational and Applied Mathematics (ICCAM-2008), Ghent, Belgium, 7–11 July, 2008Google Scholar