Some Aspects of Algorithmic Differentiation of Ordinary Differential Equations

Summary

Many problems in mechanics may be described by ordinary differential equations (ODE) and can be solved numerically by a variety of reliable numerical algorithms. For optimization or sensitivity analysis often the derivatives of final values of an initial value problem with respect to certain system parameters have to be computed. This paper discusses some subtle issues in the application of Algorithmic (or Automatic) Differentiation (AD) techniques to the differentiation of numerical integration algorithms. Since AD tools are not aware of the overall algorithm underlying a particular program, and apply the chain rule of differential calculus at the elementary operation level, we investigate how the derivatives computed by AD tools relate to the mathematically desired derivatives in the presence of numerical artifacts such as stepsize control in the integrator. As it turns out, the computation of the final time step is of critical importance. This work illustrates that AD tools compute the derivatives of the program employed to arrive at a solution, not just the derivatives of the solution that one would have arrived at with strictly mathematical means, and that, while the two may be different, highlevel algorithmic insight allows for the reconciliation of these discrepancies.