Numerical Methods for Discounted Systems of Infinite Horizon

In the previous chapters we presented both the theoretical background of optimal control theory and some interesting examples from the application fields of drugs, corruption, and terror. What remains is the question of how to actually compute the optimal solutions. How should we apply the theoretical results to retrieve algorithms for numerical calculations? A broad range of numerical algorithms can be used to solve optimal control problems, and presenting the underlying theory is far beyond the scope of this book. We therefore restrict our considerations to the special class of autonomous, discounted infinite time horizon problems.We concentrate on this restricted class of optimal control models because they are the most commonly investigated problems in an economic context. Furthermore, we provide a collection of MATLAB files (OCMat toolbox)1 enabling the numerical calculations of such optimal control problems.

Several approaches can be chosen to solve optimal control problems. The method presented here uses Pontryagin's Maximum Principle to establish the corresponding canonical system. In its essence, solving an optimal control problem is translated to the problem of analyzing the canonical system (see Definition 3.6). Before we go into further detail we have to introduce some notational specifics and general techniques that are used throughout this chapter.