Abstract
The third chapter of the book presents the main numerical methods that are useful in calculating solutions to the optimal control problems of earlier chapters when analytic methods fail. The main technique is policy improvement, but this requires an effective translation of an optimization problem for a controlled diffusion into an optimization problem for a controlled Markov chain, and various techniques for this are discussed.
Keywords
- Markov Chain
- Optimal Control Problem
- Stochastic Control Problem
- Policy Improvement
- Stochastic Optimal Control Problem
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We explain the method assuming that the state variable is one-dimensional, but the methodology works also in higher dimensions.
- 2.
For an infinite horizon problem, the value is not time dependent, so the time derivative \(\dot{V}\) of \(V\) does not appear.
- 3.
The method works with the obvious changes also for discrete-time Markov chains.
- 4.
In fact, exact maximization is not required in (3.5); finding some control which improves things will be enough provided we do not cycle round the algorithm not picking up enough gain. It is hard to state a sufficient set of conditions, but an inspection of the proof of Theorem 3.1 shows how the idea would work.
- 5.
Of course, in two dimensions, we could apply the second smooth interpolation recipe to the situation considered first, just by turning the picture through \(45^\circ \), but this does not work in higher dimensions. The convex set we inscribed the circle into is the unit \(\ell ^1\)-ball in the first situation, with \(2d\) vertices in \(d\) dimensions, whereas in the second situation we are working with the unit \(\ell ^\infty \)-ball, with \(2^d\) vertices.
- 6.
We could allow the coefficients of the SDE to depend on time, but for notational simplicity we eschew this apparent generality.
- 7.
Quite possibly this condition is not needed; at the time of writing, this issue is not decided.
- 8.
\(\ldots \) for simplicity assumed to exist and be unique \(\ldots \)
- 9.
The non-CRRA utility example of Section 2.8 is such an example.
- 10.
In higher dimensions, the story is much more complicated.
- 11.
Unique up to scalar multiples...
- 12.
See Theorem V.50.7 in [34].
- 13.
We explain the method assuming that the state variable is one-dimensional, but the methodology works also in higher dimensions.
- 14.
For an infinite horizon problem, the value is not time dependent, so the time derivative \(\dot{V}\) of \(V\) does not appear.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Rogers, L.C.G. (2013). Numerical Solution. In: Optimal Investment. SpringerBriefs in Quantitative Finance. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35202-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-35202-7_3
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35201-0
Online ISBN: 978-3-642-35202-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)