Dynamic portfolio choice: a simulation-and-regression approach

Abstract

Simulation-and-regression algorithms have become a standard tool for solving dynamic programs in many areas, in particular financial engineering and computational economics. In virtually all cases, the regression is performed on the state variables, for example on current market prices. However, it is possible to regress on decision variables as well, and this opens up new possibilities. We present numerical evidence of the performance of such an algorithm, in the context of dynamic portfolio choices in discrete-time (and thus incomplete) markets. The problem is fundamentally the one considered in some recent papers that also use simulations and/or regressions: discrete time, multi-period reallocation, and maximization of terminal utility. In contrast to that literature, we regress on decision variables and we do not rely on Taylor expansions nor derivatives of the utility function. Only basic tools are used, bundled in a dynamic programming framework: simulations—which can be black-boxed—as a representation of exogenous state variable dynamics; regression surfaces, as non-anticipative representations of expected future utility; and nonlinear or quadratic optimization, to identify the best portfolio choice at each time step. The resulting approach is simple, highly flexible and offers good performance in time and precision.

Appendix: Introductory example: regression on a state variable

In this appendix, we extend the introductory example of Sect. 3.2 to the case where we regress on a state variable as well. To keep it compact and clear, the section is written with variables, not numbers.

Consider a two period example for a portfolio consisting of one risky and one risk-free asset with a constant gross risk-free rate $R_{f}$. The investor, endowed with current wealth $W_{0}$ at time $t=0$, wants to maximize the utility of his wealth at time $t=2$, with an optimal asset allocation at time $t=0$ and $t=1$, without any shortsales or borrowing.

Denote the grid of portfolio weights used here as

$$\begin{aligned} x=\left\{ \begin{array}{cccc} x_{1},&x_{2},&x_{3},&x_{4} \end{array} \right\} , \end{aligned}$$

while the grid of wealths is:

$t=0$	$t=1$
	$w_{1,1}$
$w_{0}$
	$w_{2,1}$

In this example, the return process has an AR(1) structure. Hence, when forming a portfolio at $t=0$, the known return at $t=0$ is a state variable.¹⁵ To keep the example simple, only two sample paths of returns are simulated at each period:

Path	Period 0	Period 1	Period 2
a	$R_{0}^{(a)}$	$R_{1}^{(a)}$	$R_{2}^{(a)}$
b	$R_{0}^{(b)}$	$R_{1}^{(b)}$	$R_{2}^{(b)}$

In a second step, the recursive dynamic programming procedure is implemented with the above quantities. Starting at $t=1$, one period before maturity, we perform the following operations, for each state of wealth at this time period. We first generate a sample of wealth at $t=2$. For example, using the wealth level $w_{1,1}$, it is possible to generate a wealth at $t=2$ using the first weight and path a of the simulated return with

$$\begin{aligned} W\left( x_{1},R_{2}^{(a)}\right) =w_{1,1}\times \left( x_{1}R_{2}^{(a)}+R_{f}\right) . \end{aligned}$$

Using all the possible combinations of simulated returns at $t=2$ and portfolio weights on the grid, the end of period sample of wealths at $t=2$ is:

$$\begin{aligned} \left[ \begin{array}{cccccc} W\left( x_{1},R_{2}^{(a)}\right)&\cdots&W\left( x_{4},R_{2} ^{(a)}\right)&W\left( x_{1},R_{2}^{(b)}\right)&\cdots&W\left( x_{4},R_{2}^{(b)}\right) \ \end{array} \right] ^\intercal . \end{aligned}$$

Computing the utility of these end of period wealths with the utility function u(W), the following linear regression system can be formed:

$$\begin{aligned} \left[ \begin{array}{c} u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \\ u\left( W\left( \cdot \right) \right) \end{array} \right] =\left[ \begin{array}{cccccc} 1 &{}\quad x_{1} &{}\quad x_{1}^{2} &{}\quad R_{1}^{(a)} &{}\quad \left( R_{1}^{(a)}\right) ^{2} &{}\quad x_{1}R_{1}^{(a)}\\ 1 &{}\quad x_{2} &{}\quad x_{2}^{2} &{}\quad R_{1}^{(a)} &{}\quad \left( R_{1}^{(a)}\right) ^{2} &{}\quad x_{2}R_{1}^{(a)}\\ 1 &{}\quad x_{3} &{}\quad x_{3}^{2} &{}\quad R_{1}^{(a)} &{}\quad \left( R_{1}^{(a)}\right) ^{2} &{}\quad x_{3}R_{1}^{(a)}\\ 1 &{}\quad x_{4} &{}\quad x_{4}^{2} &{}\quad R_{1}^{(a)} &{}\quad \left( R_{1}^{(a)}\right) ^{2} &{}\quad x_{4}R_{1}^{(a)}\\ 1 &{}\quad x_{1} &{}\quad x_{1}^{2} &{}\quad R_{1}^{(b)} &{}\quad \left( R_{1}^{(b)}\right) ^{2} &{}\quad x_{1}R_{1}^{(b)}\\ 1 &{}\quad x_{2} &{}\quad x_{2}^{2} &{}\quad R_{1}^{(b)} &{}\quad \left( R_{1}^{(b)}\right) ^{2} &{}\quad x_{2}R_{1}^{(b)}\\ 1 &{}\quad x_{3} &{}\quad x_{3}^{2} &{}\quad R_{1}^{(b)} &{}\quad \left( R_{1}^{(b)}\right) ^{2} &{}\quad x_{3}R_{1}^{(b)}\\ 1 &{}\quad x_{4} &{}\quad x_{4}^{2} &{}\quad R_{1}^{(b)} &{}\quad \left( R_{1}^{(b)}\right) ^{2} &{}\quad x_{4}R_{1}^{(b)} \end{array} \right] \left[ \begin{array}{c} \beta _{1}\\ \beta _{2}\\ \beta _{3}\\ \beta _{4}\\ \beta _{5}\\ \beta _{6} \end{array} \right] +\left[ \begin{array}{c} \varepsilon _{1}\\ \varepsilon _{2}\\ \varepsilon _{3}\\ \varepsilon _{4}\\ \varepsilon _{5}\\ \varepsilon _{6}\\ \varepsilon _{7}\\ \varepsilon _{8} \end{array} \right] , \end{aligned}$$

where the elements on the left-hand side are the utility of wealths at $t=2$, and the lines of the independent variable matrix are formed with a constant and all first and second degree monomials obtained using any of the portfolio weights or of the returns at $t=1$ (the state variables). In this system, $\varepsilon _{i}$ are random errors with zero expected value, and $\beta _{1}$ to $\beta _{6}$ are unknown coefficients, whose values can be estimated with an ordinary least square regression. Using these estimated coefficients, the approximate expected utility, conditional on the first value of the state variable can be written as a continuous function of the portfolio weights, which can be used to find the optimal portfolio weight as:

$$\begin{aligned} \widehat{x}_{1,1}^{\left(a\right) }=\underset{0\le x\le 1}{\arg \max }{\text { } }\widehat{\beta }_{1}+\widehat{\beta }_{2}x+\widehat{\beta }_{3}x^{2} +\widehat{\beta }_{4}R_{1}^{(a)}+\widehat{\beta }_{5}\left( R_{1}^{(a)}\right) ^{2}+\widehat{\beta }_{6}xR_{1}^{(a)} \end{aligned}$$

which obtains a function value at the optimum of $\widehat{v}_{1,1}^{(a)}$ which correspond to the regression surface value. Using the optimal weight $\widehat{x}_{1,1}^{( a) }$ and the realized return $R_{2}^{( a) }$, it is possible to compute the realized value $\overline{v}_{1,1}^{( a) }=u(w_{1,1}\cdot (\widehat{x}_{1,1}R_{2}^{( a)}+R_{f}) )$.

Similarly, the approximate expected utility, conditional on the second value of our state variable can be written as a continuous function of the portfolio weights, which can be used to find the optimal portfolio weight as:

$$\begin{aligned} \widehat{x}_{1,1}^{\left( b\right) }=\underset{0\le x\le 1}{\arg \max }\,\,\widehat{\beta }_{1}+\widehat{\beta }_{2}x+\widehat{\beta }_{3}x^{2} +\widehat{\beta }_{4}R_{1}^{(b)}+\widehat{\beta }_{5}\left( R_{1}^{(b)}\right) ^{2}+\widehat{\beta }_{6}xR_{1}^{(b)} \end{aligned}$$

which yields a function value at the optimum of $\widehat{v}_{1,1}^{(b)}$ corresponding to the regression surface value. It is possible to compute the realized value as $\overline{v}_{1,1}^{( b) }=u( w_{1,1}\cdot ( \widehat{x}_{1,1}R_{2}^{( b) }+R_{f}) )$. Similar calculations for the other state of wealth leads to the table below which summarizes the computed quantities for each state of wealth:

$w_{k,1}$	$\widehat{x}_{k,1}^{( j) }$	$\widehat{v}_{k,1}^{( j) }$	$W( \widehat{x}_{k,1},R_{2}^{(j)})$	$\overline{v}_{k,1}^{( j)}$
$w_{1,1}$	$\widehat{x}_{1,1}^{(a)}$	$\widehat{v}_{1,1}^{(a)}$	$W( \widehat{x}_{1,1},R_{2}^{(a)})$	$\overline{v}_{1,1}^{(a)}$
	$\widehat{x}_{1,1}^{(b)}$	$\widehat{v}_{1,1}^{(b)}$	$W( \widehat{x}_{1,1},R_{2}^{(a)})$	$\overline{v}_{1,1}^{(b)}$
$w_{2,1}$	$\widehat{x}_{2,1}^{(a)}$	$\widehat{v}_{2,1}^{(a)}$	$W( \widehat{x}_{2,1},R_{2}^{(a)})$	$\overline{v}_{2,1}^{(a)}$
	$\widehat{x}_{2,1}^{(b)}$	$\widehat{v}_{2,1}^{(b)}$	$W( \widehat{x}_{2,1},R_{2}^{(a)})$	$\overline{v}_{2,1}^{(b)}$

Using the above values, the optimal weight at $t=0$ can be computed using a similar procedure. We first compute a sample of wealth at $t=1$ using all combinations of simulated returns and portfolio weights. This sample is

$$\begin{aligned} \left[ \begin{array}{cccccc} W\left( x_{1},R_{1}^{(a)}\right)&\cdots&W\left( x_{4},R_{1} ^{(a)}\right)&W\left( x_{1},R_{1}^{(b)}\right)&\cdots&W\left( x_{4},R_{1}^{(b)}\right) \ \end{array} \right] ^\intercal . \end{aligned}$$

Using these, we can generate a sample of points on the value function at $t=1$ that will be used as dependant variables in a regression. This can be done by linear interpolation with the pairs of wealths and realized values (the quantities summarized in the above table). For example, the value corresponding to $W( x_{1},R_{1}^{(a)})$ is obtained by a linear interpolation computed with the quantities below:

Wealth	Value
$w_{1,1}$	$\overline{v}_{1,1}^{(a)}$
$W( x_{1},R_{1}^{(a)})$	−
$w_{2,1}$	$\overline{v}_{2,1}^{(a)}$

while a value corresponding to $W( x_{4},R_{1}^{(b)})$ would be computed with

Wealth	Value
$w_{1,1}$	$\overline{v}_{1,1}^{(b)}$
$W( x_{4},R_{1}^{(b)})$	−
$w_{2,1}$	$\overline{v}_{2,1}^{(b)}$

Let us denote these interpolated values by $v( x_{1},R_{1} ^{(1)})$ and $v( x_{4},R_{1}^{(b)})$. With similar computations for the other values of the sample of simulated wealth at $t=1$, we obtain the following linear system

$$\begin{aligned} \left[ \begin{array}{c} v\left( x_{1},R_{1}^{(a)}\right) \\ v\left( x_{2},R_{1}^{(a)}\right) \\ v\left( x_{3},R_{1}^{(a)}\right) \\ v\left( x_{4},R_{1}^{(a)}\right) \\ v\left( x_{1},R_{1}^{(b)}\right) \\ v\left( x_{2},R_{1}^{(b)}\right) \\ v\left( x_{3},R_{1}^{(b)}\right) \\ v\left( x_{4},R_{1}^{(b)}\right) \end{array} \right] =\left[ \begin{array}{cccccc} 1 &{}\quad x_{1} &{}\quad x_{1}^{2} &{}\quad R_{0}^{(a)} &{}\quad \left( R_{0}^{(a)}\right) ^{2} &{}\quad x_{1}R_{0}^{(a)}\\ 1 &{}\quad x_{2} &{}\quad x_{2}^{2} &{}\quad R_{0}^{(a)} &{}\quad \left( R_{0}^{(a)}\right) ^{2} &{}\quad x_{2}R_{0}^{(a)}\\ 1 &{}\quad x_{3} &{}\quad x_{3}^{2} &{}\quad R_{0}^{(a)} &{}\quad \left( R_{0}^{(a)}\right) ^{2} &{}\quad x_{3}R_{0}^{(a)}\\ 1 &{}\quad x_{4} &{}\quad x_{4}^{2} &{}\quad R_{0}^{(a)} &{}\quad \left( R_{0}^{(a)}\right) ^{2} &{}\quad x_{4}R_{0}^{(a)}\\ 1 &{}\quad x_{1} &{}\quad x_{1}^{2} &{}\quad R_{0}^{(b)} &{}\quad \left( R_{0}^{(b)}\right) ^{2} &{}\quad x_{1}R_{0}^{(b)}\\ 1 &{}\quad x_{2} &{}\quad x_{2}^{2} &{}\quad R_{0}^{(b)} &{}\quad \left( R_{0}^{(b)}\right) ^{2} &{}\quad x_{2}R_{0}^{(b)}\\ 1 &{}\quad x_{3} &{}\quad x_{3}^{2} &{}\quad R_{0}^{(b)} &{}\quad \left( R_{0}^{(b)}\right) ^{2} &{}\quad x_{3}R_{0}^{(b)}\\ 1 &{}\quad x_{4} &{}\quad x_{4}^{2} &{}\quad R_{0}^{(b)} &{}\quad \left( R_{0}^{(b)}\right) ^{2} &{}\quad x_{4}R_{0}^{(b)} \end{array} \right] \left[ \begin{array}{c} \beta _{1}\\ \beta _{2}\\ \beta _{3}\\ \beta _{4}\\ \beta _{5}\\ \beta _{6} \end{array} \right] +\left[ \begin{array}{c} \varepsilon _{1}\\ \varepsilon _{2}\\ \varepsilon _{3}\\ \varepsilon _{4}\\ \varepsilon _{5}\\ \varepsilon _{6}\\ \varepsilon _{7}\\ \varepsilon _{8} \end{array} \right] , \end{aligned}$$

and the optimal portfolio allocation as:

$$\begin{aligned} \widehat{x}_{1,0}^{\left( 1\right) }=\underset{0\le x\le 1}{\arg \max }\,\,\widehat{\beta }_{1}+\widehat{\beta }_{2}x+\widehat{\beta }_{3}x^{2} +\widehat{\beta }_{4}R_{0}^{(a)}+\widehat{\beta }_{5}\left( R_{0}^{(a)}\right) ^{2}+\widehat{\beta }_{6}xR_{0}^{(a)}. \end{aligned}$$

We note here that only one optimization is performed since only one value of the state variable (the known value) is relevant at this point. This value has been placed, by convention, in the first sample path when simulating the returns in the preliminary step.

Table 8 Example 1, portfolio weights

Table 9 Example 2, portfolio weights

Table 10 Example 3, portfolio weights

Table 11 Example 4, portfolio weights, four-assets case

Table 12 Example 4, portfolio weights, five-assets case

Table 13 Example 4. Certainty equivalents

Table 14 Example 5, Optimal portfolio weights under mean return versus CVaR tradeoff—three risky assets, one period

Table 15 Example 5, sub-optimality levels with respect to the polynomial degrees

Fig. 1

Example 5: Efficient set frontier of performances of portfolios obtained from solving the mean-CVaR model with different risk aversion levels when using four different resolution schemes for the inner dynamic programming step. Note that the “times” and “plus” markers overlap in two locations which emphasizes the fact that these methods are incapable of identifying portfolios that trade-off between expected excess wealth and its conditional value at risk through the procedure that is described