## Abstract

The Least-Squares Monte Carlo (LSMC) method has gained popularity in recent years due to its ability to handle multi-dimensional stochastic control problems, including problems with state variables affected by control. However, when applied to the stochastic control problems in the multi-period expected utility models, such as finding optimal decisions in life-cycle expected utility models, the regression fit tends to contain errors which accumulate over time and typically blow up the numerical solution. In this paper we propose to transform the value function of the problems to improve the regression fit, and then using either the *smearing estimate* or *smearing estimate with controlled heteroskedasticity* to avoid the re-transformation bias in the estimates of the conditional expectations calculated in the LSMC algorithm. We also present and utilise recent improvements in the LSMC algorithms such as control randomisation with policy iteration to avoid accumulation of regression errors over time. Presented numerical examples demonstrate that transformation method leads to an accurate solution. In addition, in the forward simulation stage of the control randomisation algorithm, we propose a re-sampling of the state and control variables in their full domain at each time *t* and then simulating corresponding state variable at \(t+1\), to improve the exploration of the state space that also appears to be critical to obtain a stable and accurate solution for the expected utility models.

This is a preview of subscription content, access via your institution.

## Notes

A basis function is an element of a particular basis for a function space, where the full function space can be expressed as a linear combination of some chosen functions.

Jensen’s inequality states that for a random variable

*Z*and a concave function \(\psi \), \(\psi ({\mathbb {E}}[Z]) \ge {\mathbb {E}}[\psi (Z)].\)The tower property states that when conditioning twice, with respect to nested \(\sigma \)-algebras, the smaller amount of information always prevails such that \({\mathbb {E}}[{\mathbb {E}}[Z|{\mathcal {F}}_{t+1}]|{\mathcal {F}}_{t}] = {\mathbb {E}}[Z|{\mathcal {F}}_t]\)

## References

Aïd R, Campi L, Langrené N, Pham H (2014) A probabilistic numerical method for optimal multiple switching problems in high dimension. SIAM J Financ Math 5:191–231

Andréasson JG, Shevchenko PV (2017) Assessment of policy changes to means-tested age pension using the expected utility model: implication for decisions in retirement. Risks 5:47:1-47:21

Andréasson JG, Shevchenko PV (2021) Optimal annuitisation, housing and reverse mortgage in retirement in the presence of means-tested public pension’. Preprint, available at https://ssrn.com/abstract=2985830. Accessed 10 May 2021

Andréasson JG, Shevchenko PV, Novikov A (2017) Optimal consumption, investment and housing with means-tested public pension in retirement. Insur Math Econ 75:32–47

Baser O (2007) Modeling transformed health care cost with unknown Heteroskedasticity. Appl Econ Res Bull 1:1–6

Belomestny D (2011) Pricing Bermudan options by nonparametric regression: optimal rates of convergence for lower estimates. Finance Stoch 15:655–683

Belomestny D, Kolodko A, Schoenmakers J (2010) Regression methods for stochastic control problems and their convergence analysis. SIAM J Control Optim 48:3562–3588

Blake D, Wright D, Zhang Y (2014) Age-dependent investing: optimal funding and investment strategies in defined contribution pension plans when members are rational life cycle financial planners. J Econ Dyn Control 38:105–124

Brandt MW, Goyal A, Santa-Clara P, Stroud JR (2005) A simulation approach to dynamic portfolio choice with an application to learning about return predictability. Rev Financ Stud 18:831–873

Broadie M, Glasserman P (2004) A stochastic mesh method for pricing high-dimensional American options. J Comput Finance 7:35–72

Carroll RJ, Ruppert D (1988) Transformation and weighting in regression, 1st edn. Springer Us

Chen W, Langrené N, Tarnopolskaya T (2015) Switching surfaces for optimal natural resource extraction under uncertainty. In: Weber T, McPhee MJ, Anderssen RS (eds) MODSIM2015, 21st International Congress on modelling and simulation, pp 1063–1069, Modelling and Simulation Society of Australia and New Zealand. www.mssanz.org.au/modsim2015/E6/chen.pdf. Accessed 14 June 2017

Cox JC, Ross S, Rubinstein M (1979) Option pricing: a simplified approach. J Financ Econ 7:229–263

Denault M, Delage E, Simonato J-G (2017) Dynamic portfolio choice: a simulation-and-regression approach. Optim Eng 9:1–38

Denault M, Simonato J-G, Stentoft L (2013) A simulation-and-regression approach for stochastic dynamic programs with endogenous state variables. Comput Oper Res 40:2760–2769

Duan N (1983) Smearing estimate: a nonparametric retransformation method. J Am Stati Assoc 78:605–610

Emms P (2012) Lifetime investment and consumption using a defined-contribution pension scheme. J Econ Dyn Control 36:1303–1321

Fabozzi FJ, Paletta T, Tunaru R (2017) An improved least squares Monte Carlo valuation method based on heteroscedasticity. Eur J Oper Res 263:698–706

Garlappi L, Skoulakis G (2010) Solving consumption and portfolio choice problems: the state variable decomposition method. Rev Financ Stud 23:3346–3400

Glasserman P, Bin Yu (2004) Simulation for American options: regression now or regression later? Monte Carlo and Quasi-Monte Carlo Methods 2002. Springer Berlin Heidelberg, Berlin, pp 213–226

Greene WH (2008) Econometric analysis, 6th edn. Pearson Prentice Hall, New Jersey

Harvey A (1976) Estimating regression models with multiplicative heteroscedasticity. Econometrica 44:461–465

Hoff PD, Niu X (2012) A covariance regression model. Stat Sin 22:1–31

Kharroubi I, Langrené N, Pham H (2014) A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization. Monte Carlo Methods Appl 20:145–165

Kharroubi I, Langrené N, Pham H (2015) Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps. Ann Appl Probab 25:2301–2338

Kingston G, Thorp S (2005) Annuitization and asset allocation with HARA utility. J Pension Econ Finance 4:225

Langrene N, Tarnopolskaya T, Chen W, Zhu Z, Cooksey M (2015) New regression Monte Carlo methods for high-dimensional real options problems in minerals industry. In: Weber T, McPhee MJ, Anderssen RS (eds) MODSIM2015, 21st International Congress on modelling and simulation, Modelling and Simulation Society of Australiew Zealand, pp 1077–1083. http://www.mssanz.org.au/modsim2015/E6/langrene.pdf. Accessed 14 June 2017

Longstaff FA, Schwartz ES (2001) Valuing American options by simulation: a simple least-squares approach. Rev Financ Stud 14:113–147

Muller H-G, Stadtmuller U (1987) Estimation of heteroscedasticity in regression analysis. Ann Stat 15:610–625

Nadarajah S, Margot F, Secomandi N (2017) Comparison of least squares Monte Carlo methods with applications to energy real options. Eur J Oper Res 256:196–204

Nadarajah S, Secomandi N (2017) Relationship between least squares Monte Carlo and approximate linear programming. Oper Res Lett 45:409–414

Novikov AA, Shiryaev AN (2005) On an effective solution of the optimal stopping problem for random walks. Theory Probab Appl 49:344–354

Shevchenko PV, Luo X (2016) A unified pricing of variable annuity guarantees under the optimal stochastic control framework. Risks 4:22:1-22:31

Smyth GK (1989) Generalized linear models with varying dispersion. J R Soc Stat Soc 51:47–60

Tsitsiklis JN, Van Roy B (2001) Regression methods for pricing complex American-style options. IEEE Trans Neural Netw 12:694–703

Zhang R, Langrené N, Tian Y, Zhu Z, Klebaner F, Hamza K (2018) Local control regression: improving the least squares Monte Carlo method for portfolio optimization’. Working Paper 1803.11467 https://arxiv.org

Zhang R, Nicolas Langrené Yu, Tian ZZ, Klebaner F, Hamza K (2019) Dynamic portfolio optimization with liquidity cost and market impact: a simulation-and-regression approach. Quant Finance 19(3):519–532

Zhou X, Huazhen L, Eric J (2008) Nonparametric heteroscedastic transformation regression models for skewed data with an application to health care costs. J R Stat Soc Ser B (Stat Methodol) 70:1029–1047

## Acknowledgements

This research was supported by the CSIRO-Monash Superannuation Research Cluster in Australia. Pavel Shevchenko acknowledges the support of Australian Research Council’s Discovery Projects funding scheme (project number DP160103489) and travel support from the Institute of Statistical Mathematics, Japan. We also wish to thank Nicolas Langrené for valuable discussions and comments on the LSMC method.

## Author information

### Authors and Affiliations

### Corresponding author

## Additional information

### Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Appendices

### Appendix 1. Duan’s smearing estimate

The results in this section are based on [16]. Denote the non-transformed observations \(Y_i, i=1,\ldots ,n\) and the transformed observations \(\eta _i, i=1,\ldots ,n\) such that \(\eta _i = g(Y_i)\), \(Y_i = h(\eta _i)\), i.e. \(h := g^{-1}\). Assume *g* (the transformation) and *h* (the re-transformation) are known monotonic and continuously differentiable functions, such as a CRRA utility function \(h(x)=x^\gamma /\gamma \), \(\ \gamma <0\). Consider the linear regression carried out on the transformed observations

where \(\varvec{\beta }\) is the vector of coefficients, \(\varvec{X}_i\) is the vector of covariates and \(\epsilon _i\) are the independent and identically distributed residuals from some zero mean distribution \(F(\cdot )\) with finite variance. The error terms do not need to have a known distribution, although they are expected to have zero mean and constant variance. Now, if the re-transformation is applied to the prediction of the transformed variables we would get an incorrect estimate due to Jensen’s inequality, because \({\mathbb {E}}[Y] \le h({\mathbb {E}}[\varvec{\beta }' \varvec{X} + \epsilon ])\) if *h* is a concave function such as a utility function.

Smearing estimate attempts to approximate the non-transformed expectation

after estimating the regression coefficients \(\widehat{\varvec{\beta }}\) using the empirical distribution function of the residuals \({\widehat{\epsilon }}_i = \eta _i-\widehat{\varvec{\beta }}'\varvec{X}_i\):

where \({\mathbb {I}} \{ \cdot \}\) is the indicator symbol that equals 1 if the statement in brackets \(\{\cdot \}\) is true and 0 otherwise. The estimated expectation of *Y* can then be found as

To illustrate, suppose we consider regression \(\ln Y_i = \varvec{\beta }' \varvec{X}_i + \epsilon _i\) and we want to estimate \({\mathbb {E}}[Y^\gamma /\gamma ]\), then the smearing estimate is

The smearing estimate works well for non-normal errors and can accommodate for heteroskedasticity, provided it is not related to a covariate.

### Appendix 2. Controlled Heteroskedasticity

Consider a simple model with heteroskedasticity, such as \(Y = \varvec{\beta }' \varvec{X} + \epsilon \) where \(\varvec{X}\) is a vector of covariates, \(\varvec{\beta }\) is a vector of regression coefficients, \({\mathbb {E}}[\epsilon ]=0\) and \(\mathrm {var}[\epsilon ]=\sigma ^2 c(\varvec{X})\). There are various ways to estimate function \(c(\varvec{X})\) that is causing heteroskedasticity. In particular, we adopt a popular method from Harvey [22] (also see Baser [5], Greene [21, chapter 8]). Assume \(c(\varvec{X})=e^{\varvec{{\mathcal {L}}' X}}\) to avoid negative values, where \(\varvec{{\mathcal {L}}} = {\mathcal {L}}_0, {\mathcal {L}}_1,\ldots , {\mathcal {L}}_K\) is another vector of regression coefficients. Thus

and we can write

where \(a = \ln (\sigma ^2) + {\mathcal {L}}_0\). The parameter estimates are found by two-stage procedure. First, we find the ordinary least squares estimate \(\widehat{\varvec{\beta }}\) and calculate the observed residuals \({\widehat{\epsilon }}=Y-\widehat{\varvec{\beta }}'\varvec{X}\). Then we perform the ordinary linear regression (35) where unobserved \(\epsilon \) are replaced with \({\widehat{\epsilon }}\) to estimate the variance function \({\widehat{\sigma }}^2 \exp (\widehat{\varvec{{\mathcal {L}}}}' \varvec{X})\). Finally, using estimated variance, \(\varvec{\beta }\) is approximated by the weighted least squares method. The process can be iterated to improve the estimates.

Other methods to estimate *c*(*X*) include random effect representation [23], kernel estimates [29] or via link functions [34].

### Appendix 3. Solution to multiperiod utility model

In this section we derive the analytical solution for optimal drawdown and risky asset allocation in the multiperiod utility model considered in Sect. 5.2. The objective is to find

and corresponding optimal values of controls \(\alpha =(\alpha _0,\ldots ,\alpha _{N-1})\) and \(\delta =(\delta _0,\ldots ,\delta _{N-1})\) with \(\alpha _t\in (0,1)\) and \(\delta _t\in {\mathbb {R}}\), \(t=0,\ldots ,N-1\), when

This problem can be solved with the backward in time recursion of the Bellman equation

starting with the value function at the terminal time \(t=N\), \( V_N(X_N) = {X_N^\gamma }/{\gamma }\). The optimal values of control are calculated as

Denote the stochastic component in the transition function as \(\xi _{t+1}(\delta _t) = e^{\delta _t Z_{t+1}+(1-\delta _t) r}\) and define

so that \(V_t(x)=\sup _{\alpha _t,\delta _t} {\overline{V}}_t(x;\alpha _t,\delta _t)\).

At time \(t=N-1\),

To find the optimal values of \(\alpha _{N-1}\) and \(\delta _{N-1}\), the first-order conditions for a regular interior maximum are

Using standard closed-form integrals, the second condition gives

substituting which into the first condition gives

It is simple calculus to show that at \(\alpha _{N-1}=\alpha _{N-1}^*\) and \(\delta _{N-1}=\delta _{N-1}^*\),

i.e. sufficient conditions for a regular interior maximum are satisfied.

Finally, using \(\alpha ^*_{N-1}\) and \(\delta ^*_{N-1}\) we find the value function

which will be used in the next iteration.

Similarly, at time \(t=N-2\) we obtain

Now, it is easy to see the solution for any *t* follows the pattern

where we set \(\alpha ^*_N=1\). This claim is easily proved by induction principle (i.e. assuming that claim holds for \(t+1\) one can show that it holds for *t*).

Note that in the above equations, we can use closed-form formula

Also, note that the closed form solution for consumption only problem, considered in Sect. 5.1, can be obtained by setting \(\delta _t=1\) in the above formulas in this section.

## Rights and permissions

## About this article

### Cite this article

Andréasson, J.G., Shevchenko, P.V. A bias-corrected Least-Squares Monte Carlo for solving multi-period utility models.
*Eur. Actuar. J.* **12**, 349–379 (2022). https://doi.org/10.1007/s13385-021-00288-9

Received:

Revised:

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s13385-021-00288-9

### Keywords

- Dynamic programming
- Least-Squares Monte Carlo
- Control randomisation
- Stochastic control
- Life-cycle expected utility modelling

### JEL Classification

- D91 (Intertemporal Household Choice; Life Cycle Models and Saving)
- G11 (Portfolio Choice; Investment Decisions)
- C61 (Optimization Techniques; Programming Models; Dynamic Analysis)