Abstract
This work studies the deep learning-based numerical algorithms for optimal hedging problems in markets with general convex transaction costs. Our main focus is on how these algorithms scale with the length of the trading time horizon. Based on the comparison results of the FBSDE solver by Han, Jentzen, and E (2018) and the Deep Hedging algorithm by Buehler, Gonon, Teichmann, and Wood (2019), we propose a Stable-Transfer Hedging (ST-Hedging) algorithm, to aggregate the convenience of the leading-order approximation formulas and the accuracy of the deep learning-based algorithms. Our ST-Hedging algorithm achieves the same state-of-the-art performance in short and moderately long time horizon as FBSDE solver and Deep Hedging, and generalize well to long time horizon when previous algorithms become suboptimal. With the transfer learning technique, ST-Hedging drastically reduce the training time, and shows great scalability to high-dimensional settings. This opens up new possibilities in model-based deep learning algorithms in economics, finance, and operational research, which takes advantage of the domain expert knowledge and the accuracy of the learning-based methods.
Similar content being viewed by others
Notes
Although \(\gamma \) is being small, it often comes in pair with \(\varphi _t\) hence still stays economically meaningful.
In Gonon et al. (2021), calibration is done for an equilibrium model of two agents where the asset returns depend on the risk aversion of both agents. In our single-agent model we just take the asset’s parameters to provide a more realistic numerical analysis. Although the models considered here are Bachelier, it can be generalized to Geometric Brownian motion models and the whole calculation and derivation follow if we switch our current analysis from shares positions to money in stock positions, with the unit of the transaction costs changed accordingly.
References
Acharya, V. V., & Pedersen, L. H. (2005). Asset pricing with liquidity risk. Journal of Financial Economics, 77(2), 375–410.
Ahrens, L. (2015). On using shadow prices for the asymptotic analysis of portfolio optimization under proportional transaction costs.
Almgren, R. F. (2003). Optimal execution with nonlinear impact functions and trading-enhanced risk. Applied Mathematics Finance, 10(1), 1–18.
Almgren, R. F., & Chriss, N. (2001). Optimal execution of portfolio transactions. Journal of Risk, 3, 5–40.
Almgren, R.F., & Li, T.M. (2016). Option hedging with smooth market impact. Market Microstucture Liquid, 2(1).
Almgren, R.F., Thum, C., Hauptmann, E., & Li, H. (2005). Direct estimation of equity market impact. RISK, July.
Amihud, Y., Mendelson, H., & Pedersen, L. H. (2006). Liquidity and asset prices. Foundations and Trends in Finance, 1(4), 269–364.
Ankirchner, S., & Kruse, T. (2015). Optimal position targeting with stochastic linear-quadratic costs. Banach Center Publications, 104(1), 9–24.
Bank, P., Soner, H. M., & Voß, M. (2017). Hedging with temporary price impact. Mathematics Finance Economy, 11(2), 215–239.
Bayraktar, E., Cayé, T., & Ekren, I. (2018). Asymptotics for small nonlinear price impact: A PDE homogenization approach to the multidimensional case. Mathematics Finance, to appear.
Beck, C., Hutzenthaler, M., Jentzen, A., & Kuckuck, B. (2020). An overview on deep learning-based approximation methods for partial differential equations. arXiv:2012.12348.
Becker, S., Cheridito, P., & Jentzen, A. (2019). Deep optimal stopping. Journal of Machine Learning Research, 20, 74.
Bouchard, B., Fukasawa, M., Herdegen, M., & Muhle-Karbe, J. (2018). Equilibrium returns with transaction costs. Finance Stochalm, 22(3), 569–601.
Buehler, H., Gonon, L., Teichmann, J., & Wood, B. (2019). Deep hedging. Quantitative Finance, 19(8), 1271–1291.
Buehler, H., Gonon, L., Teichmann, J., Wood, B., Mohan, B., & Kochems, J. (2019). Deep hedging: Hedging derivatives under generic market frictions using reinforcement learning. Swiss Finance Institute Research Paper, (19–80).
Casgrain, P., Ning, B., & Jaimungal, S. (2019). Deep q-learning for nash equilibria: Nash-dqn. arXiv:1904.10554.
Cayé, T., Herdegen, M., & Muhle-Karbe, J. (2018) Trading with small nonlinear price impact. Annals of Applied Probability, to appear.
Cayé, T., Herdegen, M., & Muhle-Karbe, J. (2019). Trading with small nonlinear price impact. Annals of Applied Probability, to appear.
Choi, J. H., & Larsen, K. (2015). Taylor approximation of incomplete Radner equilibrium models. Finance Stochalm, 19(3), 653–679.
De Lataillade, J., Deremble, C., Potters, M., & Bouchaud, J.-P. (2012). Optimal trading with linear costs. RISK, 1(3), 1246–2047.
Delarue, F. (2002). On the existence and uniqueness of solutions to fbsdes in a non-degenerate case. Stochastic Processes and their Applications, 99(2), 209–286.
Dumas, B., & Luciano, E. (1991). An exact solution to a dynamic portfolio choice problem under transactions costs. Journal of Finance, 46(2), 577–595.
Ekeland, I., & Temam, R. (1999). Convex analysis and variational problems. Philadelphia, PA: SIAM.
Garleanu, N., & Pedersen, L. H. (2013). Dynamic trading with predictable returns and transaction costs. Journal of Finance, 68(6), 2309–2340.
Garleanu, N., & Pedersen, L. H. (2016). Dynamic portfolio choice with frictions. Journal of Economic Theory, 165, 487–516.
Gonon, L., Muhle-Karbe, J., & Shi, X. (2021). Asset pricing with general transaction costs: Theory and numerics. Mathematical Finance, 31(2), 595–648.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. Cambridge, MA: MIT Press.
Grohs, P., Hornung, F., Jentzen, A., & von Wurstemberger, P. (2018). A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of black-scholes partial differential equations. Preprint.
Guasoni, P., Rásonyi, M., et al. (2015). Hedging, arbitrage and optimality with superlinear frictions. Annals of Applied Probability, 25(4), 2066–2095.
Guasoni, P., & Weber, M. (2017). Dynamic trading volume. Mathematics of Finance, 27(2), 313–349.
Guasoni, P., & Weber, M. H. (2020). Nonlinear price impact and portfolio choice. Mathematical Finance, 30(2), 341–376.
Han, J., Jentzen, A., & Solving, W. E. (2018). High-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34), 8505–8510.
Han, J., & Long, J. (2020). Convergence of the deep bsde method for coupled fbsdes. Probability, Uncertainty and Quantitative Risk, 5(1), 1–33.
Han, J., & Weinan, E. (2016). Deep learning approximation for stochastic control problems.
Herdegen, M., & Muhle-Karbe, J. (2018). Stability of radner equilibria with respect to small frictions. Finance and Stochastics, 22(2), 443–502.
Hu, R. (2019) Deep fictitious play for stochastic differential games. arXiv:1903.09376.
Huré, C., Pham, H., Bachouch, A., Langrené, N. (2018). Deep neural networks algorithms for stochastic control problems on finite horizon, part i: convergence analysis. arXiv:1812.04300.
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448–456.
Janeček, K., & Shreve, S. E. (2010). Futures trading with transaction costs. Illinois Journal of Mathematics, 54(4), 1239–1284.
Kallsen, J., & Li, S. (2013). Portfolio optimization under small transaction costs: A convex duality approach. arXiv:1309.3479.
Kallsen, J., & Muhle-Karbe, J. (2017). The general structure of optimal investment and consumption with small transaction costs. Mathematics of Finance, 27(3), 659–703.
Kohlmann, M., & Tang, S. (2002). Global adapted solution of one-dimensional backward stochastic Riccati equations, with application to the mean-variance hedging. Stochastic Process of Application, 97(2), 255–288.
Lillo, F., Farmer, J. D., & Mantegna, R. N. (2003). Master curve for price-impact function. Nature, 421, 129–130.
Liu, H. (2004). Optimal consumption and investment with transaction costs and multiple risky assets. The Journal of Finance, 59(1), 289–338.
Min, M., & Hu, R. (2021). Signatured deep fictitious play for mean field games with common noise. arXiv:2106.03272.
Moallemi, C.C., & Wang, M. (2021). A reinforcement learning approach to optimal execution.
Moreau, L., Muhle-Karbe, J., & Soner, H. M. (2017). Trading with small price impact. Mathematics of Finance, 27(2), 350–400.
Muhle-Karbe, J., Shi, X., & Yang, C. (2020) An equilibrium model for the cross-section of liquidity premia. Preprint, November.
Mulvey, J. M., Sun, Y., Wang, M., & Ye, J. (2020). Optimizing a portfolio of mean-reverting assets with transaction costs via a feedforward neural network. Quantitative Finance, 20(8), 1239–1261.
Nevmyvaka, Y., Feng, Y., & Kearns, M. (2006). Reinforcement learning for optimized trade execution. In Proceedings of the 23rd international conference on Machine learning, pages 673–680, 2006.
Reppen, A.M., & Soner, H.M. (2020). Bias-variance trade-off and overlearning in dynamic decision problems. arXiv:2011.09349.
Ruf, J., & Wang, W. (2021). Hedging with linear regressions and neural networks. Journal of Business & Economic Statistics, (just-accepted):1–33.
Sannikov, Y., & Skrzypacz, A. (2016). Dynamic trading: price inertia and tront-running. Preprint.
Shi, X. (2020). Equilibrium asset pricing with transaction costs. PhD thesis, Carnegie Mellon University, Pittsburgh, PA, USA.
Shreve, S.E., & Soner, H.M. (1994). Optimal investment and consumption with transaction costs. The Annals of Applied Probability, pages 609–692.
Soner, H. M., & Touzi, N. (2013). Homogenization and asymptotics for small transaction costs. SIAM Journal on Control and Optimization, 51(4), 2893–2921.
Sutton, R.S., & Barto, A.G. (2018). Reinforcement learning: An introduction. MIT Press.
Wang, H., Zariphopoulou, T., & Zhou, X. Y. (2020). Reinforcement learning in continuous time and space: A stochastic control approach. Journal of Machine Learning Research, 21, 198–1.
Wang, H., & Zhou, X. Y. (2020). Continuous-time mean-variance portfolio selection: A reinforcement learning framework. Mathematical Finance, 30(4), 1273–1308.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The authors thank the two anonymous reviewers for their careful readings and suggestions, and thank Steven Campell, Lukas Gonon, Jiequn Han, Ruimeng Hu, Steven Kou, Johannes Muhle-Karbe, Junru Shao, Mete Soner, and Xunyu Zhou for useful comments and fruitful discussions. Part of the work is supported by the MA Mentored Research Program in Statistics at Columbia University.
Appendices
Appendix A: General Asymptotics Results
As already emphasized above, a general existence proof for the FBSDE system (2.13)–(2.14) remains a challenging open problem. To obtain tractable results with the general transaction costs under Assumption 2.1 and obtain explicit approximation trading strategies as in Cayé et al. (2018); Guasoni and Weber (2020), we focus on the financial market with the following assumptions:
Assumption A.1
-
(i)
The processes \(\Lambda \) and \(\sigma ^2\) are bounded away from zero;
-
(ii)
The processes \({\bar{a}}\), \({\bar{b}}\), \(\Lambda \), and \(\sigma \) are essentially bounded Itô processes.
In this section, we establish the asymptotic optimal strategy for the frictional mean-variance preference (2.3). For better readability, we introduce the construction of optimal strategy under Assumption A.1 in Appedix A.1. The proof for the main approximation result is in Appendix A.
1.1 Asymptotically optimal trading strategies
We show that, under Assumption (), the smallness assumption on the transaction costs level \(\lambda \) should also be a relative quantity with respect to the trading time horizon T. The following two results are the main ingredients for the asymptotic trading strategy.
A nonlinear ODE The first ingredient to cook up the leading-order approximation is the solution to a nonlinear ODE, which is also essential to the analysis of Gonon et al. (2021), Lemma 3.4 and the formal analysis of Shi (2020), Lemma 2.5.
Lemma A.2
Suppose the instantaneous transaction cost G satisfies Assumption 2.1. Then the ordinary differential equation
has a unique solution g on \(\mathbb {R}\) such that \(xg(x)\le 0\) for all \(x\in \mathbb {R}\). Moreover, g is odd, non-increasing on \(\mathbb {R}\) and g satisfies the growth conditions
where \(G^*\) is the Legendre transform of G.
Remark A.3
If the time horizon T is large, then the solution to the optimal trading strategy should become stationary. Such a stationary solution should in turn solve the essential nonlinear ODE (3.5). Far from the terminal time T, it is natural to expect that the correct solution is still identified by the same growth condition in the space variable as (A.1).
For power functions \(G(x) = |x|^q /q\), \(q\in (1,2]\), the Legendre transform is
where \(p = q/(q-1)\) is the conjugate of q. In this case, with proper inner and outer rescaling coefficients, (3.5) is exactly the same ODE which plays an important role in Lemma 19 and Lemma 21 in Guasoni and Weber (2020) and Lemma 3.1 in Cayé et al. (2019).
A fast mean-reverting SDE The second ingredient is the existence and uniqueness of a strong solution to a fast mean-reverting SDE.
Lemma A.4
Let g be the solution to the ODE (3.5) from Lemma A.2. There exists a unique strong solution of the SDE
Moreover, this process is a recurrent diffusion.
Remark A.5
When \(G(x) = x^2/2\) and \({\bar{b}}\), \({\bar{a}}\), \(\sigma \) and \(\Lambda =1\) are all constants, the solution to (3.5) is \(g(x;\gamma ,\sigma ,{\bar{a}},\lambda ) = -\sqrt{\gamma \sigma ^2 \lambda }x\), and the dynamics (A.2) becomes
This is an Ornstein-Uhlenbeck process, which is mean-reverting. In general, the requirement of \(xg(x)\le 0\) ensures that the dynamic (A.2) is indeed mean-reverting and converges to an ergodic limit.
With these two ingredients on hand, we now present our first results in the following theorem:
Theorem A.6
Let g be the solution to (3.5) and \(\left( \Delta _t\right) _{t\ge 0}\) the solution to (A.2). Then under Assumption 2.1 and Assumption A.1, for all competing admissible strategies \({\dot{\psi }}\), we have
Theorem A.6 shows that, under Assumption A.1 the smallness is not only an absolute quantity on \(\sqrt{\lambda }\), but also on the relative quantity \(\sqrt{\lambda }/T\), i.e. the smallness of \(\lambda \) should also be relative quantity with respect to the trading time horizon. Notice that the order of the smallness is derived as a coarse upper bound for every transaction costs function G that satisfies Assumption 2.1. In fact, with the specific form of the transaction costs G, we can have finer estimations, as in the example of quadratic costs shown in Corollary B.2. For the general power costs case and proportional costs case, we refer the reader to the discussion of Theorem 3.3 in Cayé et al. (2019) and Theorem 4.2 in Gonon et al. (2021).
1.2 Proof of Sect. A.1
The proof of Lemma A.2 follows the same procedure as the proof of Lemma 3.4 in Gonon et al. (2021), and Lemma A.4 follows the same procedure as the proof of Lemma 3.5 in Gonon et al. (2021).
Here we provide some auxiliary results on the function g from (3.5), which are immediate results from the proof of Lemma 3.4 in Gonon et al. (2021).
Corollary A.7
Let g be the solution to (3.5) from Lemma A.2. Then the following holds:
-
1.
There exists a constant \(C_G>0\) that only depends on G such that for all \(x\in \mathbb {R}\),
$$\begin{aligned} |g(x;\gamma ,\sigma ,{\bar{a}},\lambda )| \le C_G \sqrt{\lambda }\left( \sqrt{\lambda } + \sqrt{\gamma \sigma ^2} |x|\right) . \end{aligned}$$(A.4) -
2.
There exists a constant \(K_G>0\) that only depends on G such that for all \(x\in \mathbb {R}\),
$$\begin{aligned} |g'(x;\gamma ,\sigma ,{\bar{a}},\lambda )| \le \sqrt{\gamma \sigma ^2\lambda } K_G. \end{aligned}$$(A.5)
Corollary A.8
Let g be the solution to (3.5) from Lemma A.2 and let the process \(\Delta \) be the strong solution to (A.2) from Lemma A.4.
-
1.
We have the following uniform moment bounds
$$\begin{aligned} \sup _{T\ge 0}\mathbb {E}\left[ |\Delta _T|^k\right] <\infty , \qquad \text{ for } \text{ all } \quad k\in \mathbb {N}. \end{aligned}$$(A.6) -
2.
There exists \(M>0\), such that for an arbitrary process X with dynamic
$$\begin{aligned} dX_t = \mu _t^X \textrm{d}t + \sigma _t^X \textrm{d}W_t, \end{aligned}$$the following inequality holds a.s.:
Proof of Theorem A.6
With the strategy, we write
hence with (2.10), we have
Consider a competing admissible strategy \(\psi \) and, to ease notation, set
hence
Equation (A.8) and the convexity of G yield
We now analyze the terms on the right-hand side of (A.9). The inequality (A.7) from Lemma A.8 in turn yields
Here, the local martingale part is a true martingale. Indeed, by Hölder’s inequality, the integrability condition \(\varphi \sigma \in \mathbb {H}^2\) and the boundedness of \(g'\) established in Corollary A.7,
Also taking into account that
we can therefore use (A.10) to replace the second and the third terms on the right-hand side of (A.9), obtaining
The Cauchy-Schwartz inequality yields
Moreover, it follows thatϕ
Together with the transversality condition (2.5), it follows that
and again by Hölder’s inequality,
Therefore, the trading rate \(\dot{\varphi }\) is indeed asymptotically optimal as:
\(\square \)
Appendix B: Asymptotic results for quadratic costs
When the transaction costs is considered to be quadratic, i.e. \(G(x) = x^2/2\), we have a linear function as \(\left( G'\right) ^{-1} (x) = x\). Hence the forward equation (2.13) becomes linear with respect to the backward component Y, and the existence and uniqueness can be established as in Delarue (2002), Kohlmann and Tang (2002), provided the coefficients satisfies certain regularities. However, for general function G satisfying Assumption (), the generator for the forward component is not globally Lipschitz hence no general theory is available for the FBSDE system (2.13)–(2.14).
1.1 A concrete example
As already emphasized above, a general existence proof for the FBSDE system (2.13)–(2.14) remains a challenging open problem. Let us just briefly sketch how the nonlinear FBSDE (2.13)–(2.14) reduces to a nonlinear PDE, with the following assumptions on the market:
Assumption B.1
-
(i)
the frictionless strategy satisfies that \({{\bar{b}}}_t=0\) and \({{\bar{a}}}_t = {{\bar{a}}}\);
-
(ii)
the volatility process of the stock price remain constant \(\sigma >0\) in the models with and without transaction costs;
-
(iii)
the cost parameter is constant (\(\Lambda =1\)).
Under Assumption B.1, the forward-backward system (2.13)–(2.14) in turn becomes autonomous,
When the transaction costs is quadratic, i.e. \( G(x) = x^2/2\), we can create the optimal strategy explicitly. Accordingly, the approximation can be made more precise, and we summarize the results as follows:
Corollary B.2
For \(G(x) = x^2/2\), define the strategy
Then under Assumption B.1, for all competing admissible strategies \({\dot{\psi }}\), we have
where \(\Delta \) is the following Ornstein-Uhlenbeck process:
Remark B.3
The optimality of \({\dot{\varphi }}\) by (B.3) is derived in Theorem 4.5 (Muhle-Karbe et al. 2020). The approximation order is a straight comparison between the candidate trading rate given by \(-\sqrt{{\gamma \sigma ^2}/{\lambda }}\ \Delta \) and the optimal trading rate \({\dot{\varphi }}\).
When there is no liquidity risk, i.e. when \(\Lambda =1\) throughout the trading horizon, the smallness assumption on the liquidity \(\lambda \) is purely a relative quantity, comparing to the trading horizon T. Here with quadratic trading costs, the approximation is \(\lambda /T\), which is finer than the overall approximation \(\sqrt{\lambda }/T\).
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Shi, X., Xu, D. & Zhang, Z. Deep learning algorithms for hedging with frictions. Digit Finance 5, 113–147 (2023). https://doi.org/10.1007/s42521-023-00075-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42521-023-00075-z