On stochastic control for time changed Lévy dynamics

Controlled stochastic differential equations driven by time changed Lévy noises do not enjoy the Markov property in general, but can be treated in the framework of general martingales. From the modelling point of view, time changed noises constitute a feasible way to include time dependencies at noise level and still keep a reasonably simple structure. Furthermore, they are easy to simulate, with the result that time change Lévy dynamics attract attention in various fields of application. In this work we survey an approach to stochastic control via maximum principle for time changed Lévy dynamics. We emphasise the role and use of different information flows in tackling the various control problems. We show how these techniques can be extended to include Volterra type dynamics and the control of forward–backward systems of equations. Our techniques make use of the stochastic non-anticipating (NA) derivative in a general martingale framework.


Introduction
From the modelling point of view, the main idea of time changing a stochastic process is to represent a stochastic function X = X t , t ≥ 0, by a process L = L t , t ≥ 0, with well-known structure serving as a base, and a (possibly stochastic) perturbation = t , t ≥ 0, of the time line This is a way to change the speed of motion along the trajectories of L, like treating the time line as an elastic band. This modelling potential as well as the easiness in simulation have σ s dw s , t ≥ 0, with respect to a Brownian motion w. Then, the result above leads to the representation where W is another Brownian motion. These considerations reflect the scaling property of Brownian motion, namely, cW t d = W c 2 t (c > 0). In this respect, it is also interesting to remark that, among all Lévy processes, besides Brownian motion, only α-stable processes share a property similar to (1). Indeed Rosinski and Woyczynski [43,Theorem 3.1] proved that stochastic integrals X t = t 0 σ s dL s , with respect to the α-stable Lévy process L (α ∈ (0, 2]) can be represented as where L is another α-stable Lévy process. Here again we observe that α-stable processes enjoy the scaling property σ L t d = L σ α t . Representations (1) and (2) triggered the use of time change in stochastic volatility modelling, which started with the works [5,8,13,14] and it is still of large interest nowadays. See, e.g., [11,15,34,48]. For a given stochastic process X , the problem of the very existence of a representation in the form X t d = W t in terms of a Brownian motion and a family of stopping times t , t ≥ 0, is the so-called Skorokhod embedding problem, which has driven researchers to discuss different solutions and to provide extensions beyond W as base, eventually to include also jumps. For a survey on the subject we can refer to [38]. On the other side, the problem of determining the time change given the observations of X and fixed the base process (as a Brownian motion W , or even a Lévy process L) is known as the recovery problem and it was first studied in [50], see also [45].
From the probabilistic perspective, two classes of time change processes have turned to be successful: the subordinators (i.e. non-decreasing Lévy processes) and the absolutely continuous time change processes, characterised by t = t 0 λ s ds, with λ being a non-negative integrable process representing the time change rate. While with the case of subordinators, the time changed Lévy process X := L is still a Lévy process, with the case of absolutely continuous time changes (with non-constant rate), the process X exits the Lévy family and even the Markov family, giving in this way chances of describing forms of time dependences within the dynamics. Time changed Lévy processes with absolutely continuous time change process are widely used in applications to finance and beyond. As illustration we can propose the representation of affine processes in terms of these time changed processes, see [36]. We recall that affine processes are popular in the modelling, e.g., of interest rate and energy futures. In the present work, we consider time change processes of the absolutely continuous type. Those noises are the stochastic driving forces of the controlled dynamics we study. The goal of this article is to discuss some techniques of stochastic optimisation for such dynamics with emphasis on the role of information and the probabilistic structure of the noise. In Sect. 2 we give a description of the framework, while in the rest of the paper we summarise an approach via maximum principle to stochastic control, providing sufficient conditions to verify optimality. This is based on [23,24,29]. In Sect. 3 we lay out the ideas and justify the technical use of different information flows. We achieve the result exploiting backward stochastic differential equations (BSDEs) driven by time changed Lévy noises. In Sect. 4 we see how to adapt these ideas to the case of Volterra type controlled dynamics. For this, a substantially modified Hamiltonian functional has to be used together with the NA-derivative [20,21]. The role of stochastic differentiation will be discussed. In Sect. 5 we extend the approach to treat forward-backward systems of equations. As a side note, we remark that there are extensions of the concept of time change to infinite dimensions such as Hilbert space valued time changes, which first appeared in [37], then also used in, e.g., [4] and recently connected to infinite dimensional stochastic volatility models as, e.g., in [12]. Also further extensions are explored in the direction of the so-called meta time changes, which are general forms of distortions of the space-time, in the sense that not only time is perturbed, but also the spacial component. These are substantially non-negative random measures, so far, extending the concept of subordination. See [9].

The noise: time-space random fields
The stochastic elements refer to the complete probability space ( , F , P). The noise is represented as a random field on the time-space, which is here given by the field X := [0, T ] × R (T < ∞), where the spacial component represents the jump size in the paths. We can define two disjoint complementary sets where the first set accommodates the random field of continuous nature (no jumps corresponds to having jump size null) and the second accommodates the random fields with jumps. We remark that our work can deal with further spacial components, for example of the type X := [0, T ] × R × R d . In this case the extra spacial component may assume different meaning: in physics, this could be the actual space, in finance, this can represent the space of available assets as, e.g., [18,22]. The noise is modelled by the random field μ, which is defined as the mixture of two components on the Borel sets B X of X described here below as conditional Gaussian measure B and conditional centered Poisson measureH .
To proceed with the formal definition, we first introduce yet another random measure, which will be directly connected with the concept of time change. We adopt the framework introduced in [29].

Definition 2
We define the random measure on B X by where the stochastic process λ = (λ B , λ H ) has non-negative, stochastically continuous components in L 1 (P × dt). The set of these processes is denoted L. The P-augmented filtration generated by is denoted Here above ν is a σ -finite measure on the Borel sets The structure of the noise (3) is linked to the two following components.

Definition 3
The conditional Gaussian measure B is a random measure on B X B such that where is the standard normal cumulative distribution function. (A2) B( 1 ) and B( 2 ) are conditionally independent given F whenever 1 and 2 are disjoint sets.
The conditional Poisson measure H is a random measure on B X H such that and H ( 2 ) are conditionally independent given F whenever 1 and 2 are disjoint sets.
Furthermore we assume that (A5) B and H are conditionally independent given F . We refer to [32] or [35] for the existence of conditional distributions in Definition 3. In (3) we use the centred conditional Poisson measure, which is the signed random measureH given byH It is easy to verify that the random measure μ in (3) is a martingale random field, see e.g. [22,Definition 2.1], with respect to both the following right-continuous information flows F and G. The first filtration is and F μ := F μ t , t ∈ [0, T ] is the P-augmented filtration generated by the values μ( ), Notice that we have G T = F T at the horizon T , while the initial information is different in the two flows: G 0 = F and F 0 is trivial. We can understand G as an enlargement of the information F by the future values of , else we can regard F as partial with respect to G. From now on, we set F := F T = G T . The filtration G has little real world application since it includes some information of the future values of . However, it will be used technically in the sequel to exploit the distributional nature of the noise from the martingale representation theorem point of view. Indeed we recall that given a square integrable martingale M with respect to some reference filtration by means of a unique stochastic integrand ϕ. Here ξ ⊥ is a stochastic remainder orthogonal to all the stochastic integrals with respect to M (Kunita-Watanabe integral representation). It is well known that ξ ⊥ is a constant whenever M is a Gaussian or a centered Poisson random measure, or a mixture of the two, and the reference filtration F M is generated by M, see [16,19] and also [21,27]. In addition we can show that in the case when the martingale M is given by μ in (3) with respect to the filtration G. See [29,Theorem 3.5]. We stress that this stochastic integral representations stand at the foundation of the solutions of BSDEs. Specifically, the following explicit stochastic representation theorem is considered in the sequel. See the original version in [20] and the versions for random fields appearing in [22,28].
by means of the NA-derivative Dξ which is defined as the L 2 ( × dP) limit of the simple predictable random fields (ϕ n ) n∈N in the form defined on some intersecting system ( nk ) k=1,...,K n ;n∈N with nk = (s nk , u nk ] × B nk and B nk belonging to a countable semiring generting B R . From (6), (7) we see that Dξ ∈ I G , which is the space of Itô integrands, i.e. G-predictable processes in L 2 (d × dP). Furthermore, by (5) it is easy to see that the NA-derivative is the dual of the Itô integral. For later use, we recall that any ϕ ∈ I G can be regarded as a stochastic process with values in Z, which is the space of functions φ : R → R such that Representation (5) is explicit, compared to (4), because it indicates, besides existence, the nature of the integrand. In fact formulae (6), (7) depend only on the random variable ξ to be represented and the framework ( , G, P, μ). This type of stochastic integral representations is in line with the well-known Clark-Ocone formula, in which the integrand is given in terms of the projections of the Malliavin derivative on the information flow. We remark that the NA-derivative coincides with the Clark-Ocone formula whenever the integrator M = μ is a Brownian motion or a centred Poisson random measure, or mixtures of the two, and when ξ is Malliavin differentiable. The domain of the Malliavin derivative is contained, but not equal, to L 2 (dP).
To conclude on this section we recall the following result [46, Theorem 3.1] (see also [32]), which explicitly connects the noise μ in (3) with the concept of time-change.

A sufficient maximum principle
Hereafter we highlight the use of information to tackle optimal control problems for dynamics driven by time-changed Lévy noises. We remark immediately that these problems cannot be solved with dynamic programming methods since the state process is, in general, not Markovian. The approach we present here is via the maximum principle. We also make use of the enlarged filtration G to exploit the structure of the noise. We refer to [29] for the original presentation of the main ideas. We aim to find the optimal controlû providing the maximal value over all admissible controls A F , where the performance functional is Here G(x, ω), ω ∈ , is a concave random variable for all x ∈ R, differentiable in x a.s. (9) is apparently standard in stochastic control, however, it is the state process X that presents elements of novelty given by the driving noise μ, which provides a non-Markovian framework in general. The state process X = X t , t ∈ [0, T ], follows the dynamics s. All partial derivatives with respect to some variable x are denoted by ∂ x here and in the sequel. Sufficient conditions for the existence of an F-adapted strong solution to (10) can be found in [33].
In line with the maximum principle (MP) approach, we introduce a Hamiltonian functional Corresponding to an admissible pair (u, which is the solution to the BSDE Here x, p, q) and we note that H is differentiable in x by the assumptions on F, b and κ. A solution to Eq. (12) is studied in [29,Theorem 4.5]. There sufficient conditions to ensure that the pair (∂ x H, ∂ x G(X T )) are standard parameters (see [29,Definition 4.1]) are given. In our context these would read: There exists Remark 1 We have to be aware that to find a solution to the BSDE (12), we work under filtration G as it is the representation theorem in the form (5) (see Theorem 1) that provides the existence of the solution as shown in [29,Theorem 4.5]. Note that if another filtration were to be used, then the BSDE would present an additional term (an orthogonal martingale) as intrinsic part of the solution. This would not be suitable to pursue a maximum principle type result (as it would involve the explicit use of the quadratic variation of the orthogonal martingale, which is not explicitly known).

Definition 4
The admissible controls in (8), (9) are F-predictable stochastic processes u : [0, T ] × → U, such that the corresponding state process X in (10) has a unique strong solution in L 2 (dt ×dP), the adjoint equation (12) has a unique solution in L 2 (dt ×dP)×I G , and The couple (u, X ) is called an admissible pair. The set of admissible controls is denoted by Once evaluating the Hamiltonian functional (11) in the solution ( p, q) to (12), we obtain a G-adapted functional. Thus we cannot apply directly the maximum principle to H to solve (8) (where actually solutions should be F-adapted) as it could be done in the case of controlled jump-diffusions, see e.g. [39]. In our context, we need then to "project" the Hamiltonian onto the information flow F regarded as "partial" information with respect to G, here we are inspired by [25]. Then we have the following result.

Theorem 3
Let λ ∈ L be fixed. Letû ∈ A F . Denote the corresponding state process asX with solution (p,q) of the adjoint equation (12). Set exists and is a concave function in x for all t ∈ [0, T ] and then (û,X ) is an optimal pair for (8), (9).
In the sequel we setb s = b s (λ s ,û s ,X s-), etc. for the coefficients associated with the admissible pair (û,X ) with solution (p,q) of the adjoint equation (12). Set b s = b s (λ s , u s , X s-) and so forth for the coefficients associated to another arbitrary admissible pair (u, X ). In additionĤ s (u, x) = H s (λ s , u, x,p s ,q s ).
In the proof we assume that the integrals are well defined. Some sufficient conditions can be given to ensure this. For instance we can assume (13)- (15) and that there exist a K 2 > 0 such that P-a.s., for all t ∈ [0, T ], λ ∈ [0, ∞) 2 , u ∈ U and all x, x ∈ R.
Proof Observe that SinceX 0 − X 0 = 0 and G is concave, we have By the Itô's formula, we have Furthermore, from the Hamiltonian functional (11), we have We can show that the integrand here above is non-negative dt × dP-a.e. applying a separating hyperplane argument (see [42,Chapter 5,Section 23]) to the concave function (16) and exploiting the maximality (17).

MP for Volterra dynamics
In a fashion similar to Sect. 3 we consider the control problem associated to the performance functional cf. (8), (9). However, we now study a state process X with Volterra type dynamics: where X 0 ∈ R and the coefficients are given by the mappings We assume b · (t, λ, u, x, ·) and κ · (t, z, λ, u, x, ·) to be F-predictable for all t ∈ [0, T ], λ ∈ [0, ∞) 2 , u ∈ U, x ∈ R and z ∈ R. We also require that they are C 1 with respect to t and to x. For later use, in order to apply the transformation rule (see [41] Theorem 3.3), we assume that for all z ∈ R λ ∈ [0, ∞) 2 , u ∈ U, x ∈ R the partial derivative of κ with respect to t (denoted ∂ t κ s (t, z, λ, u, x)) is locally bounded (uniformly in t) and satisfies for some K 3 > 0 and for each fixed s ≤ t, λ ∈ [0, ∞) 2 , u ∈ U, x ∈ R. Sufficient conditions for the existence of a strong F-adapted solution to (24) in L 2 (dt × dP) are studied in [23, Theorem 3.2].
In the Volterra framework the dynamics produce a memory effect that, together with the noise μ, makes the state process X clearly non-Markovian. So, as in Sect. 3, we propose a maximum principle approach. However, we see that we have to modify the Hamiltonian functional to accommodate the Volterra features.
Hereafter, the general space S is the space of measurable functions on [0, T ] with values in S. We introduce the Hamiltonian functional : as the mapping given by the sum of the two components As we see here above we have used the NA-derivative (6), (7). Stochastic differentiation was first used in the maximum principle approach in [1], where the controlled dynamics were jump-diffusions driven by a standard Brownian motion and centred Poisson random measure. There the Malliavin derivative was used. We recall that Malliavin calculus is tailored on the type of noise and it does not extend easily to general martingales as integrators. See, e.g., [26]. We find an extension to the case of processes with conditional independent increment in [49]. However, the very use of Malliavin calculus has the drawback that the domains of the operators involved are strict subspaces of L 2 (dP) and it is difficult to verify whether the random variables of interests belong to such domains (particularly thinking of the dependence of the variables on different controls u). To circumvent this problem [2] proposes to use an extension of the Malliavin calculus to the white noise setting. See [26] and see [3] for the case of Volterra type dynamics. The white noise framework is a generalisation of Malliavin calculus to distribution spaces and it is again strongly depending on the nature of the noise. At present there is no extension of such framework to the case of time-change Lévy noises and this is topic of current research. For those reasons, we use here the NA-derivative (6), (7) and its duality with the Itô integral, see Theorem 1. The NA-derivative is well-defined with respect to all square integrable martingales and its domain is the whole L 2 (dP). All our variables belong to such domain. For λ ∈ L and for (u, X ) in (24), we introduce the adjoint equation as the BSDE Observe that (27) is a true BSDE as the terms ∂ x H 1 in the driver ∂ x H are integrated (see [23,Remark 3.3]). Such equation is admitting G-adapted solution following [29]. Hence we can implement the same form of "projection" on the filtration F to solve the optimal control problem (22), which is studied over the following set of admissible controls.

Definition 5
The set A F of admissible controls in problem (22), (23) consists of F-predictable stochastic processes u : [0, T ] × −→ U such that X in (24) and ( p, q) in (27) have a unique strong solution in L 2 (dt × dP) and L 2 (dt × dP) × I G , respectively, and The couple (u, X ) is an admissible pair.
Theorem 4 Set λ ∈ L. Letû ∈ A F and assume that the corresponding solutionsX, (p,q) of (24) and (27) exist. Define the mapping H F as Assume that, for fixed t, the function exists and is concave. Also assume that, for all t ∈ [0, T ], Thenû is an optimal control and (û,X ) is an optimal pair for problem (22).
To ease the reading, we introduce some notation in the same style as in Sect. 3. For a givenû ∈ A · ,X represents the associated controlled dynamics. Also, b s (t) := b s (t, λ t , u t , X t ),b s (t) := b s (t, λ t ,û t ,X t ), and similarly for κ,κ, F,F, G,Ĝ. We will also write H u s := H s (λ s , u , X s ,Ŷ s ,φ s (·)), Hû s := H s (λ s ,û s ,X s ,Ŷ s ,φ s (·)) and similarly for the other functionals in (26), (28).

Proof
The arguments proceed in a fashion similar to those in Theorem 3, however the presence of the Volterra structure needs to be considered with care. Here below we present only those specific arguments treating the Volterra feature of the dynamics (24). For u ∈ A F with corresponding controlled dynamics X , we consider J (u) − J (û) = I 1 + I 2 , where From the definition of H F 0 (t) we have In the study of I 2 we exploit the concavity of G, use the Itô formula for the product and the duality between the NA-derivative and the Itô integral. Finally we obtain Now notice that, The use of Itô formula is justified by the transformation rule [41, Theorem 3.3 ] and assumption (25). Then we can substitute the above into (31). The claim is then achieved thanks to (30) following a separation hyperplane argument applied to the concave function (29).
In the arguments above we have assumed that the integrals are well defined. Sufficient conditions on the coefficients of (24) can be imposed in the same spirit as the ones proposed in Sect. 3.

MP for forward-backward systems
Inspired by the concept of recursive utility introduced in [31], we see that the performance functional in problem (22) can include evaluations involving a backward differential equation in the form where is a BSDE driven by the noise μ considered under the filtration F. The driver g · (λ, u, x, y, θ) is an F-adapted process in L 2 (dt × dP) for all λ ∈ [0, ∞) 2 , u ∈ U, x ∈ R, y ∈ R, θ ∈ Z, C 1 both with respect to s and x, and with partial derivatives in L 2 (dP). Also we assume that h(x) is F T -measurable for all x ∈ R and C 1 with respect to x, a.s. For simplicity we assume that is is bounded and with bounded derivative. These conditions are sufficient to technically guarantee the well definition of some integrals that occur later in the proofs. They can be weekend in specific models. The BSDE (33) is studied under the filtration F within the setting of general martingale noise. This yields the presence of the orthogonal martingale M as intrinsic part of the solution, which derives from the stochastic integral representations of type (4). The forward dynamics of X are given in (24) with Volterra features and satisfying the same assumptions presented in Sect. 4. Observe that (24)-(33) is a partially coupled system, the solution of which is not a challenge. First one solves (24), then the value of X T is used in the final value of (33). For the existence of a strong solution (Y , , M) to (33) we can refer, e.g., to [40].
Here above (x) is a random variable for all x ∈ R, concave and differentiable in x a.s., while F and G satisfy the same conditions introduced earlier in Sect. 3.
In [6] we see one of the first works in which a couple forward-backward system considers the backward equation with respect to a general filtration (thus presenting an orthogonal martingale part). In this work the driving noise is however a Brownian motion only. The goal of that paper is to prove existence of an optimal control. In our context we see that a maximum principle can be obtained by an adequate modification of the Hamiltonian functional, which will in turn be coupled with a forward-backward system of equations. Indeed we have given by where As in Sect. 4, the NA-derivative (6), (7) appears in presence of Volterra structures. Also we see that the functional H 0 contains elements from the backward differential equation (33) in the performance functional. For λ ∈ L and in association to the coupled system (24)-(33), we then consider the functional u, X , Y , , y 0 , ζ, p, q), and the adjoint forward-backward system (ζ, ( p, q)) here below. For θ ∈ Z, we denote with ∂ θ 0 the partial derivative with respect to θ(0) and ∇ θ z denotes the Fréchet derivative with respect to θ(z), z = 0. We also denote with d dν ∇ θ z H i (t) the Radon-Nikodym derivative of ∇ θ z H(t) with respect to ν(dz).
The process ζ satisfies stochastic forward equation Later on the value ζ 0 = y 0 = Y 0 will be associated to the initial value of the backward dynamics Y . The stochastic backward equation is For the sequel we assume: • H(t) is well defined, differentiable with respect to x, y, θ(0), u and Fréchet differentiable with respect to θ(z), z = 0.
is well defined. As already noticed in Sect. 4, we remark that (36) is a true BSDE since the term ∂ x H 1 in the driver ∂ x H is integrated (see [23,Remark 3.3]). Also the forward-backward system (35), (36) is partially coupled and studied under filtration G. The solution of (35) leads to the terminal condition in (36). Sufficient conditions for the existence of strong solutions to (35) and (36) are well known. See, e.g., [33], [29]. In this framework the admissible controls are given as follows.

Definition 6
The admissible controls in the stochastic optimal control problem with performance (32) are F-predictable stochastic processes u : [0, T ] × → U, such that the corresponding forward-backward system (X , (Y , , M)) in (24)-(33) has a unique strong solution and the adjoint forward-backward system (ζ, ( p, q)) in (35), (36) has a unique solution, and The set of admissible controls is still denoted by A F .
Following the approach already suggested in the previous sections, we present the maximum principle exploiting a form of "projection" of the Hamiltonian functional on the filtration F.

Concluding remarks
In this work we have summarised how stochastic control for time changed Lévy dynamics can be addressed in the framework of maximum principles. To exploit the structure of the time changed noise, we have used different filtrations, a technical one which is anticipating of the information on the time change process and the natural one generated by the noise. Correspondingly, we have used different types of Hamiltonian functionals. The use of different information flows has impact on the possibility to solve backward stochastic differential equations, which in turn depend on different forms of stochastic integral representations. In our work we have seen that with appropriate modifications of the Hamiltonian functionals we can deal also with Volterra type dynamics, which involve memory in the coefficients and not only in the noise via time change. The work has progressed to include forward-backward systems of equations. We remark that in Sect. 5, the forward-backward system involved a Volterra type of forward dynamics (24) and a backward stochastic differential equation (33). It is natural to ask whether one could extend this to Volterra type backward equations. It turns out that this extension in rather tricky. Indeed, if the time-change was deterministic, then we would be able to progress in our solution. In this case the filtrations involved would coincide F = G and we would not have to deal with orthogonal martingale parts in the solution of a Volterra version of Eq. (33). See [24] for details. On the opposite, if the time change is genuinely stochastic, then the Volterra version of equation (33) would present an orthogonal component, which would have to be dealt with. For this some necessary assumptions on the structure of the orthogonal martingale part would be necessary, but there are no result that would guarantee those assumption at the time being.
In this presentation we have discussed only sufficient maximum principles, which are forms of verification that an "educated" guess of possible optimal control is indeed optimal. Necessary maximum principle are useful to identify those guesses. A version of a necessary maximum principle is available in the framework of Sects. 3 and 4, see [23]. The results are not yet polished enough in the case of Volterra type dynamics as presented in Sect. 5. There as well some structure on the orthogonal martingale part of (33) would be necessary.