Stochastic optimal control problems with delays in the state and in the control via viscosity solutions and applications to optimal advertising and optimal investment problems

Abstract

In this manuscript we consider optimal control problems of stochastic differential equations with delays in the state and in the control. First, we prove an equivalent Markovian reformulation on Hilbert spaces of the state equation. Then, using the dynamic programming approach for infinite-dimensional systems, we prove that the value function is the unique viscosity solution of the infinite-dimensional Hamilton-Jacobi-Bellman equation. We apply these results to problems coming from economics: stochastic optimal advertising problems and stochastic optimal investment problems with time-to-build.

1 Introduction

In this manuscript, we consider a class of stochastic optimal control problems with infinite horizon with delays in the state and in the control. Precisely, the state equation is a stochastic delay differential equation (SDDE) in $\mathbb {R}^n$ of the form

$$\begin{aligned} {\left\{ \begin{array}{ll} dy(t) = \displaystyle \left[ a_0 y(t)+b_0 (u(t) )+ \int _{-d}^0 a_1(\xi )y(t+\xi )\,d\xi + \int _{-d}^0p_1(\xi )u(t+\xi )\,d\xi \right] dt \displaystyle \\ \ \ \ \ \ \ \ \ \ \ \ + \sigma _0(u(t))\, dW(t), \quad t \ge 0, \\ y(0)=\eta _0, \quad y(\xi )=\eta _1(\xi ),\; u(\xi )=\delta (\xi ),\;\; \xi \in [-d,0), \end{array}\right. } \end{aligned}$$

(1.1)

where u(t) is the control process taking values in a bounded subset U of $\mathbb {R}^p$, $\eta _0 \in \mathbb {R}^n,$ $\eta _1 \in L^2([-d,0];\mathbb {R}^n)$ are the initial conditions of y(t), $\delta \in L^2([-d,0); U)$ is the initial condition of u(t), W(t) is a Brownian motion with values in $\mathbb {R}^q$. Denoting $\eta =(\eta _0,\eta _1)$, the goal is to minimize, over all $u(\cdot )\in \mathcal {U}$, a functional of the form

$$\begin{aligned} \mathcal {J}(\eta ,\delta ;u(\cdot )) = \mathbb {E}\left[ \int _0^{\infty } e^{-\rho t} l(y(t),u(t)) dt \right] . \end{aligned}$$

The presence of delays is the crucial aspect of (SDDE): these appear linearly via the integral terms

$$\begin{aligned} \int _{-d}^0 a_1(\xi )y(t+\xi )\,d\xi + \int _{-d}^0p_1(\xi )u(t+\xi )\,d\xi , \end{aligned}$$

where the first represents the delay in the state and the second the one in the control. For a complete picture of optimal control problems with delays we refer the reader to de Feo et al. (2023), while here we will only recall some results. A similar problem with delays only in the state was treated by means of the dynamic programming method via viscosity solutions in de Feo et al. (2023). In this paper we aim to extend some of these results to the case of delays also in the control.

The main difficulty for delay problems is in the lack of Markovianity, which prevents a direct application of the dynamic programming method, e.g. see (de Feo et al. 2023). The approach we follow here, similarly to de Feo et al. (2023), is to lift the state equation to an infinite-dimensional Banach or Hilbert space (depending on the regularity of the data), in order to regain Markovianity.¹ This is done at a cost of of moving to infinite-dimension.

In de Feo et al. (2023) (where delays are only in the state) the classical approach of rewriting the state equation in the Hilbert space

$$\begin{aligned} X=:\mathbb {R}^n \times L^2([-d,0];\mathbb {R}^n) \end{aligned}$$

was used. However when delays in the control appear this procedure has to be extended carefully. One possible way would be to use the extended delay semigroup (including the control in the state-space of the delay semigroup), e.g. see (Bensoussan et al. 2007, Part II, Chapter 4) for deterministic problems. This approach brings the complication of having an unbounded control operator ("boundary control"). However, when the delays appear in a linear way in the state equation, an alternative approach can be used. Such approach was proposed by Vinter and Kwong (1981) for deterministic control problems (see also (Bensoussan et al. 2007, Part II, Chapter 4)) and extended in Gozzi and Marinelli (2006) to the stochastic setting. In Gozzi and Marinelli (2006) a linear state equation with additive noise is considered and an equivalent abstract representation of the state equation in the Hilbert space X is proved. In this paper we generalize this result of Gozzi and Marinelli (2006) to the following nonlinear state equation with multiplicative noise (note that such state equation is more general than (1.1)):

$$\begin{aligned} \left\{ \begin{array}{l} dy(t) = \displaystyle \left[ b_0 ( y(t),u(t) )+ \int _{-d}^0 a_1(\xi )y(t+\xi )\,d\xi + \int _{-d}^0p_1(\xi )u(t+\xi )\,d\xi \right] dt \\ \displaystyle \qquad \qquad + \sigma _0(y(t),u(t))\, dW(t), \quad t \ge 0 \\ y(0)=\eta _0, \quad y(\xi )=\eta _1(\xi ),\; u(\xi )=\delta (\xi ),\;\; \xi \in [-d,0), \end{array}\right. \end{aligned}$$

In this case the abstract state equation in X is of the form

$$\begin{aligned} {\left\{ \begin{array}{ll} dY(t) = [A Y(t)+b(Y(t),u(t))]dt + \sigma (Y(t),u(t))\,dW(t), \quad \forall t \ge 0, \\ Y(0) = x=M(\eta ,\delta ). \end{array}\right. } \end{aligned}$$

for suitable operators A, M and coefficients $b, \sigma$. See Theorem 3.2 for the equivalent abstract representation of the state equation in X.

Going back to our original problem, (1.1) can be rewritten on X as

$$\begin{aligned} {\left\{ \begin{array}{ll} dY(t) = [\mathcal {A} Y(t)+f(u(t))]dt + \sigma (u(t))\,dW(t), \quad \forall t \ge 0, \\ Y(0) = x=M(\eta ,\delta ), \end{array}\right. } \end{aligned}$$

(1.2)

for suitable $\mathcal {A},f, \sigma$ (see (4.1)). The functional $\mathcal {J}(\eta ,\delta ;u(\cdot ))$ is rewritten as

$$\begin{aligned} J(x;u(\cdot )) := \mathbb {E} \left[ \int _0^{\infty } e^{-\rho t}L(Y(t),u(t))\,dt \right] , \end{aligned}$$

for a suitable cost function L (see (3.8)). Having lifted the problem in the space X we are in a similar setting to the one of de Feo et al. (2023), hence we wish to proceed in a similar way. Indeed we want to use the theory of viscosity solutions in Hilbert spaces (see (Fabbri et al. 2017, Chapter 3)) in order to treat the optimal control problem. However, $\mathcal {A},f,\sigma$ in (1.2) have a different structure than the ones in de Feo et al. (2023). Indeed in de Feo et al. (2023) the unbounded operator is the classical delay operator, while $\mathcal {A}$ here it is (up to a bounded perturbation) its adjoint and the coefficient f here has a non-zero $L^2-$component.

At this point, in order to use the theory of viscosity solutions in Hilbert spaces of (Fabbri et al. 2017, Chapter 3), we rewrite the state equation by introducing a maximal dissipative operator $\tilde{\mathcal {A}}$ in the state equation, see Proposition 4.1. Next we introduce an operator B satisfying the so called weak B-condition (which is crucial in the theory of viscosity solutions in Hilbert spaces), see Proposition 4.3. Hence, we prove that the data of the problem satisfy some regularity conditions with respect to the norm induced by the operator $B^{1 / 2}$, see Lemma 4.6. This enables us to characterize the value function of the problem as the unique viscosity solution of the following fully non linear second order infinite-dimensional HJB equation

$$\begin{aligned} \rho v(x) - \langle \tilde{\mathcal {A}} x,Dv(x)\rangle _X + H(x,Dv(x),D^2v(x))=0, \quad \forall x \in X, \end{aligned}$$

where H is the Hamiltonian. See Theorem 5.4 for such result. To the best of our knowledge, this is the first existence and uniqueness result for fully nonlinear HJB equations in Hilbert spaces related to stochastic optimal control problems with delays in the state and in the control. This extends the corresponding result of de Feo et al. (2023) to the case of delays (also) in the control. Moreover in the present paper the delay kernels $a_1,p_1$ are such that their rows $a_1^j,p_1^j \in L^2([-d,0]; \mathbb {R}^n)$ for every $j \le n$. Instead in de Feo et al. (2023) a higher regularity of $a_1,a_2$ (where $a_2$ is the delay kernel associated to the diffusion) is required, i.e. $a_1^j,a_2^j \in W^{1,2}([-d,0]; \mathbb {R}^n)$ and $a_1^j(-d)=a_2^j(-d)=0$ for every $j \le n$ (of course $p_1=0$, i.e. only delays in the state are present). However we remark that the structure of the state equation in de Feo et al. (2023), with delays only in the state, is more general.

For a complete picture of the literature related to optimal control problems with delays only in the state we refer to de Feo et al. (2023), here we only list some results: for stochastic optimal control problems with delays only in the state e.g. see (Biffis et al. 2020; Biagini et al. 2022; de Feo and Święch 2023; Djehiche et al. 2022; Di Giacinto et al. 2011; Federico 2011; Federico and Tankov 2015; Fuhrman et al. 2010; Masiero and Tessitore 2022; Pang et al. 2019). For deterministic optimal control problems with delays only in the state e.g. see (Carlier and Tahraoui 2010; Federico et al. 2010, 2011).

Stochastic differential equations with delays also in the control are more difficult since in this case the so called structure condition, that is the requirement that the range of the control operator is contained in the range of the noise, does not hold, e.g. see (Fabbri et al. 2017, Subsection 2.6.8). This fact, together with the lack of smoothing of the transition semigroup associated to the linear part of the equation (this is a common feature also in problems with delays only in the state), prevent the use of standard techniques, based on mild solutions and on backward stochastic differential equations. However stochastic differential equations with delays only in the control, linear structure of the state equation and additive noise (the HJB equation is semilinear) were completely solved in Gozzi and Masiero (2017, 2017, 2021) by means of a partial smoothing property for the Ornstein-Uhlenbeck transition semigroup which permitted to apply a variant of the approach via mild solutions in the space of continuous functions. See also (Gozzi and Masiero 2023), [32], where this approach is extended to stochastic optimal control problems with unbounded control operators and applications to problems with delays only in the control (with delay kernel being a Radon measure) are given. Finally we refer to Federico and Tacconi (2014) for a deterministic optimal control problem with delays only in the control and linear structure of the state equation solved by means of viscosity solutions in the space $\mathbb {R} \times W^{1,2}([-d,0])$ (see also Remark 4.5).

At the end of the manuscript, we provide applications of our results to problems coming from economics. We consider a stochastic optimal advertising problem with delays in the state and in the control and controlled diffusion, generalizing the one from (Gozzi and Marinelli 2006; Gozzi et al. 2009) (see also Nerlove and Arrow 1962 for the original deterministic model). We characterize the value function as the unique viscosity solution of the fully non-linear HJB equation. We recall that, in the stochastic setting with additive noise, in Gozzi and Marinelli (2006), Gozzi et al. (2009) a verification theorem was proved in the context of classical solutions and optimal feedback strategies were derived. Moreover an explicit (classical) solution of the HJB equation was derived in a specific case. We also recall that in de Feo et al. (2023) the case with no delays in the control (i.e. $p_1=0$) was treated via viscosity solutions.

Finally, we consider a stochastic optimal investment problem with with time-to-build, inspired by (Fabbri and Federico 2014, p. 36) (see also e.g. (Bambi et al. 2012, 2017) for similar models in the deterministic setting). We characterize the value function as the unique viscosity solution of the fully non-linear HJB equation.

The paper is organized as follows. In Sect. 2 we introduce the problem and state the main assumptions. In Sect. 3 we prove an equivalent infinite-dimensional formulation for a more general state equation and we rewrite the problem in an infinite dimensional setting. In Sect. 4 we prove some preliminary estimates for solutions of the state equation and the value function. In Sect. 5 we introduce the notion of viscosity solution of the HJB equation and state a theorem about the existence and uniqueness of viscosity solutions, and characterize the value function as the unique viscosity solution. In Sect. 6 we provide applications to problems coming from economics: stochastic optimal advertising models and stochastic investment models with time-to-build.

2 Setup and assumptions

We denote by $M^{m \times n}$ the space of real valued $m \times n$-matrices and we denote by $|\cdot |$ the Euclidean norm in $\mathbb {R}^{n}$ as well as the norm of elements of $M^{m \times n}$ seen as linear operators from $\mathbb {R}^{m}$ to $\mathbb {R}^{n}$. We will write $x \cdot y$ for the inner product in $\mathbb {R}^n$.

Let $d>0$. We consider the standard Lebesgue space $L^2:=L^2([-d,0];\mathbb {R}^n)$ of square integrable functions from $[-d,0]$ to $\mathbb {R}^{n}$, denoting by $\langle \cdot ,\cdot \rangle _{L^{2}}$ the inner product in $L^2$ and by $|\cdot |_{L^{2}}$ the norm. We also consider the standard Sobolev space $W^{1,2}:=W^{1,2}([-d,0]; \mathbb {R}^n)$ of functions in $L^{2}$ admitting weak derivative in $L^{2}$, endowed with the inner product $\langle f,g\rangle _{W^{1,2}}:= \langle f,g\rangle _{L^{2}}+\langle f',g'\rangle _{L^{2}}$ and norm $|f|_{W^{1,2}}:=(|f|^2_{L^{2}}+|f'|^2_{L^{2}})^{\frac{1}{2}}$, which render it a Hilbert space. It is well known that the space $W^{1,2}$ can be identified with the space of absolutely continuous functions from $[-d,0]$ to $\mathbb {R}^{n}$.

Let $\tau =\left( \Omega , \mathcal {F},\left( \mathcal {F}_t\right) _{t \ge 0}, \mathbb {P}, W\right)$ be a reference probability space, that is $(\Omega , \mathcal {F}, \mathbb {P})$ is a complete probability space, $W=(W(t))_{t \ge 0}$ is a standard $\mathbb {R}^q$-valued Wiener process, $W(0)=0$, and $\left( \mathcal {F}_t\right) _{t \ge 0}$ is the augmented filtration generated by W. We consider the following controlled stochastic delay differential equation (SDDE)

$$\begin{aligned} {\left\{ \begin{array}{ll} dy(t) = \displaystyle \left[ a_0 y(t)+b_0 (u(t) )+ \int _{-d}^0 a_1(\xi )y(t+\xi )\,d\xi + \int _{-d}^0p_1(\xi )u(t+\xi )\,d\xi \right] dt \displaystyle \\ \ \ \ \ \ \ \ \ \ \ \ + \sigma _0(u(t))\, dW(t), \quad t \ge 0, \\ y(0)=\eta _0, \quad y(\xi )=\eta _1(\xi ),\; u(\xi )=\delta (\xi ),\;\; \xi \in [-d,0). \end{array}\right. } \end{aligned}$$

(2.1)

where

• (i) given a bounded measurable set $U\subset \mathbb {R}^{p}$, $u(\cdot )$ is the control process lying in the set

  $$\begin{aligned} \mathcal {U}_\tau= & {} \{u(\cdot ): \Omega \times [0,+\infty ) \rightarrow U: \ u(\cdot ) \ \text{ is } \ (\mathcal {F}_t)\text{-progressively } \text{ measurable } \text{ and } \text{ integrable } \text{ a.s. } \}; \end{aligned}$$

• (ii) $\eta _0 \in \mathbb {R}^n$ and $\eta _1 \in L^2$ are the initial conditions of the state y;

• (iii) $\delta \in L^2([-d,0); U)$ is the initial condition of the control $u(\cdot );$

• (iv) $b_0 :U \rightarrow \mathbb {R}^n$, $\sigma _0 :U \rightarrow M^{n\times q}$;

• (v) $a_{1}, p_1:[-d,0]\rightarrow M^{n \times n}$ and if $a_{1}^{j},p_1^j$ are the j-th row of $a_1(\cdot )$, $p_1(\cdot )$ respectively for $j=1,...,n$, then $a_1^{j}, p_1^{j}\in L^2([-d,0]; \mathbb {R}^n)$.

Remark 2.1

Similarly to de Feo et al. (2023) we cannot treat the case of pointwise delay (e.g. $a_1,p_1=\delta _{-d}$ the Dirac’s Delta). In de Feo et al. (2023) a higher regularity of $a_1,a_2$ (where $a_2$ is the delay kernel associated to the diffusion) is required, i.e. $a_1^j,a_2^j \in W^{1,2}([-d,0]; \mathbb {R}^n)$ and $a_1^j(-d)=a_2^j(-d)=0$ for every $j \le n$ (of course $p_1=0$, i.e. in de Feo et al. (2023) only delays in the state are present). See also (Federico et al. 2010, 2011) for similar restrictions in deterministic problems. Here, instead, we require less regularity, i.e. $a_1^{j}, p_1^{j}\in L^2([-d,0]; \mathbb {R}^n)$.

We will assume the following conditions.

Assumption 2.2

$b_0 :U \rightarrow \mathbb {R}^n$, $\sigma _0 :U \rightarrow M^{n\times q}$ are continuous and bounded.

Under Assumption 2.2, by (Revuz and Yor 1999, Theorem IX.2.1), for each initial data $\eta :=(\eta _0,\eta _1)\in \mathbb {R}^{n}\times L^2([-d,0];\mathbb {R}^n)$, $\delta \in L^2([-d,0);U)$, and each control $u(\cdot )\in \ \mathcal {U}$, there exists a unique (up to Indistinguishability) strong solution to (3.1) and this solution admits a version with continuous paths that we denote by $y^{\eta ,\delta ;u}$.

We consider the following infinite horizon optimal control problem. Given $\eta =(\eta _0,\eta _1)\in \mathbb {R}^{n}\times L^2$, $\delta \in L^2([-d,0); U)$, we define a cost functional of the form

$$\begin{aligned} \mathcal {J}(\eta ,\delta ;u(\cdot )) = \mathbb {E}\left[ \int _0^{\infty } e^{-\rho t} l(y^{\eta ,\delta ,u}(t),u(t)) dt \right] \end{aligned}$$

(2.2)

where $\rho >0$ is the discount factor, $l :\mathbb {R}^n \times U \rightarrow \mathbb {R}$ is the running cost. As in (Fabbri et al. 2017, p. 98, Equation (2.8)), we define

$$\begin{aligned} \mathcal {U}=\bigcup _{\tau }\mathcal {U}_\tau , \end{aligned}$$

where the union is taken over all reference probability spaces $\tau$. The goal is to minimize $\mathcal {J}(\eta ,\delta ;u(\cdot ))$ over all $u(\cdot )\in \mathcal {U}$. This is a standard setup of a stochastic optimal control problem (see Yong and Zhou 1999; Fabbri et al. 2017) used to apply the dynamic programming approach. We remark (see e.g. Fabbri et al. 2017, Section 2.3.2) that

$$\begin{aligned} \inf _{u(\cdot )\in \mathcal {U}}\mathcal {J}(\eta ,\delta ;u(\cdot ))=\inf _{u(\cdot )\in \mathcal {U}_\tau } \mathcal {J}(\eta ,\delta ;u(\cdot )) \end{aligned}$$

for every reference probability space $\tau$ so the optimal control problem is in fact independent of the choice of a reference probability space.

Assumption 2.3

• (i) $l :\mathbb {R}^n \times U \rightarrow \mathbb {R}$ is continuous.

• (ii) There exist constants $K,m>0$, such that

  $$\begin{aligned} |l(z,u)| \le K(1+|z|^m) \ \ \ \forall y\in \mathbb {R}^n, \ \forall u \in U. \end{aligned}$$

  (2.3)

• (iii) There exists a local modulus of continuity for $l(\cdot ,u)$, uniform in $u\in U$, i.e. for each $R>0$, there exists a nondecreasing concave function $\omega _{R}:\mathbb {R}^{+}\rightarrow \mathbb {R}^{+}$ such that $\lim _{r\rightarrow 0^{+}} \omega _{R}(r)=0$ and

  $$\begin{aligned} |l(z,u)-l(z',u)| \le \omega _R( |z-z'|) \end{aligned}$$

  (2.4)

  for every $z,z' \in \mathbb {R}^n$ such that $|z|,|z'| \le R$ and every $u \in U$.

We will show, suitably reformulating the state equation in an infinite dimensional framework, that the cost functional is well defined and finite for a sufficiently large discount factor $\rho >0$.

Throughout the paper we will write $C>0,\omega , \omega _R$ to indicate, respectively, a constant, a modulus continuity, and a local modulus of continuity, which may change from place to place if the precise dependence on other data is not relevant. The equality involving random variables will be intended $\mathbb {P}-$a.s..

3 Infinite dimensional Markovian representation

The optimal control problem at hand is not Markovian due to the delay. In order to regain Markovianity and approach the problem by Dynamic Programming we reformulate the state equation in an infinite-dimensional space generalizing a well-known procedure, see (Bensoussan et al. 2007, Part II, Chapter 4), Vinter and Kwong (1981) for deterministic delay equations and Gozzi and Marinelli (2006) for the stochastic case with linear state equation and additive noise.

In this section, in place of (2.1), we will consider the following more general state equation:

$$\begin{aligned} \left\{ \begin{array}{l} dy(t) = \displaystyle \left[ b_0 ( y(t),u(t) )+ \int _{-d}^0 a_1(\xi )y(t+\xi )\,d\xi + \int _{-d}^0p_1(\xi )u(t+\xi )\,d\xi \right] dt \\ \displaystyle \qquad \qquad + \sigma _0(y(t),u(t))\, dW(t), \quad t \ge 0 \\ y(0)=\eta _0, \quad y(\xi )=\eta _1(\xi ),\; u(\xi )=\delta (\xi ),\;\; \xi \in [-d,0), \end{array}\right. \end{aligned}$$

(3.1)

where, in this case, $b_0 :\mathbb {R}^n \times U \rightarrow \mathbb {R}^n$, $\sigma _0 :\mathbb {R}^n \times U \rightarrow M^{n\times q}$, while all the other terms satisfy the same conditions as in (2.1). In this setting we consider the following assumptions.

Assumption 3.1

$b_0 :\mathbb {R}^n \times U \rightarrow \mathbb {R}^n$, $\sigma _0 :\mathbb {R}^n \times U \rightarrow M^{n\times q}$ are continuous and there exist constants $L,C>0$ such that

$$\begin{aligned}&|b_0(y,u)-b(y',u)|\le L |y-y'|, \\&|b_0(y,u)|\le C(1+|y|), \\&|\sigma _0(y,u)-\sigma _0(y',u)| \le L |y-y'|, \\&|\sigma _0(y,u)| \le C(1+|y|), \end{aligned}$$

for every $y,y' \in \mathbb {R}^n$, $u \in U$.

Under Assumption 3.1, by (Revuz and Yor 1999, Theorem IX.2.1), for each initial data $\eta :=(\eta _0,\eta _1)\in \mathbb {R}^{n}\times L^2([-d,0];\mathbb {R}^n)$, $\delta \in L^2([-d,0);U)$, and each control $u(\cdot )\in \ \mathcal {U}$, there exists a unique (up to indistinguishability) strong solution to (3.1) and this solution admits a version with continuous paths that we denote by $y^{\eta ,\delta ;u}$.

We define $X:= \mathbb {R}^n \times L^2$. An element $x\in X$ is a couple $x= (x_0,x_1)$, where $x_0 \in \mathbb {R}^n$, $x_{1}\in L^{2}$; sometimes, we will write $x=\begin{bmatrix} x_0\\ x_1 \end{bmatrix}.$ The space X is a Hilbert space when endowed with the inner product

$$\begin{aligned} \langle x,z\rangle _{X}&:= x_0\cdot z_0 + \langle x_1,z_1 \rangle _{L^2}= x_0 z_0 + \int _{-d}^0 x_1(\xi )\cdot z_1(\xi ) \,d\xi , \\ x&=(x_{0},x_{1})\in X, \ z=(z_{0},z_{1})\in X. \end{aligned}$$

The induced norm, denoted by $|\cdot |_X$, is then

$$\begin{aligned} |x|_{X} = \left( |x_0|^2 + \int _{-d}^0 |x_1(\xi )|_{L^{2}}^2\,d\xi \right) ^{1/2}, \ \ \ x=(x_0,x_1) \in X. \end{aligned}$$

For $R>0$, we set the following notation for the open balls of radius R in X, $\mathbb {R}^n,$ and $L^2$, respectively:

$$\begin{aligned} B_R&:=\{x \in X: |x|_{X}< R\}, \ \ \ B_R^0:=\{x_0 \in \mathbb {R}^n: |x_0|< R\}, \\ B_R^1&:=\{x_1 \in L^2[-d,0]: |x_1|_{L^{2}} < R\}, \end{aligned}$$

We denote by $\mathcal {L}(X)$ the space of bounded linear operators from X to X, endowed with the operator norm

$$\begin{aligned} |L|_{\mathcal {L}(X)}=\sup _{|x|_{X}=1} |Lx|_{X}. \end{aligned}$$

An operator $L \in \mathcal {L}(X)$ can be seen as

$$\begin{aligned} Lx=\begin{bmatrix} L_{00} &{} L_{01}\\ L_{10} &{} L_{11} \end{bmatrix}\begin{bmatrix} x_0\\ x_1 \end{bmatrix}, \quad x=(x_0,x_1) \in X, \end{aligned}$$

where $L_{00} :\mathbb {R}^n \rightarrow \mathbb {R}^n$, $L_{01} :L^2 \rightarrow \mathbb {R}^n$, $L_{10} :\mathbb {R}^n \rightarrow L^2$, $L_{11} :L^2 \rightarrow L^2$ are bounded linear operators. Moreover, given two separable Hilbert spaces $(H, \langle \cdot , \cdot \rangle _H), (K, \langle \cdot , \cdot \rangle _K)$, we denote by $\mathcal {L}_1(H,K)$ the space of trace-class operators endowed with the norm

$$\begin{aligned} |L|_{\mathcal {L}_1(H,K)}=\inf \left\{ \sum _{i \in \mathbb {N}} |a_i|_H |b_i|_{K}: Lx=\sum _{i \in \mathbb {N}} b_i \langle a_i,x \rangle , a_i \in H, b_i \in K, \forall i \in \mathbb {N} \right\} . \end{aligned}$$

We also denote by $\mathcal {L}_2(H,K)$ the space of Hilbert-Schmidt operators from H to K endowed with the norm

$$\begin{aligned} |L|_{\mathcal {L}_2(H,K)}=(\text {Tr}(L^*L))^{1/2}. \end{aligned}$$

When $H=K$ we simply write $\mathcal {L}_1(H)$, $\mathcal {L}_2(H)$. We denote by S(H) the space of self-adjoint operators in $\mathcal {L}(H)$. If $Y,Z\in S(H)$, we write $Y\le Z$ if $\langle Yx,x \rangle \le \langle Zx,x \rangle$ for every $x \in H$.

Let us define the unbounded linear operator $A:D(A)\subset X\rightarrow X$ as follows:

$$\begin{aligned} A x = \begin{bmatrix} x_1(0) \\ -x_1' \end{bmatrix} , \ \ \ \ D(A) = \left\{ x \in X: x_1 \in W^{1,2}([-d,0]; \mathbb {R}^n), \ x_1(-d)=0\right\} . \end{aligned}$$

The adjoint of A is the operator $A^*:D(A^*)\subset X\rightarrow X$ (e.g., see (Federico et al. 2010, Proposition 3.4))

$$\begin{aligned} A^*x= \begin{bmatrix} 0 \\ x_1'\end{bmatrix}, \quad D(A^*) = \left\{ x \in X: x_1 \in W^{1,2}([-d,0],\mathbb {R}^n), \ x_1(0)=x_0\right\} . \end{aligned}$$

Note that $A^*$ is the generator of the delay semigroup, see, e.g., (Bensoussan et al. 2007, Part II, Chapter 4). For problems with delays (also) in the control appearing in a linear way in the state equation, its adjoint, i.e. A, is used to reformulate the problem in the space X (see, e.g., (Bensoussan et al. 2007, Part II, Chapter 4)). Indeed, A is the generator of a strongly continuous semigroup $e^{tA}$ on X, whose explicit expression (see, e.g., (Federico and Tacconi 2014, Eq. (73))) is

$$\begin{aligned} e^{At}x=\begin{bmatrix} x_0+\int _{(-t) \vee (-d)}^0 x_1(\xi )d \xi ,\\ \Phi (t)x_1 \end{bmatrix}, \quad x =(x_0,x_1) \in X, \end{aligned}$$

(3.2)

where $\Phi (t)$ is the semigroup of truncated right shift in $L^{2}$:

$$\begin{aligned} {[}\Phi (t) f](\xi )=1_{[-d,0]}(\xi -t) f(\xi -t) \quad \forall f \in L^2. \end{aligned}$$

Now define $b :X \times U \rightarrow X$ by

$$\begin{aligned} b(x,u)=\begin{bmatrix} b_0(x_0,u)\\ a_1x_0+p_1 u \end{bmatrix} \quad \forall x=(x_0,x_1) \in X, u \in U \end{aligned}$$

and $\sigma :X \times U \rightarrow \mathcal {L}(\mathbb {R}^q,X)$ by

$$\begin{aligned} \sigma (x,u)w=\begin{bmatrix} \sigma _0(x_0,u)\\ 0 \end{bmatrix} \quad \forall x=(x_0,x_1) \in X, u \in U, w \in \mathbb {R}^q. \end{aligned}$$

Consider the infinite dimensional SDE

$$\begin{aligned} {\left\{ \begin{array}{ll} dY(t) = [A Y(t)+b(Y(t),u(t))]dt + \sigma (Y(t),u(t))\,dW(t), \\ Y(0) = x \in X. \end{array}\right. } \end{aligned}$$

(3.3)

By (Fabbri et al. 2017, Theorem 1.127), for each $u(\cdot ) \in \mathcal {U}$, (3.3) admits a unique mild solution; that is, there exists a unique progressively measurable $X-$valued process $Y=(Y_{0},Y_{1})$ such that

$$\begin{aligned} Y(t) =e^{{tA}}x+\int _{0}^{t} e^{(t-s)A} b(Y(s),u(s)) d s+\int _{0}^{t} e^{(t-s)A}\sigma (Y(s),u(s))d W(s). \end{aligned}$$

(3.4)

Define the linear operator $M :X \times L^2([-d,0],U) \rightarrow X$ by

$$\begin{aligned} M (\alpha ,\beta ) := (\alpha _0,m(\alpha _1,\beta ) ), \ \ \alpha =(\alpha _0,\alpha _1)\in X, \ \beta \in L^2([-d,0],U), \end{aligned}$$

where

$$\begin{aligned} m(\alpha _1,\beta ) (\xi ):= \int _{-d}^\xi a_1(\zeta ) \alpha _1(\zeta -\xi )\,d\zeta + \int _{-d}^\xi p_1(\zeta ) \beta (\zeta -\xi )\,d\zeta , \ \ \ \ \xi \in [-d,0]. \end{aligned}$$

Theorem 3.2

Let Assumption 3.1 hold. We have the following claims.

• (i) Let $Y^{x,u}$ be the unique mild solution to (3.3) with initial datum $x \in X$ and control $u(\cdot )\in \mathcal {U}$. For every $t \ge d$

  $$\begin{aligned} Y^{x,u}(t) =(Y^{x,u}_0(t),Y^{x,u}_1(t))= M \left( \left( Y^{x,u}_0(t),Y^{x,u}_0(t+\cdot ) \right) ,u(t+\cdot ) \right) .\end{aligned}$$

• (ii) Let $y^{\eta ,\delta ;u}$ be the solution to SDDE (3.1) with initial data $\eta ,\delta$ and under the control $u(\cdot )\in \mathcal {U}$, and let $x = M(\eta _{0},\eta _{1},\delta ).$ Then, for every $t\ge 0$,

  $$\begin{aligned} Y^{x,u}(t) =(Y^{x,u}_0(t),Y^{x,u}_1(t)) = M \left( \left( y^{\eta ,\delta ,u}(t),y^{\eta ,\delta , u}(t+\cdot ) \right) ,u(t+\cdot ) \right) . \end{aligned}$$

  In particular, for every $t\ge 0$,

  $$\begin{aligned} y^{\eta ,\delta ,u}(t)=Y^{x,u}_0(t). \end{aligned}$$

Proof

• (i) Using (3.2), we can rewrite the two components of (3.4) as

  $$\begin{aligned}&\begin{bmatrix} Y_0^{x,u}(t)\\ Y_1^{x,u}(t) \end{bmatrix}\\&\quad =\begin{bmatrix} x_0 +\int _{(-t) \vee (-d)}^0 x_1(\xi )d \xi + \int _0^t \left[ b_0(Y_0^{x,u}(s),u(s))\right. \\ \left. + \int _{(-(t-s)) \vee (-d)}^0 a_1(\xi )Y_0^{x,u}(s)+ p_1(\xi )u(s) d \xi \right] ds\\ + \int _0^t \sigma _0(Y_0^{x,u}(s),u(s)) dW(s)\\ \Phi (t) x_{1}+\int _{0}^{t}\Phi (t-s) a_{1}Y_{0}^{x,u}(s)d s+\int _{0}^{t}\Phi (t-s) p_{1} u(s)d s \end{bmatrix}. \end{aligned}$$

  Then,

  $$\begin{aligned} Y_{1}^{x,u}(t)(\xi )&= 1_{[-d,0]}(\xi -t) x_1(\xi -t)+\int _{0}^{t}1_{[-d,0]}(\xi -t+s) a_{1}(\xi -t+s) Y_{0}^{x,u}(s) d s\nonumber \\&\quad +\int _{0}^{t} 1_{[-d,0]}(\xi -t+s) p_{1}(\xi -t+s) u(s) ds \nonumber \\&= 1_{[-d,0]}(\xi -t) x_1(\xi -t)+\int _{(\xi -t)\vee -d}^{\xi } a_{1}(\alpha ) Y_{0}^{x,u}(t+\alpha -\xi ) d \alpha \nonumber \\&\quad +\int _{(\xi -t)\vee -d}^{\xi } p_{1}(\alpha ) u(t+\alpha -\xi ) d\alpha . \end{aligned}$$

  (3.5)

  For $t \ge d$, we have $\xi -t \le -d$, so that

  $$\begin{aligned} Y_{1}^{x,u}(t)(\xi )= & {} \int _{-d}^{\xi } a_{1}(\alpha ) Y_{0}^{x,u}(t+\alpha -\xi ) d \alpha +\int _{-d}^{\xi } p_{1}(\alpha ) u(t+\alpha -\xi ) d \alpha \\= & {} m\left( Y^{x,u}_{0}(t+\xi ), u(t+\xi ) \right) \end{aligned}$$

  from which we get the first claim.

• (ii) Let $x=(x_0,x_1)=M\left( \eta _{0}, \eta _1, \delta \right)$. For $\xi -t \in [-r, 0]$, $\xi \in [-r, 0]$ we have:

  $$\begin{aligned} x_1(\xi -t)=Y^{x,u}_{1}(0)(\xi -t){} & {} =\int _{-d}^{\xi -t} a_{1}(\alpha ) \eta (t+\alpha -\xi ) d \alpha \\{} & {} \quad \ +\int _{-d}^{\xi -t} p_{1}(\alpha ) \delta (t+\alpha -\xi ) d \alpha , \end{aligned}$$

  so that inserting it into (3.5) we have:

  $$\begin{aligned} Y_{1}^{x,u}(t)(\xi )=\int _{-d}^{\xi } a_{1}(\alpha ) \tilde{Y}^{x,u}_{0}(t+\alpha -\xi ) d \alpha +\int _{-d}^{\xi } p_{1}(\alpha ) u(t+\alpha -\xi ) d \alpha , \end{aligned}$$

  (3.6)

  where we have defined ${\tilde{Y}}^{x,u}_0$ to be the extension of $Y^{x,u}_0$ to $[-d,0)$ by

  $$\begin{aligned} \tilde{Y}^{x,u}_{0}(s)= {\left\{ \begin{array}{ll}\eta _1(s), &{} s \in [-d, 0), \\ Y^{x,u}_{0}(s), &{} s \ge 0.\end{array}\right. } \end{aligned}$$

  From (3.6) we have:

  $$\begin{aligned} Y^{x,u}(t)&=(Y^{x,u}_0(t),Y^{x,u}_1(t))=M(\tilde{Y}_{0}^{x,u}(t),\tilde{Y}_{0}^{x,u}(t+\cdot ),u(t+\cdot ))\nonumber \\&=(\tilde{Y}_{0}^{x,u}(t),m(\tilde{Y}_{0}^{x,u}(t+\cdot ),u(t+\cdot )). \end{aligned}$$

  (3.7)

  To conclude the proof, by uniqueness of strong solutions to (3.1), we need to prove that ${\tilde{Y}}_{0}^{x,u}$ satisfies (3.1). On the other hand, by (Gawarecki and Mandrekar 2010, Theorem 3.2), $Y^{x,u}(t)$ is also a weak solution of (3.3), i.e. it satisfies

  $$\begin{aligned} \langle Y^{x,u}(t),h\rangle _X&= \langle x,h\rangle _X +\int _0^t \langle Y^{x,u}(t),A^* h\rangle _X ds +\int _0^t \langle b(Y^{x,u}(t),u(t)), h\rangle _X ds\\&\quad +\int _0^t \langle \sigma (Y^{x,u}(t),u(t)), h \rangle _X dW(s), \ \ \ \ \forall t \ge 0, \ \forall h \in D(A^*). \end{aligned}$$

  For $k\in \mathbb {N}\setminus \{0\}$, let $h^k=(1,h_1^k)$ be defined by

  $$\begin{aligned} h_1^k(\xi )= \int _{-d}^\xi k \textbf{1}_{[-1/k,0]}(r)dr, \ \ \ \ \ \xi \in [-d,0]. \end{aligned}$$

  Then,

  $$\begin{aligned} {\left\{ \begin{array}{ll} h^k\in D(A^*)\ \ \ \forall k\in \mathbb {N}\setminus \{0\},\\ h_1^k(\xi )\rightarrow 0 \ \ \text{ as } \ k \rightarrow \infty \ \ \text{ for } \text{ a.e. } \ \xi \in [-d,0],\\ \lim _{k \rightarrow \infty }\int _{-d}^0 \frac{dh_1^k}{d\xi }(\xi ) z(\xi ) d \xi = z(0), \ \ \ \forall z \in {C}([-d,0];\mathbb {R}^{n}). \end{array}\right. } \end{aligned}$$

  Therefore, for every $t \ge 0$, we have:

  $$\begin{aligned}&Y^{x,u}_0(t)+\left\langle Y^{x,u}_1(t),h_1^k \right\rangle _{L^2}\\&\quad = x_0+ \langle x_1,h_1^k \rangle _{L^2}+ \int _0^t\left\langle Y^{x,u}_1(s),\frac{dh_1^k}{d\xi } \right\rangle _{L^2} ds + \int _0^t b_0(Y^{x,u}_0(s),u(s))ds\\&\qquad +\int _0^t \left\langle a_1 Y^{x,u}_0(s)+p_1 u(s) , h_1^k\right\rangle _{L^2} ds + \int _0^t \sigma _0(Y^{x,u}_0(s),u(s))dW(s). \end{aligned}$$

  Note that, for every $t \ge 0$, on a has $Y_1(t)(\cdot ) \in {C}([-d,0];\mathbb {R}^{n})$, since $Y_1(t)(\cdot )$ is the sum of the convolutions of $L^2$-functions. Then, taking $k\rightarrow + \infty$ in the equation above, we get

  $$\begin{aligned} Y^{x,u}_0(t)&= x_0+ \int _0^t Y^{x,u}_1(s)(0)ds + \int _0^t b_0(Y^{x,u}_0(s),u(s))ds\\&\quad + \int _0^t \sigma _0(Y^{x,u}_0(s),u(s))dW(s). \end{aligned}$$

  By (3.6), with $\xi =0$ we have:

  $$\begin{aligned} Y^{x,u}_{1}(s)(0) =\int _{-d}^{0} a_{1}(\xi ) {\tilde{Y}}^{x,u}_{0}(s+\xi ) d \xi +\int _{-d}^{0} p_{1}(\xi ) u(s+\xi ) d \xi , \end{aligned}$$

  so that, for every $t \ge 0$,

  $$\begin{aligned} Y^{x,u}_0(t)&= x_0+ \int _0^t \left[ \int _{-d}^{0} a_{1}(\xi ) {\tilde{Y}}^{x,u}_{0}(s+\xi ) d \xi +\int _{-d}^{0} p_{1}(\xi ) u(s+\xi ) d \xi \right] ds \\&\quad + \int _0^t b_0(Y^{x,u}_0(s),u(s))ds + \int _0^t \sigma _0(Y^{x,u}_0(s),u(s))dW(s). \end{aligned}$$

  Recalling the definition of ${\tilde{Y}}_0^{x,u(\cdot )}$, this says that ${\tilde{Y}}_{0}^{x,u(\cdot )}$ satisfies (3.1), so we conclude.$\square$

3.1 Objective functional

Using Theorem 3.2, we can give a Markovian reformulation on the Hilbert space X of the optimal control problem.

We present such result for an optimal control problem with the more general state equation (3.1). Consider the functional $\mathcal {J}$ defined by (2.2) with $y^{\eta ,u(\cdot ),\delta (\cdot )}$ being the solution of (3.1) (in place of (2.1)). Denoting by $Y^{x,u}$ a mild solution of (3.3) for general initial datum $x \in X$ and control $u(\cdot ) \in \mathcal {U}$ and introducing the functional

$$\begin{aligned} J(x;u(\cdot )) := \mathbb {E} \left[ \int _0^{\infty } e^{-\rho t}L(Y^{x,u}(t),u(t))\,dt \right] , \end{aligned}$$

(3.8)

where

$$\begin{aligned} L: X \times U\rightarrow \mathbb {R}, \ \ \ L(x,u) = l(x_0,u), \end{aligned}$$

the original functional $\mathcal {J}$ and J are related through

$$\begin{aligned} \mathcal {J}(\eta ,\delta ;u(\cdot ))=J(M(\eta ,\delta );u(\cdot )). \end{aligned}$$

We then consider the problem of optimizing J under (3.3) and define the value function V for this problem:

$$\begin{aligned} V(x) = \inf _{u\in \mathcal {U}} J(x;u(\cdot )). \end{aligned}$$

(3.9)

For what we said, an optimal control $u^*(\cdot )\in \mathcal {U}$ for the functional $J(x;\cdot )$ with $x=M(\eta ,\delta )$ is also optimal for $\mathcal {J}(\eta ,\delta ;\cdot )$. Hence, from now on, we focus on the optimization problem (3.9).

4 B-continuity

In order to get B-continuity of the value function V, needed to employ the theory of viscosity solutions in infinite dimension, we consider the simpler state equation (2.1). In this case, the state equation (2.1) can be rewritten in infinite dimension as

$$\begin{aligned} {\left\{ \begin{array}{ll} dY(t) = [\mathcal {A} Y(t)+f(u(t))]dt + \sigma (u(t))\,dW(t), \\ Y(0) = x=M(\eta ,\delta ) \in X, \end{array}\right. } \end{aligned}$$

(4.1)

where

$$\begin{aligned}{} & {} \mathcal {A}: D(\mathcal {A})=D(A) \subset X \rightarrow X, \ \ \ \ \ \mathcal {A} x =Ax+\begin{bmatrix} a_0 \\ a_1 \end{bmatrix}x_{0}=\begin{bmatrix} a_0 x_0+ x_1(0)\\ a_1 x_0 -x_1' \end{bmatrix},\\{} & {} f:U \rightarrow X, \ \ \ f(u)=\begin{bmatrix} b_0(u)\\ p_1 u \end{bmatrix}, \end{aligned}$$

and

$$\begin{aligned} \sigma :U \rightarrow \mathcal {L}(\mathbb {R}^q,X), \ \ \ \ \sigma (u)w=\begin{bmatrix} \sigma _0(u)w\\ 0 \end{bmatrix}. \end{aligned}$$

Indeed, since $\mathcal {A}$ is the sum of A with a bounded linear operator, by (Engel and Nagel 2000, Corollary 1.7), we have that (3.3) with specifications

$$\begin{aligned} b_0(x_0,u)=a_0x_0+b_0(u), \ \ \ \sigma _0(x_0,u)=\sigma _0(u), \end{aligned}$$

(4.2)

and (4.1) are equivalent and have the same (unique) mild solution $Y^{x,u}(t)$.

4.1 Reformulation with a maximal dissipative operator

The aim of this subsection is to rewrite (4.1) with a maximal dissipative operator $\tilde{\mathcal {A}}$ in place of $\mathcal {A}$. The need for that is that we want to use the viscosity solutions theory in infinite dimension to treat the HJB equation associated to the optimal control problem, which requires for the comparison theorem the presence of a maximal dissipative operator in the equation (see (Fabbri et al. 2017, Chapter 3)). The operator $\tilde{\mathcal {A}}$ is constructed by means of a suitable shift of the operator $\mathcal {A}$.

Proposition 4.1

There exists $\mu _0 >0$ such that $\tilde{\mathcal {A}}_\mu :D(\tilde{\mathcal {A}}_\mu ) =D(A), \subset X \rightarrow X$ defined by

$$\begin{aligned} \tilde{\mathcal {A}}_\mu x:=\mathcal {A} x-\mu x=\begin{bmatrix} a_0x_{0}-\mu x_0+ x_1(0)\\ a_1 x_0 -\mu x_1 -x_1' \end{bmatrix} = Ax + \begin{bmatrix} a_0x_0-\mu x_0\\ a_1 x_0 -\mu x_1 \end{bmatrix}, \quad x \in D(\tilde{\mathcal {A}}_\mu ) \end{aligned}$$

is maximal dissipative for every $\mu \ge \mu _0$.

Proof

Step 1. We prove that $\tilde{\mathcal {A}}_\mu$ is dissipative. Let $x \in D(A)=\left\{ x \in X: x_1 \in W^{1,2}([-d,0]; \mathbb {R}^n), \ x_1(-d)=0\right\} ,$ then

$$\begin{aligned} \langle \mathcal {A} x,x \rangle _X&=(a_0 x_0+x_1(0) ) \cdot x_0+ \int _{-d}^0 a_1(\xi )x_0 \cdot x_1(\xi )d\xi - \int _{-d}^0 x_1'(\xi ) \cdot x_1(\xi )d\xi \\&=(a_0 x_0+x_1(0) ) \cdot x_0 + x_0 \cdot \int _{-d}^0 a_1(\xi )^T x_1(\xi )d\xi - \frac{1}{2} x_1(0)^2 + \frac{1}{2} x_1(-d)^2\\&=(a_0 x_0+x_1(0) ) \cdot x_0 + x_0 \cdot \int _{-d}^0 a_1(\xi )^T x_1(\xi )d\xi - \frac{1}{2} x_1(0)^2 \\&\le a_0 x_0 \cdot x_0 + \frac{1}{2}|x_0|^2 + x_0 \cdot \int _{-d}^0 a_1(\xi )^T x_1(\xi )d\xi \\&\le (|a_0|+1) |x_0|^2 +\frac{1}{2}|a_1|^2_{L^2_{-d}}|x_1|^2_{L^2_{-d}} \le \mu _0 |x|^2, \end{aligned}$$

where $\mu _0:=\max \left\{ |a_0|+1,\frac{1}{2} |a_1|^2_{L^2}\right\}$. This implies that $\tilde{\mathcal {A}}_\mu$ is dissipative for every $\mu \ge \mu _0$.

Step 2. We prove that $\tilde{\mathcal {A}}_\mu$ is maximal for $\mu \ge \mu _0$. For that we show that there exists $\lambda >0$ such that $R(\lambda I - \tilde{\mathcal {A}}_\mu )=X$. This means that, for every $y\in X$, there exists a solution $x \in D(\tilde{\mathcal {A}}_\mu )$ to the equation

$$\begin{aligned} \lambda x -\tilde{\mathcal {A}}_\mu x=y. \end{aligned}$$

(4.3)

Fix $\lambda >0$. Let then $y \in X$. The equation above rewrites as

$$\begin{aligned} {\left\{ \begin{array}{ll} (\lambda +\mu ) x_0-a_0 x_0 - x_1 (0) =y_0,\\ {\left\{ \begin{array}{ll} (\lambda +\mu ) x_1(\xi ) - a_1(\xi ) x_0 + x_1'(\xi ) =y_1(\xi ), \quad \xi \in [-d,0]\\ x_1(-d)=0. \end{array}\right. } \end{array}\right. } \end{aligned}$$

This system is uniquely solvable. Indeed, from $(\lambda +\mu ) x_0-a_0 x_0 - x_1 (0) =y_0$, we get uniquely

$$\begin{aligned} x_1(0)=(\lambda +\mu ) x_0-a_0 x_0 - y_0. \end{aligned}$$

(4.4)

On the other hand,

$$\begin{aligned} {\left\{ \begin{array}{ll} (\lambda +\mu ) x_1(\xi ) - a_1(\xi ) x_0 + x_1'(\xi ) =y_1(\xi ), \quad \xi \in [-d,0]\\ x_1(-d)=0. \end{array}\right. } \end{aligned}$$

yields the unique solution

$$\begin{aligned} x_1(\xi )= \int _{-d}^\xi e^{-(\lambda +\mu ) (\xi - r)}(y_1 (r) + a_1(r) x_0)dr. \end{aligned}$$

Taking $\xi =0$ in this equality and equating with (4.4), we have:

$$\begin{aligned} (\lambda +\mu ) x_0-a_0 x_0 - y_0 = \int _{-d}^0 e^{(\lambda +\mu ) r} y_1(r)dr+\int _{-d}^0 e^{(\lambda +\mu ) r} a_1(r)dr x_0. \end{aligned}$$

Then, for $\mu \ge \mu _0$

$$\begin{aligned} x_0= \left[ (\lambda +\mu ) I_{\mathbb {R}^{n}} -a_0-\int _{-d}^0 e^{(\lambda +\mu ) r} a_1(r)dr \right] ^{-1}\left[ y_0 + \int _{-d}^0 e^{(\lambda +\mu )r} y_1(r)dr \right] . \end{aligned}$$

Therefore, we have proved that, for every $\mu \ge \mu _0$ and $y \in X$, there exists a unique solution $x \in D(\tilde{\mathcal {A}}_\mu )$ to (4.3) given by

$$\begin{aligned} x=\begin{bmatrix} x_0\\ x_1 \end{bmatrix}= \begin{bmatrix} \left[ (\lambda +\mu ) I_{\mathbb {R}^{n}} -a_0-\int _{-d}^0 e^{(\lambda +\mu ) r} a_1(r)dr \right] ^{-1}\left[ y_0 + \int _{-d}^0 e^{(\lambda +\mu ) r} y_1(r)dr \right] \\ \int _{-d}^\cdot e^{-(\lambda +\mu ) (\cdot - r)}(y_1 (r) + a_1(r) x_0)dr \end{bmatrix}. \end{aligned}$$

(4.5)

The claim follows. $\square$

Now, we fix $\mu > \mu _0$ and denote

$$\begin{aligned} \tilde{\mathcal {A}}:=\tilde{\mathcal {A}}_\mu =\mathcal {A} - \mu I. \end{aligned}$$

We may rewrite SDE (4.1) as

$$\begin{aligned} {\left\{ \begin{array}{ll} dY(t) = \left[ \tilde{\mathcal {A}} Y(t)+{\tilde{b}}(Y(t),u(t)) \right] dt + \sigma (u(t))\,dW(t), \\ Y(0) = x \in X, \end{array}\right. } \end{aligned}$$

(4.6)

where

$$\begin{aligned}&{\tilde{b}} :X \times U \rightarrow X, \ \ \ \ {\tilde{b}}(x,u) = \mu x+ f(u)=\begin{bmatrix} \mu x_0+b_0(u)\\ \mu x_1+p_1 u \end{bmatrix},\\&x=(x_0,x_1) \in X, \ u \in U. \end{aligned}$$

Similarly to before, since the operator $\tilde{\mathcal {A}}$ is the sum of A with a bounded operator, by (Engel and Nagel 2000, Corollary 1.7), we have that (4.6) is equivalent to (4.1) (and so also to (3.3) with the condition (4.2)) and all them have the same (unique) mild solution, that in terms of $\tilde{\mathcal {A}}$ writes as

$$\begin{aligned} Y(t) =e^{{\tilde{\mathcal {A}} t}}x+\int _{0}^{t} e^{\tilde{\mathcal {A}} (t-s)} {\tilde{b}}(Y(s),u(s)) d s+\int _{0}^{t} e^{\tilde{\mathcal {A}}(t-s)}\sigma (u(s))d W(s). \end{aligned}$$

(4.7)

4.2 Weak B-condition

In this subsection, we recall the concept of weak B-condition for the maximal dissipative operator $\tilde{\mathcal {A}}$ and introduce an operator B satisfying it. This concept is fundamental in the theory of viscosity solutions in Hilbert spaces, see (Fabbri et al. 2017, Chapter 3), which will be used in this paper.

Definition 4.2

(Fabbri et al. 2017, Definition 3.9) We say that $B \in \mathcal {L}(X)$ satisfies the weak B-condition for $\tilde{\mathcal {A}}$ if the following hold:

• (i) B is strictly positive, i.e. $\langle Bx,x \rangle _{X} >0$ for every $x \ne 0$;

• (ii) B is self-adjoint;

• (iii) $\tilde{C}^* B \in \mathcal {L}(X)$;

• (iv) There exists $c_0 \ge 0$ such that

  $$\begin{aligned} \langle \tilde{\mathcal {A}}^* Bx,x \rangle _{X} \le c_0 \langle Bx,x \rangle _{X}, \quad \forall x \in X. \end{aligned}$$

Let $\tilde{\mathcal {A}}^{-1}$ be the inverse of the operator $\tilde{\mathcal {A}}$. Its explicit expression can be derived as in the proof of Proposition 4.1:

$$\begin{aligned}{} & {} \tilde{\mathcal {A}}^{-1} z = \begin{bmatrix} -\left[ \mu I_{\mathbb {R}^{n}} -a_0-\int _{-d}^0 e^{\mu r} a_1(r)dr \right] ^{-1}\left[ z_0 + \int _{-d}^0 e^{\mu r} z_1(r)dr \right] \\ \int _{-d}^\cdot e^{-\mu (\cdot - r)}(-z_1 (r) + a_1(r) z_0)dr \end{bmatrix},\nonumber \\{} & {} z=(z_0,z_1) \in X. \end{aligned}$$

(4.8)

Notice that $\tilde{\mathcal {A}}^{-1} \in \mathcal {L}(X)$ and it is compact. Define now the compact operator

$$\begin{aligned} B:=(\tilde{\mathcal {A}}^{-1})^*\tilde{\mathcal {A}}^{-1}=(\tilde{\mathcal {A}}^*)^{-1}\tilde{\mathcal {A}}^{-1}. \end{aligned}$$

(4.9)

We are going to show that B satisfies the weak-B condition for $\tilde{\mathcal {A}}$.

Proposition 4.3

B defined in (4.9) satisfies the weak-B condition for $\tilde{\mathcal {A}}$ with $c_0=0$.

Proof

Clearly, $B \in \mathcal {L}(X)$, $\tilde{\mathcal {A}}^* B= \tilde{\mathcal {A}}^{-1} \in \mathcal {L}(X)$, and B is self adjoint. Moreover, B is strictly positive; indeed

$$\begin{aligned} \langle Bx,x \rangle _X= \langle \tilde{\mathcal {A}}^{-1}x,\tilde{\mathcal {A}}^{-1} \rangle _X= |\tilde{\mathcal {A}}^{-1}x|^2 \ge 0, \ \ \ \ \forall x\in X, \end{aligned}$$

and it is easy to check that, whenever $x \ne 0$, we have that $|\tilde{\mathcal {A}}^{-1}x|>0$. Finally, by dissipativity of $\tilde{\mathcal {A}}$, we have:

$$\begin{aligned} \langle \tilde{\mathcal {A}}^* Bx,x \rangle _X = \langle \tilde{\mathcal {A}}^{-1}x ,x \rangle _X = \langle y, \tilde{\mathcal {A}}y \rangle _X \le 0 \end{aligned}$$

with $y= \tilde{\mathcal {A}}^{-1}x$. $\square$

Observe that

$$\begin{aligned} B x= \begin{bmatrix} B_{00} &{}\quad B_{01} \\ B_{10} &{}\quad B_{11} \end{bmatrix} \begin{bmatrix} x_0 \\ x_1 \end{bmatrix}, \ \ \ \ x=(x_0,x_1)\in X. \end{aligned}$$

(4.10)

By strict positivity of B, we have that $B_{00}$ and $B_{11}$ are strictly positive. We introduce the $|\cdot |_{-1}$-norm on X by

$$\begin{aligned} \nonumber |x|_{-1}^2&:=\langle B^{1/2}x, B^{1/2}x\rangle _{X}= \langle Bx,x\rangle _{X}\\&=\langle (\tilde{\mathcal {A}}^{-1})^*\tilde{\mathcal {A}}^{-1}x,x\rangle _{X}=\langle \tilde{\mathcal {A}}^{-1}x,\tilde{\mathcal {A}}^{-1}x\rangle _{X}= |\tilde{\mathcal {A}}^{-1}x|^{2}_{X} \quad \forall x \in X. \end{aligned}$$

(4.11)

We define

$$\begin{aligned} X_{-1}:= \ \text{ the } \text{ completion } \text{ of } X \text{ under } \ |\cdot |_{-1}, \end{aligned}$$

which is a Hilbert space endowed with the inner product

$$\begin{aligned} \langle x,y\rangle _{-1}:=\langle B^{1/2}x,B^{1/2}y\rangle _{X}= \langle Bx,y\rangle _{X}= \langle \tilde{\mathcal {A}}^{-1}x,\tilde{\mathcal {A}}^{-1}y\rangle _{X}. \end{aligned}$$

Notice that $|x|_{-1}\le |\tilde{\mathcal {A}}^{-1}|_\mathcal {L(X)}|x|_X$; in particular, we have $(X,|\cdot |) \hookrightarrow (X_{-1},|\cdot |_{-1})$. Strict positivity of B ensures that the operator $B^{1 /2}$ can be extended to an isometry

$$\begin{aligned} B^{1 /2} :(X_{- 1},|\cdot |_{-1}) \rightarrow (X,|\cdot |_{X}). \end{aligned}$$

By (4.11) and an application of (Da Prato and Zabczyk 2014, Proposition B.1), we have $\text{ Range }(B^{1/2})=\text{ Range }((\tilde{\mathcal {A}}^{-1})^*)$. Since $\text{ Range }((\tilde{\mathcal {A}}^{-1})^*)=D(\tilde{\mathcal {A}}^*)$, we have

$$\begin{aligned} \text{ Range }\big (B^{1/2}\big )=D(\tilde{\mathcal {A}}^*). \end{aligned}$$

(4.12)

By (4.12), the operator $\tilde{\mathcal {A}}^* B^{1/2}$ is well defined on the whole space X. Moreover, since $\tilde{\mathcal {A}}^*$ is closed and $B^{1/2}\in \mathcal {L}(X)$, $\tilde{\mathcal {A}}^* B^{1/2}$ is a closed operator. Thus, by the closed graph theorem, we have

$$\begin{aligned} \tilde{\mathcal {A}}^* B^{1/2} \in \mathcal {L}(X). \end{aligned}$$

(4.13)

Remark 4.4

In the infinite dimensional theory of viscosity solutions it is only required that $\tilde{\mathcal {A}}^*B \in \mathcal {L}(X)$ (condition (iii) of Definition 4.2). Such an operator can be constructed for any maximal dissipative operator $\tilde{\mathcal {A}}$ (see, e.g., (Fabbri et al. 2017, Theorem 3.11)). Similarly to de Feo et al. (2023) in the case of the present paper, in addition, we also have $\tilde{\mathcal {A}}^* B^{1/2} \in \mathcal {L}(X)$. Such stronger condition may be helpful in order to get differentiability properties of the value function (see (de Feo et al. 2023, Proof of Theorem 6.5)) or in order to construct optimal feedback laws (see de Feo and Święch 2023).

Since B is a compact, self-adjoint and strictly positive operator on X, by the spectral theorem B admits a set of eigenvalues $\{\lambda _i \}_{i \in \mathbb {N}} \subset (0, +\infty )$ such that $\lambda _{i}\rightarrow 0^{+}$ and a corresponding set $\{f_i \}_{i \in \mathbb {N}} \subset X$ of eigenvectors forming an orthonormal basis of X. By taking $\{e_i \}_{i \in \mathbb {N}}$ defined by $e_i:=\frac{1 }{\sqrt{\lambda }_i} f_i$, we then get an orthonormal basis of $X_{-1}$. We set $X^N:= \text{ Span } \{f_1,...f_N\}= \text{ Span } \{e_1,...e_N\}$ for $N\ge 1$, and let $P_N :X \rightarrow X$ be the orthogonal projection onto $X_N$ and $Q_N:= I - P_N$. Since $\{e_i \}_{i \in \mathbb {N}}$ is an orthogonal basis of $X_{-1}$, the projections $P_N,Q_N$ extend to orthogonal projections in $X_{-1}$ and we will use the same symbols to denote them.

We notice that

$$\begin{aligned} B P_N =P_N B, \quad B Q_N =Q_N B. \end{aligned}$$

(4.14)

Therefore, since $|BQ_N|_\mathcal {L(X)}=|Q_NB|_\mathcal {L(X)}$ and B is compact, we get

$$\begin{aligned} \lim _{N \rightarrow \infty }|BQ_N|_\mathcal {L(X)} =0. \end{aligned}$$

(4.15)

Remark 4.5

• (i) We remark that the following inequality

  $$\begin{aligned} |x_0| \le C_R |x|_{-1} \quad \forall x=(x_0,x_1) \in B_R \subset X, \end{aligned}$$

  (4.16)

  (which was proven in de Feo et al. (2023)) here is false. We first claim that such inequality does not hold on unbounded sets of X. Indeed, we provide the following counter example which is similar to the one in Federico and Tacconi (2014). Let $n=1$ and consider

  $$\begin{aligned} x^{N}=\left( x_{0}^{N}, x_{1}^{N}\right) , \quad x_{0}^{N}=1, \quad x_{1}^{N}=-N \textbf{1}_{[-1 / N, 0]}, \quad N\ge 1 \text{. } \end{aligned}$$

  For N big enough $-1 / N>-d$, then we have

  $$\begin{aligned} \left| x^{N}\right| _{-1}{} & {} =\left| \tilde{\mathcal {A}}^{-1} x^{N}\right| =0+\int _{-\frac{1}{N}}^{0}\left| \int _{-\frac{1}{N}}^{\xi } n d s\right| ^{2} d \xi =\int _{-\frac{1}{N}}^{0} N^{2}\left( \xi +\frac{1}{N}\right) ^{2} d \xi \\{} & {} =\frac{1}{3 N} \longrightarrow 0. \end{aligned}$$

  Therefore, we have $\left| x_{0}^{N}\right| =1$ and $\left| x^{N}\right| _{-1} \rightarrow 0$.

  Now note that if the inequality (4.16) holded on bounded sets $B_R$ then it would hold on the whole X with $C_R=C>0$ which is a contradiction. Indeed assume the inequality holds on bounded sets $B_R$. Then it is true for every $x=(x_0,x_1)$ with $|x|\le 1$, i.e. we have

  $$\begin{aligned} |x_0|\le C|x|_{-1}, \quad |x|\le 1 \end{aligned}$$

  (4.17)

  Now let any $y=(y_0,y_1) \in X$, $y \ne 0$, define $x=y/|y|=(y_0/|y|,y_1/|y|)$, $|x|=1$ so that (4.17) holds and then by multiplying the inequality by |y| we have

  $$\begin{aligned} |y_0|\le C|y|_{-1}, \end{aligned}$$

  which is a contradiction.

• (ii) We point out that the fact that here (4.17) is false is not in contradiction with de Feo et al. (2023), where such inequality was shown to be true. Indeed we recall that the operators $\tilde{\mathcal {A}}$ here and in de Feo et al. (2023) are not the same: $\tilde{\mathcal {A}}$ here is the adjoint (up to some bounded perturbation) of the operator $\tilde{\mathcal {A}}$ in de Feo et al. (2023).

  In Federico and Tacconi (2014), where delays appear in the control similarly to here, the inequality is then proved in the smaller space $\mathbb {R} \times W^{1,2}([-d,0])$, see (Federico and Tacconi 2014, remark 5.4) and this is enough since the optimal control problem is deterministic. This would not be possible here due to the presence of the Brownian motion, whose trajectories are not absolutely continuous. Indeed, it would not be possible to have $Y_1(t) \in W^{1,2}$ as in Federico and Tacconi (2014).

4.3 B-continuity of the value function

In this subsection, we prove some estimates for solutions of the state equation and on the cost functional in order to prove the B-continuity of the value function.

Lemma 4.6

There exists $C>0$ and a local modulus of continuity $\omega _R$ such that

$$\begin{aligned}&|{\tilde{b}}(x, u)-{\tilde{b}}(y, u)|_X = \mu |x-y|_X \end{aligned}$$

(4.18)

$$\begin{aligned}&|{\tilde{b}}(x, u)-{\tilde{b}}(y, u)|_{-1}^2 =\langle {\tilde{b}}(x, u)-{\tilde{b}}(y, u), B(x-y)\rangle = \mu |x-y|_{-1}^{2} \end{aligned}$$

(4.19)

$$\begin{aligned}&|{\tilde{b}}(x, u)|_X \le C(1+|x|_X) \end{aligned}$$

(4.20)

$$\begin{aligned}&|\sigma (u)|_{\mathcal {L}_{2}(\mathbb {R}^q, X)} \le C \end{aligned}$$

(4.21)

$$\begin{aligned}&|L(x, u)-L(y, u)| \le \omega _R\left( |x-y|_{-1}\right) \end{aligned}$$

(4.22)

$$\begin{aligned}&|L(x, u)| \le C\left( 1+|x|_X^{m}\right) \end{aligned}$$

(4.23)

for every $x,y \in X, u \in U$, $R>0$.

Moreover

$$\begin{aligned} \lim _{N \rightarrow \infty } \sup _{u \in U} \text {Tr}\left[ \sigma (u) \sigma (u)^{*} B Q_{N}\right] =0 \end{aligned}$$

(4.24)

Proof

• (4.18) follows immdediately by the definition of ${\tilde{b}}(x,u)$.

• Similarly for (4.19) we have:

  $$\begin{aligned} |{\tilde{b}}(x, u)-{\tilde{b}}(y, u)|_{-1}^2&=\langle {\tilde{b}}(x, u)- {\tilde{b}}(y, u), B(x-y)\rangle \\&= \mu \langle x-y, B(x-y)\rangle = \mu |x-y|_{-1}^2. \end{aligned}$$

• Recalling the definition of ${\tilde{b}}(x,u)$, (4.20) follows the boundedness of $b_0(u)$ and the boundedness of U.

• (4.21) follows by the boundedness of $\sigma _0 :U \rightarrow M^{n \times q}$ since

  $$\begin{aligned} |\sigma (u)|_{\mathcal {L}_{2}(\mathbb {R}^q, X)}^2 =\sum _{i=1}^n |\sigma (u) r_i|^2=\sum _{i=1}^n |\sigma _0(u) r_i|^2\le n |\sigma _0(u)|^2 \le C \end{aligned}$$

  where $\{r_i \}_{i=1}^q$ is the classical orthonormal basis of $\mathbb {R}^q$.

• For (4.22) note that by (2.4) the function $L(x,u)=l(x_0,u)$ is weakly sequentially continuous uniformly in $u \in U$ (as $x_0 \in \mathbb {R}^n$), i.e.

  $$\begin{aligned} \sup _{u \in U} |L(y,u)-L(x,u)|=\sup _{u \in U} |l(y_0,u)-l(x_0,u)|\le \omega _R(|y_0-x_0|)\rightarrow 0 \end{aligned}$$

  for every $y \rightharpoonup x$.

  Then the inequality follows by application of (Fabbri et al. 2017, Lemma 3.6 (iii)) which can easily be extended to functions weakly sequentially continuous uniformly with respect to the control parameter.

• (4.23) finally follows by the definition of L and (2.3).

• We notice that by (Fabbri et al. 2017, Appendix B) and (4.21), we have:

  $$\begin{aligned}&|\text {Tr}\left[ \sigma (u) \sigma (u)^{*} B Q_{N}\right] |\\&\quad \le |\sigma (u) \sigma (u)^{*} BQ_N|_{\mathcal {L}_1(\mathbb {R}^q,X)} \le |\sigma (u) \sigma (u)^{*}|_{\mathcal {L}_1(\mathbb {R}^q,X)} |B Q_N|_{\mathcal {L}(X)} \\&\quad = |\sigma (u)|_{\mathcal {L}_2(\mathbb {R}^q,X)}^2 |B Q_N|_{\mathcal {L}(X)} \le C |B Q_N|_{\mathcal {L}(X)}. \end{aligned}$$

  Now taking the supremum over u and letting $N \rightarrow \infty$ by (4.15) we have (4.24).$\square$

Remark 4.7

We remark that the fact that (4.16) is false (see Remark 4.5) was the reason that lead us to consider the simpler state equation (2.1), in place of the more general state equation (3.1), in order to use the approach via viscosity solutions: regarding $b_0,$ for instance, if (4.16) were true, we could have considered a more general $b_0$ of the form $b_0(x_0,u),$ satisfying

$$\begin{aligned} |b_0(x_0,u)-b_0(y_0,u)| \le C |x_0-y_0|. \end{aligned}$$

Indeed, using (4.16), we would have had

$$\begin{aligned} |b_0(x_0,u)-b_0(y_0,u)| \le C_R|x-y|_{-1}, \end{aligned}$$

from which we would have proved

$$\begin{aligned} \langle {\tilde{b}}(x, u)-{\tilde{b}}(y, u), B(x-y)\rangle \le C_R |x-y|_{-1}^{2}. \end{aligned}$$

Similarly we could have allowed $\sigma _0(x_0,u)$ to depend also on $x_0$. This would have been enough in order to prove the B-continuity of V and the validity of the hypotheses of the comparison theorem (Fabbri et al. 2017, Theorem 3.56) when the state equation is of the form (3.1). This approach was used in de Feo et al. (2023), where this inequality was proved to be true.

We recall (Fabbri et al. 2017, Proposition 3.24). Set

$$\begin{aligned} \rho _{0}:= {\left\{ \begin{array}{ll} 0, &{} \text{ if } m =0,\\ C m+\frac{1}{2} C^{2} m, &{} \text{ if } 0<m<2, \\ C m+\frac{1}{2} C^{2} m(m-1), &{} \text{ if } m \ge 2, \end{array}\right. } \end{aligned}$$

where C is the constant appearing in (4.20) and (4.21), and m is the constant from Assumption 2.3 and (4.23).

Proposition 4.8

((Fabbri et al. 2017, Proposition 3.24)) Let Assumption 2.2 holds and let $\lambda >\rho _0$. Let Y(t) be the mild solution of (4.6) with initial datum $x \in X$ and control $u(\cdot ) \in \mathcal {U}$. Then, there exists $C_\lambda >0$ such that

$$\begin{aligned} \mathbb {E}\left[ |Y(t)|_X^m\right] \le C_\lambda \left( 1+|x|_X^m\right) e^{\lambda t}, \quad \forall t \ge 0. \end{aligned}$$

We need the following assumption.

Assumption 4.9

$\rho >\rho _{0}.$

Proposition 4.10

Let Assumptions 2.2, 2.3, and 4.9 hold. There exists $C>0$ such that

$$\begin{aligned} |J(x;u(\cdot ))|\le C(1+|x|_X^m) \quad \forall x \in X, \ \forall u(\cdot ) \in \mathcal {U}. \end{aligned}$$

Hence,

$$\begin{aligned} |V(x)|\le C(1+|x|_X^m), \ \ \ \forall x \in X. \end{aligned}$$

Proof

By (4.23) and Proposition 4.8 applied with $\lambda =(\rho +\rho _0)/2$, we have

$$\begin{aligned} |J(x,u(\cdot ))| \le C \int _0^\infty e^{-\rho t} \mathbb {E} [ (1+|Y(t)|_X)^m] dt\le C (1+|x|_X^m), \ \ \ \forall x \in X, \ \forall u(\cdot ) \in \mathcal {U}. \end{aligned}$$

The estimate on V follows from this. $\square$

Next, we show continuity properties of V. We recall first the notion of B-continuity (see (Fabbri et al. 2017, Definition 3.4))

Definition 4.11

Let $B \in \mathcal {L}(X)$ be a strictly positive self-adjoint operator. A function $u: X \rightarrow \mathbb {R}$ is said to be B-upper semicontinuous (respectively, B-lower semicontinuous) if, for any sequence $\left\{ x_{n}\right\} _{n \in \mathbb {N}}\subset X$ such that $x_{n} \rightharpoonup x \in X$ and $B x_{n} \rightarrow B x$ as $n \rightarrow \infty$, we have

$$\begin{aligned} \limsup _{n \rightarrow \infty } u\left( x_{n}\right) \le u(x) \ \ \ \text{(respectively, } \ \liminf _{n \rightarrow \infty } u\left( x_{n}\right) \ge u(x)). \end{aligned}$$

A function $u: X \rightarrow \mathbb {R}$ is said to be B-continuous if it is both B-upper semicontinuous and B-lower semicontinuous.

We remark that, since our operator B is compact, in our case B-upper/lower semicontinuity is equivalent to the weak sequential upper/lower semicontinuity, respectively.

Proposition 4.12

Let Assumptions 2.2, 2.3, and 4.9 hold. For every $R>0,$ there exists a modulus of continuity $\omega _R$ such that

$$\begin{aligned} |V(x)-V(y)| \le \omega _R(|x-y|_{-1}) , \ \ \ \forall x,y \in X, \ \text{ s.t. } \ |x|_X, |y|_X \le R. \end{aligned}$$

(4.25)

Hence V is B-continuous and thus weakly sequentially continuous.

Proof

We prove the estimate

$$\begin{aligned} |J(x,u)-J(y,u)| \le \omega _R(|x-y|_{-1}) \quad \forall x,y \in X: \ |x|_X, |y|_X \le R, \forall u(\cdot )\in \mathcal {U}, \end{aligned}$$

as in (Fabbri et al. 2017, Proposition 3.73), since the assumptions of the latter are satisfied due to Lemma 4.6. Then, (4.25) follows. As for the last claim, we observe that by (4.25) and by (Fabbri et al. 2017, Lemma 3.6(iii)), V is B-continuous in X. $\square$

We point out that V may not be continuous with respect to the $|\cdot |_{-1}$ norm in the whole X.

5 The value function as unique viscosity solution to HJB equation

In this section we prove that the value function V is the unique viscosity solution of the infinite dimensional HJB equation.

Given $v \in C^1(X)$, we denote by Dv(x) its Fréchet derivative at $x \in X$ and we write

$$\begin{aligned} Dv(x)= \begin{bmatrix} D_{x_0}v(x) \\ D_{x_1}v(x) \end{bmatrix}, \end{aligned}$$

where $D_{x_0}v(x), D_{x_1}v(x)$ are the partial Fréchet derivatives. For $v \in C^2(X)$, we denote by $D^2 v(x)$ its second order Fréchet derivative at $x \in X$ which we will often write as

$$\begin{aligned} D^2v(x)= \begin{bmatrix} D^2_{x_0^2}v(x) &{} D^2_{x_0x_1}v(x) \\ D^2_{x_1x_0}v(x) &{} D^2_{x_1^2}v(x) \end{bmatrix}. \end{aligned}$$

We define the Hamiltonian function $H:X\times X\times S(X)\rightarrow \mathbb {R}$ by

$$\begin{aligned} H(x,r,Z)&:= -\inf _{u\in U} \left\{ \langle {\tilde{b}}(x,u),r\rangle + {1\over 2}{\textrm{Tr}}(\sigma (u)\sigma (u)^*Z) + L(x,u) \right\} \\&= -\mu x_0 \cdot r_0 -\mu \langle x_1, r_1\rangle _{L^2} \\&\quad -\inf _{u\in U} \left\{ b_0(u) \cdot r_0 + \langle p_1 u, r_1 \rangle _{L^2} + {1\over 2} {\textrm{Tr}}\left[ \sigma _0(u)\sigma _0(u)^T Z_{00} \right] + l(x_0,u) \right\} \\&= -\mu x_0 \cdot r_0 -\mu \langle x_1, r_1 \rangle _{L^2} \\&\quad +\sup _{u\in U} \left\{ -b_0(u) \cdot r_0 - \langle p_1 u, r_1\rangle _{L^2} - {1\over 2} {\textrm{Tr}}\left[ \sigma _0(u)\sigma _0(u)^T Z_{00} \right] - l(x_0,u) \right\} \\&=:{\tilde{H}}(x,r,Z_{00}), \end{aligned}$$

for every $x,r \in X, Z \in \mathcal {S}(X).$

By (Fabbri et al. 2017, Theorem 3.75) the Hamiltonian H satisfies the following properties.

Lemma 5.1

Let Assumptions 2.2 and 2.3 hold.

• (i) H is uniformly continuous on bounded subsets of $X \times X \times S(X)$.

• (ii) For every $x,r \in X$ and every $Y,Z \in S(X)$ such that $Z \le Y$, we have

  $$\begin{aligned} H(x,r,Y)\le H(x,r,Z). \end{aligned}$$

  (5.1)

• (iii) For every $x,r \in X$ and every $R>0$, we have

  $$\begin{aligned} \lim _{N \rightarrow \infty } \sup \Big \{ |H(x,r,Z+\lambda BQ_N)-H(x,r,Z)|: \ |Z_{00}|\le R, \ |\lambda | \le R \Big \}=0. \end{aligned}$$

  (5.2)

• (iv) For every $R>0$ there exists a modulus of continuity $\omega _R$ such that

  $$\begin{aligned} H \left( z,\frac{B(z-y)}{\varepsilon },Z \right) -H \left( y,\frac{B(z-y)}{\varepsilon },Y \right) \ge -\omega _R\left( |z-y|_{-1}\left( 1+\frac{|z-y|_{-1}}{\varepsilon }\right) \right) \end{aligned}$$

  (5.3)

  for every $\varepsilon >0$, $y,z \in X$ such that $|y|_X,|z|_X\le R$, $Y,Z \in \mathcal {S}(X)$ satisfying

  $$\begin{aligned} Y=P_N Y P_N \quad Z=P_N Z P_N \end{aligned}$$

  and

  $$\begin{aligned} \frac{3}{\varepsilon }\left( \begin{array}{cc}B P_{N} &{} 0 \\ 0 &{} B P_{N}\end{array}\right) \le \left( \begin{array}{cc}Y &{} 0 \\ 0 &{} -Z\end{array}\right) \le \frac{3}{\varepsilon }\left( \begin{array}{cc}B P_{N} &{} -B P_{N} \\ -B P_{N} &{} B P_{N}\end{array}\right) . \end{aligned}$$

• (v) If $C>0$ is the constant in (4.20) and (4.21), then, for every $x \in X, p, q \in X, Y,Z \in \mathcal {S}(X)$,

  $$\begin{aligned} | H(x, r+q,Y+Z)-H(x, r, Y)| \le C\left( 1+|x|_X\right) |q|_X+\frac{1}{2}C^2\left( 1+|x|_X\right) ^{2}|Z_{00}|. \end{aligned}$$

  (5.4)

The HJB equation associated with the optimal control problem is the infinite dimensional PDE

$$\begin{aligned} \rho v(x) - \langle \tilde{\mathcal {A}} x,Dv(x)\rangle _X + H(x,Dv(x),D^2v(x))=0, \quad \forall x \in X. \end{aligned}$$

(5.5)

We recall the definition of B-continuous viscosity solution from Fabbri et al. (2017).

Definition 5.2

• (i) $\phi :X \rightarrow \mathbb {R}$ is a regular test function if

  $$\begin{aligned} \phi \in \Phi&:= \{ \phi \in C^2(X): \phi \textit{ is }B\textit{-lower semicontinuous and }\phi , \\&\qquad \qquad D\phi , D^2\phi , A^* D\phi \textit{ are uniformly continuous on }X\}; \end{aligned}$$

• (ii) $g :X \rightarrow \mathbb {R}$ is a radial test function if

  $$\begin{aligned} g \in \mathcal {G}&:= \{ g \in C^2(X): g(x)=g_0(|x|_X) \textit{ for some } g_0 \\&\qquad \qquad \in C^2([0,\infty )) \textit{ non-decreasing}, g_{0}'(0)=0 \}. \end{aligned}$$

Note that, if $g\in \mathcal {G}$, we have

$$\begin{aligned} D g(x)=\left\{ \begin{array}{l} g_0^{\prime }(|x|_{X}) \frac{x}{|x|_{X}}, \quad \ \ \ \text{ if } \ x \ne 0, \\ 0, \quad \quad \ \ \ \ \ \ \ \ \ \ \ \ \,\,\, \ \text{ if } \ x=0. \end{array}\right. \end{aligned}$$

(5.6)

Definition 5.3

• (i) A locally bounded B-upper semicontinuous function $v:X\rightarrow \mathbb {R}$ is a viscosity subsolution of (5.5) if, whenever $v-\phi -g$ has a local maximum at $x \in X$ for $\phi \in \Phi , g \in \mathcal {G}$, then

  $$\begin{aligned} \rho v(x) - \langle x,\tilde{\mathcal {A}}^* D\phi (x)\rangle _{X} + H(x,D\phi (x)+Dg(x) ,D^2\phi (x)+D^2g(x))\le 0. \end{aligned}$$

• (ii) A locally bounded B-lower semicontinuous function $v:X\rightarrow \mathbb {R}$ is a viscosity supersolution of (5.5) if, whenever $v+\phi +g$ has a local minimum at $x \in X$ for $\phi \in \Phi$, $g \in \mathcal {G}$, then

  $$\begin{aligned} \rho v(x) + \langle x,\tilde{\mathcal {A}}^* D\phi (x)\rangle _{X} + H(x,-D\phi (x)-Dg(x) ,-D^2\phi (x)-D^2g(x))\ge 0. \end{aligned}$$

• (iii) A viscosity solution of (5.5) is a function $v:X\rightarrow \mathbb {R}$ which is both a viscosity subsolution and a viscosity supersolution of (5.5).

Define $\mathcal {S}:=\{u :X \rightarrow \mathbb {R}: \exists k\ge 0 \ \text{ satisfying } (5.7) \text{ and } {\tilde{C}}\ge 0 \,\text{ such } \text{ that }\, |u(x)|\le {\tilde{C}} (1+|x|_X^k)\},$ where

$$\begin{aligned} {\left\{ \begin{array}{ll} k<\frac{\rho }{C+\frac{1}{2} C^{2}}, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \quad \text{ if } \ \frac{\rho }{C+\frac{1}{2} C^{2}} \le 2, \\ C k+\frac{1}{2} C^{2} k(k-1)<\rho , \quad \text{ if } \frac{\rho }{C+\frac{1}{2} C^{2}}>2, \end{array}\right. } \end{aligned}$$

(5.7)

and C is the constant appearing in (4.20) and (4.21).

We can now state the theorem characterizing V as the unique viscosity solution of (5.5) in $\mathcal {S}$.

Theorem 5.4

Let Assumptions 2.2, 2.3, and 4.9 hold. The value function V is the unique viscosity solution of (5.5) in the set $\mathcal {S}$.

Proof

Notice that $V \in \mathcal {S}$ by Proposition 4.10.

The proof of the fact that V is the unique viscosity solution of the HJB equation can be found in (Fabbri et al. 2017, Theorem 3.75) as all assumptions of this theorem are satisfied due to Lemma 5.1. $\square$

Remark 5.5

We remark that, similarly to de Feo et al. (2023), Theorem 5.4 also holds in the deterministic case, i.e. when $\sigma (x,u)=0$. (in which case we may take $\rho _0=Cm$ and $k<\rho /C$ in (5.7)). The theory of viscosity solutions handles well degenerate HJB equations, i.e. when the Hamiltonian satisfies

$$\begin{aligned} H(x,p,Y) \le H(x,p,Z) \end{aligned}$$

for every $Y,Z \in S(X)$ such that $Z\le Y$. Hence viscosity solutions can be used in connection with the dynamic programming method for optimal control of stochastic differential equations in the case of degenerate noise in the state equation, in particular, when it completely vanishes (deterministic case). This is not possible using the mild solutions approach (see de Feo et al. 2023; Fabbri et al. 2017).

Remark 5.6

In this work we could not prove the partial differentiability of V with respect to $x_0$ as in (de Feo et al. 2023, Theorem 6.5). Indeed in (de Feo et al. 2023, Theorem 6.5) a key assumption was

$$\begin{aligned} |V(y)-V(x)| \le K_R|y-x|_{-1}, \quad \forall x, y \in X,|x|_X,|y|_X \le R. \end{aligned}$$

(5.8)

In de Feo et al. (2023) this condition holds under some standard assumptions, see (de Feo et al. 2023, Example 6.2). In particular the cost $l(\cdot ,u)$ is assumed to be Lipschitz (uniformly in u).

However in the present paper we could not prove (5.8): indeed due to Remark 4.5 the following inequality (which holds true in de Feo et al. (2023)) is false

$$\begin{aligned} |x_0| \le C |x|_{-1} \quad \forall x=(x_0,x_1) \in X. \end{aligned}$$

(5.9)

This means that even for a Lipschitz $l(\cdot ,u)$ (uniformly in u) we cannot use (5.9) in the following way

$$\begin{aligned} |L(x, u)-L(y, u)|=|l(x_0,u)-l(y_0,u)| \le |x_0-y_0| \le C|x-y|_{-1} \quad \forall x,y \in X, \end{aligned}$$

(5.10)

to get the Lipschitzianity of L with respect to $|\cdot |_{-1}$ (as it was done in de Feo et al. (2023)). The best we could get in this paper is (4.22), i.e.

$$\begin{aligned} |L(x, u)-L(y, u)| \le \omega _R\left( |x-y|_{-1}\right) , \end{aligned}$$

but this is of course not enough in order to prove (5.8). Hence we could not proceed as in the proof of (de Feo et al. 2023, Theorem 6.5).

We finally remark that the same reason prevented us to apply $C^{1,1}-$regularity results from de Feo et al. (2023), where $L(\cdot ,u)$ was assumed to be Lipschitz with respect to $|\cdot |_{-1}$, uniformly in u.

6 Applications

In this section we provide applications of our results to problems coming from economics.

6.1 Optimal advertising with delays

The following model is a generalization of the ones in Gozzi and Marinelli (2006), Gozzi et al. (2009) to the case of controlled diffusion. We recall that in de Feo et al. (2023) the case with no delays in the control (i.e. $p_1=0$) was treated.

The model for the dynamics of the stock of advertising goodwill y(t) of the product is given by the following controlled SDDE

$$\begin{aligned} {\left\{ \begin{array}{ll} dy(t) = \displaystyle \left[ a_0 y(t)+b_0 u(t )+ \int _{-d}^0 a_1(\xi )y(t+\xi )\,d\xi + \int _{-d}^0p_1(\xi )u(t+\xi )\,d\xi \right] dt \\ \quad +\left[ \sigma _0 + \gamma _0 u(t) \right] dW(t), \quad t \ge 0, \\ y(0)=\eta _0, \quad y(\xi )=\eta _1(\xi ),\; u(\xi )=\delta (\xi ),\;\; \xi \in [-d,0), \end{array}\right. } \end{aligned}$$

where $d>0$, the control process u(t) models the intensity of advertising spending and W is a real-valued Brownian motion. Moreover

• (i) $a_0 \le 0$ is a constant factor of image deterioration in absence of advertising;

• (ii) $b_0 \ge 0$ is a constant advertising effectiveness factor;

• (iii) $a_1 \le 0$ is a given deterministic function satisfying the assumptions used in the previous sections and represent the distribution of the forgetting time;

• (iv) $p_1 \ge 0$ is a given deterministic function satisfying the assumptions used in the previous sections and it is the density function of the time lag between the advertising expenditure and the corresponding effect on the goodwill level;

• (v) $\sigma _0>0$ is a fixed uncertainty level in the market;

• (vi) $\gamma _0> 0$ is a constant uncertainty factor which multiplies the advertising spending;

• (vii) $\eta _0 \in \mathbb {R}$ is the level of goodwill at the beginning of the advertising campaign;

• (viii) $\eta _1 \in L^2([-d,0];\mathbb {R})$ is the history of the goodwill level.

• (ix) $\delta \in L^2([-d,0]; U)$ is the history of the advertising spending.

Again, we use the same setup of the stochastic optimal control problem as the one in Sect. 2 and the control set U is here $U= [0,{\bar{u}}]$ for some ${\bar{u}}>0$. The optimization problem is

$$\begin{aligned} \inf _{u \in \mathcal {U}}\mathbb {E} \left[ \int _0^\infty e^{-\rho s} l(y(s),u(s)) d s\right] , \end{aligned}$$

where $\rho >0$ is a discount factor, $l(x,u)=h(u)-g(x)$, with a continuous and convex cost function $h :U \rightarrow \mathbb {R}$ and a continuous and concave utility function $g :\mathbb {R} \rightarrow \mathbb {R}$ which satisfies Assumption 2.3.

We are then in the setting of Sect. 4. Therefore we can use Theorem 5.4 to characterize the value function V as the unique viscosity solution to (5.5).

6.2 Optimal investment models with time-to-build

The following model is inspired by (Fabbri and Federico 2014, p. 36). See also, e.g., (Bambi et al. 2012, 2017) for similar models in the deterministic setting.

Let us consider a state process y(t),  representing the stock capital of a certain enterprise at time t, and a control process $u(t) \ge 0$, representing the investment undertaken at time t to increase y. We assume that the dynamics of y(t) is given by the following SDDE

$$\begin{aligned} {\left\{ \begin{array}{ll} d y(t)= \left[ b_0 (u(t))+ \int _{-d}^0 p_1(\xi ) u (t+\xi ) d \xi \right] d t+\sigma _0( u(t) )d W(t), \quad t \ge 0, \\ y(0)=\eta _0, \quad u(\xi )=\delta (\xi ),\;\; \xi \in [-d,0), \end{array}\right. } \end{aligned}$$

where

• (i) $b_0 :U \rightarrow [0,\infty )$ is a continuous bounded function representing an instantaneous effect of the investment on the capital;

• (ii) $p_1 \ge 0$ is a given deterministic function satisfying the assumptions used in the previous sections and representing the density function of the time-to-build between the investment and the corresponding effect on the stock capital;

• (iii) $\sigma _0 :U \rightarrow [0,\infty )$ is a a continuous bounded function representing the uncertainty of achievement of the investment plans.

• (iv) $\eta _0 \in \mathbb {R}$ is the initial level of the capital;

• (v) $\delta \in L^2([-d,0];U)$ is the history of the investment spending.

Again, we use the same setup of the stochastic optimal control problem of Sect. 2 and the control set U here is $U= [0,{\bar{u}}]$ for some ${\bar{u}}>0$. The goal is to maximize, over all $u (\cdot ) \in \mathcal {U},$ the expected integral of the discounted future profit flow in the form

$$\begin{aligned} \mathbb {E}\left[ \int _0^{\infty } e^{-\rho t}( F(y(t))-C(u(t))) d t\right] , \end{aligned}$$

where $F: \mathbb {R} \rightarrow \mathbb {R}$ is a production function and $C: U \rightarrow \mathbb {R}$ is a cost function. We assume that F, C satisfy Assumption 2.3. The optimization problem is equivalent to minimize, over all $u (\cdot ) \in \mathcal {U},$

$$\begin{aligned} \mathbb {E} \left[ \int _0^\infty e^{-\rho t} l(y(t),u(t)) d t\right] , \end{aligned}$$

where $l(x,u):=C(u)- F(x)$. We are then in the setting of Sect. 2 (with $a_1=0$). Therefore we can use Theorem 5.4 to characterize the value function V as the unique viscosity solution to (5.5).