Energy transition under scenario uncertainty: a mean-field game of stopping with common noise

Abstract

We study the impact of transition scenario uncertainty, namely that of future carbon price and electricity demand, on the pace of decarbonization of the electricity industry. To this end, we develop a theory of optimal stopping mean-field games with non-Markovian common noise and partial observation. For mathematical tractability, the theory is formulated in discrete time and with common noise restricted to a finite probability space. We prove the existence of Nash equilibria for this game using the linear programming approach. We then apply the general theory to build a discrete time model for the long-term dynamics of the electricity market subject to common random shocks affecting the carbon price and the electricity demand. We consider two classes of agents: conventional producers and renewable producers. The former choose an optimal moment to exit the market and the latter choose an optimal moment to enter the market by investing into renewable generation. The agents interact through the market price determined by a merit order mechanism with an exogenous stochastic demand. We illustrate our model by an example inspired by the UK electricity market, and show that scenario uncertainty leads to significant changes in the speed of replacement of conventional generators by renewable production.

Appendices

Complementary results on the linear programming approach for MFGs of optimal stopping

In this appendix, we provide a theoretical analysis of the linear programming approach in discrete time in an abstract and general case. We show by a duality argument that the relaxed set is equal to the closed convex hull of the set induced by Markovian stopping times. In particular, this result allows us to derive a probabilistic representation of the relaxed set in terms of randomized stopping times (in the sense of [16]).

Markovian and randomized stopping times in the canonical space

We consider a discrete time setting with horizon $T\in \mathbb N^*$ and we let $I=\{0, \ldots , T\}$ be the set of time indices and $I^*=\{1, \ldots , T\}$. We are given a nonempty compact metric space (E, d) and consider two canonical spaces: $\Omega _X=E^I$ and ${\overline{\Omega }} = E^I\times I$, where $E^I$ is the set of functions from I to E endowed with the pointwise convergence topology (which coincides with the uniform convergence topology here). Denote by ${\mathcal {B}}(\Omega _X)$ and ${\mathcal {B}}(\overline{\Omega })$ the respective Borel $\sigma$-algebras. Let X be the identity map on $\Omega _X$ and let ${\overline{X}}$ and $\overline{\tau }$ be the projections on ${\overline{\Omega }}$, i.e. $\overline{X}({\overline{\omega }})=x$ and ${\overline{\tau }}({\overline{\omega }})=\theta$ for each ${\overline{\omega }}=(x, \theta )\in {\overline{\Omega }}$. On $(\Omega _X, {\mathcal {B}}(\Omega _X))$ we define the filtration ${\mathcal {F}}^X_t:=\sigma (X_s:s\le t)$, for $t\in I$, and on $({\overline{\Omega }}, {\mathcal {B}}({\overline{\Omega }}))$ we define the filtrations ${\mathcal {F}}^{{\overline{X}}}_t:=\sigma ({\overline{X}}_s:s\le t)$ and ${\mathcal {F}}^{{\overline{\tau }}}_t:=\sigma (\{{\overline{\tau }}=0\}, \ldots , \{{\overline{\tau }}=t\})$, for $t\in I$. Note that ${\mathbb {F}}^{{\overline{\tau }}}$ is the smallest filtration for which ${\overline{\tau }}$ is a stopping time. Define $\overline{{\mathcal {F}}}_t:={\mathcal {F}}^{{\overline{X}}}_t\vee {\mathcal {F}}^{{\overline{\tau }}}_t$, for $t\in I$. For any probability $\overline{\mathbb {P}}$ on $({\overline{\Omega }}, {\mathcal {B}}({\overline{\Omega }}))$, we denote by $\mathbb F^{{\overline{X}}, \overline{\mathbb {P}}}=({\mathcal {F}}_t^{{\overline{X}}, \overline{\mathbb {P}}})_{t\in I}$ and $\overline{\mathbb F}^{\overline{\mathbb {P}}}=(\overline{{\mathcal {F}}}^{\overline{\mathbb {P}}}_t)_{t\in I}$ the filtrations

$$\begin{aligned} {\mathcal {F}}_t^{{\overline{X}}, \overline{\mathbb {P}}}:={\mathcal {F}}_t^{{\overline{X}}}\vee {\mathcal {N}}_{\overline{\mathbb {P}}}(\overline{{\mathcal {F}}}_T), \quad \overline{{\mathcal {F}}}_t^{\overline{\mathbb {P}}}:=\overline{{\mathcal {F}}}_t \vee {\mathcal {N}}_{\overline{\mathbb {P}}}(\overline{{\mathcal {F}}}_T). \end{aligned}$$

We are given transition kernels $(\pi _t)_{t\in I^*}$ on E and an initial law $m_0^*$ on E. We assume that the transition kernels are continuous as maps from E to ${\mathcal {P}}(E)$. Let $\nu \in {\mathcal {P}}(E^I)$ be the unique law of the Markov chain with transition kernels $(\pi _t)_{t\in I^*}$ and initial law $m_0^*$. We denote by $\mathbb F^{X, \nu }=({\mathcal {F}}_t^{X, \nu })_{t\in I}$ the filtration

$$\begin{aligned} {\mathcal {F}}_t^{X, \nu }:={\mathcal {F}}_t^{X}\vee {\mathcal {N}}_{\nu }({\mathcal {F}}_T^X). \end{aligned}$$

Definition 12

Denote by ${\mathcal {T}}$ the set of measurable functions $\tau :\Omega _X\rightarrow I$ such that $\tau$ is an $\mathbb F^{X, \nu }$-stopping time.

Definition 13

Let ${\mathcal {A}}_0$ be the set of probabilities $\overline{\mathbb {P}}$ on $({\overline{\Omega }}, {\mathcal {B}}({\overline{\Omega }}))$ such that

• 1. Under $\overline{\mathbb {P}}$, ${\overline{X}}$ is a Markov chain with initial distribution $m_0^*$ and transition kernels $(\pi _t)_{t\in I^*}$.

• 2. ${\overline{\tau }}$ is an $\mathbb F^{{\overline{X}}, \overline{\mathbb {P}}}$-stopping time.

Lemma 10

(Characterization of ${\mathcal {A}}_0$ by Markovian stopping times) It holds that

$$\begin{aligned} {\mathcal {A}}_0=\{\nu (dx)\delta _{\tau (x)}(d\theta ): \tau \in {\mathcal {T}}\}. \end{aligned}$$

Proof

Let $\overline{\mathbb {P}}\in {\mathcal {A}}_0$. By the first condition, the first marginal of $\overline{\mathbb {P}}$ coincides with $\nu$. Since ${\overline{\tau }}$ is ${\mathcal {F}}_T^{{\overline{X}}, \overline{\mathbb {P}}}$-measurable, there exists some measurable function $\tau :\Omega _X\rightarrow I$ such that $\overline{\mathbb {P}}({\overline{\tau }}= \tau ({\overline{X}}))=1$. In particular we can disintegrate $\overline{\mathbb {P}}$ as $\overline{\mathbb {P}}(dx, d\theta )=\nu (dx)\delta _{\tau (x)}(d\theta )$. Using that $\overline{\tau }$ is an $\mathbb F^{{\overline{X}}, \overline{\mathbb {P}}}$-stopping time and that ${\overline{\tau }}= \tau ({\overline{X}})$ $\overline{\mathbb {P}}$-a.s., we get that $\tau ({\overline{X}})$ is an $\mathbb F^{{\overline{X}}, \overline{\mathbb {P}}}$-stopping time. Moreover, for any $t\in I$ and $B\in {\mathcal {F}}_t^{\overline{X}, \overline{\mathbb {P}}}$, it is easy to check that $(X, \tau (X))^{-1}(B)\in {\mathcal {F}}_t^{X, \nu }$. In particular, taking $B=\{\tau (\overline{X})\le t\}$, we get $\{\tau (X)\le t\}\in {\mathcal {F}}_t^{X, \nu }$, implying that $\tau \in {\mathcal {T}}$.

Now let $\tau \in {\mathcal {T}}$ and define $\overline{\mathbb {P}}(dx, d\theta ):=\nu (dx)\delta _{\tau (x)}(d\theta )$. Since $\overline{\mathbb {P}}\circ \overline{X}^{-1}=\nu$, the first condition is verified. Define the set $C:=\{\overline{\tau }=\tau ({\overline{X}})\}\in \overline{{\mathcal {F}}}_T$ which has probability 1 under $\overline{\mathbb {P}}$. We have

$$\begin{aligned} \{\tau ({\overline{X}})\le t\} = \{x\in \Omega _X:\tau (x)\le t\}\times I = (C_1\times I)\cup (C_2\times I), \end{aligned}$$

where $C_1\in {\mathcal {F}}_t^X$ and $C_2\in {\mathcal {N}}_\nu ({\mathcal {F}}_T^X)$ are such that $\{x\in \Omega _X:\tau (x)\le t\}=C_1\cup C_2$. We can deduce that $C_1\times I\in {\mathcal {F}}_t^{\overline{X}}$ and $C_2\times I\in {\mathcal {N}}_{\overline{\mathbb {P}}}(\overline{{\mathcal {F}}}_T)$, so that $\{\tau ({\overline{X}})\le t\}\in {\mathcal {F}}_t^{\overline{X}, \overline{\mathbb {P}}}$. Finally,

$$\begin{aligned} \{\overline{\tau }\le t\}=(\{\tau ({\overline{X}})\le t\} \cap C)\cup (\{\overline{\tau }\le t\}\cap C^c)\in {\mathcal {F}}_t^{\overline{X}, \overline{\mathbb {P}}}. \end{aligned}$$

$\square$

Let ${\mathcal {L}}$ be the generator associated to $(\pi _t)_{t\in I^*}$. For any $\varphi \in C(I\times E)$, define the process $M(\varphi )$ by $M_0(\varphi )=\varphi (0, {\overline{X}}_0)$ and

$$\begin{aligned} M_t(\varphi ) = \varphi (t, {\overline{X}}_t) - \sum _{s=0}^{t-1}{\mathcal {L}}(\varphi )(s, {\overline{X}}_s), \quad t\in I^*. \end{aligned}$$

By taking the conditional expectation, one can show the following lemma.

Lemma 11

(Equivalence between the Markov property and the martingale problem) Let $\overline{\mathbb G}$ be a filtration including ${\mathbb {F}}^{\overline{X}}$ and $\overline{\mathbb {P}}$ be a probability on $(\overline{\Omega }, {\mathcal {B}}(\overline{\Omega }))$ such that $\overline{\mathbb {P}}\circ \overline{X}_0^{-1}=m_0^*$. Then, the following are equivalent:

• 1. Under $\overline{\mathbb {P}}$, ${\overline{X}}$ is a $\overline{\mathbb G}$-Markov chain with transition kernels $(\pi _t)_{t\in I^*}$.

• 2. Under $\overline{\mathbb {P}}$, for any $\varphi \in C(I\times E)$, the process $M(\varphi )$ is a $\overline{\mathbb G}$-martingale.

Definition 14

Let ${\mathcal {A}}_1$ be the set of probabilities $\overline{\mathbb {P}}$ on $(\overline{\Omega }, {\mathcal {B}}({\overline{\Omega }}))$ such that under $\overline{\mathbb {P}}$, ${\overline{X}}$ is an $\overline{\mathbb F}$-Markov chain with initial distribution $m_0^*$ and transition kernels $(\pi _t)_{t\in I^*}$. By Lemma 11, ${\mathcal {A}}_1$ is exactly the set of probability measures $\overline{\mathbb {P}}$ on $({\overline{\Omega }}, {\mathcal {B}}({\overline{\Omega }}))$ such that $\overline{\mathbb {P}}\circ \overline{X}_0^{-1}=m_0^*$ and under $\overline{\mathbb {P}}$, for any $\varphi \in C(I\times E)$, the process $M(\varphi )$ is an $\overline{\mathbb F}$-martingale.

Lemma 12

We have the inclusion ${\mathcal {A}}_0\subset {\mathcal {A}}_1$.

Proof

Let $\overline{\mathbb {P}}\in {\mathcal {A}}_0$. Since ${\overline{\tau }}$ is an ${\mathbb {F}}^{{\overline{X}}, \overline{\mathbb {P}}}$-stopping time, we have that $\overline{{\mathbb {F}}}^{\overline{\mathbb {P}}}=\mathbb F^{{\overline{X}}, \overline{\mathbb {P}}}$. For any $\varphi \in C(E)$, we obtain

$$\begin{aligned} \mathbb E^{\overline{\mathbb {P}}}[\varphi ({\overline{X}}_t)|\overline{{\mathcal {F}}}_{t-1}]=\mathbb E^{\overline{\mathbb {P}}}[\varphi ({\overline{X}}_t)|\overline{{\mathcal {F}}}_{t-1}^{\overline{\mathbb {P}}}]=\mathbb E^{\overline{\mathbb {P}}}[\varphi ({\overline{X}}_t)|{\mathcal {F}}_{t-1}^{{\overline{X}}, \overline{\mathbb {P}}}]=\mathbb E^{\overline{\mathbb {P}}}[\varphi ({\overline{X}}_t)|{\overline{X}}_{t-1}]. \end{aligned}$$

$\square$

Lemma 13

(Immersion property) If $\overline{\mathbb {P}}\in {\mathcal {A}}_1$, then $\mathbb F^{{\overline{X}}}$ is immersed in $\overline{\mathbb F}$ under $\overline{\mathbb {P}}$.

Proof

We need to show that for all $t\in I$, under $\overline{\mathbb {P}}$, $\overline{{\mathcal {F}}}_t$ is conditionally independent of ${\mathcal {F}}^{{\overline{X}}}_T$ given ${\mathcal {F}}^{{\overline{X}}}_t$. Let $\varphi _t:{\overline{\Omega }}\rightarrow \mathbb R$ be bounded and $\overline{{\mathcal {F}}}_t$-measurable, $\psi _t:E^I\rightarrow \mathbb R$ be bounded and ${\mathcal {F}}_t^{{\overline{X}}}$-measurable, $\psi _{t+}:E^I\rightarrow \mathbb R$ be bounded and $\sigma ({\overline{X}}_{t+1}, \ldots , {\overline{X}}_T)$-measurable and $\phi _{t}:E^I\rightarrow \mathbb R$ be bounded and ${\mathcal {F}}_t^{{\overline{X}}}$-measurable. We have

$$\begin{aligned} \mathbb E^{\overline{\mathbb {P}}}[\varphi _t({\overline{X}}, {\overline{\tau }})\psi _t({\overline{X}})\psi _{t+}({\overline{X}})\phi _{t}({\overline{X}})]&= \mathbb E^{\overline{\mathbb {P}}}[\varphi _t({\overline{X}}, {\overline{\tau }})\psi _t({\overline{X}})\mathbb E^{\overline{\mathbb {P}}}[\psi _{t+}({\overline{X}})|\overline{{\mathcal {F}}}_t]\phi _{t}({\overline{X}})]\\&= \mathbb E^{\overline{\mathbb {P}}}[\varphi _t({\overline{X}}, {\overline{\tau }})\psi _t({\overline{X}})\mathbb E^{\overline{\mathbb {P}}}[\psi _{t+}({\overline{X}})|{\mathcal {F}}_t^{{\overline{X}}}]\phi _{t}({\overline{X}})]\\&= \mathbb E^{\overline{\mathbb {P}}}[\mathbb E^{\overline{\mathbb {P}}}[\varphi _t({\overline{X}}, \overline{\tau })|{\mathcal {F}}_t^{{\overline{X}}}]\mathbb E^{\overline{\mathbb {P}}}[\psi _t(\overline{X})\psi _{t+}({\overline{X}})|{\mathcal {F}}_t^{\overline{X}}]\phi _{t}({\overline{X}})]. \end{aligned}$$

This shows that

$$\begin{aligned} \mathbb E^{\overline{\mathbb {P}}}[\varphi _t({\overline{X}}, {\overline{\tau }})\psi _t({\overline{X}})\psi _{t+}({\overline{X}})|{\mathcal {F}}_t^{{\overline{X}}}]=\mathbb E^{\overline{\mathbb {P}}}[\varphi _t({\overline{X}}, {\overline{\tau }})|{\mathcal {F}}_t^{{\overline{X}}}]\mathbb E^{\overline{\mathbb {P}}}[\psi _t({\overline{X}})\psi _{t+}({\overline{X}})|{\mathcal {F}}_t^{{\overline{X}}}]. \end{aligned}$$

This is sufficient to prove the claim. $\square$

Applying a discrete time version of Proposition 1.10 in Chapter 1 in [14], we get the following corollary.

Corollary 14

We have the equality ${\mathcal {A}}_1=\{\nu (dx)\kappa (x, d\theta ):\kappa \in {\mathcal {K}}\}$, where ${\mathcal {K}}$ is the set of transition kernels from $\Omega _X$ to I such that for all $t\in I$ and $B\in \sigma (\{0\}, \ldots , \{t\})$, the mapping $x\mapsto \kappa (x, B)$ is ${\mathcal {F}}_t^{X, \nu }$-measurable.

In particular, the set ${\mathcal {A}}_1$ can be interpreted as the set of randomized stopping times since each kernel $\kappa \in {\mathcal {K}}$ gives a probability to stop given the observation of the underlying process (in an adapted way through the filtration of the process). The set ${\mathcal {A}}_0$ corresponds to the set of Markovian stopping times in the sense that up to null sets, the stopping rule is determined by the information given by the underlying Markov process and the kernel is given by a Dirac measure (cf Lemma 10).

Proposition 15

The set ${\mathcal {A}}_1$ is compact and convex.

Proof

The relative compactness of ${\mathcal {A}}_1$ follows since ${\mathcal {P}}({\overline{\Omega }})$ is compact. Let us show that ${\mathcal {A}}_1$ is closed. Let $(\overline{\mathbb {P}}_n)_{n\ge 1}\subset {\mathcal {A}}_1$ converging weakly to some $\overline{\mathbb {P}}$. Using the continuity of the map $(x, \theta )\mapsto x_0$ and passing to the limit, we get $\overline{\mathbb {P}}\circ {\overline{X}}_0^{-1}=m_0^*$. Let $\varphi \in C(E)$ and $\psi \in C({\overline{\Omega }})$ an $\overline{{\mathcal {F}}}_{t-1}$-measurable function, then

$$\begin{aligned} \mathbb E^{\overline{\mathbb {P}}}[(M_t(\varphi )-M_{t-1}(\varphi ))\psi ({\overline{X}}, {\overline{\tau }})]=\lim _{n\rightarrow \infty }\mathbb E^{\overline{\mathbb {P}}_n}[(M_t(\varphi )-M_{t-1}(\varphi ))\psi ({\overline{X}}, {\overline{\tau }})]=0. \end{aligned}$$

This shows that ${\mathcal {A}}_1$ is closed and henceforth compact. The convexity follows by the linearity of the conditions with respect to the probability measure. $\square$

For any $\overline{\mathbb {P}}\in {\mathcal {A}}_1$, we define the occupation measures

$$\begin{aligned}{} & {} m_t^{\overline{\mathbb {P}}}(B):=\overline{\mathbb {P}}({\overline{X}}_t\in B, t<{\overline{\tau }}), \quad B\in {\mathcal {B}}(E),\quad t\in I\setminus \{T\},\\{} & {} \mu _t^{\overline{\mathbb {P}}}(B):=\overline{\mathbb {P}}({\overline{X}}_t\in B, {\overline{\tau }}=t), \quad B\in {\mathcal {B}}(E), \quad t\in I. \end{aligned}$$

Definition 15

Define the sets

$$\begin{aligned} {\mathcal {R}}_0:=\{(\mu ^{\overline{\mathbb {P}}}, m^{\overline{\mathbb {P}}}): \overline{\mathbb {P}}\in {\mathcal {A}}_0\},\qquad {\mathcal {R}}_1:=\{(\mu ^{\overline{\mathbb {P}}}, m^{\overline{\mathbb {P}}}): \overline{\mathbb {P}}\in {\mathcal {A}}_1\}. \end{aligned}$$

Proposition 16

The set ${\mathcal {R}}_1$ is compact and convex.

Proof

For the compactness it suffices to show that ${\mathcal {A}}_1\ni \overline{\mathbb {P}}\mapsto (\mu ^{\overline{\mathbb {P}}}, m^{\overline{\mathbb {P}}})$ is continuous, since ${\mathcal {A}}_1$ is compact. Let $(\overline{\mathbb {P}}_n)_{n\ge 1}\subset {\mathcal {A}}_1$ converging weakly to $\overline{\mathbb {P}}\in {\mathcal {A}}_1$. For any $\varphi \in C(E)$ and $t\in I{\setminus }\{T\}$,

$$\begin{aligned} \int _E\varphi (x)m_t^{\overline{\mathbb {P}}_n}(dx)=\mathbb E^{\overline{\mathbb {P}}_n}[\varphi ({\overline{X}}_t)\mathbb {1}_{t<{\overline{\tau }}}]\underset{n\rightarrow \infty }{\longrightarrow }\mathbb E^{\overline{\mathbb {P}}}[\varphi ({\overline{X}}_t)\mathbb {1}_{t<{\overline{\tau }}}]=\int _E\varphi (x)m_t^{\overline{\mathbb {P}}}(dx), \end{aligned}$$

where we used the continuity of the function ${\overline{\Omega }}\ni (x, \theta )\mapsto \varphi (x_t)\mathbb {1}_{t<\theta }$. This shows that for each $t\in I{\setminus }\{T\}$, $(m_t^{\overline{\mathbb {P}}_n})_{n\ge 1}$ converges to $m_t^{\overline{\mathbb {P}}}$. By the same argument, for each $t\in I$, $(\mu _t^{\overline{\mathbb {P}}_n})_{n\ge 1}$ converges to $\mu _t^{\overline{\mathbb {P}}}$. We conclude that ${\mathcal {R}}_1$ is compact. By the linearity of the same mapping ${\mathcal {A}}_1\ni \overline{\mathbb {P}}\mapsto (\mu ^{\overline{\mathbb {P}}}, m^{\overline{\mathbb {P}}})$ and the convexity of ${\mathcal {A}}_1$, we get that ${\mathcal {R}}_1$ is convex. $\square$

We define the relaxed set in this framework as follows.

Definition 16

Let ${\mathcal {R}}$ be the set of pairs $(\mu , m)\in {\mathcal {P}}^{sub}(E)^I\times {\mathcal {P}}^{sub}(E)^{I{\setminus }\{T\}}$ such that for all $\varphi \in C(I\times E)$,

$$\begin{aligned} \sum _{t=0}^T\int _{E}\varphi (t, x) \mu _t(dx) =\int _{E} \varphi (0, x)m_0^*(dx) + \sum _{t=0}^{T-1}\int _{E}{\mathcal {L}}(\varphi )(t, x) m_t(dx). \end{aligned}$$

Proposition 17

We have the inclusions ${\mathcal {R}}_0\subset {\mathcal {R}}_1\subset {\mathcal {R}}$. In particular $\overline{\text {conv}}({\mathcal {R}}_0)\subset {\mathcal {R}}_1\subset {\mathcal {R}}$.

Proof

The first inclusion is a consequence of the inclusion ${\mathcal {A}}_0\subset {\mathcal {A}}_1$ proved in Lemma 12. For the second inclusion, let $\overline{\mathbb {P}}\in {\mathcal {A}}_1$ and consider the associated measures $(\mu ^{\overline{\mathbb {P}}}, m^{\overline{\mathbb {P}}})$. Using that for any $\varphi \in C(I\times E)$, $M(\varphi )$ is an $\overline{\mathbb F}$-martingale under $\overline{\mathbb {P}}$ and ${\overline{\tau }}$ is an $\overline{\mathbb F}$-stopping time, we get

$$\begin{aligned} \mathbb E^{\overline{\mathbb {P}}}[M_{{\overline{\tau }}}(\varphi )|\overline{{\mathcal {F}}}_0]=\varphi (0, {\overline{X}}_0), \end{aligned}$$

which implies by taking the expectation and replacing the expression of $M_{{\overline{\tau }}}(\varphi )$,

$$\begin{aligned} \mathbb E^{\overline{\mathbb {P}}}[\varphi ({\overline{\tau }}, {\overline{X}}_{{\overline{\tau }}})] = \mathbb E^{\overline{\mathbb {P}}}[\varphi (0, {\overline{X}}_0)] + \mathbb E^{\overline{\mathbb {P}}}\left[ \sum _{t=0}^{{\overline{\tau }}-1}{\mathcal {L}}(\varphi )(t, {\overline{X}}_t)\right] . \end{aligned}$$

By the definition of the measures we obtain

$$\begin{aligned} \sum _{t=0}^T\int _E\varphi (t, x)\mu _t^{\overline{\mathbb {P}}}(dx)=\int _E\varphi (0, x)m_0^*(dx) + \sum _{t=0}^{T-1}\int _E{\mathcal {L}}(\varphi )(t, x)m_t^{\overline{\mathbb {P}}}(dx). \end{aligned}$$

The last inclusions follow by Proposition 16. $\square$

Probabilistic representation

We want to show that the sets ${\mathcal {R}}$ and ${\mathcal {R}}_1$ are equal. We will follow similar arguments to [30] in order to show that the closed convex hull of ${\mathcal {R}}_0$ is equal to ${\mathcal {R}}$, which is sufficient to obtain the desired equality according to Proposition 17.

Equality of the values. We are given two functions $f\in C(I{\setminus } \{T\}\times E)$ and $g\in C(I\times E)$. We place ourselves on the filtered probability space $(\Omega _X, {\mathcal {B}}(\Omega _X), \mathbb F^X, \nu )$. For each $(t, x)\in I\times E$, define the probability measure on $\nu _{t, x}\in (\Omega _X, {\mathcal {B}}(\Omega _X))$ by

$$\begin{aligned} \nu _{t, x}(dx_0, \ldots , dx_T)=\prod _{s=0}^t\delta _x(dx_s)\prod _{\ell =t+1}^T\pi _\ell (x_{\ell -1}; dx_\ell ). \end{aligned}$$

Denote by ${\mathcal {T}}_{t, x}$ the set of $\{t, \ldots , T\}$-valued $\mathbb F^{X, \nu _{t, x}}$-stopping times. Define the value function

$$\begin{aligned} v(t, x):=\sup _{\tau \in {\mathcal {T}}_{t, x}}\mathbb E^{\nu _{t, x}}\left[ \sum _{s=t}^{\tau -1}f(s, X_s) + g(\tau , X_{\tau })\right] . \end{aligned}$$

The value function verifies the dynamic programming principle (Theorem 1.9 in Chapter 1 in [55]):

$$\begin{aligned} v(T, x)= & {} g(T, x), \quad x\in E,\\ v(t, x)= & {} \max \left\{ f(t, x) + \int _{E}v(t+1, x')\pi _{t+1}(x, dx'), g(t, x)\right\} , \quad x\in E, \; t\in I\setminus \{T\}. \end{aligned}$$

By backward induction, since g, f and the transition kernels are continuous, we obtain that $v\in C(I\times E)$. Define the quantities

$$\begin{aligned} V^S(f, g)&:=\sup _{(\mu , m)\in {\mathcal {R}}_0}\sum _{t=0}^{T-1}\int _Ef(t, x)m_t(dx) + \sum _{t=0}^T\int _Eg(t, x)\mu _t(dx)\\&=\sup _{\tau \in {\mathcal {T}}}\mathbb E^\nu \left[ \sum _{t=0}^{\tau -1}f(t, X_t) + g(\tau , X_{\tau })\right] ,\\ V^{LP}(f, g)&:=\sup _{(\mu , m)\in {\mathcal {R}}}\sum _{t=0}^{T-1}\int _Ef(t, x)m_t(dx) + \sum _{t=0}^T\int _Eg(t, x)\mu _t(dx). \end{aligned}$$

By Proposition 17, we have $V^S\le V^{LP}$. The Snell envelope $Y=(Y_t)_{t\in I}$ associated to (f, g) is $Y_t=v(t, X_t)+\sum _{s=0}^{t-1}f(s, X_s)$. We say that $\tau ^\star \in {\mathcal {T}}$ is an (f, g)-optimal stopping time if

$$\begin{aligned} V^S(f, g)=\mathbb E^\nu \left[ \sum _{t=0}^{\tau ^\star -1}f(t, X_t) + g(\tau ^\star , X_{\tau ^\star })\right] . \end{aligned}$$

The random variable $\tau _{\text {min}}:=\inf \{t\in \{0, \ldots , T\}: v(t, X_t) =g(t, X_t)\}$ is an (f, g)-optimal stopping time and $(Y_{t\wedge \tau _{\text {min}}})_{t\in I}$ is an $\mathbb F^X$-martingale. In particular,

$$\begin{aligned} V^S(f, g)=\mathbb E^\nu [Y_{\tau _{\text {min}}}]=\mathbb E^\nu [v(0, X_0)]=\int _E v(0, x)m_0^*(dx). \end{aligned}$$

Now, for all $(\mu , m)\in {\mathcal {R}}$, using v as a test function in the constraint, we get

$$\begin{aligned} V^S(f, g)&=\sum _{t=0}^T\int _E v(t, x)\mu _t(dx) -\sum _{t=0}^{T-1}\int _E {\mathcal {L}}(v)(t, x)m_t(dx) \\&\ge \sum _{t=0}^T\int _E g(t, x)\mu _t(dx) +\sum _{t=0}^{T-1}\int _E f(t, x)m_t(dx), \end{aligned}$$

where the inequality is a consequence of the dynamic programming principle. Taking the supremum over $(\mu , m)\in {\mathcal {R}}$, we deduce that $V^S(f, g)\ge V^{LP}(f, g)$, which implies that $V^S(f, g)= V^{LP}(f, g)$.

Theorem 18

The set ${\mathcal {R}}$ is equal to the closed convex hull of ${\mathcal {R}}_0$. As a consequence, ${\mathcal {R}}={\mathcal {R}}_1$, meaning that we can represent any $(\mu , m)\in {\mathcal {R}}$ with some randomized stopping time.

Proof

Assume that there exists some $(\mu ^0, m^0)\in {\mathcal {R}}{\setminus } \overline{\text {conv}}({\mathcal {R}}_0)$. Recall that for any compact metric space K, the topological dual of ${\mathcal {M}}_s(K)$ endowed with the weak topology $\sigma ({\mathcal {M}}_s(K), C(K))$ is C(K). In particular, by Theorem 3.4 (b) p. 59 in [59], there exists some $(f^0, g^0)\in C(I{\setminus }\{T\}\times E)\times C(I\times E)$ and $c\in \mathbb R$ such that for all $(\mu , m)\in \overline{\text {conv}}({\mathcal {R}}_0)$,

$$\begin{aligned}{} & {} \sum _{t=0}^{T-1}\int _E f^0(t, x)m_t(dx) + \sum _{t=0}^T\int _E g^0(t, x)\mu _t(dx)\\{} & {} <c <\sum _{t=0}^{T-1}\int _E f^0(t, x)m_t^0(dx) + \sum _{t=0}^T\int _E g^0(t, x)\mu _t^0(dx) \le V^{LP}(f^0, g^0). \end{aligned}$$

Taking the supremum over $(\mu , m)\in {\mathcal {R}}_0$, we get $V^{S}(f^0, g^0)<V^{LP}(f^0, g^0)$, which is a contradiction. $\square$

Other technical results

1.1 Exchangeable random variables and De Finetti’s theorem

We recall a version of De Finetti’s theorem adapted to our setting. Let (E, d) and $(F, \rho )$ be two complete and separable metric spaces. Consider a complete probability space $(\Omega , {\mathcal {F}}, \mathbb P)$, a sequence of E-valued random variables $(X_n)_{n\ge 1}$ and an F-valued random variable Y.

Definition 17

We say that $(X_n)_{n\ge 1}$ is i.i.d. given Y if the following two conditions are satisfied.

• 1. For all $n\ge 1$ and all $(B_k)_{k\le n}\in {\mathcal {B}}(E)^n$,

  $$\begin{aligned} \mathbb P\left( \cap _{k=1}^n\{X_k\in B_k\}|Y\right) =\prod _{k=1}^n\mathbb P(X_k\in B_k|Y)\quad a.s. \end{aligned}$$

• 2. For all $n, k\ge 1$ and all $B\in {\mathcal {B}}(E)$, $\mathbb P(X_n\in B|Y)=\mathbb P(X_k\in B|Y)$ a.s.

If $(X_n)_{n\ge 1}$ is i.i.d. given Y, then one can show that the sequence $(X_n, Y)_{n\ge 1}$ is exchangeable. In particular, using the same argument as in [39], we obtain that for all measurable functions $\psi :E\times F\mapsto \mathbb R$ such that $\mathbb E[|\psi (X_1, Y)|]<\infty$,

$$\begin{aligned} \lim _{n\rightarrow \infty }\frac{1}{n}\sum _{k=1}^n\psi (X_k, Y)={\mathbb {E}}[\psi (X_1, Y)|{\mathcal {G}}]\quad a.s. \end{aligned}$$

(17)

where ${\mathcal {G}}$ is the exchangeable $\sigma$-algebra. Using Theorem 3 in [53], one can replace ${\mathcal {G}}$ in (17) by $\sigma (Y)$ in order to derive the following result.

Theorem 19

If $(X_n)_{n\ge 1}$ is i.i.d. given Y, then, for all measurable functions $\psi :E\times F\mapsto \mathbb R$ such that ${\mathbb {E}}[|\psi (X_1, Y)|]<\infty$, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\frac{1}{n}\sum _{k=1}^n\psi (X_k, Y)=\mathbb E[\psi (X_1, Y)|Y]\quad a.s. \end{aligned}$$

1.2 Dynkin’s formula for randomized stopping times

Proposition 20

Let $\varphi \in C(I\times E)$ and $\kappa$ be a randomized stopping time. Then,

$$\begin{aligned} \sum _{t=0}^T\mathbb E[\varphi (t, X_t)\kappa (\{t\})]=\mathbb E[\varphi (0, X_0)] + \sum _{t=0}^{T-1}\mathbb E[{\mathcal {L}}(\varphi )(t, X_t)\kappa (\{t+1, \ldots , T\})]. \end{aligned}$$

Proof

Let $\overline{\Omega }:=\Omega \times I$, $\overline{{\mathcal {F}}}:={\mathcal {F}}\otimes 2^I$, $\overline{{\mathbb {P}}}(d\omega , ds):=\mathbb P(d\omega )\kappa (\omega , ds)$. Consider ${\overline{X}}_t(\omega , s):=X_t(\omega )$ and $\theta (\omega , s):= s$. Define the filtration $\mathbb F^\theta :=({\mathcal {F}}^\theta _t)_t$ with ${\mathcal {F}}_t^\theta :=\sigma (\{\theta =0\}, \ldots , \{\theta =t\})$. Now, construct the filtration $\overline{{\mathbb {F}}}:=(\overline{{\mathcal {F}}}_t)_t$, with $\overline{{\mathcal {F}}}_t:={\mathcal {F}}_t\otimes {\mathcal {F}}^\theta _t$. By construction $\theta$ is an $\overline{\mathbb F}$-stopping time. We are going to show that $\overline{X}:=(\overline{X}_t)_t$ is an $\overline{{\mathbb {F}}}$-Markov chain. Let $\psi \in C(E)$. We show first that

$$\begin{aligned} \overline{\mathbb E}[\psi (\overline{X}_{t})|\overline{{\mathcal {F}}}_{t-1}]=\mathbb E[\psi (X_t)|{\mathcal {F}}_{t-1}],\quad \overline{\mathbb P}-a.s. \end{aligned}$$

Let $B\in {\mathcal {F}}_{t-1}$ and $C\in {\mathcal {F}}_{t-1}^\theta$. We have

$$\begin{aligned} \overline{\mathbb E}[\psi (\overline{X}_{t})\mathbb {1}_{B\times C}]&=\mathbb E[\psi (X_t)\mathbb {1}_{B}\kappa (C)]=\mathbb E[{\mathbb {E}}[\psi (X_t)|{\mathcal {F}}_{t-1}]\mathbb {1}_{B}\kappa (C)]\\&=\overline{{\mathbb {E}}}[\mathbb E[\psi (X_t)|{\mathcal {F}}_{t-1}]\mathbb {1}_{B\times C}], \end{aligned}$$

where in the second equality we used the measurability property of $\kappa$. We finally have

$$\begin{aligned} \overline{\mathbb E}[\psi (\overline{X}_{t})|\overline{{\mathcal {F}}}_{t-1}]=\mathbb E[\psi (X_t)|{\mathcal {F}}_{t-1}]=\int _E\psi (x)\pi _{t}(\overline{X}_{t-1}; dx). \end{aligned}$$

This shows that $\overline{X}$ is an $\overline{\mathbb F}$-Markov chain. Now, by Dynkin’s formula,

$$\begin{aligned} \overline{\mathbb E}[\varphi (\theta , \overline{X}_{\theta })]=\overline{\mathbb E}[\varphi (0, \overline{X}_0)] + \overline{\mathbb E}\left[ \sum _{t=0}^{\theta -1}{\mathcal {L}}(\varphi )(t, {\overline{X}}_t)\right] . \end{aligned}$$

By applying Fubini’s theorem we obtain the desired result. $\square$

1.3 A continuity property for the price function

Lemma 21

Let (E, d) be a metric space. Consider the function $P:E\rightarrow \mathbb R_+$ defined by

$$\begin{aligned} P(x):=\inf \{p\ge 0:r(x)\le s(x, p)\}\wedge p_{\text {max}}, \quad x\in E, \end{aligned}$$

where $r:E\rightarrow \mathbb R_+$ and $s:E\times {\mathbb {R}}_+\rightarrow \mathbb R_+$ are continuous, for each $x\in E$, $p\mapsto s(x, p)$ is increasing with $s(x, 0)=0$ and there exists ${\tilde{p}}\ge 0$ such that $\inf _{x\in E}s(x, {\tilde{p}})\ge \sup _{x\in E}r(x)$. Then there exists for each $x\in E$ a unique $p^\star (x)\in [0, {\tilde{p}}]$ such that

$$\begin{aligned} r(x)=s(x, p^\star (x)), \quad P(x)=p^\star (x)\wedge p_{\text {max}}, \quad x\in E. \end{aligned}$$

Moreover, $x\mapsto p^\star (x)$ is continuous and as a consequence P is also continuous.

Proof

For each $x\in E$, since $r(x)\le s(x, {\tilde{p}})$, we have that ${\tilde{p}}\in \{p\ge 0:r(x)\le s(x, p)\}$. Using that $s(x, 0)=0\le r(x)$ and $p\mapsto s(x, p)$ is continuous and increasing we must have that there exists a unique $p^\star (x)\in [0, \tilde{p}]$ such that $r(x)=s(x, p^\star (x))$ and for all $p<p^\star (x)$, $r(x)>s(x, p^\star (x))$. This allows us to write $P(x)=p^\star (x)\wedge p_{\text {max}}$. Let us show that $p^\star (x)$ is continuous. Let $(x_n)_{n\ge 1}\subset E$ converging to $x\in E$. Since $[0, {\tilde{p}}]$ is compact, there exists at least one limit point of the sequence $(p^\star (x_n))_{n\ge 1}$. Let $p_0$ be a limit point of the above sequence and assume without loss of generality that the whole sequence converges to that point. Using the continuity of r and s and passing to the limit in the equality $r(x_n)=s(x_n, p^\star (x_n))$, we obtain $r(x)=s(x, p_0)$, and henceforth $p_0=p^\star (x)$. $\square$