1 Introduction and preliminary

In 1990, Pardoux and Peng [2] firstly put forward the theory of nonlinear backward stochastic differential equations (BSDEs for short) and proved an existence and uniqueness result under the Lipschitz assumption. Since then, many scholars have paid more attention to the existence and uniqueness of solutions for BSDEs under weaker assumptions, such as Lepeltier and San Martin [3], Mao [4], Jia [5] and Fan [6]. In particular, Pardoux [7] investigated the existence and uniqueness of \(L^{2}\) solutions for BSDEs when the generator g satisfies monotonicity, continuity and general growth conditions with respect to y.

On the other hand, many scholars were devoted their work to investigating BSDEs driven by different noises and its applications such as Cohen and Szpruch [8], Nualart and Schoutens [9], Zhou [10] and their references therein. In particular, Di Nunno and Sjursen [1] introduced a new class of BSDEs driven by time-changed Lévy noises. In fact, it derives from a conditional Brownian motion and a doubly stochastic Poisson random field of the following type:

$$ y_{t} = \xi + \int_{t}^{T} g (s,\lambda_{s} ,y_{s},z_{s} )\,ds - \int_{t}^{T} \int_{R} z_{s} (x)\mu(ds,dx) ,\quad t \in[0,T], $$
(1.1)

where the terminal time \(T>0\), the terminal condition \(\xi \in L^{2} (\Omega,\mathcal{F}_{T},P)\), the generator \(g(w,t,\lambda,y,z):\Omega \times[0,T] \times[0, + \infty)^{2} \times R \times\Phi \mapsto R\) is \(\mathcal{G}_{t}\)-measurable for all \(t\in[0, T ]\) and μ is the mixture of a conditional Brownian measure B on \([0, T ]\times\{0\}\) and a centered doubly stochastic Poisson measure H̃ on \([0, T ] \times R_{0}\) with

$$\mu(\Delta): = B \bigl(\Delta \cap[0,T] \times\{ 0\} \bigr) + \tilde{H} \bigl( \Delta \cap[0,T] \times R_{0} \bigr),\quad \Delta\subset[0,T]\times R. $$

They proved the existence and uniqueness of solution under the following conditions (L1)-(L2):

  1. (L1)
    $$\bigl\vert {g (w, t,\lambda,y,z ) - g \bigl(w, t,{\lambda ,y',z} \bigr)} \bigr\vert \le C \bigl\vert y-y' \bigr\vert , $$
  2. (L2)
    $$\bigl\vert {g (w, t,\lambda,y,z ) - g \bigl(w, t,{\lambda ,y,z'} \bigr)} \bigr\vert \le C \bigl\Vert z - z' \bigr\Vert _{\lambda}, $$

    where \(C>0\) is a constant and

    $$\bigl\Vert z-z' \bigr\Vert _{\lambda }^{2}: = \bigl\vert z(0)-z'(0) \bigr\vert ^{2} \lambda^{B} + \int_{R_{0} } { \bigl\vert z(x)-z'(x) \bigr\vert ^{2} q(dx)}\lambda^{H}. $$

Moreover, they also obtained a sufficient maximum principle for a general optimal control problem.

Motivated by the above work, our paper aims to discuss the existence and uniqueness of solutions for BSDEs driven by time-changed Lévy noises when the generator g satisfies monotonicity, continuity and general growth conditions with respect to y, which generalize the existence and uniqueness result of [1]. Moreover, we introduce the stability result for the first time under this structure.

In the sequel, we shall introduce the preliminary to establish our desired result. For more details, please refer to [1] and the references therein.

Let \(( {\Omega,{\mathcal {F}},P} )\) be a complete probability space and \(\lambda:= (\lambda^{B}, \lambda^{H})\in L^{1}(\Omega\times[0, T ]; (R^{+})^{2})\) be a pair of stochastic process whose components are nonnegative and continuous in probability. Let \(X = ([0, T ] \times\{0\}) \cup([0, T ] \times R_{0})\), where \(R_{0} = R\setminus\{0\}\) and \(\mathcal{B}_{X}\) denotes the Borel σ-algebra on X. Define the random measure Λ on X by

$$\begin{aligned} \Lambda(\Delta) :=& \int_{0}^{T} {1_{\{ (t,0) \in\Delta\} } (t) \lambda_{t}^{B} \,dt} + \int_{0}^{T} \int_{R_{0} } 1_{\Delta}(t,x)q(dx)\lambda_{t}^{H} \,dt \\ :=& \Lambda^{B} \bigl(\Delta \cap[0,T] \times\{ 0\} \bigr) + \Lambda^{H} \bigl(\Delta \cap[0,T] \times R_{0} \bigr), \end{aligned}$$

as the mixture of measures on disjoint sets, \(\Delta\subseteq X\). Here, q is a deterministic and σ-finite measure on the Borel sets of \(R_{0}\) satisfying \(\int_{R_{0} } {x^{2} q(dx)} < + \infty\).

Here, \(\mathcal{F}^{\Lambda}\) represents the σ-algebra which is generated by the values of Λ. B and H are called signed random measures on the Borel sets of \([0, T ] \times\{0\}\) and \([0, T ] \times R_{0}\), respectively. And they satisfy Definition 2.1 of [1]. Let \(\tilde{H}: = H - \Lambda^{H}\) be a signed random measure given by

$$\tilde{H}(\Delta) = H(\Delta) -\Lambda^{H}(\Delta),\quad \Delta\subset [0, T ] \times R_{0}. $$

Definition 1

[1]

We define the signed random measure μ on the Borel subsets of X by

$$\mu(\Delta):= B \bigl(\Delta\cap[0, T ] \times\{0\} \bigr) +\tilde{H} \bigl( \Delta \cap[0, T ] \times R_{0} \bigr),\quad \Delta\subseteq X. $$

Based on the definitions of B and H, μ is the mixture of a conditional Brownian measure B on \([0, T ] \times\{0\}\) and a centered doubly stochastic Poisson measure H̃ on \([0, T ] \times R_{0}\). Moreover, we know that

$$\begin{aligned}& E \bigl[\mu(\Delta) | \mathcal{F}^{\Lambda}\bigr] = 0,\qquad E \bigl[B( \Delta)^{2} | \mathcal{F}^{\Lambda}\bigr] = \Lambda^{B} (\Delta), \\& E \bigl[\tilde{H}(\Delta)^{2} | \mathcal{F}^{\Lambda}\bigr] = \Lambda^{H} (\Delta),\qquad E \bigl[\mu(\Delta)^{2} | \mathcal {F}^{\Lambda}\bigr] = \Lambda(\Delta), \end{aligned}$$

and

$$E \bigl[\mu(\Delta_{1} )\mu(\Delta_{2} ) | \mathcal{F}^{\Lambda}\bigr] = E \bigl[\mu(\Delta_{1} ) | \mathcal{F}^{\Lambda}\bigr]E \bigl[\mu (\Delta_{2} ) | \mathcal{F}^{\Lambda}\bigr] = 0,\quad\mbox{if } { \Delta_{1}} \cap{ \Delta_{2}}=\phi. $$

Here the random measures B and H are specific types of time-changed Brownian motion and a pure jump Lévy process.

Let \(\mathbb{F}= \{\mathcal{F}_{t}, t \in[0, T ]\}\), where \(\mathcal {F}_{t}: =\bigcap_{r >t}\mathcal{F}^{\mu}_{r}\) and \(\mathcal{F}^{\mu}_{t}= \mathcal{F}^{B}_{t}\vee\mathcal{F}^{H}_{t}\vee\mathcal{F}^{\Lambda}_{t}\). Let \(\mathbb{G}= \{\mathcal{G}_{t}, t \in[0, T ]\}\) where \(\mathcal{G}_{t} =\mathcal{F}^{\mu}_{t}\vee\mathcal{F}^{\Lambda}\). Remark that \(\mathcal {G}_{T}=\mathcal{F}_{T}\) and \(\mathcal{G}_{0}=\mathcal{F}^{\Lambda}\). Next, we introduce some definitions of spaces.

Let \(S^{2}_{\mathbb{G}}\) be the space of \(\mathbb{G}\)-adapted càdlàg processes ψ such that

$$E \Bigl[ \sup_{t \in[0,T]} \vert \psi_{t} \vert ^{2} \Bigr] < + \infty; $$

let \(L^{2}_{\mathbb{G}}\) be a subspace of \(L^{2}(\Omega\times[0, T ] \times R, \mathcal{F}\otimes\mathcal{B}_{X}, P\otimes\Lambda )\) of the random fields admitting a \(\mathbb{G}\)-predictable modification and its norm be

$$\Vert \psi \Vert _{L^{2}_{\mathbb{G}} }^{2}: = E \biggl[ { \int_{0}^{T} { \bigl\vert \psi_{s} (0) \bigr\vert ^{2} \lambda_{s}^{B} \,ds} + \int_{0}^{T} { \int_{R_{0} } { \bigl\vert \psi_{s} (x) \bigr\vert ^{2} q(dx)\lambda_{s}^{H} \,ds} } } \biggr] < + \infty; $$

let Φ be the space of functions \(z: R \rightarrow R\) such that \(\vert z(0) \vert ^{2} + \int_{R_{0} } { \vert z(x) \vert ^{2} q(dx)} < + \infty\).

Definition 2

A solution to the BSDE (1.1) is a pair of stochastic processes \((y_{t}, z_{t} )_{t\in[0,T]} \) satisfying (1.1) such that \((y_{t}, z_{t} )_{t\in[0,T]}\in S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}} \).

2 Main result

In this section, we shall state the existence and uniqueness result for solution of BSDE (1.1). Let us start with introducing the following assumptions (H1)-(H4):

  1. (H1)

    \(dP\times dt\)-a.e., for each \(z \in\Phi\), the mapping \(y \mapsto g(t,\lambda,y,z)\) is continuous. Moreover, there exists a constant \(\alpha\in R\) such that

    $$\bigl(y - y' \bigr) \bigl( {g (w,t,\lambda,y,z) - g \bigl(w,t, \lambda,y',z \bigr)} \bigr) \le\alpha \bigl\vert y - y' \bigr\vert ^{2}; $$
  2. (H2)

    there exists a continuous increasing function \(\phi:R^{+} \mapsto R^{+}\) such that \(dP\times dt\)-a.e., for each \(y \in R \),

    $$\bigl\vert g(w,t,\lambda,y,0) \bigr\vert \le \bigl\vert g(w,t,\lambda,0,0) \bigr\vert + \phi \bigl( \vert y \vert \bigr); $$
  3. (H3)

    there exists a constant \(C>0\) such that \(dP\times dt\)-a.e., for each \(y \in R \) and \(z, z'\in\Phi\)

    $$\bigl\vert {g (w, t,\lambda,y,z ) - g \bigl(w, t,{\lambda,y,z'} \bigr)} \bigr\vert \le C \bigl\Vert z - z' \bigr\Vert _{\lambda}; $$
  4. (H4)

    \(E [ { \int_{0}^{T} { \vert g(w,t,\lambda_{t},0,0) \vert ^{2} \,dt}} ] < + \infty\).

Theorem 1 is the main result of this paper.

Theorem 1

Under assumptions (H1)-(H4), for each \(\xi \in L^{2} (\Omega,\mathcal{F}_{T},P)\), BSDE (1.1) has a unique solution \((y_{t},z_{t})_{t\in[0,T]}\).

Remark 1

It is not hard to check that (L1) ⇒ (H1) and (H2). Thus, our main result extends the corresponding conclusion of [1].

Before giving the proof of Theorem 1, we establish the following two propositions (see Propositions 1-2). Proposition 1 and Proposition 2 are, respectively, the prior estimate and stability of the solutions to BSDEs, which play an important role in the proof of our main result. For convenience, we always assume that \(\alpha=0\) in (H1). Indeed if \((y_{t},z_{t})_{t\in[0,T]}\) is a solution of (1.1), then \(( \tilde{y_{t}},\tilde{z_{t}})_{t\in[0,T]}\) with \(\tilde{y_{t}}=e^{\alpha t}y_{t}\) and \(\tilde{z_{t}}=e^{\alpha t}z_{t}\) satisfies an analogous BSDE with terminal condition \(\tilde{\xi}= e^{\alpha T} \xi\) and generator

$$\tilde{g}(t,\lambda,y,z) = e^{\alpha t} g \bigl(t,\lambda,e^{ - \alpha t} y,e^{ - \alpha t} z \bigr) - \alpha y. $$

g̃ satisfies assumptions (H1)-(H4) with \(\alpha=0\). Hence in the rest of this section, we will suppose that \(\alpha=0\).

Proposition 1

Let g satisfy (H1) and (H3); let \((y_{t},z_{t})_{t\in[0,T]}\) be a solution of BSDE (1.1). Then there exists a constant \(A>0\) depending on C and T such that, for each \(0\leq u\leq t\leq T\),

$$\begin{aligned}& E \Bigl[ \sup_{r \in[t,T]} \vert y_{r} \vert ^{2} \big| \mathcal{G}_{u} \Bigr] + E \biggl[ \int_{t}^{T} \int_{R} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx) \Big| \mathcal{G}_{u} \biggr] \\& \quad \le A \biggl( E \bigl[ \vert \xi \vert ^{2} \vert \mathcal {G}_{u} \bigr] + E \biggl[ \int_{t}^{T} { \bigl\vert g (s, \lambda_{s},0,0) \bigr\vert ^{2}\,ds} \Big| \mathcal{G}_{u} \biggr] \biggr). \end{aligned}$$

Proof

From Definition 1, BSDE (1.1) is equivalent to the following BSDE:

$$\begin{aligned} y_{t} = \xi + \int_{t}^{T} {g (s,\lambda_{s} ,y_{s},z_{s} )\,ds} - \int_{t}^{T} {z_{s} (0)\,dB_{s} } - \int_{t}^{T} { \int_{R_{0}} {z_{s} (x)\tilde{H} (ds,dx)} },\quad t \in[0,T]. \end{aligned}$$

Applying the Itô formula to \(\vert y_{s} \vert ^{2}\), we get

$$\begin{aligned} d \vert y_{s} \vert ^{2} =& - 2y_{s} g(s, \lambda_{s},y_{s},z_{s} )\,ds +2y_{s} \biggl( z_{s} (0)\,dB_{s} + \int_{R_{0} } z_{s} (x)\tilde{H}(ds,dx) \biggr) \\ &{} + \biggl( z_{s} (0)\,dB_{s} + \int_{R_{0} } z_{s} (x)\tilde{H}(ds,dx) \biggr)^{2} \\ =& - 2y_{s} g(s,\lambda_{s},y_{s} ,z_{s} )\,ds + 2y_{s} \int_{R} {z_{s} (x)\mu (ds,dx)} + \int_{R} {z_{s}^{2} (x)\Lambda(ds,dx)}. \end{aligned}$$

Integrating both sides from t to T, we deduce

$$\begin{aligned}& \vert y_{t} \vert ^{2} + \int_{t}^{T} \int_{R} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx) \\& \quad = \vert \xi \vert ^{2} + 2 \int_{t}^{T} y_{s} g (s, \lambda_{s},y_{s},z_{s} )\,ds - 2 \int_{t}^{T} \int _{R} y_{s} z_{s} (x)\mu(ds,dx) . \end{aligned}$$
(2.1)

By (H1) and (H3), we can get

$$\begin{aligned} 2 y_{s} g (s,\lambda_{s},y_{s} ,z_{s} ) =& 2 y_{s} \bigl( g (s,\lambda_{s} ,y_{s},z_{s} ) - g(s,\lambda_{s},0,0 ) \bigr) + 2 y_{s} g (s,\lambda_{s},0,0) \\ \le& \frac{1}{2} { \biggl[ \bigl\vert z_{s} (0) \bigr\vert ^{2}{\lambda_{s}^{B} } + { \int_{R_{0} } { \bigl\vert z_{s} (x) \bigr\vert ^{2} q(dx)} } {\lambda_{s}^{H} } \biggr] } \\ &{} + \bigl(2C^{2}+1 \bigr) { \vert y_{s} \vert ^{2} } + { \bigl\vert g (s,\lambda_{s},0,0) \bigr\vert ^{2}}. \end{aligned}$$
(2.2)

By the Burkholder-Davis-Gundy (BDG) inequality, \(\{ {\int_{0}^{t} {\int_{R} {y_{s} z_{s} (x)\mu(ds,dx)} } } \}_{t \in[0,T]}\) is a uniformly integrable martingale. In fact, there exists a constant \(p>0\) such that

$$\begin{aligned}& E \biggl[ \sup_{r \in[0,T]} \biggl\vert \int_{0}^{r} \int_{R} y_{s} z_{s} (x)\mu(ds,dx) \biggr\vert \biggr] \\& \quad \le pE \biggl[ { \biggl( \int_{0}^{T} { { \int_{R} { \vert y_{s} \vert ^{2} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } } \biggr) } ^{1 / 2} \biggr] \\& \quad \le \frac{p}{2}E \Bigl[ { {\sup_{t \in [0,T]} \vert y_{t} \vert ^{2} } } \Bigr] + \frac{p}{2}E \biggl[ \int_{0}^{T} \int _{R} { \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} \biggr] \\& \quad< +\infty. \end{aligned}$$

Taking \(E[\cdot| \mathcal{G}_{u}]\) on both sides of (2.1) for each \(0\leq u\leq t\leq T\) and combining with the above inequality and (2.2), we can obtain

$$\begin{aligned} \frac{1}{2}E \biggl[ { { \int_{t}^{T} { \int_{R} { \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda (ds,dx)} } } \Big| \mathcal{G}_{u} } \biggr] \le X_{t}, \end{aligned}$$
(2.3)

where

$$\begin{aligned} X_{t} := &E \bigl[ { { \vert \xi \vert ^{2} } \vert \mathcal{G}_{u} } \bigr] + \bigl(2C^{2}+1 \bigr) \int_{t}^{T} {E \Bigl[ { {\sup_{r \in[s,T]} \vert y_{r} \vert ^{2}} \big| \mathcal{G}_{u} } \Bigr]\,ds} +E \biggl[ { { \int_{t}^{T} { \bigl\vert g(s,\lambda_{s} ,0,0) \bigr\vert ^{2}\,ds} } \Big| \mathcal{G}_{u} } \biggr]. \end{aligned}$$

Furthermore, it follows from the BDG inequality that there exists a positive constant d such that for each \(0\leq u\leq t\leq T\),

$$\begin{aligned}& 2E \biggl[ \sup_{r \in[t,T]} \biggl\vert \int_{r}^{T} \int_{R} {y_{s} z_{s} (x)\mu(ds,dx)} \biggr\vert \Big| \mathcal{G}_{u} \biggr] \\& \quad \le dE \biggl[ \biggl( \int_{t}^{T} \int_{R} \vert y_{s} \vert ^{2} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx) \biggr) ^{1/ 2} \Big| \mathcal{G}_{u} \biggr] \\& \quad \le\frac{1}{2}E \Bigl[ \sup_{r \in [t,T]} \vert y_{r} \vert ^{2} \big| \mathcal{G}_{u} \Bigr] + \frac{d^{2}}{2}E \biggl[ \int_{t}^{T} \int_{R} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda (ds,dx) \Big| \mathcal{G}_{u} \biggr]. \end{aligned}$$
(2.4)

By virtue of (2.2) and (2.4), it follows from (2.1) that

$$\begin{aligned}& \frac{1}{2}E \Bigl[ \sup_{r \in[t,T]} \vert y_{r} \vert ^{2} \big| \mathcal{G}_{u} \Bigr] +\frac{1}{2} E \biggl[ \int _{t}^{T} \int_{R} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx) \Big| \mathcal{G}_{u} \biggr] \\& \quad \le X_{t} + \frac{d^{2}}{2} E \biggl[ \int_{t}^{T} \int _{R} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx) \Big| \mathcal{G}_{u} \biggr]. \end{aligned}$$

Thus, from the above inequality and (2.3), we have

$$E \Bigl[ { {\sup_{r \in[t,T]} \vert y_{r} \vert ^{2} } \big| \mathcal{G}_{u}} \Bigr] + E \biggl[ { { \int_{t}^{T} { \int_{R} { \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } } \Big| \mathcal{G}_{u}} \biggr] \le \bigl(2 + 2d^{2} \bigr)X_{t}. $$

By letting

$$h_{t}:= E \Bigl[ {\sup_{r \in[t,T]} \vert y_{r} \vert ^{2} \big| \mathcal{G}_{u}} \Bigr] + E \biggl[ { { \int_{t}^{T} { \int _{R} { \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } } \Big| \mathcal{G}_{u}} \biggr] $$

and noticing the definition of \(X_{t}\), we know that, for each \(0\leq u\leq t\leq T\),

$$h_{t} \le \bigl(2 + 2d^{2} \bigr) \biggl( {E \bigl[ { { \vert \xi \vert ^{2} } \vert \mathcal{G}_{u}} \bigr] + E \biggl[ { { \int_{t}^{T} { \bigl\vert g (s,\lambda _{s},0,0) \bigr\vert ^{2} \,ds} } \Big| \mathcal{G}_{u} } \biggr] + \bigl(2C^{2} + 1 \bigr) \int _{t}^{T} {h_{s}\, ds}} \biggr). $$

Finally, Gronwall’s inequality yields, for each \(t \in[0, T]\),

$$h_{t} \le e^{(2 + 2d^{2} )(2C^{2} + 1)(T - t)} \biggl( \bigl(2 + 2d^{2} \bigr)E \bigl[ { { \vert \xi \vert ^{2} } \vert {\mathcal {G}}_{u} } \bigr] + \bigl(2 + 2d^{2} \bigr)E \biggl[ \int_{t}^{T} { \bigl\vert g(s,\lambda_{s} ,0,0) \bigr\vert ^{2} \,ds} \Big| {\mathcal {G}}_{u} \biggr] \biggr), $$

which completes the proof of Proposition 1. □

Proposition 2 is a stability result. For each \(n \geq1\), let \((y_{t}, z_{t} )_{t\in[0,T]} \) and \((y^{n}_{t}, z^{n}_{t} )_{t\in[0,T]} \) be, respectively, a solution of the BSDE (1.1) and the following BSDE depending on the parameter n:

$$\begin{aligned} y^{n}_{t} = \xi^{n} + \int_{t}^{T} {g^{n} \bigl(s, \lambda_{s},y^{n}_{s},z^{n}_{s} \bigr)\,ds} - \int _{t}^{T} { \int_{R} {z^{n}_{s} (x)\mu(ds,dx)} }. \end{aligned}$$

Furthermore, we introduce the following assumptions (B1) and (B2):

  1. (B1)

    \(\xi^{n} \in L^{2} (\Omega,\mathcal{F}_{T},P)\) and all \(g^{n}\) satisfy (H1) and (H3);

  2. (B2)

    \(\lim_{n \to\infty} E [ \vert \xi^{n} - \xi \vert ^{2} + \int_{0}^{T} \vert g^{n} (s,\lambda_{s},y_{s},z_{s} ) - g (s,\lambda_{s},y_{s},z_{s} ) \vert ^{2} \,ds] = 0\).

Proposition 2

Under assumptions (B1) and (B2), we have

$$ \lim_{n \to\infty} E \biggl[ {\sup_{t \in[0,T]} \bigl\vert y_{t}^{n} - y_{t} \bigr\vert ^{2} + \int_{0}^{T} { \int_{R} { \bigl\vert z^{n}_{s} (x)-z_{s} (x) \bigr\vert ^{2} \Lambda (ds,dx)} }} \biggr]=0. $$

Proof

For each \(n\geq1\), let \(\hat{y}_{\cdot}^{n} = y_{\cdot}^{n} - y_{\cdot}, \hat{z}_{\cdot}^{n} = z_{\cdot}^{n} - z_{\cdot}\) and \(\hat{\xi}^{n} = \xi^{n} - \xi\). Then

$$\begin{aligned} \hat{y}^{n}_{t} = \hat{\xi}^{n} + \int_{t}^{T} { \bigl[g^{n} \bigl(s, \lambda_{s},y^{n}_{s},z^{n}_{s} \bigr)-g (s,\lambda_{s},y_{s},z_{s} ) \bigr] \,ds} - \int_{t}^{T} { \int_{R} {\hat{z}^{n}_{s} (x)\mu(ds,dx)} }. \end{aligned}$$

Applying Itô’s formula to \(\vert \hat{y}_{s}^{n} \vert ^{2}\) on \([t,T]\), we have

$$\begin{aligned}& \bigl\vert \hat{y}_{t}^{n} \bigr\vert ^{2} + \int_{t}^{T} \int_{R} \bigl\vert \hat{z}_{s}^{n} (x) \bigr\vert ^{2} \Lambda (ds,dx) \\& \quad = \bigl\vert \hat{\xi}^{n} \bigr\vert ^{2} + 2 \int_{t}^{T} {\hat{y}_{s}^{n} \bigl[g^{n} \bigl(s,\lambda_{s},y^{n}_{s} ,z^{n}_{s} \bigr)-g (s,\lambda_{s} ,y_{s},z_{s} ) \bigr]\,ds} \\& \qquad{} - 2 \int_{t}^{T} \int_{R} \hat{y}_{s}^{n} \hat{z}_{s}^{n} (x)\mu (ds,dx) . \end{aligned}$$

By (H1) with \(\alpha=0\) and (H3), we have

$$\begin{aligned}& 2 \hat{y}_{s}^{n} \bigl[g^{n} \bigl(s, \lambda_{s},y^{n}_{s},z^{n}_{s} \bigr)-g (s,\lambda_{s},y_{s},z_{s} ) \bigr] \\& \quad \le 2C \bigl\vert \hat{y}_{s}^{n} \bigl\vert \bigr\Vert \hat{z}_{s}^{n} \bigr\Vert _{\lambda_{s}} + 2 \hat{y}_{s}^{n} \bigl[g^{n} (s,\lambda_{s} ,y_{s},z_{s} )-g (s,\lambda_{s} ,y_{s},z_{s} ) \bigr] \\& \quad \le \frac{1}{2} \biggl[ \bigl\vert \hat{z}_{s}^{n} (0) \bigr\vert ^{2} {\lambda_{s}^{B} } + \int _{R_{0} } \bigl\vert \hat{z}_{s}^{n} (x) \bigr\vert ^{2} q(dx) \lambda_{s}^{H} \biggr] + \bigl(2C^{2}+1 \bigr) \bigl\vert \hat{y}_{s}^{n} \bigr\vert ^{2} \\& \qquad{} + \bigl\vert g^{n} (s,\lambda_{s},y_{s} ,z_{s} )-g (s,\lambda_{s},y_{s} ,z_{s} ) \bigr\vert ^{2}. \end{aligned}$$

Thus,

$$\begin{aligned}& \bigl\vert \hat{y}_{t}^{n} \bigr\vert ^{2} + \frac{1}{2} \int_{t}^{T} \int_{R} { \bigl\vert \hat{z}_{s}^{n} (x) \bigr\vert ^{2} \Lambda(ds,dx)} \\& \quad \leq \bigl\vert \hat{\xi}^{n} \bigr\vert ^{2} - 2 \int_{t}^{T} \int_{R} {\hat{y}_{s}^{n} \hat{z}_{s}^{n} (x)\mu(ds,dx)} + \bigl(2C^{2}+1 \bigr) \int_{t}^{T} \bigl\vert \hat{y}_{s}^{n} \bigr\vert ^{2} \,ds \\& \qquad {} + \int_{t}^{T} \bigl[g^{n} (s, \lambda_{s},y_{s},z_{s} )-g (s, \lambda_{s},y_{s},z_{s} ) \bigr]^{2}\,ds. \end{aligned}$$
(2.5)

Taking the mathematical expectation on both sides of (2.5), by Gronwall’s inequality, we can obtain

$$\begin{aligned} &E \bigl[ \bigl\vert \hat{y}_{t}^{n} \bigr\vert ^{2} \bigr] \leq KE \biggl[ \bigl\vert \hat{\xi}^{n} \bigr\vert ^{2} + \int_{0}^{T} { \bigl[g^{n} (s, \lambda_{s},y_{s},z_{s} )-g (s, \lambda_{s},y_{s},z_{s} ) \bigr]^{2} \,ds} \biggr]. \end{aligned}$$

Then by (2.5) with \(t =0\) and the previous inequality, we can get

$$\begin{aligned} &E \biggl[ \int_{0}^{T} { \int_{R} { \bigl\vert \hat{z}_{s}^{n} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } \biggr] \leq K E \biggl[ \bigl\vert \hat{\xi}^{n} \bigr\vert ^{2} + \int_{0}^{T} { \bigl[g^{n} (s, \lambda_{s},y_{s},z_{s} )-g (s, \lambda_{s},y_{s},z_{s} ) \bigr]^{2} \,ds} \biggr]. \end{aligned}$$

Finally, taking the supremum over \(t \in[0,T]\) on both sides of (2.5) and the mathematical expectation, and applying the BDG inequality to the supremum of the martingale on the right-hand side, we have

$$\begin{aligned} &E \Bigl[{\sup_{t \in[0,T]} } \bigl\vert \hat{y}_{t}^{n} \bigr\vert ^{2} \Bigr] \leq K E \biggl[ \bigl\vert \hat{\xi}^{n} \bigr\vert ^{2} + \int_{0}^{T} { \bigl[g^{n}(s, \lambda_{s},y_{s},z_{s} )-g (s, \lambda_{s},y_{s},z_{s} ) \bigr]^{2} \,ds} \biggr]. \end{aligned}$$

Note that the above K is a positive constant depending on C and T. Thus, in view of (B2), we complete the proof of Proposition 2. □

Now, we give the proof of Theorem 1.

Proof of Theorem 1

The uniqueness part of Theorem 1 is established immediately by Proposition 2. Hence, we just need to prove the existence part, which can be divided into three steps.

Step 1. We shall prove that, for each \(\xi \in L^{2} (\Omega ,\mathcal{F}_{T},P)\), if there exists a constant \(M>0\) such that

$$ dP\mbox{-a.s.,}\quad \vert \xi \vert \leq M \quad\mbox{and}\quad dP\times dt\mbox{-a.e.,}\quad \bigl\vert g(t,\lambda,0,0) \bigr\vert \leq M, $$
(2.6)

and g satisfy (L1) and (H3), then there exists a unique solution to BSDE (1.1).

For \((u_{t},v_{t})_{t\in[0,T]}\in S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\), define

$$y_{t}:= E \biggl[ \xi + \int_{t}^{T} g(s,\lambda_{s} ,u_{s},v_{s} )\,ds \Big| \mathcal{G}_{t} \biggr]. $$

Moreover, by the martingale representation theorem

$$\begin{aligned} y_{t} =& E \biggl[ { {\xi + \int_{0}^{T} {g(s,\lambda_{s} ,u_{s},v_{s} )\,ds} } \Big| \mathcal{G}_{t}} \biggr] - \int_{0}^{t} {g(s,\lambda_{s} ,u_{s},v_{s} )\,ds} \\ =& E \biggl[ { {\xi + \int_{0}^{T} {g(s,\lambda_{s} ,u_{s},v_{s} )\,ds} } \Big| \mathcal{F}^{\Lambda}} \biggr] + \int_{0}^{t} { \int_{R} {z_{s}{(x)} \mu(ds,dx)} } - \int_{0}^{t} {g(s,\lambda_{s} ,u_{s},v_{s} )\,ds}, \end{aligned}$$

where \(z_{\cdot}\in L^{2}_{\mathbb{G}}\). It follows from (2.6) that \(y_{\cdot}\in S^{2}_{\mathbb{G}}\). Thus, \((y_{t},z_{t})_{t\in[0,T]}\) is a solution of the BSDE:

$$y_{t} = \xi + \int_{t}^{T} {g (s,\lambda_{s} ,u_{s},v_{s} )\,ds} - \int_{t}^{T} { \int_{R} {z_{s} (x)\mu(ds,dx)} }. $$

Therefore, we can define the map Ξ: \(S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\rightarrow S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\) by putting \(\Xi((u_{t},v_{t}))=(y_{t},z_{t})\). Let \((u_{\cdot},v_{\cdot})\), \((u'_{\cdot},v'_{\cdot})\in S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}} \), \((y_{\cdot},z_{\cdot})=\Xi((u_{\cdot},v_{\cdot}))\), \((y'_{\cdot},z'_{\cdot})=\Xi((u'_{\cdot},v'_{\cdot}))\). Let \((\bar{u}_{\cdot},\bar{v}_{\cdot})=(u_{\cdot}-u'_{\cdot},v_{\cdot}-v'_{\cdot})\), \((\bar{y}_{\cdot},\bar{z}_{\cdot})=(y_{\cdot}-y'_{\cdot},z_{\cdot}-z'_{\cdot})\). Applying Itô formula to \(e^{\beta s} \vert \bar{y}_{s} \vert ^{2}\), we can obtain, for each \(\beta \in R\),

$$\begin{aligned}& e^{\beta t} E \bigl[ \vert \bar{y}_{t} \vert ^{2} \bigr] + E \biggl[ { \int_{t}^{T} {e^{\beta s} \beta \vert \bar{y}_{s} \vert ^{2} \,ds} + \int_{t}^{T} { \int_{R} {e^{\beta s} \bigl\vert \bar{z}_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } } \biggr] \\& \quad \le2CE \biggl[ { \int_{t}^{T} {e^{\beta s} \vert \bar{y}_{s} \vert \bigl( \vert \bar{u}_{s} \vert + \Vert \bar{v}_{s} \Vert _{\lambda_{s} } \bigr)\,ds} } \biggr] \\& \quad \le4C^{2} E \biggl[ { \int_{t}^{T} {e^{\beta s} \vert \bar{y}_{s} \vert ^{2} \,ds} } \biggr] + \frac{1}{2}E \biggl[ { \int_{t}^{T} {e^{\beta s} \vert \bar{u}_{s} \vert ^{2} \,ds} + \int_{t}^{T} { \int_{R} {e^{\beta s} \bigl\vert \bar{v}_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } } \biggr]. \end{aligned}$$

Choose \(\beta=1+4C^{2}\), hence

$$\begin{aligned}& E \biggl[ \int_{t}^{T} {e^{\beta s} \vert \bar{y}_{s} \vert ^{2} \,ds} + \int_{t}^{T} \int_{R} {e^{\beta s} \bigl\vert \bar{z}_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} \biggr] \\& \quad \le\frac {1}{2}E \biggl[ \int_{t}^{T} {e^{\beta s} \vert \bar{u}_{s} \vert ^{2} \,ds} + \int_{t}^{T} \int_{R} {e^{\beta s} \bigl\vert \bar{v}_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} \biggr], \end{aligned}$$

from which Ξ is a contraction on \(S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\) equipped with the following norm:

$$\bigl\Vert (y_{\cdot},z_{\cdot}) \bigr\Vert _{\beta}: = \biggl( {E \biggl[ { \int_{0}^{T} {e^{\beta s} \vert y_{s} \vert ^{2} \,ds} + \int_{0}^{T} { \int_{R} {e^{\beta s} \bigl\vert z_{s} (x) \bigr\vert ^{2} \Lambda(ds,dx)} } } \biggr]} \biggr)^{1/2}. $$

Thus, \((y_{t},z_{t})_{t\in[0,T]}\) is a unique solution of BSDE (1.1).

Step 2. We shall prove that, for each \(\xi \in L^{2} (\Omega ,\mathcal{F}_{T},P)\) and \(V\in L^{2}_{\mathbb{G}} \), if g satisfy (H1)-(H4), then there exists a unique solution to the following BSDE:

$$\begin{aligned} y_{t} = \xi + \int_{t}^{T} {g (s,\lambda_{s} ,y_{s},V_{s} )\,ds} - \int_{t}^{T} { \int_{R} {z_{s} (x)\mu(ds,dx)} }. \end{aligned}$$
(2.7)

Firstly, we assume that (2.6) holds. For given V, \(g(s,\lambda ,y,V_{s})\) will be viewed as \(g(s,\lambda,y)\). By similar proof as Proposition 2.4 in [7], we can construct of smooth approximations \((g_{n}, n\in N)\) of g,

$$g_{n} (t,\lambda,y): = \bigl(\rho_{n} * g(t,\lambda, \cdot) \bigr) (y), $$

where \(\rho_{n}:R \mapsto R^{+}\) is a sequence of smooth functions with compact support which approximate the Dirac measure at 0 and satisfy \(\int{\rho_{n} (x)\,dx = 1}\). For each \(n\geq1\), \(g_{n}\) is smooth and monotone in y, and thus locally Lipschitz in y uniformly with respect to \((w,t,\lambda)\). Hence, we need to introduce a truncation function \(q_{p}\) in \(g_{n}\):

$$g_{n,p} (t,\lambda,y): = g_{n} \bigl(t,\lambda,q_{p} (y) \bigr), q_{p} (y): = \frac{{py}}{{ \vert y \vert \vee p}}. $$

By Step 1, \((y_{t}^{n,p},z_{t}^{n,p})_{t\in[0,T]}\) is a unique solution for BSDE (2.7) with generator \(g_{n,p}\). The sequence \((y_{t}^{n,p})_{t\in[0,T]}\) is bounded and its boundedness does not depend on p. In fact, since \(g_{n,p}\) satisfies globally Lipschitz in y, applying the Itô formula to \(\vert y_{s}^{n,p} \vert ^{2}\) on \([t,T]\), we can get

$$\bigl\vert y_{t}^{n,p} \bigr\vert ^{2} \le E \bigl[ { { \vert \xi \vert ^{2} } \vert \mathcal {G}_{t}} \bigr] + E \biggl[ { { \int_{t}^{T} { \bigl\vert g_{n,p} (s, \lambda_{s},0) \bigr\vert ^{2} \,ds} } \Big| \mathcal{G}_{t}} \biggr] + (2C+ 1) \int_{t}^{T} {E \bigl[ { { \bigl\vert y_{s}^{n,p} \bigr\vert ^{2} } \vert \mathcal{G}_{t}} \bigr]\,ds}. $$

By (2.6) and Gronwall’s inequality, for each \(r\in[t,T]\), we can obtain

$$E \bigl[ { { \bigl\vert y_{r}^{n,p} \bigr\vert ^{2} } \vert \mathcal{G}_{t}} \bigr] \le M^{2} (1 + T)e^{(2C+ 1)(T - r)}. $$

Taking \(r=t\), we obtain

$$\bigl\vert y_{t}^{n,p} \bigr\vert ^{2} \le M^{2} (1 + T)e^{(2C+ 1)T}. $$

Therefore, for p large enough, \((y_{\cdot}^{n,p},z_{\cdot}^{n,p})\) turns into \((y_{\cdot}^{n},z_{\cdot}^{n})\). From the above facts, \(g_{n}\) satisfies conditions of Proposition 1 with relative constant independent of n. So, \((y_{t}^{n},u_{t}^{n},z_{t}^{n})_{t\in[0,T]}\) is bounded, i.e.,

$$\sup_{n \in N} E \biggl[ { \int_{0}^{T} { \bigl\vert y_{s}^{n} \bigr\vert ^{2} \,ds} + \int _{0}^{T} { \bigl\vert u_{s}^{n} \bigr\vert ^{2} \,ds} + \int_{0}^{T} { \int_{R} { \bigl\vert z_{s}^{n} \bigr\vert ^{2} \Lambda(dx,ds)} } } \biggr] \le\bar{M}, $$

where \(u^{n}_{t} =g_{n}(t,\lambda_{t},y^{n}_{t})\). Thus, we can find a subsequence (named again \((y_{t}^{n},u_{t}^{n},z_{t}^{n})_{t\in[0,T]}\)) converging weakly to \((y_{t},u_{t},z_{t})_{t\in[0,T]}\). Moreover, the martingale \(\int _{t}^{T} {\int_{R} {z^{n}_{s} (x)\mu(ds,dx)} }\) converges weakly to \(\int_{t}^{T} {\int_{R} {z_{s} (x)\mu(ds,dx)} }\) in \(L^{2}(\Omega\times[0,T])\). In fact, let \(\eta\in L^{2}(\Omega,\mathcal{F}_{T};R)\), which can be written as \(\eta = E[ \eta \vert \mathcal{F}^{\Lambda}] + \int_{0}^{T} \int_{R} \varphi_{s} (x)\mu(ds,dx) \). Then

$$\begin{aligned}& E \biggl[ \eta \cdot \int_{t}^{T} \int_{R} {z_{s}^{n} (x)\mu(ds,dx)} \biggr] = E \biggl[ \int_{t}^{T} \int_{R} {z_{s}^{n} (x) \varphi_{s} (x)\Lambda (ds,dx)} \biggr] \\& \quad\rightarrow\quad E \biggl[ \int_{t}^{T} \int_{R} {z_{s} (x)\varphi_{s} (x) \Lambda(ds,dx)} \biggr] = E \biggl[ \eta \cdot \int_{t}^{T} \int_{R} z_{s} (x)\mu(ds,dx) \biggr]. \end{aligned}$$

By a similar proof to Proposition 2.4 in [7], \(u_{t}\) is equal to \(g(t,\lambda_{t}, y_{t})\).

Lastly, condition (2.6) can be taken away by a truncation procedure. For each \(x\in R\) and \(m\geq1\), let \(q^{m}(x)={xm/{( \vert x \vert \vee m)}}\). Set

$$\xi^{m}: = q^{m}(\xi)\quad\mbox{and}\quad g^{m} (t, \lambda,y): = g(t,\lambda,y) - g(t,\lambda,0) + q^{m} \bigl(g(t, \lambda,0) \bigr). $$

By (H4), for each \(m,r\geq1\), we can get

$$\lim_{m \to\infty}\sup_{r \geq1} E \bigl[ {{ \bigl\vert \xi^{m+r} - \xi^{m} \bigr\vert ^{2} }} \bigr]=0 $$

and

$$\lim_{m \to\infty}\sup_{r \geq1} E \biggl[ \int_{0}^{T} { \bigl\vert q^{m+r} \bigl(g(t,\lambda_{t},0) \bigr)-q^{m} \bigl(g(t, \lambda_{t},0) \bigr) \bigr\vert ^{2} \,dt} \biggr]=0. $$

Moreover, \(g^{m}\) satisfies (H1)-(H4) and (2.6). Thus, for each \(m\geq 1\), \((y_{t}^{m},z_{t}^{m} )_{t \in[0,T]}\) is the unique solution to the following BSDE:

$$\begin{aligned} y_{t}^{m} = \xi^{m} + \int_{t}^{T} {g^{m} \bigl(s, \lambda_{s},y^{m}_{s} \bigr)\,ds} - \int_{t}^{T} { \int_{R} {z^{m}_{s} (x)\mu(ds,dx)} }. \end{aligned}$$

Set \(\hat{y}^{m,r}_{\cdot}= y^{m+r}_{\cdot}-y^{m}_{\cdot}\), \(\hat{z}^{m,r}_{\cdot}= z^{m+r}_{\cdot}-z^{m}_{\cdot}\) and \(\hat{\xi}^{m,r}=\xi^{m+r}-\xi^{m}\). Then

$$\begin{aligned} \hat{y}_{t}^{m,r} = \hat{\xi}^{m,r} + \int_{t}^{T} {\hat{g}^{m,r} \bigl(s, \lambda_{s}, \hat{y}_{s}^{m,r} \bigr)\,ds} - \int_{t}^{T} { \int_{R} {\hat{z}^{m,r}_{s} (x)\mu (ds,dx)} }, \end{aligned}$$

where for each \((t,\lambda,y)\), \(\hat{g}^{m,r}(t,\lambda, y ):= g^{m+r}(t,\lambda, y + y^{m}_{t})- g^{m}(t,\lambda, y^{m}_{t} )\). We can check that, for each \(m, r \geq1\), \(\hat{g}^{m,r}\) satisfies (H1) and (H3),

$$\lim_{m \to\infty} \sup_{r \ge1} E \bigl[ { \bigl\vert \hat{\xi}^{m,r} - 0 \bigr\vert ^{2} } \bigr]=0\quad \mbox{and}\quad \lim_{m \to\infty} \sup_{r \ge1} E \biggl[ { \int_{0}^{T} { \bigl\vert \hat{g}^{m,r} (s,\lambda_{s},0) -\widetilde{g} (s,\lambda _{s},0) \bigr\vert ^{2} \,ds} } \biggr] = 0, $$

where for each \((t,\lambda,y)\), \(\widetilde{g} (t,\lambda ,y):\equiv0\). Thus, by Proposition 2, we can get

$$\lim_{m \to\infty} \sup_{r \ge1} E \biggl[ \mathop{ \sup}_{s \in[0,T]} \bigl\vert \hat{y}_{s}^{m,r} - 0 \bigr\vert ^{2} + \int_{0}^{T} { \int_{R} { \bigl\vert \hat{z}_{s}^{m,r}(x) - 0 \bigr\vert ^{2} \Lambda(ds,dx)} } \biggr] = 0. $$

Thus, the sequence \((y^{n}_{t},z^{n}_{t} )_{t\in[0,T]}\) converges to the solution \((y_{t},z_{t} )_{t\in[0,T]}\) in \(S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\) as \(n\to\infty\).

Step 3. We shall prove that, for each \(\xi \in L^{2} (\Omega ,\mathcal{F}_{T},P)\), if g satisfy (H1)-(H4), then there exists a unique solution to BSDE (1.1).

Clearly, we can find a mapping Ξ̂: \(S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\rightarrow S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\) by Step 2, where \(\hat{\Xi}((U_{\cdot},V_{\cdot}))= (y_{\cdot},z_{\cdot})\) with solution \((y_{\cdot},z_{\cdot}) \in S^{2}_{\mathbb{G}}\times L^{2}_{\mathbb{G}}\) satisfying BSDE (2.7). And it is a contractive mapping with the norm \(\Vert \cdot \Vert _{\beta}\) with suitable β by the same arguments as Step 1. Thus, it has a fixed point \((y_{t},z_{t} )_{t\in[0,T]}\) which is a solution of the BSDE (1.1). □