Nonparametric density estimation in compound Poisson processes using convolution power estimators

Abstract

Consider a compound Poisson process which is discretely observed with sampling interval $\Delta$ until exactly $n$ nonzero increments are obtained. The jump density and the intensity of the Poisson process are unknown. In this paper, we build and study parametric estimators of appropriate functions of the intensity, and an adaptive nonparametric estimator of the jump size density. The latter estimation method relies on nonparametric estimators of $m$th convolution powers density. The $L^2$-risk of the adaptive estimator achieves the optimal rate in the minimax sense over Sobolev balls. Numerical simulation results on various jump densities enlight the good performances of the proposed estimator.

Appendices

Appendix 1: Proofs

1.1 Proof of Proposition 21

The joint distribution of $(S_1, Z_1)$ is elementary using that the increments $X_{j\Delta }-X_{(j-1)\Delta }$ are i.i.d.. The process $(X_{j\Delta }^x= x+X_{j\Delta }, j\ge 1)$ is strong Markov. We denote by ${\mathbb {P}}_x$ its distribution on the canonical space ${\mathbb {R}}^{{\mathbb {N}}}$, denote by $(X_j, j\ge 0)$ the canonical process of ${\mathbb {R}}^{{\mathbb {N}}}$ and by ${\mathcal {F}}_j=\sigma (X_k, k\le j)$ the canonical filtration. Let $\theta :{\mathbb {R}}^{{\mathbb {N}}} \rightarrow {\mathbb {R}}^{{\mathbb {N}}}$ denote the shift operator. Consider the stopping times built on the canonical process $S_0=0$,

$$\begin{aligned} S_i= \inf \{j > S_{i-1}, X_{j}-X_{j-1}\ne 0\}, i\ge 1, \end{aligned}$$

and let

$$\begin{aligned} Z_i= X_{S_i }- X_{S_i-1 }. \end{aligned}$$

Because the $S_i$’s are built using the increments $(X_{j}-X_{j-1}, j\ge 1)$, their distributions under ${\mathbb {P}}_x$ are independent of the initial condition $x$. We have $S_i=S_{i-1}+ S_1\circ \theta _{S_{i-1}}$. The process $(X_{S_{i-1}+j} -X_{S_{i-1}}=(X_j-X_0)\circ \theta _{S_{i-1}}, j\ge 0)$ is independent of ${\mathcal {F}}_{S_{i-1}}$ and has distribution ${\mathbb {P}}_0$ and $Z_i= Z_1 \circ \theta _{S_{i-1}}$. Consequently,

$$\begin{aligned} {\mathbb {E}}_x(\varphi (S_i-S_{i-1})\psi (Z_i)|{\mathcal {F}}_{S_{i-1}})= {\mathbb {E}}_0(\varphi (S_1)\psi (Z_1)). \end{aligned}$$

By iterate conditioning, we get the result. $\square$

1.2 Proof of Proposition 22

Let us set

$$\begin{aligned} p(\Delta )=1-e^{-c\Delta }= \frac{e^{c\Delta }-1}{e^{c\Delta }}. \end{aligned}$$

An elementary computation yields:

$$\begin{aligned} c\Delta = \log { \left( \frac{x}{x-1} \right) } \quad \text{ with }\quad x:=x(\Delta )=\frac{1}{p(\Delta )}= 1+\frac{1}{e^{c\Delta }-1}>1, \end{aligned}$$

and

$$\begin{aligned} \frac{(e^{c\Delta } - 1)^m}{c\Delta }= H_m(x). \end{aligned}$$

As the standard maximum likelihood (and unbiased) estimator of $1/p(\Delta )$ computed from the sample $(S_i-S_{i-1}, i=1, \ldots ,n)$ is $S_n/n \ge 1$, we are tempted to estimate $H_m(x)$ by $H_m(S_n/n)$. This is not possible as $S_n/n$ may be equal to $1$. This is why we introduce a truncation. Set $u_0= \Delta / (e^{c_0\Delta /2}-1), \; u_1= \Delta /(e^{2c_1\Delta }-1), u= \Delta /(e^{c \Delta }-1)$. Note that

$$\begin{aligned} 1 +\frac{u_1}{\Delta }< x= 1+\frac{u}{\Delta }< 1 +\frac{u_0}{\Delta }. \end{aligned}$$

We have

$$\begin{aligned} \widehat{c_m(\Delta )}-c_m(\Delta )= H_m(S_n/n)1_{(1+ \frac{u_1}{\Delta } \le \frac{S_n}{n} \le 1+ \frac{u_0}{\Delta })} - H_m(x)= A_1+A_2 \end{aligned}$$

with

$$\begin{aligned} A_1= \left( H_m(S_n/n) - H_m(x)\right) \;1_{\left( 1+ \frac{u_1}{\Delta } \le \frac{S_n}{n} \le 1+ \frac{u_0}{\Delta }\right) }, \end{aligned}$$

and

$$\begin{aligned} A_2= - H_m(x) \left( 1_{\left( \frac{S_n}{n} <1+ \frac{u_1}{\Delta }\right) }+ 1_{\left( \frac{S_n}{n}>1+ \frac{u_0}{\Delta }\right) }\right) . \end{aligned}$$

Thus, on the set $(1+ \frac{u_1}{\Delta } \le \frac{S_n}{n} \le 1+ \frac{u_0}{\Delta })$,

$$\begin{aligned} (H_m(S_n/n) - H_m(x))^2 \le \left( \frac{S_n}{n} - x\right) ^2 \sup _{\xi \in [ 1+\frac{u_1}{\Delta }, 1+\frac{u_0}{\Delta }] }( H^{'}_m(\xi ))^2. \end{aligned}$$

As

$$\begin{aligned} H^{'}_m(\xi )= -\frac{m}{(\xi -1)^{m+1} \; \log {\frac{\xi }{\xi -1}}} +\frac{1}{\xi (\xi -1)^{m+1} \; \log ^2{\frac{\xi }{\xi -1}}}, \end{aligned}$$

we have, for $\xi \in [ 1+\frac{u_1}{\Delta }, 1+\frac{u_0}{\Delta }]$,

$$\begin{aligned} | H^{'}_m(\xi )|\le \frac{2\Delta ^{m}}{c_0 u_1^{m+1}} \left( m+ \frac{2}{u_1 c_0} \right) . \end{aligned}$$

Writing that $e^{2c_1\Delta } -1=2c_1\Delta e^{2sc_1\Delta }$ for $s\in (0,1)$, using that $2c_1\Delta \le \log (2)$, we get $1/u_1\le 4c_1$. As

$$\begin{aligned} {\mathbb {E}}\left( \frac{S_n}{n} - x\right) ^2 = \frac{1-p(\Delta )}{n p^2(\Delta ) }= \frac{e^{c\Delta }}{n(e^{c\Delta } -1)^2}, \end{aligned}$$

we get, using $e^{c\Delta }-1\ge c\Delta \ge c_0\Delta$:

$$\begin{aligned} {\mathbb {E}}A_1^2 \le C_m' \frac{\Delta ^{2(m-1)}}{n}, \text{ with } C_m'=\frac{4\sqrt{2}(4c_1)^{2(m+1)}}{c_0^4}\left( m+\frac{8c_1}{c_0}\right) ^2. \end{aligned}$$

Then, we have, setting $a_0= u_0 -u >0 , \;\; a_1= u-u_1>0$,

$$\begin{aligned}&{\mathbb {P}}\left( \frac{S_n}{n}<1+ \frac{u_1}{\Delta }\right) + {\mathbb {P}}\left( \frac{S_n}{n}>1+ \frac{u_0}{\Delta }\right) \\&\quad = {\mathbb {P}}\left( \frac{\Delta }{p(\Delta )}- \Delta \frac{S_n}{n}> a_1\right) + {\mathbb {P}}\left( \Delta \frac{S_n}{n}- \frac{\Delta }{p(\Delta )}>a_0\right) \\&\quad \le \left( \frac{1}{ a_1^2}+\frac{1}{ a_0^2}\right) \frac{\Delta ^2 \; e^{c\Delta }}{n(e^{c\Delta } -1)^2}. \end{aligned}$$

Thus, noting that $u_0-u\ge 1/(2c_1)$ and $u-u_1 \ge 1/(4\sqrt{2}c_0)$,

$$\begin{aligned} {\mathbb {E}}A_2^2 \le \left( \frac{1}{ a_1^2}+\frac{1}{ a_0^2}\right) \frac{(e^{c\Delta } -1)^{2(m-1)} e^{c\Delta }}{nc^2}\le C''_m\frac{\Delta ^{2(m-1)}}{n}, \end{aligned}$$

(23)

where

$$\begin{aligned} C''_m= 4\sqrt{2}\left[ 8c_0^2+ c_1^2\right] \frac{(4c_1)^{2(m-1)}}{c_0^2}. \end{aligned}$$

The proof is complete with $C_m=2(C'_m+ C''_m)$. $\square$

1.3 Proof of Proposition 41

Consider $f$ integrable with $\Vert f\Vert _1=\int |f|$ and square integrable such that $\int (1+x^2)^\alpha |f^*(x)|^2dx \le R$. Then

$$\begin{aligned}&\int (1+x^2)^\alpha |g^*(x)|^2 dx \\&\quad = \left( \frac{e^{-c \Delta }}{1-e^{-c \Delta }}\right) ^2 \sum _{m,k \ge 1} \frac{(c \Delta )^m}{m!}\frac{(c \Delta )^k}{k!} \int (1+x^2)^\alpha [f^*(x)]^{m}[f^*(-x)]^{k}dx \\&\quad \le \left( \frac{e^{-c \Delta }}{1-e^{-c \Delta }}\right) ^2 \sum _{m,k \ge 1} \frac{(c \Delta )^m}{m!}\frac{(c \Delta )^k}{k!} \Vert f\Vert _1^{m+k-2} \int (1+x^2)^\alpha |f^*(x)|^2dx\\&\quad \le R \left( \frac{e^{-c \Delta }}{1-e^{-c \Delta }}\right) ^2 \frac{1}{\Vert f\Vert _1^2} \left( \sum _{m \ge 1} \frac{(c \Delta )^m}{m!} \Vert f\Vert _1^{m}\right) ^2\\&\quad = R \left( \frac{e^{-c \Delta }}{1-e^{-c \Delta }}\frac{\exp (c\Delta \Vert f\Vert _1)-1}{\Vert f\Vert _1} \right) ^2 :=R(\Delta )<+\infty \end{aligned}$$

As $f$ is a density, $\Vert f\Vert _1=1$ and $R(\Delta )=R$. This implies the announced result for $g$. $\square$

1.4 Proof of Proposition 42

Recall that $f^*= \sum _{m\ge 1} ((-1)^{m+1}/m) c_m(\Delta )(g^*)^m$ (see (6)–(7)). Let $f_\ell$ be such that $f^{*}_\ell = f^* \; 1_{[-\pi \ell , \pi \ell ]}$ and $f_{K,\ell }$ be such that

$$\begin{aligned} f_{K,\ell }^{*}=1_{[-\pi \ell , \pi \ell ]} \sum _{m=1}^{K+1}\frac{(-1)^{m+1}}{m} c_m(\Delta ) (g^*)^m. \end{aligned}$$

Recall that $\widetilde{f_{K,\ell }}$ (see (11)) is such that

$$\begin{aligned} (\widetilde{f_{K,\ell }})^{*}=1_{[-\pi \ell , \pi \ell ]} \sum _{m=1}^{K+1}\frac{(-1)^{m+1}}{m} c_m(\Delta ) \widehat{(g^*)^m}. \end{aligned}$$

We distinguish the first term of this development from the other ones and set

$$\begin{aligned} \widetilde{f_{K,\ell }}= \widetilde{f_{K,\ell }}^{(1)} + \widetilde{{\mathcal {R}} f_{K,\ell }}, \text{ with } \widetilde{f_{K,\ell }}^{(1)} = c_1(\Delta ) \widehat{g^{\star 1}_{\ell }}=c_1(\Delta ) \widehat{g_{\ell }}. \end{aligned}$$

(24)

Analogously, with $g_\ell$ such that $g^{*}_\ell = g^* \; 1_{[-\pi \ell , \pi \ell ]}$,

$$\begin{aligned} f_{K,\ell }= f_{K,\ell }^{(1)}+ {\mathcal {R}}f_{K,\ell }, \text{ with } f_{K,\ell }^{(1)}= c_1(\Delta )g_{\ell } \end{aligned}$$

(25)

The following decomposition of the $L^2$-norm holds:

$$\begin{aligned} \left\| f-\widehat{f_{K,\ell }}\right\|&\le \left\| f-f_\ell \right\| +\left\| f_\ell -f_{K,\ell }\right\| + \left\| f_{K,\ell }^{(1)}-\widetilde{f_{K,\ell }}^{(1)}\right\| \\&+ \left\| {\mathcal {R}}f_{K,\ell }-\widetilde{{\mathcal {R}}f_{K,\ell }}\right\| +\left\| \widetilde{f_{K,\ell }}-\widehat{f_{K,\ell }}\right\| , \end{aligned}$$

which involves two bias terms and two stochastic error terms. The first bias term is the usual deconvolution bias term:

$$\begin{aligned} \Vert f-f_\ell \Vert ^2 = \frac{1}{2 \pi } \int \limits _{_{|t|\ge \pi \ell }}|f^*(t)|^2 dt \end{aligned}$$

Noting that

$$\begin{aligned} f_\ell ^*-f_{K,\ell }^*= 1_{[-\pi \ell , \pi \ell ]} \sum _{m=K+2}^{\infty }\frac{(-1)^{m+1}}{m} c_m(\Delta ) (g^*)^m, \end{aligned}$$

we get, using that $|g^*(t)|\le 1$ and $\Vert g\Vert \le \Vert f\Vert$ (see Proposition 41):

$$\begin{aligned} 2\pi \Vert f_\ell -f_{K,\ell }\Vert ^2&= \Vert f_\ell ^*-f_{K,\ell }^*\Vert ^2 = \int \limits _{-\pi \ell }^{\pi \ell } \left| \sum _{m=K+2}^{\infty }\frac{(-1)^{m+1}}{m} c_m(\Delta ) (g^*)^m(t)\right| ^2dt \nonumber \\&\le \int \limits _{-\pi \ell }^{\pi \ell } \left( \sum _{m \ge K+2}\frac{1}{m} c_m(\Delta ) |g^*(t)| \right) ^2 dt \nonumber \\&\le 2\pi \Vert g\Vert ^2 \left( \sum _{m \ge K+2}\frac{1}{m} c_m(\Delta )\right) ^2\nonumber \\&\le \frac{2\pi \Vert f\Vert ^2 }{(c\Delta )^2(K+2)^2}\left( \frac{(e^{c\Delta }-1)^{K+2}}{2-e^{c\Delta }}\right) ^2\nonumber \\&\le \frac{4\pi \Vert f\Vert ^2 (\sqrt{2}c\Delta )^{2K+2}}{( (K+2)^2(2-e^{2\Delta }))^2}\le 2\pi A_K \Delta ^{2K+2}, \end{aligned}$$

(26)

where in the last line, we have used $1/(2-e^{c\Delta })^2\le 1/(2-\sqrt{2})^2\le 3$ and $e^{c\Delta }-1 \le \sqrt{2} c\Delta$ and $A_K$ is given in (18).

To study the next term, we recall that, ${\mathbb {E}}(|\widehat{(g^*)(t)}-(g^*)(t)|^2)\le 1/n$. Then we get

$$\begin{aligned} 2\pi {\mathbb {E}}\left( \left\| f_{K,\ell }^{(1)}-\widetilde{f_{K,\ell }}^{(1)}\right\| ^2\right)&= \int \limits _{-\pi \ell }^{\pi \ell } {\mathbb {E}}\left( \left| c_1(\Delta ) [\widehat{(g^*)(t)}-(g^*)(t)]\right| ^2\right) dt \nonumber \\&\le \frac{2\pi \ell [c_1(\Delta )]^2}{n} \le \frac{4\pi \ell }{n} \end{aligned}$$

(27)

since $c_1(\Delta )\le \sqrt{2}$.

Hereafter, we use inequality (15) of Proposition 31.

$$\begin{aligned}&2\pi {\mathbb {E}}\left( \left\| {\mathcal {R}}f_{K,\ell }-\widetilde{{\mathcal {R}}f_{K,\ell }}\right\| ^2\right) \\&\quad = \int \limits _{-\pi \ell }^{\pi \ell } {\mathbb {E}}\left( \left| \sum _{m=2}^{K+1}\frac{(-1)^{m+1}}{m} c_m(\Delta ) [\widehat{(g^*)^m(t)}-(g^*)^m(t)]\right| ^2\right) dt \\&\quad \le \int \limits _{-\pi \ell }^{\pi \ell }(K+1) \sum _{m=2}^{K+1}\frac{1}{m^2} [c_m(\Delta )]^2 {\mathbb {E}}\left( |\widehat{(g^*)^m(t)}-(g^*)^m(t)|^2 \right) dt\\&\quad \le 2\pi K \sum _{m=2}^{K+1} \frac{{\mathcal {E}}_m}{m^2} [c_m(\Delta )]^2\left( \frac{\ell }{n^m} + \frac{ \Vert g\Vert ^2}{n}\right) \end{aligned}$$

This yields, since $c_m(\Delta )\le (\sqrt{2})^m(c\Delta )^{m-1}$ and $\ell /n \le 1$,

$$\begin{aligned} {\mathbb {E}}\left( \Vert {\mathcal {R}}f_{K,\ell }-\widetilde{{\mathcal {R}}f_{K,\ell }}\Vert ^2\right) \le \frac{D_K}{n} \end{aligned}$$

(28)

with

$$\begin{aligned} D_K= K \sum _{m=2}^{K+1} \frac{2^m c^{2(m-1)}{\mathcal {E}}_m}{m^2} \Delta ^{2(m-1)}\left( \frac{1}{n^{m-2}} + \Vert g\Vert ^2\right) \end{aligned}$$

For the last term, we use Proposition 22, with the fact that the estimators $\widehat{c_m(\Delta )}$ and $\widehat{(g^*)^m(t)}$ are independent, and write

$$\begin{aligned}&2\pi {\mathbb {E}}\left( \left\| \widetilde{f_{K,\ell }}-\widehat{f_{K,\ell }}\right\| ^2\right) \\ \nonumber&\quad = \int \limits _{-\pi \ell }^{\pi \ell } {\mathbb {E}}\left( \left| \sum _{m=1}^{K+1}\frac{(-1)^{m+1}}{m}\left( \widehat{c_m(\Delta )} - c_m(\Delta )\right) \widehat{(g^*)^m(t)}\right| ^2dt\right) \nonumber \\&\quad \le 2 \int \limits _{-\pi \ell }^{\pi \ell } {\mathbb {E}}\left( \left| \sum _{m=1}^{K+1}\frac{(-1)^{m+1}}{m}\left( \widehat{c_m(\Delta )} - c_m(\Delta )\right) [\widehat{(g^*)^m(t)}-(g^*)^m(t)]\right| ^2dt\right) \nonumber \\&\qquad +\,\, 2 \int \limits _{-\pi \ell }^{\pi \ell } {\mathbb {E}}\left( \left| \sum _{m=1}^{K+1}\frac{(-1)^{m+1}}{m}\left( \widehat{c_m(\Delta )} - c_m(\Delta )\right) (g^*)^m(t)\right| ^2dt\right) \nonumber \\&\quad \le 2(K+1) \sum _{m=1}^{K+1} \frac{1}{m^2} \left\{ {\mathbb {E}}\left[ \left( \widehat{c_m(\Delta )} \!-\! c_m(\Delta )\right) ^2\right] \!\int _{-\pi \ell }^{\pi \ell } {\mathbb {E}}\left[ |\widehat{(g^*)^m(t)}\!-\!(g^*)^m(t)|^2\right] dt\right. \nonumber \\&\qquad \left. +\,\,{\mathbb {E}}\left[ \left( \widehat{c_m(\Delta )} - c_m(\Delta )\right) ^2\right] \int _{-\pi \ell }^{\pi \ell } |g^*(t)|^{2m}dt\right\} \nonumber \\&\quad \le 2(K+1)\left\{ \frac{C_1}{n} \left( \frac{2\pi \ell }{n} + 2\pi \Vert g\Vert ^2\right) \right. \nonumber \\&\qquad \left. +\,\, \sum _{m=2}^{K+1} \frac{C_m\Delta ^{2(m-1)}}{m^2}\left[ \frac{{\mathcal {E}}_m}{n} \int _{-\pi \ell }^{\pi \ell }\left( \frac{1}{n^m}+ \frac{1}{n}|g^*(t)|^2\right) dt + \frac{1}{n} \Vert g^*\Vert ^2\right] \right\} . \end{aligned}$$

Therefore

$$\begin{aligned} 2\pi {\mathbb {E}}\left( \left\| \widetilde{f_{K,\ell }}-\widehat{f_{K,\ell }}\right\| ^2\right) \le \frac{2\pi E_K}{n} \end{aligned}$$

(29)

using that $\ell /n \le 1$ and

$$\begin{aligned} E_K=2(K+1) \left[ C_1(1+\Vert g\Vert ^2) + \sum _{m=2}^{K+1} \frac{C_m}{m^2}\Delta ^{2(m-1)}{\mathcal {E}}_m \left( \frac{1}{n^{m-1}}+2 \Vert g\Vert ^2\right) \right] . \end{aligned}$$

This ends the proof of the result with $D_K+E_K\le B_K$ and $\Vert g\Vert \le \Vert f\Vert$. $\square$

1.5 Proof of Theorem 41

Consider the contrast

$$\begin{aligned} \gamma _n(t)=\Vert t\Vert ^2 -2\langle t, \widehat{f_{K, L_n}}\rangle , \end{aligned}$$

and for $\ell =1, \ldots , L_n$, the increasing sequence of spaces

$$\begin{aligned} S_\ell =\{ t\in {\mathbb {L}}^2\cap {\mathbb {L}}^1({\mathbb {R}}), \; \mathrm{supp}(t^*)\subset [-\pi \ell , \pi \ell ]\}. \end{aligned}$$

Note that, for $\ell \le L_n$ and $t\in S_\ell$, $\gamma _n(t) =\Vert t\Vert ^2 -2\langle t, \widehat{f_{K,\ell }}\rangle ,$ and

$$\begin{aligned} \arg \min _{t\in S_\ell }\gamma _n(t) = \widehat{f_{K,\ell }}, \quad \text{ with } \quad \gamma _n(\widehat{f_{K,\ell }})=-\Vert \widehat{f_{K,\ell }}\Vert ^2. \end{aligned}$$

For $\ell ,\ell *\le L_n, \,s\in S_\ell$ and $t\in S_{\ell *}$, the following decomposition holds:

$$\begin{aligned} \gamma _n(t)-\gamma _n(s)= \Vert t-f\Vert ^2 -\Vert s-f\Vert ^2 -2\langle t-s, \widehat{f_{K,L_n}}-f\rangle \end{aligned}$$

and $\langle t-s, \widehat{f_{K,L_n}}-f\rangle = \langle t-s, \widehat{f_{K,L_n}}-f_{L_n}\rangle$. By definition of the estimator,

$$\begin{aligned} \gamma _n(\widehat{f_{K,\hat{\ell }}})+ \mathrm{pen}(\hat{\ell }) \le \gamma _n(\widehat{f_{K,\ell }}) +\mathrm{pen}( \ell ) \le \gamma _n(f_{\ell }) +\mathrm{pen}( \ell ). \end{aligned}$$

Thus, we obtain, $\forall \ell \in \{1, \dots , L_n\}$,

$$\begin{aligned} \Vert \widehat{f_{K,\hat{\ell }}}-f\Vert ^2&\le \Vert f_{\ell }-f\Vert ^2 + \mathrm{pen}(\ell )+ 2 \left\langle \widehat{f_{K,\hat{\ell }}} -f_{\ell }, \widehat{f_{K,L_n}} - f_{L_n}\right\rangle - \mathrm{pen}(\hat{\ell }) \nonumber \\&\le \Vert f_{\ell }-f\Vert ^2 + \mathrm{pen}(\ell )+ \frac{1}{4} \Vert \widehat{f_{K,\hat{\ell }}} -f_{\ell }\Vert ^2 \nonumber \\&+\,\, 4 \sup _{t\in S_{\ell }+S_{\hat{\ell }}, \Vert t\Vert =1} \left\langle t, \widehat{f_{K,L_n}} - f_{L_n}\right\rangle ^2 - \mathrm{pen}(\hat{\ell }) \end{aligned}$$

(30)

Then using

$$\begin{aligned} \frac{1}{4} \left\| \widehat{f_{K,\hat{\ell }}} -f_{\ell }\right\| ^2 \le \frac{1}{2} \left\| \widehat{f_{K,\hat{\ell }}} -f\right\| ^2 + \frac{1}{2} \Vert f -f_{\ell }\Vert ^2, \end{aligned}$$

(31)

and decompositions (24) and (25), we get

$$\begin{aligned} \left\langle t, \widehat{f_{K,L_n}} - f_{L_n}\right\rangle&= \left\langle t, \widehat{f_{K,L_n}} - \widetilde{f_{K,L_n}} \right\rangle + \left\langle t, \widetilde{f_{K,L_n}}^{(1)} - f_{K, L_n}^{(1)}\right\rangle \nonumber \\&+\,\, \left\langle t, {\mathcal {R}}\widetilde{f_{K,L_n}} - {\mathcal {R}}f_{K, L_n}\right\rangle + \left\langle t, f_{K,L_n}-f_{L_n}\right\rangle . \end{aligned}$$

By the Cauchy-Schwarz Inequality and for $\Vert t\Vert =1$, we have

$$\begin{aligned} \left\langle t, \widehat{f_{K,L_n}} - f_{L_n}\right\rangle ^2&\le 4 \left\| \widehat{f_{K,L_n}} - \widetilde{f_{K,L_n}}\right\| ^2 + 4 \left\| {\mathcal {R}}\widetilde{f_{K,L_n}} - {\mathcal {R}}f_{K, L_n}\right\| ^2\nonumber \\&+\,\, 4 \Vert f_{K,L_n} - f_{L_n}\Vert ^2 + 4 \left\langle t, \widetilde{f_{K,L_n}}^{(1)}- f_{K,L_n}^{(1)}\right\rangle ^2. \end{aligned}$$

(32)

Thus, inserting (31) and (32) in (30) yields

$$\begin{aligned} \frac{1}{2} \left\| \widehat{f_{K,\hat{\ell }}}-f\right\| ^2&\le \frac{3}{2} \Vert f_{\ell }-f\Vert ^2 + 16 \Vert f_{K,L_n} - f_{L_n}\Vert ^2\nonumber \\&+\,\, 16 \left\| \widehat{f_{K,L_n}} - \widetilde{f_{K,L_n}}\right\| ^2+ 16\left\| {\mathcal {R}}\widetilde{f_{K,L_n}} - {\mathcal {R}}f_{K, L_n}\right\| ^2 + \mathrm{pen}(\ell ) \nonumber \\&+\,\, 16 \sup _{t\in S_{\ell \vee \hat{\ell }}, \Vert t\Vert =1} \left\langle t, \widetilde{f_{K,L_n}}^{(1)} - f_{K,L_n}^{(1)}\right\rangle ^2 - \mathrm{pen}(\hat{\ell }) \end{aligned}$$

Here, the bounds of Proposition 42 can be applied. Indeed (26), (28) and (29) are uniform with respect to $\ell$ and imply

$$\begin{aligned} \Vert f_{K,L_n} - f_{L_n}\Vert ^2\le A_K\Delta ^{2(K+2)},\; {\mathbb {E}}\left( \Vert {\mathcal {R}}\widetilde{f_{K,L_n}} - {\mathcal {R}}f_{K, L_n}\Vert ^2\right) \le D_K/n, \end{aligned}$$

and

$$\begin{aligned} {\mathbb {E}}\left( \left\| \widehat{f_{K,L_n}} - \widetilde{f_{K,L_n}}\right\| ^2\right) \le E_K/n. \end{aligned}$$

Below, we prove using the Talagrand Inequality that

$$\begin{aligned} {\mathbb {E}}\left( \sup _{t\in S_{\ell \vee \hat{\ell }}, \Vert t\Vert =1} \langle t, \widetilde{f_{K,L_n}}^{(1)} - f_{K,L_n}^{(1)}\rangle ^2 -p(\ell ,\hat{\ell })\right) _+\le \frac{C'}{n}, \end{aligned}$$

(33)

where $p(\ell ,\ell ')=8\ell \vee \ell '/n$ and $16 p(\ell , \ell ')\le \mathrm{pen}(\ell )+ \mathrm{pen}(\ell ')$ as soon as $\kappa \ge \kappa _0=16\times 8$.

Thus, we get ${\mathbb {E}}(16 p(\ell , \hat{\ell })-\mathrm{pen}(\hat{\ell }))\le \mathrm{pen}(\ell )$ and

$$\begin{aligned} {\mathbb {E}}\left( \left\| \widehat{f_{K,\hat{\ell }}}-f\right\| ^2\right) \le 4\Vert f-f_\ell \Vert ^2 + 4\mathrm{pen}(\ell ) + 32 A_K\Delta ^{2(K+2)} + 32\frac{B_K}{n} + \frac{32 C'}{n}. \end{aligned}$$

Proof of 33

We consider $t\in S_{\ell ^*}$ for $\ell ^*=\ell \vee \ell '$ with $\ell , \ell ' \le L_n$ and (see (24) and (25))

$$\begin{aligned} \nu _n(t)= c_1(\Delta ) \langle t, {\hat{g}}_{L_n} -g_{L_n}\rangle = \frac{1}{n}\sum _{k=1}^n (\psi _t(Z_k)-{\mathbb {E}}(\psi _t(Z_k))) \end{aligned}$$

where

$$\begin{aligned} \psi _t(z)= \frac{c_1(\Delta )}{2\pi }\int t^*(u)e^{iuz}du=c_1(\Delta ) t(z). \end{aligned}$$

We apply the Talagrand Inequality recalled in Sect. 8, and to this aim, we compute the quantities $M, H, v$. First

$$\begin{aligned} \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1}\sup _z |\psi _t(z)|\le \frac{c_1(\Delta )}{2\pi }\sqrt{2\pi \ell ^*} \times \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1}\Vert t^*\Vert =c_1(\Delta ) \sqrt{\ell ^*}:=M. \end{aligned}$$

The density of $Z_1$ is $g$ which satisfies

$$\begin{aligned} \Vert g\Vert _{\infty } \le \sum _{m\ge 1} \frac{1}{e^{c \Delta }-1} \frac{(c \Delta )^m}{m!} \Vert f^{\star \;m}\Vert _{\infty }\le \Vert f\Vert _{\infty }. \end{aligned}$$

Therefore,

$$\begin{aligned} \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1}\mathrm{Var}(\psi _t(Z_1))\le c_1^2(\Delta ) \times \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1}{\mathbb {E}}(t^2(Z_1))\le c_1^2(\Delta )\Vert f\Vert _\infty :=v. \end{aligned}$$

Lastly, using the bound in (27) and the fact that for $t\in S_{\ell ^*}$,

$$\begin{aligned} \left\langle t, \widetilde{f_{K,L_n}}^{(1)} - f_{K,L_n}^{(1)}\right\rangle =\left\langle t, \widetilde{f_{K,\ell ^*}}^{(1)} - f_{K,\ell ^*}^{(1)}\right\rangle , \end{aligned}$$

we get

$$\begin{aligned} {\mathbb {E}}\left( \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1}\nu _n^2(t)\right)&= {\mathbb {E}}\left( \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1} \left\langle t, \widetilde{f_{K,\ell ^*}}^{(1)} - f_{K,\ell ^*}^{(1)}\right\rangle ^2\right) \\&\le {\mathbb {E}}\left( \left\| \widetilde{f_{K,\ell ^*}}^{(1)} - f_{K,\ell ^*}^{(1)}\right\| ^2 \right) \ \le \frac{2\ell ^*}{n}:=H^2. \end{aligned}$$

Therefore, Lemma 81 yields with $\epsilon ^2=1/2$,

$$\begin{aligned} {\mathbb {E}}\left( \sup _{t\in S_{\ell ^*}, \Vert t\Vert =1}\nu _n^2(t)-4H^2\right) \le \frac{A_1}{n}\left( e^{-A_2\ell ^*} + e^{-A_3 \sqrt{n}}\right) \end{aligned}$$

for constants $A_1, A_2, A_3$ depending on $c_1(\Delta )$ and $\Vert f\Vert _\infty$. Now since

$$\begin{aligned} \sum _{\ell '=1}^{L_n} e^{-A_2\ell \vee \ell '}=\ell e^{-A_2\ell } + \sum _{\ell <\ell '\le L_n}e^{-A_2\ell '} \end{aligned}$$

is bounded by say $B_2$ and $L_n e^{-A_3\sqrt{n}}$ is bounded by $B_3$, we get

$$\begin{aligned} {\mathbb {E}}\left( \sup _{t\in S_{\ell \vee \hat{\ell }}, \Vert t\Vert =1}\nu _n^2(t)-8\frac{\ell \vee \hat{\ell }}{n} \right) \le \sum _{\ell '} {\mathbb {E}}\left( \sup _{t\in S_{\ell \vee \ell '}, \Vert t\Vert =1}\nu _n^2(t)-4H^2\right) \le \frac{B_4}{n}. \end{aligned}$$

This ends the proof of (33) and thus of Theorem 41. $\square$

Appendix 2

The result below follows from the Talagrand concentration inequality given in Klein and Rio (2005) and arguments in Birgé and Massart (1998) (see the proof of their Corollary 2 page 354).

Lemma 81

(Talagrand Inequality) Let $Y_1, \dots , Y_n$ be independent random variables, let $\nu _{n,Y}(f)=(1/n)\sum _{i=1}^n [f(Y_i)-{\mathbb {E}}(f(Y_i))]$ and let ${\mathcal {F}}$ be a countable class of uniformly bounded measurable functions. Then for $\epsilon ^2>0$

$$\begin{aligned}&{\mathbb {E}}\Big [\sup _{f\in {\mathcal {F}}}|\nu _{n,Y}(f)|^2-2(1+2\epsilon ^2)H^2\Big ]_+ \\&\quad \le \frac{4}{K_1}\left( \frac{v}{n} e^{-K_1\epsilon ^2 \frac{nH^2}{v}} + \frac{98M^2}{K_1n^2C^2(\epsilon ^2)} e^{-\frac{2K_1 C(\epsilon ^2)\epsilon }{7\sqrt{2}}\frac{nH}{M}}\right) , \end{aligned}$$

with $C(\epsilon ^2)=\sqrt{1+\epsilon ^2}-1,\, K_1=1/6$, and

$$\begin{aligned} \sup _{f\in {\mathcal {F}}}\Vert f\Vert _{\infty }\le M, \;\;\;\; {\mathbb {E}}\Big [\sup _{f\in {\mathcal {F}}}|\nu _{n,Y}(f)|\Big ]\le H, \; \sup _{f\in {\mathcal {F}}}\frac{1}{n}\sum _{k=1}^n\mathrm{Var}(f(Y_k)) \le v. \end{aligned}$$

By standard density arguments, this result can be extended to the case where ${\mathcal {F}}$ is a unit ball of a linear normed space, after checking that $f\mapsto \nu _n(f)$ is continuous and ${\mathcal {F}}$ contains a countable dense family.