Nonparametric spectral density estimation under local differential privacy

Abstract

We consider nonparametric estimation of the spectral density of a stationary time series under local differential privacy. In this framework only an appropriately anonymised version of a finite snippet from the time series can be observed and used for inference. The anonymisation procedure can be chosen in advance among all mechanisms satisfying the condition of local differential privacy, and we propose truncation followed by Laplace perturbation for this purpose. Afterwards, the anonymised time series snippet is used to define a surrogate of the classical periodogram and our estimator is obtained by projection of this privatised periodogram to a model given by a finite dimensional subspace of $L^2([-\pi ,\pi ])$. This estimator attains nearly the same convergence rate as in the case where the original time series can be observed. However, a reduction of the effective sample size in contrast to the non-privacy framework is shown to be unavoidable. We also consider adaptive estimation and suggest to select an estimator from a set of candidate estimators by means of a penalised contrast criterion. We derive an oracle inequality which shows that the adaptive estimator attains nearly the same rate of convergence as the best estimator from the candidate set. Concentration inequalities for quadratic forms in terms of sub-exponential random variables, which have been recently derived in Götze et al. (Electron J Probab 26:1–22, 2021), turn out to be essential for our proof. Finally, we illustrate our findings in a small simulation study.

Appendices

Appendix A Proofs of Theorem 3.2 (upper bound for a fixed Fourier model ${\mathfrak {m}}$)

The following result (which is valid without any distributional assumptions on the marginal distribution of the stationary time series X) has been proven in Comte (2001).

Proposition A.1

(Comte 2001, Proposition 1) Let X be a stationary sequence with autocovariance function satisfying Assumption 1.1. Then

$$\begin{aligned} \int _{-\pi }^{\pi } (f(\omega ) - {{\textbf {E}}}I_n(\omega ))^2 \text {d}\omega \le \frac{M_1 + 39M^2}{2\pi n} =:\frac{M_2}{n}. \end{aligned}$$

Note that Assumption 1.1 holds also for the time series $Z'$ instead of X with $M_1= M_1^{Z'} = M_1^X$ and $M=M^X$ replaced with $M=M^{Z'} = M^X + 8\tau _n^2/\alpha ^2$.

Let us now consider the event

$$\begin{aligned} A :=\bigcap _{i=1}^n \{ {\widetilde{X}}_i = X_i \}, \end{aligned}$$

and its complement $A^\complement = \bigcup _{i=1}^n \{{\widetilde{X}}_i \ne X_i \}$. Denoting with $f_{\mathfrak {m}}$ the projection of f on the space $S_{\mathfrak {m}}$, we have the decomposition

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f \Vert ^2&= \Vert f_{\mathfrak {m}}- f \Vert ^2 + {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2\nonumber \\&= \Vert f_{\mathfrak {m}}- f \Vert ^2 + {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_A + {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A^\complement }. \end{aligned}$$

(A1)

The first term on the right-hand side of (A1) is already in the form of the statement of the theorem, and we have to study the terms including ${{\textbf {1}}}_A$ and ${{\textbf {1}}}_{A^\complement }$ only.

$\underline{\hbox {Bound for} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_A}$: By the very definition we have $X_i={\widetilde{X}}_i$ on A, and hence $Z_i = Z'_i = X_i + \xi _i$ for $\xi _i \sim {\mathcal {L}}(2\tau _n/\alpha )$. Hence, on the event A the identity

$$\begin{aligned} I_n^Z(\omega ) = I_n^{Z'}(\omega ) \end{aligned}$$

holds (with $I_n^{Z'}$ defined exactly as $I_n^Z$ with Z replaced with $Z'$), and we have

$$\begin{aligned} \Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_A&= \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle f - {\widehat{I}}_n, \phi _i \rangle |^2 {{\textbf {1}}}_A\\&= \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle f - (I_n^Z(\omega ) - 4\tau _n^2/(\pi \alpha ^2)), \phi _i \rangle |^2 {{\textbf {1}}}_A\\&= \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle f - (I_n^{Z'}(\omega ) - 4\tau _n^2/(\pi \alpha ^2)), \phi _i \rangle |^2 {{\textbf {1}}}_A\\&\le \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle f - ( I_n^{Z'}(\omega ) - 4\tau _n^2/(\pi \alpha ^2)) ,\phi _i \rangle |^2\\&= \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle f^{Z'} - I_n^{Z'}, \phi _i \rangle |^2. \end{aligned}$$

where the last identity is established in (10). Now,

$$\begin{aligned} \Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_A&\le 2 \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} (|\langle f^{Z'} - {{\textbf {E}}}I_n^{Z'},\phi _i \rangle |^2 + |\langle {{\textbf {E}}}I_n^{Z'} - I_n^{Z'}, \phi _i \rangle |^2 )\\&= 2 \Vert (f^{Z'} - {{\textbf {E}}}I_n^{Z'})_{\mathfrak {m}}\Vert ^2 + 2 \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{Z'} - I_n^{Z'}, \phi _i \rangle |^2\\&\le 2 \Vert f^{Z'} - {{\textbf {E}}}I_n^{Z'} \Vert ^2 + 2 \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{Z'} - I_n^{Z'}, \phi _i \rangle |^2. \end{aligned}$$

In order to bound the term $\Vert f^{Z'} - {{\textbf {E}}}I_n^{Z'} \Vert ^2$, we use Proposition A.1 in order to obtain:

$$\begin{aligned} \Vert f^{Z'} - {{\textbf {E}}}I_n^{Z'} \Vert ^2 = \int _{-\pi }^{\pi } (f^{Z'} - {{\textbf {E}}}I_n^{Z'}(\omega ))^2 \text {d}\omega&\le \max \left( \frac{M_1}{\pi n}, \frac{39 (M^{Z'})^2}{\pi n} \right) \nonumber \\&\lesssim \max \left( \frac{1}{n}, \frac{\tau _n^4}{n \alpha ^4} \right) . \end{aligned}$$

(A2)

Let us now consider the expression ${{\textbf {E}}}\sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{Z'} - I_n^{Z'}, \phi _i \rangle |^2$. We write

$$\begin{aligned} I_n^{Z'}(\omega ) = I_n^X + I_n^\xi + {\widetilde{I}}_n, \end{aligned}$$

where

$$\begin{aligned} I_n^X(\omega )&= \frac{1}{2\pi n} \left|\sum _{t=1}^n (X_t - {\bar{X}}_n) e^{-\text {i}t \omega } \right|^2,\\ I_n^\xi (\omega )&= \frac{1}{2\pi n} \left|\sum _{t=1}^n (\xi _t - {\overline{\xi }}_n) e^{-\text {i}t \omega } \right|^2, \qquad \text {and}\\ {\widetilde{I}}_n(\omega )&= \frac{1}{2\pi n} \left( \sum _{t=1}^n (X_t - {\bar{X}}_n) e^{-\text {i}t \omega } \right) \left( \sum _{t=1}^n (\xi _t - {\bar{\xi }}_n) e^{\text {i}t \omega } \right) \\&\quad + \frac{1}{2\pi n} \left( \sum _{t=1}^n (X_t - {\bar{X}}_n) e^{\text {i}t \omega } \right) \left( \sum _{t=1}^n (\xi _t - {\bar{\xi }}_n) e^{-\text {i}t \omega } \right) . \end{aligned}$$

Noting that ${{\textbf {E}}}{\widetilde{I}}_n = 0$ we obtain

$$\begin{aligned} \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{Z'} - I_n^{Z'}, \phi _i \rangle |^2&\lesssim \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{X} - I_n^{X}, \phi _i \rangle |^2 + \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{\xi } - I_n^{\xi }, \phi _i \rangle |^2 + \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {\widetilde{I}}_n, \phi _i \rangle |^2. \end{aligned}$$

The first term appears already in the non-private case. In order to bound it, note that we can write the empirical coefficient ${\widehat{a}}_i^X$ also in the form

$$\begin{aligned} {\widehat{a}}_i^X = \frac{1}{n} \sum _{j=-\lfloor \frac{n-1}{2} \rfloor }^{\lfloor \frac{n}{2} \rfloor } I_n^X(\omega _j) \bar{\phi }_i(\omega _j) \end{aligned}$$

where $\omega _j = \frac{2\pi j}{n}$, $j=-\lfloor \frac{n-1}{2} \rfloor ,\ldots ,\lfloor \frac{n}{2} \rfloor$ are the Fourier frequencies (at this point, the fact that we restrict ourselves to the complex Fourier basis is used). Hence,

$$\begin{aligned} {{\textbf {E}}}\sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{X} - I_n^{X}, \phi _i \rangle |^2&= \frac{1}{n^2} \sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} \sum _{j=-\lfloor \frac{n-1}{2} \rfloor }^{\lfloor \frac{n}{2} \rfloor } \sum _{k=-\lfloor \frac{n-1}{2} \rfloor }^{\lfloor \frac{n}{2} \rfloor } {\text {Cov}}(I_n(\omega _j),I_n(\omega _k))\bar{\phi }_i(\omega _j) \phi _i(\omega _k) \\&\lesssim \frac{D_{\mathfrak {m}}}{n} \end{aligned}$$

where we used Theorem 10.3.2 in Brockwell and Davis (1991) (this results also relies on the specific linear process structure assumed). The same reasoning can be applied to the term incorporating $I_n^\xi$ (indeed, one can even directly use Proposition 10.3.2, (ii) from Brockwell and Davis (1991) instead of Theorem 10.3.2, (ii)):

$$\begin{aligned} {{\textbf {E}}}\sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{\xi } - I_n^{\xi }, \phi _i \rangle |^2&\lesssim \frac{D_{\mathfrak {m}}\tau _n^4}{n\alpha ^4}. \end{aligned}$$

Finally, a direct calculation shows that

$$\begin{aligned} {{\textbf {E}}}\sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {\widetilde{I}}_n, \phi _i \rangle |^2 \lesssim \frac{D_{\mathfrak {m}}\tau _n^2}{n\alpha ^2}. \end{aligned}$$

Putting together the obtained estimates, we get

$$\begin{aligned} {{\textbf {E}}}\sum _{i \in {\mathcal {I}}_{\mathfrak {m}}} |\langle {{\textbf {E}}}I_n^{Z'} - I_n^{Z'}, \phi _i \rangle |^2 \le C \left[ \frac{D_{\mathfrak {m}}}{n} \vee \frac{D_{\mathfrak {m}}\tau _n^4}{n\alpha ^4} \right] . \end{aligned}$$

(A3)

Combining (A2) and (A3), we obtain

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_A \lesssim \left[ \frac{D_{\mathfrak {m}}}{n} \vee \frac{D_{\mathfrak {m}}\tau _n^4}{n\alpha ^4} \right] . \end{aligned}$$

$\underline{\hbox {Bound for }{{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A^\complement }}$: By the Cauchy–Schwarz inequality, we have

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A^\complement }&\le ( {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^4 )^{1/2} \cdot ( {{\textbf {P}}}(A^\complement ) )^{1/2}, \end{aligned}$$

(A4)

and we analyse the two factors on the right-hand side separately. First,

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^4&\le {{\textbf {E}}}\Vert f + \frac{4\tau _n}{\pi \alpha ^2} - I_n^Z \Vert ^4. \end{aligned}$$

Now,

$$\begin{aligned} \Vert f + \frac{4\tau _n}{\pi \alpha ^2} - I_n^Z \Vert ^4&\le 4 \pi ^2 \Vert f + \frac{4\tau _n}{\pi \alpha ^2} - I_n^Z \Vert _\infty ^4\\&\le 32\pi ^2 \Vert f + \frac{4\tau _n}{\pi \alpha ^2} \Vert _\infty ^4 + 32 \pi ^2 \Vert I_n^Z \Vert _\infty ^4\\&\le 32\pi ^2 \left( \frac{1}{2\pi } \sum _{k \in {\mathbb {Z}}} |\gamma (k) |+ \frac{4\tau _n}{\pi \alpha ^2} \right) ^4 + 32\pi ^2 \Vert I_n^Z \Vert _\infty ^4. \end{aligned}$$

Furthermore, using $|{\widetilde{X}}_t |\le \tau _n$,

$$\begin{aligned} {{\textbf {E}}}[ \Vert I_n^Z \Vert _\infty ^4 ]&= {{\textbf {E}}}\left[ \frac{1}{(2\pi n)^4} \Vert \sum _{t=1}^n ({\widetilde{X}}_t - \bar{{\widetilde{X}}}_n) e^{-\text {i}t \bullet } + \sum _{t=1}^n (\xi _t - {\overline{\xi }}_n) e^{-\text {i}t \bullet } \Vert _\infty ^8 \right] \\&\le \frac{2^7}{(2\pi n)^4} \cdot {{\textbf {E}}}\left[ \Vert \sum _{t=1}^n ({\widetilde{X}}_t - \bar{{\widetilde{X}}}_n) e^{-\text {i}t \bullet } \Vert _\infty ^8 \right] + \frac{2^7}{(2\pi n)^4} {{\textbf {E}}}\left[ \Vert \sum _{t=1}^n (\xi _t - {\overline{\xi }}_n) e^{-\text {i}t \bullet } \Vert _\infty ^8 \right] \\&\le \frac{2^7}{(2\pi n)^4} \cdot {{\textbf {E}}}\left[ \left( \sum _{t=1}^{n} |{\widetilde{X}}_t - \bar{{\widetilde{X}}}_n |\right) ^8 \right] + \frac{2^7}{(2\pi n)^4} \cdot {{\textbf {E}}}\left[ \left( \sum _{t=1}^n |\xi _t |+ n |{\overline{\xi }}_n |\right) ^8 \right] \\&\le \frac{2^{15} n^8\tau _n^8}{(2\pi n)^4} + \frac{2^7}{(2\pi n)^4} \cdot {{\textbf {E}}}\left[ \left( 2 \sum _{t=1}^n |\xi _t |\right) ^8 \right] \\&\le \frac{2^{11} n^4 \tau _n^8}{\pi ^4} + \frac{2^{15}}{(2\pi n)^4} \cdot {{\textbf {E}}}\left[ \left( \sum _{t=1}^n |\xi _t |\right) ^8 \right] \\&= \frac{2^{11} n^4 \tau _n^8}{\pi ^4} + \frac{2^{19} \tau _n^8 (n+7) \cdot (n+6) \cdot \ldots \cdot n}{\pi ^4 n^4 \alpha ^8}\\&\lesssim \frac{n^4 \tau _n^8}{1 \wedge \alpha ^8} \end{aligned}$$

where we have also used that $\sum _{t=1}^{n} |\xi _t |\sim \Gamma (n,\alpha /(2\tau _n))$ together with the fact that the kth moment of a $\Gamma (n,\beta )$-distributed random variable is equal to $(n + k - 1)\cdot \ldots \cdots n/\beta ^k$. Thus,

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^4 \lesssim \left[ \left( \sum _{k \in {\mathbb {Z}}} |\gamma (k)|+ \frac{\tau _n^2}{\alpha ^2}\right) ^4 + \frac{n^4 \tau _n^8}{1 \wedge \alpha ^8} \right] . \end{aligned}$$

(A5)

Putting this bound into (A4), we note that it is sufficient to show that ${{\textbf {P}}}(A^\complement ) \lesssim n^{-6}$ to obtain the claimed bound. We will derive such a bound in the following using Assumption 3.1. For n sufficiently large (namely $\tau _n > 2\mu$ has to hold which is equivalent to $n \ge \exp ((\mu ^2/14)^{1/(1+\delta )})$) we obtain from (13) that

$$\begin{aligned} {{\textbf {P}}}(A^\complement )&\le \sum _{i=1}^{n} {{\textbf {P}}}(X_i \ne {\widetilde{X}}_i)\\&= \sum _{i=1}^{n} {{\textbf {P}}}(|X_i |> \tau _n)\\&\le \sum _{i=1}^{n} {{\textbf {P}}}(|X_i - \mu |> \tau _n/2)\\&\le 2\sum _{i=1}^{n} e^{-\frac{\tau _n^2}{8 \nu }}\\&\le 2n e^{-\frac{\tau _n^2}{8 \nu }}. \end{aligned}$$

With $\tau _n^2 = 56 \log ^{1+\delta }(n)$ (our definition), $\log ^\delta (n) \ge \nu$ for n large enough and for such n we obtain ${{\textbf {P}}}(A^\complement ) \lesssim n^{-6}$. Combining this estimate with (A4) and (A5), we obtain the bound

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A^\complement } \lesssim \frac{\tau _n^4}{n} \vee \frac{\tau _n^4}{n\alpha ^4}. \end{aligned}$$

Putting the obtained bounds for the terms ${{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A}$ and ${{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A^\complement }$ into the right-hand side of (A1) yields the claim of the theorem.

Proofs of Sect. 4

This section is devoted to the proof of the minimax lower bound showing an inevitable reduction of the effective sample size from n to $n\alpha ^2$ for $\alpha \le 1$. The proof of Theorem 4.1 is given in Subsection B.2. Before we prepare the ground for this proof by deriving a data processing inequality connecting the Kullback–Leibler divergence of the original snippet $X_{1:n}$ and its privatised version $Z_{1:n}$ obtained via an arbitrary privacy mechanism Q. This data processing inequality (inequality (B9) below) complements Corollary 3 from Duchi et al. (2018) where only the case of independent raw data $X_1,\ldots ,X_n$ was dealt with.

1.1 Preliminaries

In the following ${{\textbf {P}}}^{Z_{1:n}}$ (${{\textbf {Q}}}^{Z_{1:n}}$) denotes the distribution of the privatised sample $Z_{1:n}$ when ${{\textbf {P}}}^{X_{1:n}}$ (${{\textbf {Q}}}^{X_{1:n}}$) is the distribution of the raw data $X_{1:n}$ and Q the privacy mechanism that is arbitrary but fixed throughout. First, we apply the chain rule for the Kullback–Leibler divergence, the standard data-processing inequality (12) from Duchi et al. (2018) under local differential privacy, the first Pinsker inequality, and the definition of the conditional Kullback–Leibler divergence in order to obtain

$$\begin{aligned}&\text {KL}({{\textbf {P}}}^{Z_{1:n}}\parallel {{\textbf {Q}}}^{Z_{1:n}})\end{aligned}$$

(B6)

$$\begin{aligned}&\quad = \sum _{i=1}^n \int _{{\mathcal {Z}}^{i-1}} \text {KL}( {{\textbf {P}}}^{Z_i\mid Z_{1:i-1}=z_{1:i-1}}\parallel {{\textbf {Q}}}^{Z_i\mid Z_{1:i-1}=z_{1:i-1}}) {{\textbf {P}}}^{Z_{1:i-1}}(\text {d}z_{1:i-1})\nonumber \\&\quad \le 4(e^\alpha - 1)^2 \sum _{i=1}^n \int _{{\mathcal {Z}}^{i-1}} \text {TV}^2 ({{\textbf {P}}}^{X_i\mid Z_{1:i-1}=z_{1:i-1}}\parallel {{\textbf {Q}}}^{X_i\mid Z_{1:i-1}=z_{1:i-1}}) {{\textbf {P}}}^{Z_{1:i-1}}(\text {d}z_{1:i-1})\nonumber \\&\quad \le 2(e^\alpha - 1)^2 \sum _{i=1}^n \int _{{\mathcal {Z}}^{i-1}} \text {KL}({{\textbf {P}}}^{X_i\mid Z_{1:i-1}=z_{1:i-1}}\parallel {{\textbf {Q}}}^{X_i\mid Z_{1:i-1}=z_{1:i-1}}) {{\textbf {P}}}^{Z_{1:i-1}}(\text {d}z_{1:i-1}) \nonumber \\&\quad = 2(e^\alpha - 1)^2 \sum _{i=1}^n \text {KL}( {{\textbf {P}}}^{X_i\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i\mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}). \end{aligned}$$

(B7)

We now would like to show that

$$\begin{aligned} \text {KL}&( {{\textbf {P}}}^{X_i\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i\mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}) \nonumber \\&\le \text {KL}( {{\textbf {P}}}^{X_i\mid X_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i\mid X_{1:i-1}} \mid {{\textbf {P}}}^{X_{1:i-1}}). \end{aligned}$$

(B8)

In order to derive (B8), note that

$$\begin{aligned} \text {KL}({{\textbf {P}}}^{X_i\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i \mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}})&= \text {KL}({{\textbf {P}}}^{X_i,Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i,Z_{1:i-1}} )\\&\quad -\, \text {KL}({{\textbf {P}}}^{Z_{1:i-1}} \parallel {{\textbf {Q}}}^{Z_{1:i-1}}) \end{aligned}$$

and

$$\begin{aligned} \text {KL}({{\textbf {P}}}^{X_i\mid X_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i\mid X_{1:i-1}} \mid {{\textbf {P}}}^{X_{1:i-1}})&= \text {KL}({{\textbf {P}}}^{X_{1:i}} \parallel {{\textbf {Q}}}^{X_{1:i}}) - \text {KL}({{\textbf {P}}}^{X_{1:i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}}). \end{aligned}$$

Moreover, since the privacy mechanism Q is fixed,

$$\begin{aligned}&\text {KL}({{\textbf {P}}}^{X_{1:i-1}}\parallel {{\textbf {Q}}}^{X_{1:i-1}}) = \text {KL}({{\textbf {P}}}^{X_{1:i-1},Z_{1:i-1}}\parallel {{\textbf {Q}}}^{X_{1:i-1}, Z_{1:i-1}})\\&\quad = \text {KL}({{\textbf {P}}}^{Z_{1:i-1}} \parallel {{\textbf {Q}}}^{Z_{1:i-1}}) + \text {KL}({{\textbf {P}}}^{X_{1:i-1}\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}\mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}) \end{aligned}$$

and hence

$$\begin{aligned}&\text {KL}({{\textbf {P}}}^{X_i\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i \mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}) = \text {KL}({{\textbf {P}}}^{X_i,Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i,Z_{1:i-1}} ) \\&\hspace{1em}- \text {KL}({{\textbf {P}}}^{X_{1:i-1}}\parallel {{\textbf {Q}}}^{X_{1:i-1}}) + \text {KL}({{\textbf {P}}}^{X_{1:i-1}\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}\mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}). \end{aligned}$$

Consequently, it remains to show that

$$\begin{aligned}\scriptstyle \text {KL}({{\textbf {P}}}^{X_{1:i}}\parallel {{\textbf {Q}}}^{X_{1:i}}) \ge \text {KL}({{\textbf {P}}}^{X_i,Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i,Z_{1:i-1}} ) + \text {KL}({{\textbf {P}}}^{X_{1:i-1}\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}\mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}). \end{aligned}$$

Because

$$\begin{aligned} \text {KL}({{\textbf {P}}}^{X_{1:i}}\parallel {{\textbf {Q}}}^{X_{1:i}})&= \text {KL}({{\textbf {P}}}^{X_i,Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i,Z_{1:i-1}})\\&\hspace{1em}+ \text {KL}({{\textbf {P}}}^{X_{1:i-1} \mid X_i,Z_{i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}\mid X_i,Z_{1:i-1}} \mid {{\textbf {P}}}^{X_i,Z_{1:i-1}}), \end{aligned}$$

this is equivalent to

$$\begin{aligned}&\text {KL}({{\textbf {P}}}^{X_{1:i-1} \mid X_i,Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}\mid X_i,Z_{1:i-1}} \mid {{\textbf {P}}}^{X_i,Z_{1:i-1}})\\&\hspace{3cm}\ge \text {KL}({{\textbf {P}}}^{X_{1:i-1}\mid Z_{1:i-1}} \parallel {{\textbf {Q}}}^{X_{1:i-1}\mid Z_{1:i-1}} \mid {{\textbf {P}}}^{Z_{1:i-1}}), \end{aligned}$$

and this last inequality holds true because of the fact that conditioning increases divergence. This finishes the proof of (B8).

Now, putting (B8) into (B7) and using the chain rule again we obtain

$$\begin{aligned} \text {KL}({{\textbf {P}}}^{Z_{1:n}}\parallel {{\textbf {Q}}}^{Z_{1:n}})&\le 2(e^\alpha - 1)^2 \sum _{i=1}^n \text {KL}( {{\textbf {P}}}^{X_i\mid X_{1:i-1}} \parallel {{\textbf {Q}}}^{X_i\mid X_{1:i-1}} \mid {{\textbf {P}}}^{X_{1:i-1}})\\&= 2(e^\alpha - 1)^2 \text {KL}( {{\textbf {P}}}^{X_{1:n}} \parallel {{\textbf {Q}}}^{X_{1:n}} ) \end{aligned}$$

which in combination with the standard data processing inequality $\text {KL}({{\textbf {P}}}^{Z_{1:n}}\parallel {{\textbf {Q}}}^{Z_{1:n}}) \le \text {KL}({{\textbf {P}}}^{X_{1:n}}\parallel {{\textbf {Q}}}^{X_{1:n}})$ (see, for instance, Polyanskiy and Wu (2019), Theorem 6.2) yields the claim

$$\begin{aligned} \text {KL}({{\textbf {P}}}^{Z_{1:n}}\parallel {{\textbf {Q}}}^{Z_{1:n}}) \le \min \{ 2(e^\alpha - 1)^2, 1 \} \, \text {KL}({{\textbf {P}}}^{X_{1:n}}\parallel {{\textbf {Q}}}^{X_{1:n}}). \end{aligned}$$

(B9)

1.2 Proof of Theorem 4.1

With $B :=\sum _{j \in {\mathbb {Z}}} \beta _j^{-2} < \infty$ put $\zeta = \min \left\{ (2\pi )/(B \eta ), 1/(2\eta ), \pi /2 \right\}$ and $m_\alpha = \min \{ 1, 2(e^\alpha -1 )^2\}$. For any ${\varvec{\theta }}= (\theta _j)_{0 \le j \le k_n^{*}} \in \{ \pm 1 \}^{k_n^{*}+1}$, we consider the function $f^{\varvec{\theta }}$ defined through

$$\begin{aligned} f^{\varvec{\theta }}&= \frac{2L}{3} + \theta _0 \left( \frac{L^2 \zeta }{9 n m_\alpha } \right) ^{1/2} + \left( \frac{L^2 \zeta }{9 n m_\alpha } \right) ^{1/2} \sum _{1 \le |j |\le k_n^{*}} \theta _{|j |} \varvec{\text {e}}_j\\&= \frac{2L}{3} + \left( \frac{L^2 \zeta }{9 n m_\alpha } \right) ^{1/2} \sum _{0 \le |j |\le k_n^{*}} \theta _{|j |} \varvec{\text {e}}_j. \end{aligned}$$

Let us first check whether the functions $f^{\varvec{\theta }}$ belong to the set ${\mathcal {F}}(\beta , L)$ of admissible functions for any ${\varvec{\theta }}\in \{ \pm 1 \}^{k_n^{*}+ 1}$. First, $f^ {\varvec{\theta }}$ is a real-valued function since $f^{\varvec{\theta }}_j = f^{\varvec{\theta }}_{-j}$ holds for all j and all ${\varvec{\theta }}$ by construction. Second, $f^{\varvec{\theta }}$ is non-negative since

$$\begin{aligned} \left\Vert \left( \frac{L^2 \zeta }{9 n m_\alpha } \right) ^{1/2} \sum _{0 \le |j |\le k_n^{*}} \theta _{|j |} \varvec{\text {e}}_j \right\Vert _ \infty&\le \left( \frac{L^2 \zeta }{9 n m_\alpha } \right) ^{1/2} \sum _{0 \le |j|\le k_n^{*}} \frac{1}{\sqrt{2\pi }}\\&= \left( \frac{L^2 \zeta }{18\pi } \right) ^{1/2} \left( \sum _{0 \le |j|\le k_n^{*}} \beta _j^{-2} \right) ^{1/2} \cdot \left( \sum _{0 \le |j|\le k_n^{*}} \frac{\beta _j^2}{n m_\alpha } \right) ^{1/2}\\&\le \left( \frac{L^2 \zeta B}{18\pi } \right) ^{1/2} \cdot \left( \beta _{k_n^{*}}^2 \cdot \frac{2k_n^{*}+ 1}{n m_\alpha } \right) ^{1/2}\\&\le \left( \frac{L^2 \zeta B \eta }{18\pi } \right) ^{1/2}\\&\le \frac{L}{3}, \end{aligned}$$

and hence we even have $f^{\varvec{\theta }}\ge L/3 \ge 0$ (the fact that the functions $f^{\varvec{\theta }}$ are uniformly bounded from below will also be used below). Third, $\sum _{j \in {\mathbb {Z}}} |f^{\varvec{\theta }}_j |^2 \beta _j^2 \le L^2$ for any ${\varvec{\theta }}\in \{ \pm 1 \}^{k_n^{*}+ 1}$ thanks to the estimate

$$\begin{aligned} \sum _{j \in {\mathbb {Z}}} |f^{\varvec{\theta }}_j |^2 \beta _j^2&= \sum _{0 \le |j|\le k_n^{*}} |f^{\varvec{\theta }}_j |^2 \beta _j^2\\&= \left[ \frac{2L}{3} + \theta _0 \left( \frac{L^2 \zeta }{9n m_\alpha } \right) ^{1/2} \right] ^2 + \frac{L^2\zeta }{9} \sum _{1 \le |j|\le k_n^{*}} \frac{\beta _j^2}{n m_\alpha }\\&\le \frac{8L^2}{9} + \frac{2L^2 \zeta }{9n m_\alpha } + \frac{L^2 \zeta }{9} \cdot \beta _{k_n^{*}}^2 \cdot \frac{2k_n^{*}}{n m_\alpha }\\&\le \frac{8L^2}{9} + \frac{2L^2 \zeta }{9} \cdot \beta _{k_n^{*}}^2 \cdot \frac{2k_n^{*}+ 1}{n m_\alpha }\\&\le L^2. \end{aligned}$$

Combining the derived properties ensures $f^{\varvec{\theta }}\in {\mathcal {F}}(\beta ,L)$.

In the sequel we denote with ${{\textbf {P}}}_{\varvec{\theta }}^{Z_{1:n}}$ the law of the snippet $Z_{1:n}$ when $(X_t)_{t \in {\mathbb {Z}}}$ is a stationary time series with zero mean and spectral density $f^{\varvec{\theta }}$, and the privacy mechanism Q that generates $Z_{1:n}$ from $X_{1:n}$ is arbitrary among those satisfying the $\alpha$-LDP constraint.

Let ${\widetilde{f}}\in L^2([-\pi ,\pi ])$ be an arbitrary estimator defined in terms of the snippet $Z_{1:n}$. Its maximal risk can be bounded from below by reduction to a finite set of hypotheses:

$$\begin{aligned}&\sup _{f \in {\mathcal {F}}(\beta , L)} {{\textbf {E}}}_f \Vert {\widetilde{f}}- f \Vert ^2\nonumber \ge \sup _{{\varvec{\theta }}\in \{ \pm 1 \}^{k_n^{*}+ 1} } {{\textbf {E}}}_{\varvec{\theta }}\Vert {\widetilde{f}}- f^{\varvec{\theta }}\Vert ^2\nonumber \\&\quad \ge \frac{1}{2^{k_n^{*}+ 1}} \sum _{{\varvec{\theta }}\in \{ \pm 1 \}^{k_n^{*}+1} } {{\textbf {E}}}_{\varvec{\theta }}\Vert {\widetilde{f}}- f^{\varvec{\theta }}\Vert ^2\nonumber \\&\quad \ge \frac{1}{2^{k_n^{*}+ 1}} \sum _{{\varvec{\theta }}\in \{ \pm 1 \}^{k_n^{*}+1} } \sum _{0 \le |j |\le k_n^{*}} {{\textbf {E}}}_{\varvec{\theta }}|{\widetilde{f}}_j - f^{\varvec{\theta }}_j |^2\nonumber \\&\quad = \frac{1}{2^{k_n^{*}+ 1}} \sum _{0 \le |j |\le k_n^{*}} \sum _{{\varvec{\theta }}\in \{ \pm 1 \}^{k_n^{*}+1} } \frac{1}{2} \left\{ {{\textbf {E}}}_{\varvec{\theta }}|{\widetilde{f}}_j - f^{\varvec{\theta }}_j |^2 + {{\textbf {E}}}_{{\varvec{\theta }}^{|j|}} |{\widetilde{f}}_j - f^{{\varvec{\theta }}^{|j |}}_j |^2 \right\} , \end{aligned}$$

(B10)

where for ${\varvec{\theta }}= (\theta _k) \in \{ \pm 1 \}^{k_n^{*}+ 1}$ and $j \in \llbracket -k_n^{*},k_n^{*}\rrbracket$ the element ${\varvec{\theta }}^{|j|}$ is defined by $\theta ^{|j|}_k = \theta _k$ for $k \ne |j|$ and $\theta ^{|j|}_{|j|} = -\theta _{|j|}$ (‘flip in the jth coordinate’). Recall the definition of the Hellinger affinity $\rho ({{\textbf {P}}}, {{\textbf {Q}}})$,

$$\begin{aligned} \rho ({{\textbf {P}}}, {{\textbf {Q}}}) :=\int \sqrt{\text {d}{{\textbf {P}}}\text {d}{{\textbf {Q}}}}, \end{aligned}$$

where the integral is taken with respect to any dominating measure of ${{\textbf {P}}}$ and ${{\textbf {Q}}}$. For any estimator ${\widetilde{f}}$, we have

$$\begin{aligned} \rho ({{\textbf {P}}}_{\varvec{\theta }}^{Z_{1:n}}, {{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j|}})&\le \int \frac{|{\widetilde{f}}_j - f^{\varvec{\theta }}_j |}{|f^{\varvec{\theta }}_j - f^{{\varvec{\theta }}^{|j|}}_j |} \sqrt{\text {d}{{\textbf {P}}}^{Z_{1:n}}_{\varvec{\theta }}\text {d}{{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j|}}} + \int \frac{|{\widetilde{f}}_j - f^{{\varvec{\theta }}^{|j|}}_j |}{|f^{\varvec{\theta }}_j - f^{{\varvec{\theta }}^{|j|}}_j |} \sqrt{\text {d}{{\textbf {P}}}^{Z_{1:n}}_{\varvec{\theta }}\text {d}{{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j|}}}\\&\le \left( \int \frac{|{\widetilde{f}}_j - f^{\varvec{\theta }}_j |^2}{|f^{\varvec{\theta }}_j - f^{{\varvec{\theta }}^{|j|}}_j |^2} \text {d}{{\textbf {P}}}^{Z_{1:n}}_{\varvec{\theta }}\right) ^{1/2} + \left( \int \frac{|{\widetilde{f}}_j - f^{{\varvec{\theta }}^{|j|}}_j |^2}{|f^{\varvec{\theta }}_j - f^{{\varvec{\theta }}^{|j|}}_j |^2} \text {d}{{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j |}} \right) ^{1/2}, \end{aligned}$$

from which we obtain using the elementary estimate $(a+b)^2 \le 2a^2 + 2b^2$ that

$$\begin{aligned} \frac{1}{2} |f^{\varvec{\theta }}_j - f^{{\varvec{\theta }}^{|j|}}_j |^2 \rho ^2({{\textbf {P}}}^{Z_{1:n}}_{\varvec{\theta }}, {{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j|}}) \le {{\textbf {E}}}_{\varvec{\theta }}|{\widetilde{f}}_j - f^{\varvec{\theta }}_j |^2+ {{\textbf {E}}}_{{\varvec{\theta }}^{|j|}} |{\widetilde{f}}_j - f^{{\varvec{\theta }}^{|j|}}_j |^2. \end{aligned}$$

(B11)

Taking the identity $H^2({{\textbf {P}}}, {{\textbf {Q}}}) = 2 (1 - \int \sqrt{\text {d}{{\textbf {P}}}\text {d}{{\textbf {Q}}}})$ (cf. Tsybakov 2004, p. 69) into account, finding an upper bound for the squared Hellinger distance corresponds to finding a lower bound for the affinity. We have

$$\begin{aligned} H^2({{\textbf {P}}}^{Z_{1:n}}_{\varvec{\theta }}, {{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j|}})&\le \text {KL}({{\textbf {P}}}^{Z_{1:n}}_{\varvec{\theta }}\parallel {{\textbf {P}}}^{Z_{1:n}}_{{\varvec{\theta }}^{|j|}})\\&\le m_\alpha \text {KL}({{\textbf {P}}}^{X_{1:n}}_{\varvec{\theta }}\parallel {{\textbf {P}}}^{X_{1:n}}_{{\varvec{\theta }}^{\vert j\vert }})\\&\le \frac{n m_\alpha }{4\pi (\min _{{\varvec{\theta }}} \inf _\omega f^{\varvec{\theta }}(\omega ))^2} \cdot \Vert f^{\varvec{\theta }}- f^{{\varvec{\theta }}^{|j|}} \Vert ^2\\&\le \frac{9n m_\alpha }{4\pi L^2} \cdot [ |f_j^{\varvec{\theta }}- f^{{\varvec{\theta }}^{|j|}}_j |^2 + |f_{-j}^{\varvec{\theta }}- f^{{\varvec{\theta }}^{|j|}}_{-j} |^2 ] \end{aligned}$$

by using Equation (2.19) from Tsybakov (2004), Eq. (B9), Lemma 3.4 from Bentkus (1985), and the fact that $f^{\varvec{\theta }}\ge L/3$ (the last inequality was established above). Thus, by the very definition of $\zeta$

$$\begin{aligned} H^2({{\textbf {P}}}^{Z_{1:n}}_\theta ,{{\textbf {P}}}^{Z_{1:n}}_{\theta ^{|j|}})&\le \frac{18n m_\alpha }{\pi L^2} \cdot \left( \frac{L^2 \zeta }{9n m_\alpha } \right) \le 1, \end{aligned}$$

and consequently $\rho ({{\textbf {P}}}_\theta ^{Z_{1:n}}, {{\textbf {P}}}^{Z_{1:n}}_{\theta ^{|j|}}) \ge 1/2$. Putting this estimate into (B11) and combining the result with (B10) yields

$$\begin{aligned} \sup _{f \in {\mathcal {F}}(\beta , L)} {{\textbf {E}}}\Vert {\widetilde{f}}- f \Vert ^2&\ge \frac{1}{4} \sum _{0 \le |j|\le k_n^{*}} \frac{L^2 \zeta }{9n m_\alpha }\\&= \frac{L^2 \zeta }{36} \cdot \frac{2k_n^{*}+ 1}{n m_\alpha } \end{aligned}$$

which provides the statement of the Theorem 4.1.

Proof of Theorem 5.4

We define the event A (and its complement) exactly as in the proof of Theorem 3.2, that is,

$$\begin{aligned} A = \bigcap _{i=1}^{n} \{ X_i = {\widetilde{X}}_i \}, \end{aligned}$$

and we consider the decomposition

$$\begin{aligned} {{\textbf {E}}}\Vert {\widetilde{f}}- f \Vert ^2 = {{\textbf {E}}}\Vert {\widetilde{f}}- f \Vert ^2 {{\textbf {1}}}_A + {{\textbf {E}}}\Vert {\widetilde{f}}- f \Vert ^2 {{\textbf {1}}}_{A^\complement }. \end{aligned}$$

$\underline{\hbox {Upper bound for }{{\textbf {E}}}\Vert {\widetilde{f}}- f \Vert ^2 {{\textbf {1}}}_A}$: We can write the contrast as

$$\begin{aligned} \Upsilon _n(t) = \Vert t \Vert ^2 - 2 \langle {\widehat{I}}_n, t \rangle = \Vert t-f \Vert ^2 - 2 \langle {\widehat{I}}_n - f,t \rangle - \Vert f \Vert ^2. \end{aligned}$$

By the definitions of ${\widetilde{f}}$ and ${\widehat{{\mathfrak {m}}}}$ combined with the fact that ${\widehat{f}}_{{\mathfrak {m}}}$ minimizes the contrast over the space $S_{\mathfrak {m}}$ the estimate

$$\begin{aligned} \Upsilon _n({\widetilde{f}}) + {\text {pen}}({\widehat{{\mathfrak {m}}}}) \le \Upsilon _n(f_{\mathfrak {m}}) + {\text {pen}}({\mathfrak {m}}) \end{aligned}$$

holds for all ${\mathfrak {m}}\in {\mathcal {M}}_n$, we obtain

$$\begin{aligned} \Vert f - {\widetilde{f}}\Vert ^2 - 2 \langle {\widehat{I}}_n - f, {\widetilde{f}}\rangle + {\text {pen}}({\widehat{{\mathfrak {m}}}}) \le \Vert f - f_{\mathfrak {m}}\Vert ^2 - 2 \langle {\widehat{I}}_n - f, f_{\mathfrak {m}}\rangle + {\text {pen}}({\mathfrak {m}}). \end{aligned}$$

Then, by elementary algebraic manipulations,

$$\begin{aligned} \Vert f - {\widetilde{f}}\Vert ^2&\le \Vert f - f_{\mathfrak {m}}\Vert ^2 + 2 \langle {\widehat{I}}_n - f, {\widetilde{f}}- f_{\mathfrak {m}}\rangle + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}})\\&= \Vert f - f_{\mathfrak {m}}\Vert ^2 + 2 \langle f - {{\textbf {E}}}{\widehat{I}}_n, f_{\mathfrak {m}}- {\widetilde{f}}\rangle + 2 \langle {\widehat{I}}_n - {{\textbf {E}}}{\widehat{I}}_n, {\widetilde{f}}- f_{\mathfrak {m}}\rangle \\&\hspace{1em}+ {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}). \end{aligned}$$

On the event A, we have $Z'_{1:n}=Z_{1:n}$ and ${\widehat{I}}_n = I^Z_n - \frac{4\tau _n^2}{\pi \alpha ^2} = I^{Z'}_n - \frac{4\tau _n^2}{\pi \alpha ^2}$. Hence, on A the identity

$$\begin{aligned} \langle {\widehat{I}}_n - {{\textbf {E}}}{\widehat{I}}_n, {\widetilde{f}}- f_{\mathfrak {m}}\rangle = \langle I_n^{Z'} - {{\textbf {E}}}I_n^{Z'}, {\widetilde{f}}- f_{\mathfrak {m}}\rangle \end{aligned}$$

holds. By definition of $I_n^{Z'}$, we have

$$\begin{aligned} I_n^{Z'}(\omega )&= \frac{1}{2\pi n} \left|\sum _{t=1}^n (Z_t^\prime - \overline{Z}^\prime _n) e^{-\text {i}t \omega } \right|^2\\&= \frac{1}{2\pi n} \left|\sum _{t=1}^n (X_t - {\bar{X}}_n) e^{-\text {i}t \omega } + \sum _{t=1}^n (\xi _t - {\bar{\xi }}_n) e^{-\text {i}t \omega } \right|^2\\&= \frac{1}{2\pi n} \left|\sum _{t=1}^n (X_t - {\bar{X}}_n) e^{-\text {i}t \omega } \right|^2 + \frac{1}{2\pi n} \left( \sum _{t=1}^n (X_t - {\bar{X}}_n) e^{-\text {i}t \omega } \right) \left( \sum _{t=1}^n (\xi _t - {\bar{\xi }}_n) e^{\text {i}t \omega } \right) \\&\hspace{1em}+ \frac{1}{2\pi n} \left( \sum _{t=1}^n (X_t - {\bar{X}}_n) e^{\text {i}t \omega } \right) \left( \sum _{t=1}^n (\xi _t - {\bar{\xi }}_n) e^{-\text {i}t \omega } \right) + \frac{1}{2\pi n}\left|\sum _{t=1}^n (\xi _t - {\bar{\xi }}_n) e^{-\text {i}t \omega } \right|^2\\&=:I_n^X(\omega ) + {\widetilde{I}}_n(\omega ) + I_n^\xi (\omega ) \end{aligned}$$

(${\widetilde{I}}_n$ is defined as the sum of the two ‘mixed’ terms). For ${\mathfrak {m}}, {\mathfrak {m}}' \in {\mathcal {M}}_n$, set

$$\begin{aligned} G_X({\mathfrak {m}},{\mathfrak {m}}^\prime )&= \sup _{u \in {\mathcal {B}}_{{\mathfrak {m}}, {\mathfrak {m}}^\prime }} \langle I_n^X - {{\textbf {E}}}I_n^X, u \rangle ,\\ G_\xi ({\mathfrak {m}},{\mathfrak {m}}^\prime )&= \sup _{u \in {\mathcal {B}}_{{\mathfrak {m}}, {\mathfrak {m}}^\prime }} \langle I_n^\xi - {{\textbf {E}}}I_n^\xi , u \rangle ,\\ {\widetilde{G}}({\mathfrak {m}},{\mathfrak {m}}^\prime )&= \sup _{u \in {\mathcal {B}}_{{\mathfrak {m}}, {\mathfrak {m}}^\prime }} \langle {\widetilde{I}}_n, u \rangle , \end{aligned}$$

where ${\mathcal {B}}_{{\mathfrak {m}}, {\mathfrak {m}}^\prime }$ denotes the unit ball in $S_{\mathfrak {m}}+ S_{{\mathfrak {m}}^\prime }$, and we write $G_X({\mathfrak {m}})$, $G_\xi ({\mathfrak {m}})$, and ${\widetilde{G}}({\mathfrak {m}})$ when ${\mathfrak {m}}= {\mathfrak {m}}^\prime$. We have $G_X({\mathfrak {m}},{\mathfrak {m}}^\prime ) \le G_X({\mathfrak {m}}) + G_X({\mathfrak {m}}^\prime )$, and the same type of bound holds for $G_\xi$ and ${\widetilde{G}}$. As a consequence, using the estimate $2xy \le \tau x^2 + \tau ^{-1} y^2$ for $\tau = 16$ we have

$$\begin{aligned} \Vert f - {\widetilde{f}}\Vert ^2 {{\textbf {1}}}_A&\le ( \Vert f - f_{\mathfrak {m}}\Vert ^2 + 2 \langle f - {{\textbf {E}}}I^{Z'}_n, f_{\mathfrak {m}}- {\widetilde{f}}\rangle + 2 \langle I^{Z'}_n - {{\textbf {E}}}I^{Z'}_n, {\widetilde{f}}- f_{\mathfrak {m}}\rangle + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A \\&= ( \Vert f - f_{\mathfrak {m}}\Vert ^2 + 2 \langle f - {{\textbf {E}}}I^{Z'}_n, f_{\mathfrak {m}}- {\widetilde{f}}\rangle + 2 \langle I_n^X - {{\textbf {E}}}I_n^X, {\widetilde{f}}- f_{\mathfrak {m}}\rangle +2 \langle I_n^\xi - {{\textbf {E}}}I_n^\xi , {\widetilde{f}}- f_{\mathfrak {m}}\\&\hspace{1em} + 2 \langle {\widetilde{I}}_n, {\widetilde{f}}- f_{\mathfrak {m}}\rangle + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A \\&\le ( \Vert f - f_{\mathfrak {m}}\Vert ^2 + 2 \langle f - {{\textbf {E}}}I^{Z'}_n, f_{\mathfrak {m}}- {\widetilde{f}}\rangle + 2 \Vert {\widetilde{f}}- f_{\mathfrak {m}}\Vert G_X({\mathfrak {m}}, {\widehat{{\mathfrak {m}}}}) + 2 \Vert {\widetilde{f}}- f_{\mathfrak {m}}\Vert G_\xi ({\mathfrak {m}},{\widehat{{\mathfrak {m}}}}) \\&\hspace{1em} + 2\Vert {\widetilde{f}}- f_{\mathfrak {m}}\Vert {\widetilde{G}}({\mathfrak {m}},{\widehat{{\mathfrak {m}}}}) + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A \\&= ( \Vert f - f_{\mathfrak {m}}\Vert ^2 + \tau \Vert f - {{\textbf {E}}}I^{Z'}_n \Vert ^2 + 4\tau ^{-1} \Vert f_{\mathfrak {m}}- {\widetilde{f}}\Vert ^2 + \tau G_X^2({\mathfrak {m}},{\widehat{{\mathfrak {m}}}}) + \tau G_\xi ^2({\mathfrak {m}},{\widehat{{\mathfrak {m}}}}) \\&\hspace{1em} + \tau {\widetilde{G}}^2({\mathfrak {m}},{\widehat{{\mathfrak {m}}}}) + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A \\&=( \Vert f - f_{\mathfrak {m}}\Vert ^2 + 16 \Vert f - {{\textbf {E}}}I^{Z'}_n \Vert ^2 + \frac{1}{4} \Vert f_{\mathfrak {m}}- {\widetilde{f}}\Vert ^2 + 32 G_X^2({\widehat{{\mathfrak {m}}}}) + 32 G_\xi ^2({\widehat{{\mathfrak {m}}}}) \\&\hspace{1em}+ 32 {\widetilde{G}}^2({\widehat{{\mathfrak {m}}}}) + 32 G_X^2({\mathfrak {m}}) + 32 G_\xi ^2({\mathfrak {m}}) + 32 {\widetilde{G}}^2({\mathfrak {m}}) + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A\\&\le ( 3 \Vert f - f_{\mathfrak {m}}\Vert ^2/2 + 16 \Vert f - {{\textbf {E}}}I^{Z'}_n \Vert ^2 + \frac{1}{2} {{\textbf {E}}}\Vert f - {\widetilde{f}}\Vert ^2 + 32 G_X^2({\widehat{{\mathfrak {m}}}}) + 32 G_\xi ^2({\widehat{{\mathfrak {m}}}}) \\&\hspace{1em} + 32 {\widetilde{G}}^2({\widehat{{\mathfrak {m}}}}) + 32 G_X^2({\mathfrak {m}}) + 32 G_\xi ^2({\mathfrak {m}}) + 32 {\widetilde{G}}^2({\mathfrak {m}}) + {\text {pen}}({\mathfrak {m}}) - {\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A. \end{aligned}$$

Hence,

$$\begin{aligned} \Vert f - {\widetilde{f}}\Vert ^2 {{\textbf {1}}}_A&\le ( 3 \Vert f - f_{\mathfrak {m}}\Vert ^2 + 32 \Vert f - {{\textbf {E}}}I^{Z'}_n \Vert ^2 + 64 G_X^2({\widehat{{\mathfrak {m}}}}) + 64 G_\xi ^2({\widehat{{\mathfrak {m}}}}) \\&\quad +\, 64 {\widetilde{G}}^2({\widehat{{\mathfrak {m}}}}) + 64 G_X^2({\mathfrak {m}}) + 64 G_\xi ^2({\mathfrak {m}}) + 64 {\widetilde{G}}^2({\mathfrak {m}}) + 2{\text {pen}}({\mathfrak {m}}) - 2{\text {pen}}({\widehat{{\mathfrak {m}}}}) ) {{\textbf {1}}}_A. \end{aligned}$$

If the numerical constant in the definition of the penalty is large enough, we can write ${\text {pen}}({\mathfrak {m}}) = {\text {pen}}_X({\mathfrak {m}}) + {\text {pen}}_\xi ({\mathfrak {m}}) + {\widetilde{{\text {pen}}}}({\mathfrak {m}})$ such that

$$\begin{aligned} {\text {pen}}_X({\mathfrak {m}})&\ge 32\kappa _X \Vert f \Vert _\infty (1+ C_{{\bar{r}}}^2) \frac{D_{{\mathfrak {m}}} ( 1 + L_{{\mathfrak {m}}})^2}{n},\\ {\text {pen}}_\xi ({\mathfrak {m}})&\ge 32 \kappa _\xi \frac{\tau _n^4D_{{\mathfrak {m}}} (L_{{\mathfrak {m}}}^4 + L_{\mathfrak {m}}+ \log (n) ) }{n\alpha ^4}, \qquad \text {and}\\ {\widetilde{{\text {pen}}}}({\mathfrak {m}})&\ge 32 {\widetilde{\kappa }}M^4 (1+\Vert f \Vert _\infty )^2 (L_{{\mathfrak {m}}}^4 + L_{\mathfrak {m}}+ \log (n) ) \frac{D_{{\mathfrak {m}}}}{n} \end{aligned}$$

holds for any model ${\mathfrak {m}}\in {\mathcal {M}}_n$. Summing over all potential models and taking expectations implies

$$\begin{aligned} {{\textbf {E}}}\Vert f - {\widetilde{f}}\Vert ^2 {{\textbf {1}}}_A&\le 3 \Vert f - f_{\mathfrak {m}}\Vert ^2 + 32 \Vert f - {{\textbf {E}}}I^{Z'}_n \Vert ^2 + 4 {\text {pen}}({\mathfrak {m}}) \\&\hspace{1em}+ 64 \sum _{{\mathfrak {m}}\in {\mathcal {M}}_n} {{\textbf {E}}}\left[ \left( G_X^2({\mathfrak {m}}) - {\text {pen}}_X({\mathfrak {m}})/32 \right) _+ \right] \\&\hspace{1em}+ 64 \sum _{{\mathfrak {m}}\in {\mathcal {M}}_n} {{\textbf {E}}}\left[ \left( G_\xi ^2({\mathfrak {m}}) - {\text {pen}}_\xi ({\mathfrak {m}})/32 \right) _+ \right] \\&\hspace{1em}+ 64 \sum _{{\mathfrak {m}}\in {\mathcal {M}}_n} {{\textbf {E}}}\left[ \left( {\widetilde{G}}^2({\mathfrak {m}}) - {\widetilde{{\text {pen}}}}({\mathfrak {m}})/32 \right) _+ \right] . \end{aligned}$$

The following three lemmata are proven in Section D.3.

Lemma C.1

For any fixed model ${\mathfrak {m}}\in {\mathcal {M}}_n$ and a sufficiently large constant $\kappa _X > 0$, we have

$$\begin{aligned} {{\textbf {E}}}\left[ \left( G_X^2({\mathfrak {m}}) - \kappa _X \Vert f \Vert _\infty ^2 (1 + C_{{\bar{r}}}^2)\frac{D_{{\mathfrak {m}}} (1 + L_{{\mathfrak {m}}})^2}{n} \right) _+ \right] \lesssim e^{- L_{{\mathfrak {m}}} D_{{\mathfrak {m}}}} \cdot \frac{C (C_{{\bar{r}}}, \Vert f \Vert _\infty ) }{n}. \end{aligned}$$

Lemma C.2

For any fixed model ${\mathfrak {m}}\in {\mathcal {M}}_n$ and a sufficiently large constant $\kappa _\xi > 0$, we have

$$\begin{aligned} {{\textbf {E}}}\left[ \left( G_\xi ^2({\mathfrak {m}}) - \kappa _\xi \frac{\tau _n^4D_{{\mathfrak {m}}} (L_{{\mathfrak {m}}}^4 + L_{\mathfrak {m}}+ \log (n) ) }{n\alpha ^4} \right) _+ \right] \lesssim e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \frac{C(C_{{\bar{r}}}) \tau _n^4}{n^3 \alpha ^4}. \end{aligned}$$

Lemma C.3

For any fixed model ${\mathfrak {m}}\in {\mathcal {M}}_n$ and a sufficiently large constant ${\widetilde{\kappa }}> 0$, we have

$$\begin{aligned}&{{\textbf {E}}}\left[ \left( {\widetilde{G}}^2({\mathfrak {m}}) - {\widetilde{\kappa }}M^4 (1+\Vert f \Vert _\infty )^2 (L_{{\mathfrak {m}}}^4 + L_{\mathfrak {m}}+ \log (n) ) \frac{D_{{\mathfrak {m}}}}{n} \right) _+ \right] \\&\quad \lesssim e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \frac{C(C_{{\bar{r}}}, \Vert f \Vert _\infty ) M^4}{n^3} \end{aligned}$$

where $M = 3+ 4\tau _n/\alpha$.

The expectations are bounded by Lemmata C.1, C.2, and C.3, combined with Assumption 5.3 in order to obtain

$$\begin{aligned} {{\textbf {E}}}\Vert f - {\widetilde{f}}\Vert ^2 {{\textbf {1}}}_A&\le 3 \Vert f - f_{\mathfrak {m}}\Vert ^2 + 32 \Vert f - {{\textbf {E}}}I^{Z'}_n \Vert ^2 + 4{\text {pen}}({\mathfrak {m}}) \\&\quad +\, C(C_{{\bar{r}}},\Vert f \Vert _\infty ) \max \left\{ \frac{1}{n}, \frac{\tau _n^4}{n^3 \alpha ^4} \right\} . \end{aligned}$$

Finally, by Proposition A.1 we get (using the same argument as in the proof of Theorem 3.2)

$$\begin{aligned} {{\textbf {E}}}\Vert f - {\widetilde{f}}\Vert ^2 {{\textbf {1}}}_A&\lesssim \Vert f - f_{\mathfrak {m}}\Vert ^2 + \max \left\{ \frac{\tau _n^4}{n \alpha ^4} , \frac{1}{n} \right\} + {\text {pen}}({\mathfrak {m}}) \\&\hspace{1em}+ C(C_{{\bar{r}}},\Vert f \Vert _\infty ) \max \left\{ \frac{1}{n}, \frac{\tau _n^4}{n^3 \alpha ^4} \right\} . \end{aligned}$$

Since, this estimate holds for any fixed model ${\mathfrak {m}}$, we can take the infimum over all potential models which yields

$$\begin{aligned} {{\textbf {E}}}\Vert f - {\widetilde{f}}\Vert ^2 {{\textbf {1}}}_A&\lesssim \inf _{{\mathfrak {m}}\in {\mathcal {M}}_n} \left[ \Vert f - f_{\mathfrak {m}}\Vert ^2 , {\text {pen}}({\mathfrak {m}}) \right] + \max \left\{ \frac{\tau _n^4}{n \alpha ^4} , \frac{1}{n} \right\} \\&\quad + C(C_{{\bar{r}}},\Vert f \Vert _\infty ) \max \left\{ \frac{1}{n}, \frac{\tau _n^4}{n^3 \alpha ^4} \right\} . \end{aligned}$$

$\underline{\hbox {Upper bound for }{{\textbf {E}}}\Vert {\widetilde{f}}- f \Vert ^2 {{\textbf {1}}}_{A^\complement }:}$ This term can be bounded exactly as in the upper bound for any fixed model:

$$\begin{aligned} {{\textbf {E}}}\Vert {\widehat{f}}_{\mathfrak {m}}- f_{\mathfrak {m}}\Vert ^2 {{\textbf {1}}}_{A^\complement } \lesssim \frac{\tau _n^4}{n} \vee \frac{\tau _n^4}{n\alpha ^4}. \end{aligned}$$

Concentration results for the proof of Theorem 5.4

1.1 A general chaining argument

Let ${\overline{S}}$ be a finite dimensional subspace of $L^2 \cap L^\infty$ spanned by some orthonormal basis $(\phi _i)_{i \in {\mathcal {I}}'}$. We denote the dimension $|{\mathcal {I}}' |$ of ${\overline{S}}$ with D, and define the quantity

$$\begin{aligned} {\bar{r}}_\phi = \frac{1}{\sqrt{D}} \sup _{\beta \in {\mathbb {R}}^D, \beta \ne 0} \frac{\Vert \sum _{i \in {\mathcal {I}}'} \beta _i \phi _i \Vert _\infty }{|\beta |_\infty }. \end{aligned}$$

In addition, we define ${\bar{r}}$ as the infimum of ${\bar{r}}_\phi$ taken over all possible orthonormal bases of ${\overline{S}}$.

Proposition D.1

(Proposition 1 from Birgé and Massart 1998) Let ${\overline{S}}$ be a D-dimensional linear subspace of $L^2 \cap L^\infty$ with its index ${\bar{r}}$ defined as above. Let ${\mathcal {B}}$ be any ball of radius $\sigma$ in ${\overline{S}}$ and $0< \delta < \sigma /5$. Then there exists a finite set $T \subset {\mathcal {B}}$ which is simultaneously a $\delta$-net for ${\mathcal {B}}$ with respect to the $L^2$-norm and an ${\bar{r}}\delta$-net with respect to the $L^\infty$-norm and such that $|T |\le (6\sigma /\delta )^D$.

We will apply Proposition D.1 with $\sigma = 1$. In the sequel, we will use the following chaining argument. For $0< \delta _0 < 1/5$ and any $k \in {\mathbb {N}}$, we set $\delta _k = \delta _0 2^{-k}$ and consider a sequence of $\delta _k$-nets $(T_k)_{k \in {\mathbb {N}}}$ with $T_k = T_{\delta _k}$. Then, for any $u \in {\mathcal {B}}_{{\mathfrak {m}}}$ (${\mathcal {B}}_{\mathfrak {m}}$ is defined in the proof of Theorem 5.4 as the unit ball in the space $S_{\mathfrak {m}}$), we are able to find a sequence $(u_k)_{k \ge 0}$ with $u_k \in T_k$ such that $\Vert u - u_k \Vert ^2 \le \delta _k^2$ and $\Vert u-u_k \Vert _\infty \le {\bar{r}}_{{\mathfrak {m}}} \delta _k$. In addition we set $\delta _{-1} = 1$. The proof of Proposition 1 in Birgé and Massart (1998) shows that one can take $T_{-1} = \{ 0 \}$ in this case where Proposition D.1 as stated above cannot be applied directly. Moreover, one can achieve $|T_k |\le (6/\delta _k)^{D_{\mathfrak {m}}}$ for any k. We have the following decomposition:

$$\begin{aligned} u = \sum _{k=0}^\infty (u_k - u_{k-1}). \end{aligned}$$

(D12)

From the above properties it follows that for $k \ge 0$, $\Vert u_k - u_{k-1} \Vert ^2 \le 2(\delta _k^2 + \delta _{k-1}^2) = 5\delta _{k-1}^2/2$ and $\Vert u_k - u_{k-1} \Vert _\infty \le 3 {\bar{r}}_{{\mathfrak {m}}} \delta _{k-1}/2$. These estimates will be used below without further reference.

Let us finally note that we will work with different definitions of $\delta _0$ below. For the purely Gaussian terms in Sect. D.4.1 it will turn out convenient to choose $0< \delta _0 < 1/5$ as a numerical constant independent of n whereas for the analysis of the Laplace term in Sect. D.4.2 and the mixed term D.4.3 we will need to choose $\delta _0 \asymp n^{-1}$ in order to get better rates (at the cost of slightly worse logarithmic terms). We put $H_k = \log (|T_k |)$. Then

$$\begin{aligned} H_k \le D_{{\mathfrak {m}}} \log (6/\delta _k) = D_{{\mathfrak {m}}} [\log (6/\delta _0) + k \log 2] \end{aligned}$$

which will be used below without further reference.

1.2 The Toeplitz matrix $T_n(u)$

In the following three Sects. D.4.1–D.4.3 we will consider the following Toeplitz matrix $T_n(u)$ associated with the function u given by the entries

$$\begin{aligned}{}[T_n(u)]_{j,k} = \int _{-\pi }^\pi u(\omega ) \text {e}^{\text {i}\omega (j-k)} \text {d}\omega , \qquad 1 \le j,k \le n. \end{aligned}$$

The matrix $T_n(u)$ is always Hermitian but since we consider only symmetric u, the same holds true for $T_n(u)$ (which is then real-valued).

1.3 Proof of Lemmata C.1, C.2, and C.3

We consider a sequence $(\eta _k)_{k \ge 0}$ of positive numbers and $\eta \ge \sum _{k \ge 0}\eta _k$ (these quantities will be specified later on). Then, using the decomposition (D12), we get

$$\begin{aligned} {{\textbf {P}}}(\sup _{u \in B_{{\mathfrak {m}}}} \Xi _n(u)> \eta )&={{\textbf {P}}}\left[ \exists (u_k) \in \prod T_k : \sum _{k \ge 0} \Xi _n(u_k - u_{k-1}) > \sum _{k \ge 0} \eta _k \right] \\&=:P, \end{aligned}$$

where $\Xi \in \{ \Xi _n^X, \Xi _n^\xi , {\widetilde{\Xi }}_n \}$. For any $k \ge 0$, we get from Propositions D.2, D.3, and D.4 that, for any $u_{k-1} \in T_{k-1}$ and $u_k \in T_k$,

$$\begin{aligned} {{\textbf {P}}}( \Xi _n^X(u_k - u_{k-1})> \eta _k)&\le 2 \exp \left( - c\min \left( \frac{8\pi ^2 n\eta _k^2}{45 \Vert f\Vert _\infty ^2 \delta _{k-1}^2}, \frac{4\pi n\eta _k}{9\Vert f\Vert _\infty {\bar{r}}_{{\mathfrak {m}}} \delta _{k-1}} \right) \right) ,\\ {{\textbf {P}}}( \Xi _n^\xi (u_k - u_{k-1})> \eta _k )&\le 2 \exp \left( - \frac{1}{C} \min \left( \frac{\pi ^2 n\eta _k^2 \alpha ^4}{160\tau _n^4 \delta _{k-1}^2} , \frac{ \sqrt{2\pi n\eta _k} \alpha }{4\sqrt{3/2} \tau _n \sqrt{{\bar{r}}_{{\mathfrak {m}}} \delta _{k-1}}} \right) \right) ,\\ {{\textbf {P}}}( {\widetilde{\Xi }}_n (u_k - u_{k-1}) > \eta _k )&\le 2 \exp \left( - \frac{1}{C} \min \left( \frac{4\pi ^2 n\eta _k^2}{5 \Vert f \Vert _\infty M^4 \delta _{k-1}^2} , \frac{2\sqrt{\pi n\eta _k} }{M \Vert f \Vert _\infty ^{1/2} \sqrt{3{\bar{r}}_{{\mathfrak {m}}} \delta _{k-1}}} \right) \right) . \end{aligned}$$

In any of the three cases $\Xi \in \{ \Xi _n^X, \Xi _n^\xi , \widetilde{\Xi }_n \}$ we will choose the sequence $\eta _k$ such that the argument of the exponential terms on the right-hand side is less or equal $-(H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda )$. Hence, we choose

$$\begin{aligned}\scriptstyle \eta _k = C \Vert f \Vert _\infty \delta _{k-1} \max \left( \sqrt{\frac{H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda }{n}}, \frac{{\bar{r}}_{{\mathfrak {m}}} (H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda )}{n} \right) \end{aligned}$$

in the Gaussian case ($\Xi _n = \Xi _n^X$),

$$\begin{aligned}\scriptstyle \eta _k \ge C \frac{\tau _n^2 \delta _{k-1}}{\alpha ^2}\max \left\{ \sqrt{\frac{H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda }{n}}, \frac{{\bar{r}}_{{\mathfrak {m}}} }{n} (H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda )^2 \right\} \end{aligned}$$

in the subexponential case ($\Xi _n = \Xi _n^\xi$), and

$$\begin{aligned}\scriptstyle \eta _k \ge C M^2 \delta _{k-1} (1+ \Vert f \Vert _\infty ) \max \left\{ \sqrt{\frac{H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda }{n}}, \frac{{\bar{r}}_{{\mathfrak {m}}}(H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda )^2}{n} \right\} . \end{aligned}$$

in the mixed case ($\Xi _n = {\widetilde{\Xi }}_n$). Under these choices of $(\eta _k)_{k \ge 0}$, we obtain for $\eta \ge \sum \eta _k$ (using the assumption that $D_{{\mathfrak {m}}} \ge 1$)

$$\begin{aligned} {{\textbf {P}}}(\sup _{u \in B_{{\mathfrak {m}}}} \Xi _n(u) > \eta )&\le 2 \sum _{k \ge 0} \exp (-kD_{{\mathfrak {m}}}- L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} - \lambda )\\&= 2 \exp (- L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} - \lambda ) \sum _{k\ge 0} e^{-kD_{{\mathfrak {m}}}}\\&\le 3.2 \exp (- L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} - \lambda ). \end{aligned}$$

We have to compute bounds for $\sum _{k \ge 0} \eta _k$ in each of the three cases. For the Gaussian case, we can take $0< \delta _0<1/5$ as a purely numerical constant in order to obtain

$$\begin{aligned} \left( \sum _{k \ge 0} \eta _k \right) ^2&\lesssim \Vert f \Vert _\infty ^2 \left( \sum _{k \ge 0} \delta _{k-1} \sqrt{\frac{H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda }{n}} \right. \\&\left. \hspace{1em}+ \sum _{k \ge 0} \delta _{k-1} \frac{{\bar{r}}_{{\mathfrak {m}}} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda )}{n} \right) ^2\\&\lesssim \Vert f \Vert _\infty ^2 \left( \frac{1}{n} \sum _{k \ge 0} \delta _{k-1} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda )) \right. \\&\left. \hspace{1em}+ \frac{{\bar{r}}^2_{{\mathfrak {m}}}}{n^2} ( \sum _{k \ge 0}\delta _{k-1} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda ) )^2 \right) \\&\lesssim \Vert f \Vert _\infty ^2 \left[ \left( \frac{D_{{\mathfrak {m}}} + D_{{\mathfrak {m}}} L_{{\mathfrak {m}}} + \lambda }{n} \right) + \frac{{\bar{r}}^2_{{\mathfrak {m}}}}{n^2} (D_{{\mathfrak {m}}}^2 + D_{{\mathfrak {m}}}^2L_{{\mathfrak {m}}}^2 + \lambda ^2) \right] \\&\lesssim \Vert f \Vert _\infty ^2 \left[ \frac{D_{{\mathfrak {m}}}(1 + L_{{\mathfrak {m}}})}{n} + \frac{\lambda }{n} + \frac{C_{{\bar{r}}}^2 D_{{\mathfrak {m}}}(1+ L_{{\mathfrak {m}}}^2)}{n} + \frac{{\bar{r}}^2_{{\mathfrak {m}}} \lambda ^2}{n^2} \right] \\&\le \kappa _X \Vert f \Vert _\infty ^2 (1 + C_{{\bar{r}}}^2)\frac{D_{{\mathfrak {m}}} (1 + L_{{\mathfrak {m}}})^2}{n} + 2 \left[ \frac{\lambda }{n} \vee \frac{{\bar{r}}^2_{{\mathfrak {m}}} \lambda ^2}{n^2} \right] \end{aligned}$$

for some numerical constant $\kappa _X$. Then,

$$\begin{aligned} {{\textbf {E}}}&\left[ \left( G_X^2({\mathfrak {m}}) - \kappa _X \Vert f \Vert _\infty ^2 (1 + C_{{\bar{r}}}^2)\frac{D_{{\mathfrak {m}}} (1 + L_{{\mathfrak {m}}})^2}{n} \right) _+ \right] \\&= \int _0^\infty {{\textbf {P}}}\left( G_X^2({\mathfrak {m}}) > \kappa _X \Vert f \Vert _\infty ^2 (1 + C_{{\bar{r}}}^2)\frac{D_{{\mathfrak {m}}} (1 + L_{{\mathfrak {m}}})^2}{n} + u \right) \text {d}u\\&\le e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \left( \int _{2\kappa _X \Vert f \Vert _\infty ^2/{\bar{r}}_{{\mathfrak {m}}}^2}^\infty e^{-nu/(2\kappa _X \Vert f \Vert _\infty ^2)} \text {d}u + \int _0^{2\kappa _X \Vert f \Vert _\infty ^2/{\bar{r}}_{{\mathfrak {m}}}^2} e^{-n\sqrt{u} /(2 \sqrt{\kappa _X} {\bar{r}}_{{\mathfrak {m}}} \Vert f \Vert _\infty )} \text {d}u \right) \\&\le e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \cdot \frac{2\kappa _X \Vert f \Vert _\infty ^2}{n} \left( \int _0^\infty e^{-v} \text {d}v + \frac{2{\bar{r}}_{{\mathfrak {m}}}}{n} \int _0^\infty e^{-\sqrt{v}} \text {d}v \right) \\&\lesssim e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \frac{C(C_{{\bar{r}}}, \Vert f \Vert _\infty )}{n} \end{aligned}$$

which is the claim in the Gaussian case.

In the subexponential case, we have

$$\begin{aligned} \left( \sum _{k \ge 0} \eta _k \right) ^2&\scriptstyle \lesssim \left( \frac{\tau _n^2}{\alpha ^2} \sum _k \delta _{k-1} \left[ \sqrt{\frac{H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda }{n}} + \frac{{\bar{r}}_{{\mathfrak {m}}} (H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda )^2}{n} \right] \right) ^2\\&\scriptstyle \hspace{-3em}= \frac{\tau _n^4}{\alpha ^4} \left[ \sum _{k} \delta _{k-1} \sqrt{\frac{H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda }{n}} + \frac{{\bar{r}}_{{\mathfrak {m}}}}{n} \sum _{k} \delta _{k-1} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda )^2 \right] ^2\\&\hspace{-3em}\scriptstyle \lesssim \frac{\tau _n^4}{\alpha ^4} \left( \sum _{k} \delta _{k-1} \sqrt{\frac{H_{k-1} + H_k + k D_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda }{n}} \right) ^2\\&\scriptstyle \hspace{-2em} + \frac{\tau _n^4}{\alpha ^4} \cdot \frac{{\bar{r}}_{{\mathfrak {m}}}^2}{n^2} \cdot \left( \sum _{k} \delta _{k-1} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + \lambda )^2 \right) ^2\\&\hspace{-3em}\scriptstyle \lesssim \frac{\tau _n^4}{n \alpha ^4} \delta _0^2 \sum _{k} 2^{-k} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda )\\&\scriptstyle \hspace{-2em} + \frac{\tau _n^4 {\bar{r}}_{{\mathfrak {m}}}^2}{\alpha ^4 n^2} \delta _0^2 \left[ \sum 2^{-k} (H_{k-1}^2 + H_k^2 + k^2 D_{{\mathfrak {m}}}^2 + L_{{\mathfrak {m}}}^2 D_{{\mathfrak {m}}}^2 + \lambda ^2) \right] ^2\\&\hspace{-3em}\scriptstyle \lesssim \frac{\tau _n^4}{n \alpha ^4} \delta _0^2 \sum _{k} 2^{-k} (H_{k-1} + H_k + kD_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda )\\&\hspace{-2em}\scriptstyle + \frac{\tau _n^4 {\bar{r}}_{{\mathfrak {m}}}^2}{\alpha ^4 n^2} \delta _0^2 \sum _k 2^{-k} (H_{k-1}^2 + H_k^2 + k^2 D_{{\mathfrak {m}}}^2 + L_{{\mathfrak {m}}}^2 D_{{\mathfrak {m}}}^2 + \lambda ^2)^2\\&\hspace{-3em}\scriptstyle \lesssim \frac{\tau _n^4}{n \alpha ^4} \delta _0^2 (D_{{\mathfrak {m}}} \log (1/\delta _0) + L_{{\mathfrak {m}}}D_{{\mathfrak {m}}} + \lambda )\\&\scriptstyle \hspace{-2em}+ \frac{\tau _n^4 {\bar{r}}_{{\mathfrak {m}}}^2}{n^2 \alpha ^4} \delta _0^2 \left[ L_{{\mathfrak {m}}}^2 D_{{\mathfrak {m}}}^2 + D_{{\mathfrak {m}}}^2 \log ^2(1/\delta _0) + \lambda ^2 \right] ^2\\&\hspace{-3em} \scriptstyle \lesssim \frac{\tau _n^4}{n\alpha ^4} \delta _0^2 \left[ L_{{\mathfrak {m}}} D_{{\mathfrak {m}}} + D_{\mathfrak {m}}\log (1/\delta _0) + \lambda \right] \\&\scriptstyle \hspace{-2em}+ \frac{\tau _n^4 C_{{\bar{r}}}^2}{n \alpha ^4} \delta _0^2 L_{\mathfrak {m}}^4 D_{\mathfrak {m}}^3 + \frac{\tau _n^4 C_{{\bar{r}}}^2}{n \alpha ^4} \delta _0^2 \log ^4(1/\delta _0) D_{\mathfrak {m}}^3 + \frac{\lambda ^4 \tau _n^4 {\bar{r}}_{\mathfrak {m}}^2}{\alpha ^4 n^2}\delta _0^2. \end{aligned}$$

Taking $\delta _0 = c/n$, we obtain (note again that we assume $D_{\mathfrak {m}}\le n$ for all ${\mathfrak {m}}\in {\mathcal {M}}_n$)

$$\begin{aligned} \left( \sum _{k \ge 1} \eta _k \right) ^2&\le \kappa _\xi \left\{ \frac{\tau _n^4}{n\alpha ^4} (L_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}^4 + \log ^4(n) ) D_{{\mathfrak {m}}} +\frac{\tau _n^4}{n^3 \alpha ^4} \left[ \lambda \vee \frac{\lambda ^4 {\bar{r}}_{{\mathfrak {m}}}^2}{n} \right] \right\} \end{aligned}$$

for a sufficiently large constant $\kappa _\xi =\kappa _\xi (C_{{\bar{r}}})$. Then,

$$\begin{aligned} {{\textbf {E}}}&\left[ \left( G_\xi ^2({\mathfrak {m}}) - \kappa _\xi \frac{\tau _n^4D_{{\mathfrak {m}}} (L_{{\mathfrak {m}}}^4 + L_{{\mathfrak {m}}} + \log ^4(n) ) }{n\alpha ^4} \right) _+ \right] \\&\le \int _0^\infty {{\textbf {P}}}\left( G_\xi ^2({\mathfrak {m}}) > \kappa _\xi \frac{\tau _n^4D_{{\mathfrak {m}}} (L_{{\mathfrak {m}}}^4 + L_{{\mathfrak {m}}} + \log ^4(n) ) }{n\alpha ^4} + u \right) \text {d}u\\&\le e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \left( \int _{(n/{\bar{r}}_{{\mathfrak {m}}}^2)^{1/3}}^\infty e^{-n\alpha /(\tau _n \sqrt{{\bar{r}}_{{\mathfrak {m}}}}) \cdot (u/(2\kappa _\xi ))^{1/4}} \text {d}u + \int _0^{(n/{\bar{r}}_{{\mathfrak {m}}}^2)^{1/3}} e^{-un^3\alpha ^4/(2\kappa _\xi \tau _n^4)} \text {d}u \right) \\&\le e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \cdot \left( \frac{2\kappa _\xi \tau _n^4{\bar{r}}_{{\mathfrak {m}}}^2}{n^4 \alpha ^4} + \frac{2\kappa _\xi \tau _n^4}{n^3\alpha ^4} \right) \\&\lesssim e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \frac{\tau _n^4}{n^3 \alpha ^4} \end{aligned}$$

which is the claim of Lemma C.2.

In the mixed case, apart from the dependence of the leading numerical constant on $\Vert f\Vert _\infty$ and the different dependence in terms of $\alpha$ (which is hidden in the quantity M), the obtained expressions for $\eta _k$, $k\ge 0$ are the same as in the subexponential case. Taking $\delta _0 = c/n$ for some numerical constant $0< c < 1/5$ again, we thus obtain

$$\begin{aligned} \left( \sum _{k \ge 1} \eta _k \right) ^2&\le {\widetilde{\kappa }}M^4 (1+ \Vert f \Vert _\infty )^2 \left\{ \frac{D_{{\mathfrak {m}}}}{n} ( L_{{\mathfrak {m}}} + L_{{\mathfrak {m}}}^4 + \log (n) ) + + \frac{1}{n^3} \left[ \lambda \vee \frac{\lambda ^4 {\bar{r}}_{{\mathfrak {m}}}^2}{n} \right] \right\} . \end{aligned}$$

A calculation similar to the one for the sub-exponential case yields

$$\begin{aligned}&{{\textbf {E}}}\left[ \left( ({\widetilde{G}}({\mathfrak {m}}))^2 - {\widetilde{\kappa }}M^4 (1+\Vert f \Vert _\infty )^2 (L_{\mathfrak {m}}+ L_{{\mathfrak {m}}}^4 + \log (n) ) \frac{D_{{\mathfrak {m}}}}{n} \right) _+ \right] \\&\quad \le e^{-L_{{\mathfrak {m}}}D_{{\mathfrak {m}}}} \frac{C(C_{{\bar{r}}}, \Vert f \Vert _\infty ) M^4}{n^3}. \end{aligned}$$

1.4 Auxiliary results

1.4.1 Gaussian terms

Proposition D.2

Put $\Xi _n^X(u) = \langle I_n^X - {{\textbf {E}}}I_n^X, u \rangle$. For any symmetric function u,

$$\begin{aligned} {{\textbf {P}}}(\Xi _n^X(u) \ge t) \le 2 \exp \left[ - c \min \left( \frac{4\pi ^2nt^2}{9 \Vert f\Vert _\infty ^2 \Vert u\Vert ^2}, \frac{2\pi nt}{3 \Vert f \Vert _\infty \Vert u \Vert _\infty } \right) \right] . \end{aligned}$$

Proof

Denote $X=(X_1,\ldots ,X_n)^\top$. First, we can write

$$\begin{aligned} \Xi _n^X(u) = \frac{1}{2\pi n} [ (X - {\bar{X}}_n {\textbf{1}})^\top T_n(u) (X - {\bar{X}}_n {\textbf{1}}) - {{\textbf {E}}}(X - {\bar{X}}_n {\textbf{1}})^\top T_n(u) (X - {\bar{X}}_n {\textbf{1}}) ]. \end{aligned}$$

Let H be the hyperplane orthogonal to the linear subspace generated by the vector ${\textbf{1}}$ in ${\mathbb {R}}^n$. Note that $X- {\bar{X}}_n {\textbf{1}} = P_H X = P_H \Sigma _X^{1/2} Y$ where $Y \sim {\mathcal {N}}({\textbf{0}}, E_n)$ and $\Sigma _X$ is the covariance matrix of $X_{1:n}$. Now, we the Hanson–Wright inequality (Proposition E.1) with $A = (\Sigma _X^{1/2})^\top P_H^\top T_n(u) P_H \Sigma _X^{1/2}$. Since the $Y_i$ are i.i.d. $\sim {\mathcal {N}}(0,1)$, we have $\Vert Y_i \Vert _{\psi _2} \le \sqrt{8/3} \le \sqrt{3} = K$. For the given choice of A, we need to bound the quantities $\Vert A \Vert _{\text {HS}}$ and $\Vert A \Vert _{\text {op}}$ appearing on the right-hand side of the Hanson–Wright inequality. First,

$$\begin{aligned} \Vert A \Vert _{\text {HS}}^2&= \text {tr}(A^\top A) =\text {tr}(\Sigma _X^{1/2} P_H^\top T_n(u) P_H \Sigma _X P_H^\top T_n(u) P_H \Sigma _X^{1/2})\\&= \text {tr}( P_H \Sigma _X P_H^\top T_n(u) P_H \Sigma _X P_H^\top T_n(u))\\&\le \Vert f \Vert _\infty ^2 \cdot \text {tr}( T_n(u)^2)\\&\le n \Vert f \Vert _\infty ^2 \Vert u \Vert ^2, \end{aligned}$$

where we have used the bound $\text {tr}((AB)^2) \le \rho (A)^2 \text {tr}(B^2)$, and the fact that $\text {tr}(T_n(u)^2) \le n \Vert u \Vert ^2$ from p. 284 in Comte (2001). Second,

$$\begin{aligned} \Vert A \Vert _{\text {op}}&= \Vert \Sigma _X^{1/2} P_H^\top T_n(u) P_H \Sigma _X^{1/2} \Vert _{\text {op}}\\&\le \Vert \Sigma _X^{1/2} \Vert _{\text {op}} \cdot \Vert T_n(u) \Vert _{\text {op}} \cdot \Vert \Sigma _X^{1/2} \Vert _{\text {op}}\\&= \Vert \Sigma _X \Vert _{\text {op}} \cdot \Vert T_n(u) \Vert _{\text {op}}\\&= \rho (\Sigma _X) \cdot \rho (T_n(u))\\&\le \Vert f \Vert _\infty \cdot \Vert u \Vert _\infty . \end{aligned}$$

Using these estimates, application of the Hanson–Wright inequality (Proposition E.1) yields

$$\begin{aligned} {{\textbf {P}}}( \Xi ^X(u) \ge t ) \le 2 \exp \left[ - c \min \left( \frac{4\pi ^2nt^2}{9 \Vert f\Vert _\infty ^2 \Vert u\Vert ^2}, \frac{2\pi nt}{3\Vert f \Vert _\infty \Vert u \Vert _\infty } \right) \right] . \end{aligned}$$

$\square$

1.4.2 Subexponential terms

Proposition D.3

Let $\Xi _n^\xi (u) = \langle I_n^\xi - {{\textbf {E}}}I_n^\xi , u \rangle$. For any symmetric function u,

$$\begin{aligned} {{\textbf {P}}}(\Xi _n^\xi (u) \ge t) \le 2 \exp \left( - \frac{1}{C} \min \left( \frac{\pi ^2 n t^2 \alpha ^4}{64 \tau _n^4 \Vert u \Vert ^2}, \frac{\sqrt{2\pi nt}\alpha }{4\tau _n \Vert u \Vert _\infty ^{1/2}} \right) \right) . \end{aligned}$$

Proof

Let H be the hyperplane orthogonal to the space generated by the vector ${\textbf{1}}$ in ${\mathbb {R}}^n$. Then, for $\xi = (\xi _1,\ldots ,\xi _n)^\top$, $\xi - {\overline{\xi }}_n {\textbf{1}} = P_H \xi$. We have

$$\begin{aligned} \Xi _n^\xi (u)&= \frac{1}{2\pi n} [(P_H \xi )^\top T_n(u) P_H \xi - {{\textbf {E}}}(P_H \xi )^\top T_n(u) P_H \xi ]. \end{aligned}$$

We will now use Proposition E.2 from Appendix E which is taken from Götze et al. (2021). More precisely, we would like to apply this result with our $\xi _i$ playing the role of the $X_i$, with $A = P_H^\top T_n(u) P_H$, and $\beta = 1$. We have ${{\textbf {E}}}\xi _i^2 = \sigma _i^2 = 8\tau _n^2/\alpha ^2$ for all $i \in \llbracket 1, n\rrbracket$. Moreover $\Vert \xi _i \Vert _{\Psi _1} \le 4\tau _n / \alpha$ which will play the role of M. The last estimate is easily derived using the fact that $|\xi _i |$ obeys an exponential distribution with parameter $\lambda = \alpha /(2\tau _n)$ and then considering the moment generating function for the exponential distribution. It remains to bound the quantities $\Vert A \Vert _{\text {HS}}$ and $\Vert A \Vert _{\text {op}}$. First,

$$\begin{aligned} \Vert A \Vert _{\text {HS}}^2 = \text {tr}(A^\top A)&= \text {tr}(P_H^\top T_n(u) P_H P_H^\top T_n(u) P_H)\\&= \text {tr}((P_H P_H^\top T_n(u))^2) \qquad [\text {cyclic property}]\\&= \rho (P_H P_H^\top )^2 \cdot \text {tr}(T_n(u)^2) \qquad [\text {since } \text {tr}((MN)^2) \le \rho (M)^2 \text {tr}(N^2)]\\&\le \text {tr}(T_n(u)^2). \end{aligned}$$

Using the same argument as on p. 284 in Comte (2001), we have $\text {tr}(T_n(u)^2) \le n \Vert u \Vert ^2$, and hence

$$\begin{aligned} \Vert A \Vert _{\text {HS}}^2 \le n \Vert u \Vert ^2. \end{aligned}$$

Second, for $\Vert A \Vert _{\text {op}}$ have the bound

$$\begin{aligned} \Vert A \Vert _{\text {op}} = \rho (A) \le \rho (T_n(u)) \le \Vert u \Vert _\infty . \end{aligned}$$

Thus, we finally obtain

$$\begin{aligned} {{\textbf {P}}}(\Xi _n^\xi (u) \ge t) \le 2 \exp \left( - \frac{1}{C} \min \left( \frac{\pi ^2 n t^2 \alpha ^4}{64 \tau _n^4 \Vert u \Vert ^2}, \frac{\sqrt{2\pi nt}\alpha }{4\tau _n \Vert u \Vert _\infty ^{1/2}} \right) \right) \end{aligned}$$

which is the claim assertion. $\square$

1.4.3 Mixed terms

Proposition D.4

For any symmetric function u,

$$\begin{aligned} {{\textbf {P}}}\left( {\widetilde{\Xi }}_n(u) \ge t \right) \le 2 \exp \left( - \frac{1}{C} \min \left( \frac{t^2 n}{2M^4 \Vert u \Vert ^2 \cdot \Vert f\Vert _\infty }, \left( \frac{nt}{M^2 \Vert u\Vert _\infty \cdot \Vert f \Vert _\infty ^{1/2}} \right) ^{1/2} \right) \right) \end{aligned}$$

where $M=3+4\tau _n/\alpha$.

Proof

In order to deal with the mixed term, we first write

$$\begin{aligned} \begin{pmatrix} X\\ \xi \end{pmatrix} = \begin{pmatrix} \sqrt{\Sigma _{{\textbf {X}}}} &{} {\varvec{0}}_{n}\\ {\varvec{0}}_{n} &{} {{\textbf {E}}}_{n} \end{pmatrix} \begin{pmatrix} Y\\ \xi \end{pmatrix} \end{aligned}$$

where $Y = (Y_1,\ldots ,Y_n)^\top$ is a vector of i.i.d. standard Gaussian random variables. Then, the term of interest can be written as

$$\begin{aligned} \begin{pmatrix} X^\top&\xi ^\top \end{pmatrix} \begin{pmatrix} {\varvec{0}}_{n} &{} T_n(u)\\ T_n(u) &{} {\varvec{0}}_{n} \end{pmatrix} \begin{pmatrix} X\\ \xi \end{pmatrix}&= \begin{pmatrix} Y^\top&\xi ^\top \end{pmatrix} \begin{pmatrix} \sqrt{\Sigma _{{\textbf {X}}}} &{} {\varvec{0}}_{n}\\ {\varvec{0}}_{n} &{} {{\textbf {E}}}_{n} \end{pmatrix} \begin{pmatrix} {\varvec{0}}_{n} &{} T_n(u)\\ T_n(u) &{} {\varvec{0}}_{n} \end{pmatrix} \begin{pmatrix} \sqrt{\Sigma _{{\textbf {X}}}} &{} {\varvec{0}}_{n}\\ {\varvec{0}}_{n} &{} {{\textbf {E}}}_{n} \end{pmatrix} \begin{pmatrix} Y\\ \xi \end{pmatrix}\\&= \begin{pmatrix} Y^\top&\xi ^\top \end{pmatrix} \begin{pmatrix} {\varvec{0}}_{n} &{} \sqrt{\Sigma _{{\textbf {X}}}} T_n(u) \\ T_n(u) \sqrt{\Sigma _{{\textbf {X}}}} &{} {\varvec{0}}_{n} \end{pmatrix} \begin{pmatrix} Y \\ \xi \end{pmatrix} \\&=:\begin{pmatrix} Y^\top&\xi \end{pmatrix} A \begin{pmatrix} Y\\ \xi \end{pmatrix}. \end{aligned}$$

Since all components of the vector $(Y^\top \, \xi ^\top )$ are independent, and the matrix A is symmetric, we can apply Proposition E.2 again with $\beta = 1$ as in the proof of Proposition D.3. We have ${{\textbf {E}}}Y_i^2 = 1$, ${{\textbf {E}}}\xi _i^2 = 8\tau _n^2/\alpha ^2$. As seen above $\Vert \xi _i \Vert _{\psi _1} \le 4\tau _n/\alpha$ and moreover $\Vert Y_i \Vert _{\psi _1} \le \Vert 1 \Vert _{\psi _2} \cdot \Vert Y_i \Vert _{\psi _2} \le (\log 2)^{-1/2} \cdot \sqrt{3} \le 3$. Hence, we can take $M = 3 + 4\tau _n/\alpha$. Application of Proposition (Götze et al. 2021) yields

$$\begin{aligned} {{\textbf {P}}}\left( {\widetilde{\Xi }}_n(u) \ge t \right) \le 2 \exp \left( - \frac{1}{C} \min \left( \frac{4\pi ^2 t^2 n^2}{M^4 \Vert A \Vert _{\text {HS}}^2}, \left( \frac{2 \pi nt}{M^2 \Vert A \Vert _{\text {op}}} \right) ^{1/2} \right) \right) , \end{aligned}$$

and we have to find appropriate bounds for the quantities $\Vert A \Vert _{\text {HS}}$ and $\Vert A \Vert _{\text {op}}$. Now, using the estimate H.1.g in Section II.9 from Marshall et al. (2011), p. 341, we have

$$\begin{aligned} \Vert A \Vert _{\text {HS}}^2&= \text {tr}(A^\top A) = \text {tr}\begin{pmatrix} \sqrt{\Sigma _{{\textbf {X}}}} T_n(u)^2 \sqrt{\Sigma _{{\textbf {X}}}} &{} {\varvec{0}}_{n}\\ {\varvec{0}}_{n} &{} T_n(u) \Sigma _{{\textbf {X}}}T_n(u) \end{pmatrix}\\&= \text {tr}(\sqrt{\Sigma _{{\textbf {X}}}} T_n(u)^2 \sqrt{\Sigma _{{\textbf {X}}}}) + \text {tr}(T_n(u) \Sigma _{{\textbf {X}}}T_n(u))\\&= 2 \text {tr}(\Sigma _X T_n(u)^2)\\&\le 2 n \Vert u \Vert ^2 \cdot \Vert f \Vert _\infty . \end{aligned}$$

Finally, in order to bound $\Vert A \Vert _{\text {op}}$, note that

$$\begin{aligned} \Vert A \Vert _{\text {op}}&\le \Vert T_n(u)\Vert _{\text {op}} \cdot \Vert \sqrt{\Sigma _X} \Vert _{\text {op}}\\&\le \Vert u\Vert _\infty \cdot \Vert f \Vert _\infty ^{1/2}. \end{aligned}$$

$\square$

Auxiliary results

Proposition E.1

(Hanson–Wright inequality, Rudelson and Vershynin (2013, Theorem 1.1)) Let $X=(X_1,\ldots ,X_n) \in {\mathbb {R}}^n$ be a random vector with independent components $X_i$ which satisfy ${{\textbf {E}}}X_i = 0$ and $\Vert X_i \Vert _{\psi _2} \le K$. Let A be an $n \times n$-matrix. Then, for every $t \ge 0$,

$$\begin{aligned} {{\textbf {P}}}\left( |X^\top A X - {{\textbf {E}}}X^\top A X |> t \right) \le 2 \exp \left[ - c \min \left( \frac{t^2}{K^4 \Vert A \Vert ^2_{\text {HS}}}, \frac{t}{K^2 \Vert A \Vert _{\text {op}}} \right) \right] . \end{aligned}$$

The following result generalizes Proposition E.1 because it can also deal with other exponential Orlicz norms than $\Vert \cdot \Vert _{\psi _2}$. This permits to apply the result to subexponential random variables as the Laplace noise used for our anonymization algorithm.

Proposition E.2

(Götze et al. 2021, Proposition 1.1) Let $X_1,\ldots ,X_n$ be independent random variables satisfying ${{\textbf {E}}}X_i = 0$, ${{\textbf {E}}}X_i^2 = \sigma _i^2$, $\Vert X_i \Vert _{\psi _\beta } \le M$ for some $\beta \in (0,1] \cup \{2\}$, and A be a symmetric $n \times n$ matrix. For any $t > 0$ we have

$$\begin{aligned} {{\textbf {P}}}\left( |\sum _ {i,j} a_{ij}X_i X_j - \sum _{i=1}^{n} \sigma _i^2 a_{ii} |\ge t \right) \le 2 \exp \left( - \frac{1}{C} \min \left( \frac{t^2}{M^4 \Vert A \Vert ^2_{\text {HS}}}, \left( \frac{t}{M^2 \Vert A \Vert _{\text {op}}} \right) ^{\beta /2} \right) \right) . \end{aligned}$$