Parametric estimation of hidden stochastic model by contrast minimization and deconvolution

Abstract

We study a new parametric approach for particular hidden stochastic models. This method is based on contrast minimization and deconvolution and can be applied, for example, for ecological and financial state space models. After proving consistency and asymptotic normality of the estimation leading to asymptotic confidence intervals, we provide a thorough numerical study, which compares most of the classical methods that are used in practice (Quasi-Maximum Likelihood estimator, Simulated Expectation Maximization Likelihood estimator and Bayesian estimators) to estimate the Stochastic Volatility model. We prove that our estimator clearly outperforms the Maximum Likelihood Estimator in term of computing time, but also most of the other methods. We also show that this contrast method is the most robust with respect to non Gaussianity of the error and also does not need any tuning parameter.

Appendices

Appendix A: M-estimator

Definition 1

Geometrical ergodic process

Denote by $Q^n(x,.)$ the transition kernel at step $n$ of a (discrete-time) stationary Markov chain $(X_n)_n$ which started at $x$ at time $0$. That is, $Q^n(x,F) = \mathbb P (X_n \in F | X_0 =x)$. Let $\pi$ denote the stationary law of $X_n$ and let $f$ be any measurable function. We call mixing coefficients $(\beta _n)_n$ the coefficients defined by, for each $n$:

$$\begin{aligned} \beta _n = \int \left[ \sup _{||f||_{\infty }\le 1}\left| Q^n(x,f) - \pi (f) \right| \right] \pi (dx), \end{aligned}$$

where $\pi (f) = \int f(y) \pi (dy)$. We say that a process is geometrically ergodic if the decreasing of the sequence of the mixing coefficients $(\beta _n)_n$ is geometrical, that is:

$$\begin{aligned} \exists \ 0<\eta <1, \text{ such} \text{ that} \beta _n \le \eta ^n. \end{aligned}$$

The following results are the main tools for the proof of Theorem 1.

Consider the following quantities:

$$\begin{aligned} \mathbf P _{n}h_{\theta }=\frac{1}{n}\sum _{i=1}^{n}h_{\theta }(Y_i);\quad \mathbf P _{n}S_{\theta }=\frac{1}{n}\sum _{i=1}^{n}\nabla _{\theta }h_{\theta }(Y_i) \quad \text{ and} \quad \mathbf P _{n}H_{\theta }=\frac{1}{n}\sum _{i=1}^{n}\nabla ^{2}_{\theta }h_{\theta }(Y_i) \end{aligned}$$

where $h_{\theta }(y)$ is real function from $\varTheta \times \fancyscript{Y}$ with value in $\mathbb{R }$.

Lemma 1

Uniform Law of Large Numbers (ULLN) (see Newey and McFadden 1994 for the proof).      

Let $(Y_i)$ be an ergodic stationary process and suppose that:

• 1. $h_{\theta }(y)$ is continuous in $\theta$ for all $y$ and measurable in $y$ for all $\theta$ in the compact subset $\varTheta$.

• 2. There exists a function $s(y)$(called the dominating function) such that $\left| h_{\theta }(y)\right| \le s(y)$ for all $\theta \in \varTheta$ and $\mathbb{E }[s(Y_1)]<\infty$. Then:

$$\begin{aligned} \sup _{\theta \in \varTheta }\left| \mathbf P _{n}h_{\theta }-\mathbf P h_{\theta }\right| \rightarrow 0 \quad \text{ in} \text{ probability} \text{ as} \text{ n} \rightarrow \infty . \end{aligned}$$

Moreover, $\mathbf P h_{\theta }$ is a continuous function of $\theta$.

Proposition 1

(Proposition 7.8 p. 472 in Hayashi (2000). The proof is in Newey (1987) Theorem 4.1.5.)

Suppose that:

• 1. $\theta _0$ is in the interior of $\varTheta$.

• 2. $h_{\theta }(y)$ is twice continuously differentiable in $\theta$ for any $y$.

• 3. The Hessian matrix of the application $\theta \mapsto \mathbf P h_{\theta }$ is non-singular.

• 4. $\sqrt{n}\mathbf P _{n}S_{\theta } \stackrel{}{\rightarrow } \fancyscript{N}(0, \varOmega (\theta _0))$ in law as n $\rightarrow \infty$, with $\varOmega (\theta _0)$ a positive definite matrix.

• 5. Local dominance on the Hessian: for some neighbourhood $\fancyscript{U}$ of $\theta _0$:

  $$\begin{aligned} \mathbb{E }\left[ \sup _{\theta \in \fancyscript{U} }\left\| \nabla _{\theta }^{2}h_{\theta }(Y_{1})\right\| \right] <\infty , \end{aligned}$$

so that, for any consistent estimator $\hat{\theta }$ of $\theta _0$ we have: $\mathbf P _{n}H_{\hat{\theta }} \rightarrow \mathbb{E }[\nabla ^{2}_{\theta }h_{\theta }(Y_1)]$ in probability as n $\rightarrow \infty$.

Then, $\hat{\theta }$ is asymptotically normal with asymptotic covariance matrix given by:

$$\begin{aligned} \varSigma (\theta _{0})=\mathbb{E }[\nabla ^{2}_{\theta }h_{\theta }(Y_1)]^{-1} \varOmega (\theta _{0})\mathbb{E }[\nabla ^{2}_{\theta }h_{\theta }(Y_1)]^{-1} \end{aligned}$$

where the differential $\nabla ^{2}_{\theta }h_{\theta }(Y_1)$ is taken at point $\theta =\theta _0$.

Proposition 2

(The proof is in Jones 2004)

Let $Y_i$ be an ergodic stationary Markov chain and let g: $\fancyscript{Y}\, \rightarrow \, \mathbb R$ a borelian function. Suppose that $Y_i$ is geometrically ergodic and $\mathbb{E }\left[ |g(Y_1)|^{2+\delta }\right] <\infty$ for some $\delta >0$. Then, when $n \rightarrow \infty$,

$$\begin{aligned} \sqrt{n}(\mathbf P _{n}g-\mathbf P g)\stackrel{}{\rightarrow }\fancyscript{N}(0,\sigma ^{2}_{g}) \text{ in} \text{ law,} \end{aligned}$$

where $\sigma ^{2}_{g}:=Var\left[ (g(Y_{1})\right] +2\sum _{j=1}^{\infty }Cov \left( g(Y_{1}), g(Y_{j})\right) <\infty$.

Appendix B: Proofs of theorem 1

For the reader convenience we split the proof of Theorem 1 into three parts: in Sect. 2.7, we give the proof of the existence of our contrast estimator defined in (1.1). In Sect. 2.7, we prove the consistency, that is, the first part of Theorem 1. Then, we prove the asymptotic normality of our estimator in Sect. 2.7, that is, the second part of Theorem 1. The Sect. 2.7 is devoted to Corollary 1. Finally, in Sect. 2.7 we prove that Theorem 1 applies for the AR(1) and SV models.

1.1 B.1 Proof of the existence and measurability of the M-estimator

By assumption, the function $\theta \mapsto \left\| l_{\theta }\right\| _{2}^2$ is continuous. Moreover, $l^{*}_{\theta }$ and then $u^{*}_{l_{\theta }}(x)=\frac{1}{2\pi }\int e^{ixy}\frac{l^{*}_{\theta }(-y)}{f^{*}_{\varepsilon }(y)}dy$ are continuous w.r.t $\theta$. In particular, the function $m_{\theta }(\mathbf{y }_{i})=\left\| l_{\theta }\right\| _{2}^2-2y_{i+1}u^{*}_{l_{\theta }}(y_{i})$ is continuous w.r.t $\theta$. Hence, the function $\mathbf P _{n}m_{\theta }=\frac{1}{n}\sum _{i=1}^{n}m_{\theta }(\mathbf{Y }_i)$ is continuous w.r.t $\theta$ belonging to the compact subset $\varTheta$. So, there exists $\tilde{\theta }$ belongs to $\varTheta$ such that:

$$\begin{aligned} \inf _{\theta \in \varTheta }\mathbf P _{n}m_{\theta }=\mathbf P _{n}m_{\tilde{\theta }}.\square \end{aligned}$$

1.2 B.2 Proof of the consistency

By assumption $l_{\theta }$ is continuous w.r.t $\theta$ for any $x$ and measurable w.r.t $x$ for all $\theta$ which implies the continuity and the measurability of the function $\mathbf P _{n}m_{\theta }$ on the compact subset $\varTheta$. Furthermore, the local dominance assumption (C) implies that $\mathbb{E }\left[ \sup _{\theta \in \varTheta }\left| m_{\theta }(\mathbf{Y }_i)\right| \right]$ is finite. Indeed,

$$\begin{aligned} \left| m_{\theta }(\mathbf{y }_{i})\right|&= \left| \left\| l_{\theta }\right\| _{2}^2-2y_{i+1}u^{*}_{l_{\theta }}(y_{i})\right| \\&\le \left\| l_{\theta }\right\| _{2}^2+2\left| y_{i+1}u^{*}_{l_{\theta }}(y_{i})\right| . \end{aligned}$$

As $\left\| l_{\theta }\right\| _{2}^2$ is continuous on the compact subset $\varTheta ,\, \sup _{\theta \in \varTheta }\left\| l_{\theta }\right\| _{2}^2$ is finite. Therefore, $\mathbb{E }\left[ \sup _{\theta \in \varTheta }\left| m_{\theta }(\mathbf{Y }_i)\right| \right]$ is finite if $\mathbb{E }\left[ \sup _{\theta \in \varTheta }\left| Y_{i+1}u^{*}_{l_{\theta }}(Y_i)\right| \right]$ is finite. Lemma ULLN 1 gives us the uniform convergence in probability of the contrast function: for any $\varepsilon >0$:

$$\begin{aligned} \lim _{n\rightarrow +\infty } \mathbb P \left( \sup _{\theta \in \varTheta }\left| \mathbf P _{n}m_{\theta }-\mathbf P m_{\theta }\right| \le \varepsilon \right) =1. \end{aligned}$$

Combining the uniform convergence with Theorem 2.1 p. 2121 chapter 36 in Hansen and Horowitz (1997) yields the weak (convergence in probability) consistency of the estimator.$\square$

Remark 5

In most applications, we do not know the bounds for the true parameter. So the compactness assumption is sometimes restrictive, one can replace the compactness assumption by: $\theta _0$ is an element of the interior of a convex parameter space $\varTheta \subset \mathbb R ^{r}$. Then, under our assumptions except the compactness, the estimator is also consistent. The proof is the same and the existence is proved by using convex optimization arguments. One can refer to Hayashi (2000) for this discussion.

1.3 B.3 Proof of the asymptotic normality

The proof is based on the following Lemma:

Lemma 2

Suppose that the conditions of the consistency hold. Suppose further that:

• 1. $\mathbf{Y }_i$ geometrically ergodic.

• 2. (Moment condition): for some $\delta >0$ and for each $j\in \left\{ 1,\ldots ,r\right\} \!:$

  $$\begin{aligned} \mathbb{E }\left[ \left| \frac{{\partial }m_{\theta }(\mathbf{Y }_{1})}{{\partial }\theta _j}\right| ^{2+\delta }\right] <\infty . \end{aligned}$$

• 3. (Hessian Local condition): For some neighbourhood $\fancyscript{U}$ of $\theta _0$ and for $j,k\in \left\{ 1,\ldots , r\right\} \!:$

  $$\begin{aligned} \mathbb{E }\left[ \sup _{\theta \in \fancyscript{U} }\left| \frac{{\partial }^2m_{\theta }(\mathbf{Y }_{1})}{{\partial }\theta _j {\partial }\theta _k}\right| \right] < \infty . \end{aligned}$$

  Then, $\widehat{\theta }_{n}$ defined in Eq. (9) is asymptotically normal with asymptotic covariance matrix given by:

$$\begin{aligned} \varSigma (\theta _{0})=V_{\theta _0}^{-1} \varOmega (\theta _{0})V_{\theta _0}^{-1} \end{aligned}$$

where $V_{\theta _0}$ is the Hessian of the application $\mathbf P m_{\theta }$ given in Eq. (7).

Proof

The proof follows from Proposition 1 and Proposition 2 and by using the fact that by assumption we have $\mathbb{E }[\nabla _{\theta }^{2}m_{\theta }(\mathbf{Y }_1)]=\nabla _{\theta }^{2}\mathbb{E }[m_{\theta }(\mathbf{Y }_1)]$. $\square$

It just remains to check that the conditions (2) and (3) of Lemma 2 hold under our assumptions (T).

Moment condition: As the function $l_{\theta }$ is twice continuously differentiable w.r.t $\theta$, for all $\mathbf{y }_{i} \, \in \, \mathbb{R }^2$, the application $m_{\theta }(\mathbf{y }_{i}):\ \theta \in \varTheta \mapsto m_{\theta }(\mathbf{y }_{i})=||l_{\theta }||_{2}^2 - 2y_{i+1}u^*_{l_{\theta }}(y_{i})$ is twice continuously differentiable for all $\theta \, \in \, \varTheta$ and its first derivatives are given by:

$$\begin{aligned} \nabla _{\theta }m_{\theta }(\mathbf{y }_{i})= \nabla _{\theta }||l_{\theta }||_{2}^2 - 2y_{i+1}\nabla _{\theta }u^*_{l_{\theta }}(y_{i}). \end{aligned}$$

By assumption, for each $j\in \left\{ 1,\ldots ,r\right\} ,\, \frac{{\partial }l_{\theta }}{{\partial }\theta _j} \in \mathbb L _{1}(\mathbb{R })$, therefore one can apply the Lebesgue Derivation Theorem and Fubini’s Theorem to obtain:

$$\begin{aligned} \nabla _{\theta }m_{\theta }(\mathbf{y }_{i}) =\left[ \nabla _{\theta }||l_{\theta }||_{2}^2 - 2y_{i+1}u^*_{\nabla _{\theta }l_{\theta }}(y_{i}) \right] . \end{aligned}$$

(16)

Then, for some $\delta >0$:

$$\begin{aligned} \left| \nabla _{\theta }m_{\theta }(\mathbf{y }_{i})\right| ^{2+\delta }&= \left| \nabla _{\theta }||l_{\theta }||_{2}^2 - 2y_{i+1}u^*_{\nabla _{\theta }l_{\theta }}(y_{i}) \right| ^{2+\delta }\nonumber \\&\le C_{1}\left| \nabla _{\theta }||l_{\theta }||_{2}^2\right| ^{2+\delta }+C_{2}\left| y_{i+1}u^*_{\nabla _{\theta }l_{\theta }}(y_{i}) \right| ^{2+\delta }, \end{aligned}$$

(17)

where $C_{1}$ and $C_{2}$ are two positive constants. By assumption, the function $||l_{\theta }||_{2}^2$ is twice continuously differentiable w.r.t $\theta$. Hence, $\nabla _{\theta }||l_{\theta }||_{2}^2$ is continuous on the compact subset $\varTheta$ and the first term of Eq. (17) is finite. The second term is finite by the moment assumption (T).

Hessian Local dominance: For $j,k \in \left\{ 1,\ldots ,r\right\} ,\, \frac{{\partial }^2 l_{\theta }}{{\partial }\theta _j {\partial }\theta _k} \in \mathbb L _{1}(\mathbb{R })$, the Lebesgue Derivation Theorem gives:

$$\begin{aligned} \nabla ^{2}_{\theta }m_{\theta }(\mathbf{y }_{i})=\nabla ^{2}_{\theta }||l_{\theta }||_{2}^2 - 2y_{i+1}u^*_{\nabla ^{2}_{\theta }l_{\theta }}(y_{i}), \end{aligned}$$

and, for some neighbourhood $\fancyscript{U}$ of $\theta _0$:

$$\begin{aligned} \mathbb{E }\left[ \sup _{\theta \in \fancyscript{U} }\left\| \nabla _{\theta }^{2}m_{\theta }(\mathbf{Y }_i)\right\| \right] \le \sup _{\theta \in \fancyscript{U}}\left\| \nabla ^{2}_{\theta }||l_{\theta }||_{2}^2\right\| +2\mathbb{E }\left[ \sup _{\theta \in \fancyscript{U}}\left\| Y_{i+1}u^{*}_{\nabla _{\theta }^{2}l_{\theta }}(Y_i)\right\| \right] . \end{aligned}$$

The first term of the above equation is finite by continuity and by compactness argument. And, the second term is finite by the Hessian local dominance assumption (T). $\square$

1.4 B.4 Proof of corollary 1

By replacing $\nabla _{\theta }m_{\theta }(\mathbf{Y }_{1})$ by its expression (16), we have:

$$\begin{aligned} \varOmega _{0}(\theta )&= \mathbb{V }ar\left[ \nabla _{\theta }||l_{\theta }||_{2}^2 - 2Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right] \\&= 4 \mathbb{V }ar\left[ Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right] \\&= 4\left[ \mathbb{E }\left[ Y_{2}^2 \left( u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right) \left( u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right) ^{\prime }\right] -\mathbb{E }\left[ Y_{2} u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})\right] \mathbb{E }\left[ Y_{2} u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})\right] ^{\prime }\right] . \end{aligned}$$

Furthermore, by Eq. (1) and by independence of the centered noise $(\varepsilon _2)$ and $(\eta _2)$, we have:

$$\begin{aligned} \mathbb{E }\left[ Y_{2} u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})\right] = \mathbb{E }\left[ b_{\phi _{0}}(X_1) u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})\right] \!. \end{aligned}$$

Using Fubini’s Theorem and Eq. (1) we obtain:

$$\begin{aligned} \mathbb{E }\left[ b_{\phi _{0}}(X_1) u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})\right]&= \mathbb{E }\left[ b_{\phi _{0}}(X_1) \int e^{iY_{1}z} u_{\nabla _{\theta }l_{\theta }}(z) dz \right] \nonumber \\&= \mathbb{E }\left[ b_{\phi _{0}}(X_1) \int \frac{1}{2\pi }\frac{1}{f_{\varepsilon }^*(z)}e^{iY_{1}z} (\nabla _{\theta }l_{\theta })^*(-z)dz \right] \nonumber \\&= \frac{1}{2\pi } \int \mathbb{E }\left[ b_{\phi _{0}}(X_1)e^{i(X_1+\varepsilon _1)z} \right] \frac{1}{f_{\varepsilon }^*(z)} (\nabla _{\theta }l_{\theta })^*(-z) dz \nonumber \\&= \frac{1}{2\pi } \int \frac{\mathbb{E }\left[ e^{i\varepsilon _1z}\right] }{f_{\varepsilon }^*(z)} \mathbb{E }\left[ b_{\phi _{0}}(X_1)e^{iX_1z}\right] (\nabla _{\theta }l_{\theta })^*(-z) dz\nonumber \\&= \frac{1}{2\pi } \mathbb{E }\left[ b_{\phi _{0}}(X_1) \int e^{iX_1z} (\nabla _{\theta }l_{\theta })^*(-z) dz \right] \nonumber \\&= \frac{1}{2\pi } \mathbb{E }\left[ b_{\phi _{0}}(X_1) \left( (\nabla _{\theta }l_{\theta })^*(-X_1)\right) ^*\right] \nonumber \\&= \mathbb{E }\left[ b_{\phi _{0}}(X_1) \nabla _{\theta }l_{\theta }(X_1)\right] . \end{aligned}$$

(18)

Hence,

$$\begin{aligned} \varOmega _{0}(\theta )=4\left( P_{2}-P_{1}\right) \!, \end{aligned}$$

where

$$\begin{aligned}&P_{1}=\mathbb{E }\left[ b_{\phi _{0}}(X_1) \nabla _{\theta }l_{\theta }(X_1)\right] \mathbb{E }\left[ b_{\phi _{0}}(X_1) \nabla _{\theta }l_{\theta }(X_1)\right] ^{\prime },\\&P_2=\mathbb{E }\left[ Y_{2}^2 \left( u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right) \left( u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right) ^{\prime }\right] \!. \end{aligned}$$

Calculus of the covariance matrix of Corollary (1): By replacing $(\nabla _{\theta }m_{\theta }(Y_{1}))$ by its expression (16) we have:

$$\begin{aligned} \varOmega _{j-1}(\theta )&= {\mathbb{C }ov}\left( \!\nabla _{\theta }||l_{\theta }||_{2}^2 - 2Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}), \nabla _{\theta }||l_{\theta }||_{2}^2 - 2Y_{j+1}u^*_{\nabla _{\theta }l_{\theta }}(Y_{j}) \right) \!,\\&= 4{\mathbb{C }ov}\left( Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}), Y_{j+1}u^*_{\nabla _{\theta }l_{\theta }}(Y_{j})\right) ,\\&= 4\left[ \!\mathbb{E }\left( \!Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})Y_{j+1}u^*_{\nabla _{\theta }l_{\theta }}(Y_{j})\!\right) \!-\!\mathbb{E }\left( \!Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1})\!\right) \mathbb{E }\left( \!Y_{j+1}u^*_{\nabla _{\theta }l_{\theta }}(Y_{j})\right) ^{\prime }\right] . \end{aligned}$$

By using Eq. (18) and the stationary property of the $Y_i$, one can replace the second term of the above equation by:

$$\begin{aligned} \mathbb{E }\left[ b_{\phi _{0}}(X_1)\nabla _{\theta }l_{\theta }(X_1)\right] \mathbb{E }\left[ b_{\phi _{0}}(X_1)\nabla _{\theta }l_{\theta }(X_1)\right] ^{\prime }. \end{aligned}$$

Furthermore, by using Eq. (1) we obtain:

$$\begin{aligned} \mathbb{E }\left[ Y_{2}Y_{j+1}u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{1}) u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{j})\right]&= \mathbb{E }\left[ b_{\phi _{0}}(X_1)b_{\phi _{0}}(X_j)u^{*}_{\nabla _{\theta }l_{\theta }} (Y_{1})u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{j})\right] \nonumber \\&+ \mathbb{E }\left[ b_{\phi _{0}}(X_1)\left( \eta _{j+1}+\varepsilon _{j+1}\right) u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{1})u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{j})\right] \nonumber \end{aligned}$$

(19)

$$\begin{aligned} \\&+ \mathbb{E }\left[ b_{\phi _{0}}(X_j)\left( \eta _2+\varepsilon _2\right) u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{1})u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{j})\right] \end{aligned}$$

(20)

$$\begin{aligned}&+ \mathbb{E }\left[ \left( \eta _2+\varepsilon _2\right) \left( \eta _{j+1}+\varepsilon _{j+1}\right) u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{1})u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{j})\right] .\nonumber \\ \end{aligned}$$

(21)

By independence of the centered noise, the term (19), (20) and (21) are equal to zero. Now, if we use Fubini’s Theorem we have:

$$\begin{aligned} \mathbb{E }\left[ b_{\phi _{0}}(X_1)b_{\phi _{0}}(X_j)u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{1})u^{*}_{\nabla _{\theta }l_{\theta }}(Y_{j})\right] \!=\!\mathbb{E }\left[ b_{\phi _{0}}(X_1) b_{\phi _{0}}(X_j) \nabla _{\theta }l_{\theta }(X_1) \nabla _{\theta }l_{\theta }(X_j)\right] .\nonumber \\ \end{aligned}$$

(22)

Hence, the covariance matrix is given by:

$$\begin{aligned} \varOmega _{j-1}(\theta )&= 4\left( \mathbb{E }\left[ b_{\phi _{0}}(X_1)b_{\phi _{0}}(X_j)\left( \nabla _{\theta }l_{\theta }(X_1)\right) \left( \nabla _{\theta }l_{\theta }(X_j)\right) ^{\prime }\right] \right. \\&\left. -\mathbb{E }\left[ b_{\phi _{0}}(X_1)\left( \nabla _{\theta }l_{\theta }(X_1)\right) \right] \mathbb{E }\left[ b_{\phi _{0}}(X_1)\left( \nabla _{\theta }l_{\theta }(X_1)\right) \right] ^{\prime }\right) \\&= 4\left( \tilde{C}_{j-1}-\mathbb{E }\left[ b_{\phi _{0}}(X_1)\left( \nabla _{\theta }l_{\theta }(X_1)\right) \right] \mathbb{E }\left[ b_{\phi _{0}}(X_1)\left( \nabla _{\theta }l_{\theta }(X_1)\right) \right] ^{\prime }\right) \\&= 4\left( \tilde{C}_{j-1}-P_{1}\right) \!. \end{aligned}$$

Finally, we obtain: $\varOmega (\theta )=\varOmega _{0}(\theta )+2\sum _{j >1}^{\infty }\varOmega _{j-1}(\theta )$ with $\varOmega _{0}(\theta )=4\left( P_2-P_{1}\right)$ and $\varOmega _{j-1}(\theta )=4\bigl (\tilde{C}_{j-1}-P_{1}\bigr )$.

Expression of the Hessian matrix $V_{\theta }$ : We have:

$$\begin{aligned} \mathbf P m_{\theta } = ||l_{\theta }||_{2}^2 - 2\left\langle l_{\theta }, l_{\theta _0}\right\rangle . \end{aligned}$$

(23)

For all $\theta$ in $\varTheta$, the application $\theta \mapsto \mathbf P m_{\theta }$ is twice differentiable w.r.t $\theta$ on the compact subset $\varTheta$. And for $j\in \left\{ 1,\ldots ,r\right\}$:

$$\begin{aligned} \frac{{\partial }\mathbf P m}{{\partial }\theta _j}(\theta )&= 2 \left\langle \frac{{\partial }l_{\theta }}{{\partial }\theta _j}, l_{\theta }\right\rangle -2 \left\langle \frac{{\partial }l_{\theta }}{{\partial }\theta _j}, l_{\theta _0}\right\rangle \\&= 2 \left\langle \frac{{\partial }l_{\theta }}{{\partial }\theta _j}, l_{\theta }-l_{\theta _0}\right\rangle ,\\&= 0 \text{ at} \text{ the} \text{ point} \theta _{0}, \end{aligned}$$

and for $j,k \in \left\{ 1,\ldots ,r\right\}$:

$$\begin{aligned} \frac{{\partial }^2 \mathbf P m}{{\partial }\theta _j {\partial }\theta _k}(\theta )&= 2 \left( \left\langle \frac{{\partial }^2 l_{\theta }}{{\partial }\theta _j \theta _k}, l_{\theta }- l_{\theta _0}\right\rangle +\left\langle \frac{{\partial }l_{\theta }}{{\partial }\theta _k}, \frac{{\partial }l_{\theta }}{{\partial }\theta _j}\right\rangle \right) _{j,k}\\&= 2 \left( \left\langle \frac{{\partial }l_{\theta }}{{\partial }\theta _k}, \frac{{\partial }l_{\theta }}{{\partial }\theta _j}\right\rangle \right) _{j,k} \text{ at} \text{ the} \text{ point} \theta _{0}. \end{aligned}$$

Appendix C: Proof of the applications

1.1 C.1 The Gaussian AR(1) model with measurement noise

1.1.1 C.1.1 Contrast function

We have:

$$\begin{aligned} l_{\theta }(x)=\frac{1}{\sqrt{2\pi \gamma ^{2}}} \phi x \exp \left( -\frac{1}{2\gamma ^2}x^2\right) . \end{aligned}$$

So that:

$$\begin{aligned} ||l_{\theta }||_{2}^{2}=\int |l_{\theta }(x)|^{2}dx =\frac{\phi ^{2}\gamma }{4\sqrt{\pi }}, \end{aligned}$$

and the Fourier Transform of $l_{\theta }$ is given by:

$$\begin{aligned} l^{*}_{\theta }(y)&= \int e^{iyx}l_{\theta }(x)dx = \int e^{iyx} \frac{1}{\sqrt{2\pi \gamma ^{2}}} \phi x \exp \left( -\frac{1}{2\gamma ^2}x^2\right) dx\\&= -i\phi \mathbb{E }\left[ iGe^{iyG}\right] = -i\phi \frac{{\partial }}{{\partial }y}\mathbb{E }\left[ e^{iyG}\right] \quad \text{ where} G\sim \fancyscript{N}(0,\gamma ^2)\\&= -i\phi \frac{{\partial }}{{\partial }y} \left[ e^{-\frac{y^2}{2}\gamma ^2}\right] \\&= i\phi y \gamma ^2e^{-\frac{y^2}{2}\gamma ^2}. \end{aligned}$$

As $\varepsilon _i$ is a centered Gaussian noise with variance $\sigma _{\varepsilon }^2$, we have:

$$\begin{aligned} f_{\varepsilon }(x)=\frac{1}{\sqrt{2\pi \sigma _{\varepsilon }^2}} \exp \left( -\frac{1}{2\sigma _{\varepsilon }^2}x^2\right) \text{ and} f^*_{\varepsilon }(x)= \exp \left( -\frac{1}{2}x^2 \sigma _{\varepsilon }^2\right) . \end{aligned}$$

Define:

$$\begin{aligned} \displaystyle u_{l_{\theta }}(y) =\frac{1}{2\pi } \frac{l^*_{\theta }(-y)}{f^*_{\varepsilon }(y)}. \end{aligned}$$

Then:

$$\begin{aligned} u^{*}_{l_{\theta }}(y)&= \frac{1}{2\pi }\int \frac{l^*_{\theta }(-x)}{f^*_{\varepsilon }(x)} e^{iyx}dx = \frac{-i}{2\pi } \phi \gamma ^2 \int x e^{iyx} \exp \left( \frac{x^2}{2}\sigma _{\varepsilon }^2\right) \exp \left( \frac{-x^2}{2}\gamma ^2\right) dx\\&= \frac{-i}{2\pi }\phi \gamma ^2 \frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)^{1/2}} \int xe^{iyx} (\gamma ^2-\sigma _{\varepsilon }^2)^{1/2}\exp \left( -\frac{1}{2}x^2(\gamma ^2-\sigma _{\varepsilon }^2)\right) dx\\&= \! -\! \frac{1}{\sqrt{2\pi }}\phi \gamma ^2 \frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)^{1/2}} \mathbb{E }\left[ \!iGe^{iyG}\!\right] \!=\! -\!\frac{1}{\sqrt{2\pi }}\phi \gamma ^2 \frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)^{1/2}} \frac{{\partial }}{{\partial }y}\mathbb{E }\left[ \!e^{iyG}\!\right] \\&= -\frac{1}{\sqrt{2\pi }}\phi \gamma ^2 \frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)^{1/2}} \frac{{\partial }}{{\partial }y} \left[ e^{-\frac{y^2}{2(\gamma ^2-\sigma _{\varepsilon }^2)}}\right] \\&= \frac{1}{\sqrt{2\pi }} \phi \gamma ^2 \frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)^{3/2}}y e^{-\frac{y^2}{2(\gamma ^2-\sigma _{\varepsilon }^2)}}, \end{aligned}$$

where $G\sim \fancyscript{N}\left( 0,\frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)} \right)$. We deduce that the function $m_{\theta }(\mathbf{y }_{i})$ is given by:

$$\begin{aligned} m_{\theta }(\mathbf{y }_{i})&= ||l_{\theta }||_{2}^{2}-2y_{i+1}u^*_{l_{\theta }}(y_{i})\\&= \frac{\phi ^{2}\gamma }{4\sqrt{\pi }} - 2y_{i} y_{i+1} \frac{1}{\sqrt{2\pi }}\phi \gamma ^2 \frac{1}{(\gamma ^2-\sigma _{\varepsilon }^2)^{3/2}}\exp \left( -\frac{y_{i}^2}{2(\gamma ^2-\sigma _{\varepsilon }^2)}\right) . \end{aligned}$$

Then, the contrast estimator defined in (1.1) is given by:

$$\begin{aligned} \widehat{\theta }_n&= \arg \min _{\theta \in \varTheta }\mathbf P _{n}m_{\theta }\\&= \arg \min _{\theta \in \varTheta }\left\{ \frac{\phi ^{2}\gamma }{4\sqrt{\pi }}-\sqrt{\frac{2}{\pi }}\frac{\phi \gamma ^{2}}{n(\gamma ^{2}-\sigma _{\varepsilon }^2)^{3/2}}\sum _{j=1}^{n} Y_{j+1} Y_{j}\exp \left( -\frac{1}{2} \frac{Y^{2}_{j}}{(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}\right) \right\} \!.\quad \square \end{aligned}$$

1.1.2 C.1.2 Checking assumptions of Theorem 1

Mixing properties. If $|\phi |<1$, the process $\mathbf{Y }_i$ is geometrically ergodic. For further details, we refer to Dedecker et al. (2007).

Regularity conditions: It remains to prove that the assumptions of Theorem 1 hold. It is easy to see that the only difficulty is to check the moment condition and the local dominance (C)-(T) and the uniqueness assumption (CT). The others assumptions are easily to verify since the function $l_{\theta }(x)$ is regular in $\theta$ belonging to $\varTheta$.

(CT): The limit contrast function $\displaystyle \mathbf P m_{\theta } : \theta \in \varTheta \mapsto \mathbf P m_{\theta }$ given by:

$$\begin{aligned} \theta \mapsto \mathbf P m_{\theta }&= ||l_{\theta }||_{2}^{2}-2\left\langle l_{\theta }, l_{\theta _{0}}\right\rangle \\&= \frac{\phi ^{2}\gamma }{4\sqrt{\pi }}-\sqrt{\frac{2}{\pi }}\frac{\phi \phi _{0}\gamma ^{2}\gamma _{0}^{2}}{(\gamma ^{2}+\gamma _{0}^2)^{\frac{3}{2}}}, \end{aligned}$$

is differentiable for all $\theta$ in $\varTheta$ and $\nabla _{\theta }\mathbf P m_{\theta }=0_{\mathbb{R }^2}$ if and only if $\theta$ is equal to $\theta _0$ . More precisely its first derivatives are given by:

$$\begin{aligned} \frac{\partial \mathbf P m_{\theta }}{\partial \phi }&= \frac{1}{4\sqrt{\pi }}\frac{\phi \gamma (2-\phi ^2)}{(1-\phi ^2)}-\sqrt{\frac{2}{\pi }}\phi _0\gamma _0^{2}(\gamma ^2+\gamma _0^2)^{-3/2}\\&\quad \left( \frac{\gamma ^2+\gamma ^2\phi ^2}{(1-\phi ^2)}-\frac{3\phi ^2\gamma ^4}{(1-\phi ^2)(\gamma ^2+\gamma _0^2)}\right) ,\\ \frac{\partial \mathbf P m_{\theta }}{\partial \sigma ^2}&= \frac{\phi ^2}{8\sqrt{\pi }\sigma (1-\phi ^2)^{1/2}}-\sqrt{\frac{2}{\pi }}\frac{\phi _0\gamma _0^{2}}{(1-\phi ^2)(\gamma ^2+\gamma _0^2)^{3/2}}\left( \phi -\frac{3\phi \gamma ^2}{(\gamma ^2+\gamma _0^2)}\right) , \end{aligned}$$

and

$$\begin{aligned} \nabla _{\theta }\mathbf P m_{\theta }=0_{\mathbb{R }^2} \Leftrightarrow \theta =\theta _0 \end{aligned}$$

The partial derivatives of $l_{\theta }$ w.r.t $\theta$ are given by:

$$\begin{aligned}&\frac{{\partial }l_{\theta }}{{\partial }\phi }(x)= \left( \left( \frac{-\phi ^2}{1-\phi ^2}+1\right) x + \frac{\phi ^2}{(1-\phi ^2)\gamma ^2}x^3\right) \frac{1}{\sqrt{2\pi \gamma ^2}}e^{-\frac{x^2}{2\gamma ^2}},\\&\frac{{\partial }l_{\theta }}{{\partial }\sigma ^2}(x)=\left( -\frac{\phi }{2(1-\phi ^2)\gamma ^2}x +\frac{\phi }{2(1-\phi ^2)\gamma ^4}x^3 \right) \frac{1}{\sqrt{2\pi \gamma ^2}}e^{-\frac{x^2}{2\gamma ^2}}. \end{aligned}$$

For the reader convenience let us introduce the following notations:

$$\begin{aligned}&a_1= \frac{-\phi ^2}{(1-\phi ^2)}+1 = \frac{1-2\phi ^2}{(1-\phi ^2)} \quad \text{ and}\quad a_2= \frac{\phi ^2}{(1-\phi ^2)\gamma ^2},\end{aligned}$$

(24)

$$\begin{aligned}&b_1= \frac{-\phi }{2(1-\phi ^2)\gamma ^2}\quad \text{ and}\quad b_2= \frac{\phi }{2(1-\phi ^2)\gamma ^4}. \end{aligned}$$

(25)

We rewrite:

$$\begin{aligned} \nabla _{\theta }l_{\theta }(x)&= \left( \frac{{\partial }l_{\theta }}{{\partial }\phi }(x), \frac{{\partial }l_{\theta }}{{\partial }\sigma ^2}(x)\right) ^{^{\prime }}\\&= \left( (a_1x+a_2x^3)\times g_{0,\gamma ^2}(x), (b_1x+b_2x^3)\times g_{0,\gamma ^2}(x)\right) ^{^{\prime }}, \end{aligned}$$

where the function $g_{0,\gamma ^2}$ defines the normal probability density of a centered random variable with variance $\gamma ^2$. Now, we can use Corollary 1 to compute the Hessian matrix $V_{\theta _0}$:

$$\begin{aligned} V_{\theta _0}&= 2 \begin{pmatrix} \left\| \frac{{\partial }l_{\theta }}{{\partial }\phi }\right\| _{2}^{2} &{} \left\langle \frac{{\partial }l_{\theta }}{{\partial }\phi }, \frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}\right\rangle \\ \left\langle \frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}, \frac{{\partial }l_{\theta }}{{\partial }\phi }\right\rangle &{} \left\| \frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}\right\| _{2}^{2} \end{pmatrix}\nonumber \\&= \frac{1}{\gamma _0 \sqrt{\pi }}\nonumber \\&\begin{pmatrix} a_{1}^{2}\mathbb{E }[X^{2}]+2a_{1}a_{2}\mathbb{E }[X^{4}]+a_{2}^{2}\mathbb{E }[X^{6}] &{} a_{1}b_{1}\mathbb{E }[X^{2}]+a_{1}b_{2}\mathbb{E }[X^{4}]+a_{2}b_{1}\mathbb{E }[X^{4}]+a_{2}b_{2}\mathbb{E }[X^{6}] \\ a_{1}b_{1}\mathbb{E }[X^{2}]+a_{1}b_{2}\mathbb{E }[X^{4}]+a_{2}b_{1}\mathbb{E }[X^{4}]+a_{2}b_{2}\mathbb{E }[X^{6}]&{} b_{1}^{2}\mathbb{E }[X^{2}]+2b_{1}b_{2}\mathbb{E }[X^{4}]+b_{2}^{2}\mathbb{E }[X^{6}] \end{pmatrix},\nonumber \\ \end{aligned}$$

(26)

with $X \sim \fancyscript{N}\bigl (0,\frac{\gamma _{0}^2}{2}\bigr )$. By replacing the terms $a_{1}, a_{2}, b_{1}$ and $b_{2}$ at the point $\theta _0$ we obtain:

$$\begin{aligned} V_{\theta _{0}} =\frac{1}{8\sqrt{\pi }(1-\phi _0^2)^2} \begin{pmatrix} \gamma _{0}(7\phi _0^4-4\phi _0^2+4) &{} \frac{-5\phi ^{5}_0+3\phi ^{3}_0+2\phi _{0}}{2\gamma _{0}(1-\phi ^{2}_0)}\\ \frac{-5\phi ^{5}_0+3\phi ^{3}_0+2\phi _0}{2\gamma _{0}(1-\phi ^{2}_0)} &{} \frac{7\phi _0^2}{4\gamma _{0}^3} \end{pmatrix}, \end{aligned}$$

(27)

which has a positive determinant equal to $0.0956$ at the true value $\theta _{0}=(0.7,0.3)$. Hence, $V_{\theta _{0}}$ is non-singular. Furthermore, the strict convexity of the function $\mathbf P m_{\theta }$ gives that $\theta _0$ is a minimum.

(C): (Local dominance): We have:

$$\begin{aligned} \mathbb{E }\left[ \sup _{\theta \in \varTheta }\left| Y_{2}u^{*}_{l_{\theta }}(Y_{1})\right| \right]&= \frac{1}{\sqrt{2\pi }}\mathbb{E }\left[ \sup _{\theta \in \varTheta }\left| \frac{\phi \gamma ^2}{(\gamma ^2-\sigma _{\varepsilon }^2)^{(3/2)}}Y_{2}Y_{1}\exp \left( -\frac{Y_{1}^2}{2(\gamma ^2-\sigma _{\varepsilon }^2)}\right) \right| \right] . \end{aligned}$$

The multivariate normal density of the pair $\mathbf{Y }_{1}=(Y_{1},Y_{2})$ denoted $g_{(0, \fancyscript{J}_{\theta _0})}$ is given by:

$$\begin{aligned} \frac{1}{2\pi } \det {(\fancyscript{J}_{\theta _0})}^{-1/2} \exp \left( -\frac{1}{2}\mathbf{y }_{1}^{^{\prime }} \fancyscript{J}_{\theta _0}^{-1}\mathbf{y }_{1}\right) , \end{aligned}$$

with:

$$\begin{aligned} \fancyscript{J}_{\theta _0}\!=\!\begin{pmatrix} \sigma _{\varepsilon }^2+\gamma ^2_0 &{} \phi _0 \gamma ^2_0\\ \phi _0 \gamma ^2_0 &{} \sigma _{\varepsilon }^2+\gamma ^2_0 \end{pmatrix} \quad \text{ and} \quad \fancyscript{J}_{\theta _0}^{-1}\!=\!\frac{1}{(\sigma _{\varepsilon }^2+\gamma ^2_0 )^{2}-\gamma ^4_0\phi ^{2}_0}\begin{pmatrix} \sigma _{\varepsilon }^2+\gamma ^2_0 &{} -\phi _0 \gamma ^2_0\\ -\phi _0\gamma ^2_0 &{} \sigma _{\varepsilon }^2+\gamma ^2_0 \end{pmatrix}. \end{aligned}$$

By definition of the parameter space $\varTheta$ and as all moments of the pair $\mathbf{Y }_{1}$ exist, the quantity $\mathbb{E }\left[ \sup _{\theta \in \varTheta }\left| Y_{2}u^{*}_{l_{\theta }}(Y_{1})\right| \right]$ is finite.

Moment condition (T): We recall that:

$$\begin{aligned} \nabla _{\theta }l_{\theta }(x)&= \left( \frac{{\partial }l_{\theta }}{{\partial }\phi }(x), \frac{{\partial }l_{\theta }}{{\partial }\sigma ^2}(x)\right) ^{^{\prime }}\\&= \left( (a_1x+a_2x^3)\times g_{0,\gamma ^2}(x), (b_1x+b_2x^3)\times g_{0,\gamma ^2}(x)\right) ^{^{\prime }}. \end{aligned}$$

The Fourier transforms of the first derivatives are:

$$\begin{aligned} \left( \frac{\partial l_{\theta }}{\partial \phi }(x)\right) ^{*}&=\int \exp \left( i xy\right) \left( a_1y+a_2y^3\right) \times g_{0,\gamma ^2}(y) dy\\&=\!-ia_1\mathbb{E }\left[ \!iG\exp \left( i xG\right) \!\right] \!+\!ia_2\mathbb{E }\left[ \!-iG^{3}\exp \left( i xG\right) \!\right] \text{ where} \text{ G} \!\sim \! \fancyscript{N}(0,\gamma ^{2}) \\&=-ia_{1}\frac{{\partial }}{{\partial }x}\mathbb{E }\left[ \exp \left( i xG\right) \right] +ia_{2}\frac{{\partial }^{3}}{{\partial }x^{3}}\mathbb{E }\left[ \exp \left( i xG\right) \right] \\&=-ia_{1}\frac{{\partial }}{{\partial }x}\exp \left( -\frac{x^{2}}{2}\gamma ^{2}\right) +ia_{2}\frac{{\partial }^{3}}{{\partial }x^{3}}\exp \left( -\frac{x^{2}}{2}\gamma ^{2}\right) \\&=(ia_{1}\gamma ^{2}x+3ia_{2}\gamma ^{4}x-ia_{2}\gamma ^{6}x^{3})\exp \left( -\frac{x^{2}}{2}\gamma ^{2}\right) , \end{aligned}$$

and

$$\begin{aligned} \left( \frac{\partial l_{\theta }}{\partial \sigma ^{2}}(x)\right) ^{*}=(ib_{1}\gamma ^{2}x+3ib_{2}\gamma ^{4}x-ib_{2}\gamma ^{6}x^{3})\exp \left( -\frac{x^{2}}{2}\gamma ^{2}\right) . \end{aligned}$$

We can compute the function $u_{\nabla _{\theta }l_{\theta }}(x)$:

$$\begin{aligned} u_{\frac{\partial l_{\theta }}{\partial \phi }}(x)&= \frac{1}{2\pi }\frac{\left( \frac{\partial l_{\theta }}{\partial \phi }(-x)\right) ^{*}}{f^{*}_{\varepsilon }(x)}\\&= \frac{1}{\sqrt{2\pi }}(\gamma ^{2}-\sigma ^{2}_{\varepsilon })^{1/2}\exp \left( -\frac{x^2}{2}(\gamma ^{2}-\sigma ^{2}_{\varepsilon })\right) \\&\times \left\{ \frac{1}{\sqrt{2\pi }}\frac{1}{(\gamma ^{2}-\sigma ^{2}_{\varepsilon })^{1/2}}\left( (-ia_{1}\gamma ^{2}-3ia_{2}\gamma ^{4})x+ia_{2}\gamma ^{6}x^{3}\right) \right\} \\&= -i\overline{C}\left( A_1x-A_2x^{3}\right) g_{0,\frac{1}{(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}}(x), \end{aligned}$$

with $\overline{C}=\frac{1}{\sqrt{2\pi }}\frac{1}{(\gamma ^{2}-\sigma _{\varepsilon }^2)^{1/2}}$ and $A_1=a_{1}\gamma ^{2}+3a_{2}\gamma ^{4}=\gamma ^2\frac{(1+\phi ^2)}{(1-\phi ^2)}$ and $A_2=a_{2}\gamma ^{6}=\gamma ^4\frac{\phi ^2}{(1-\phi ^2)}.$ The Fourier transform of the function $u_{\frac{\partial l_{\theta }}{\partial \phi }}(x)$ is given by:

$$\begin{aligned} u^{*}_{\frac{\partial l_{\theta }}{\partial \phi }}(x)&= -i\overline{C}\int \exp \left( iyx\right) \left( A_1y -A_2y^{3}\right) g_{\left( 0,\frac{1}{(\gamma ^{2}-\sigma _{\varepsilon }^2)}\right) }(y)dy\nonumber \\&= -\overline{C} A_1\frac{{\partial }}{{\partial }x}\mathbb{E }\left[ \exp \left( ixG\right) \right] -\overline{C} A_2\frac{{\partial }^{3}}{{\partial }x^{3}}\mathbb{E }\left[ \exp \left( ixG\right) \right] \nonumber \\&\quad \text{ where} \text{ G} \sim \fancyscript{N}\left( 0, \frac{1}{(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}\right) \nonumber \\&= -\overline{C} A_1\frac{{\partial }}{{\partial }x}\left( \exp \left( -\frac{x^{2}}{2(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}\right) \right) -\overline{C} A_2\frac{{\partial }^{3}}{{\partial }x^{3}}\left( \exp \left( -\frac{x^{2}}{2(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}\right) \right) \nonumber \\&= \left( \varPsi ^{\phi _0}_{1}x+\varPsi ^{\phi _0}_{2}x^{3}\right) \exp \left( -\frac{x^{2}}{2(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}\right) , \end{aligned}$$

(28)

with $\varPsi ^{\phi _0}_{1}=\overline{C}\left( \frac{A_1}{(\gamma ^{2}-\sigma ^{2}_{\varepsilon })}-\frac{3A_2}{(\gamma ^{2}-\sigma _{\varepsilon }^2)^{2}}\right)$ and $\varPsi ^{\phi _0}_{2}=\overline{C}\left( \frac{A_2}{(\gamma ^{2}-\sigma _{\varepsilon }^2)^{3}}\right) .$ By the same arguments, we obtain:

$$\begin{aligned} u^{*}_{\frac{\partial l_{\theta }}{\partial \sigma ^2}}(x) =\left( \varPsi ^{\sigma _0^{2}}_{1}x+\varPsi ^{\sigma _0^{2}}_{2}x^{3}\right) \exp \left( -\frac{x^{2}}{2(\gamma ^{2}-\sigma _{\varepsilon }^2)}\right) , \end{aligned}$$

(29)

with $\varPsi ^{\sigma _0^{2}}_{1}=\overline{C}\left( \frac{B_1}{(\gamma ^{2}-\sigma _{\varepsilon }^2)}-\frac{3B_2}{(\gamma ^{2}-\sigma _{\varepsilon }^2)^{2}}\right) , \varPsi ^{\sigma _0^{2}}_{2}=\overline{C}\left( \frac{B_2}{(\gamma ^{2}-\sigma _{\varepsilon }^2)^{3}}\right) , B_1=b_{1}\gamma ^{2}+3b_{2}\gamma ^{4}=\frac{\phi }{(1-\phi ^2)}$ and $B_2=b_{2}\gamma ^{6}=\gamma ^2\frac{\phi }{2(1-\phi ^2)}.$

Hence, for some $\delta >0,\, \mathbb{E }\left[ \left| Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right| ^{2+\delta }\right]$ is finite if:

$$\begin{aligned}&\mathbb{E }\left[ \left| \left( \varPsi ^{\phi _0}_{1}Y_{1}Y_{2}+\varPsi ^{\phi _0}_{2}Y_{1}^{3}Y_{2}\right) \exp \left( -\frac{Y_{1}^{2}}{2(\gamma ^{2}-\sigma _{\varepsilon }^2)}\right) \right| ^{2+\delta }\right] <\infty ,\\&\mathbb{E }\left[ \left| \left( \varPsi ^{\sigma _0^{2}}_{1}Y_{1}Y_{2}+\varPsi ^{\sigma _0^{2}}_{2}Y_{1}^{3}Y_{2}\right) \exp \left( -\frac{Y_{1}^{2}}{2(\gamma ^{2}-\sigma _{\varepsilon }^2)}\right) \right| ^{2+\delta }\right] <\infty , \end{aligned}$$

which is satisfied by the existence of all moments of the pair $\mathbf{Y }_{1}$. One can check that the Hessian local assumption (T) is also satisfied by the same arguments.

1.1.3 C.1.3 Explicit form of the covariance matrix

Lemma 3

The matrix $\varSigma (\theta _0)$ in the Gaussian AR(1) model is given by:

$$\begin{aligned} \varSigma (\theta _0)=V^{-1}_{\theta _0}\varOmega (\theta _{0})V^{-1}_{\theta _0} \end{aligned}$$

with

$$\begin{aligned} V_{\theta _{0}} =\frac{1}{8\sqrt{\pi }(1-\phi _0^2)^2} \begin{pmatrix} \gamma _0(7\phi _0^4-4\phi _0^2+4) &{} \frac{-5\phi ^{5}_0+3\phi ^{3}_0+2\phi _{0}}{2\gamma _0(1-\phi ^{2}_0)}\\ \frac{-5\phi ^{5}_0+3\phi ^{3}_0+2\phi _0}{2\gamma _0(1-\phi ^{2}_0)} &{} \frac{7\phi _0^2}{4\gamma _0^3} \end{pmatrix}, \end{aligned}$$

and

$$\begin{aligned} \varOmega (\theta _{0})=\varOmega _0(\theta _{0})+2\sum _{j>1}^{\infty }\varOmega _{j-1}(\theta _{0}) = 4\left[ P_{2}-P_1\right] +8\sum _{j>1}^{\infty }(\tilde{C}_{j-1}-P_1) \end{aligned}$$

where:

$$\begin{aligned} P_{1}=\begin{pmatrix} \frac{\phi _{0}^2\gamma _{0}^2(2-\phi _{0}^2)^{2}}{64\pi (1-\phi _{0}^2)^2} &{}\quad \frac{\phi _{0}^3(2-\phi _{0}^2)}{128\pi (1-\phi _{0}^2)^2}\\ \frac{\phi _{0}^3(2-\phi _{0}^2)}{128\pi (1-\phi _{0}^2)^2} &{}\quad \frac{\phi _{0}^4}{256\pi (1-\phi _{0}^2)^2\gamma _{0}^2} \end{pmatrix}, \end{aligned}$$

and $P_2$ is the $2\times 2$ symmetric matrix multiplied by a factor $\frac{1}{\sqrt{\pi (\gamma _0^{2}-\sigma ^{2}_{\varepsilon })}}$ and its coefficients $(P^2_{lm})_{1\le l,m \le 2}$ are given by:

$$\begin{aligned} P^2_{11}&= \left( \varPsi ^{\phi _0}_{1}\right) ^{2}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+15\left( \varPsi ^{\phi _0}_{2}\right) ^{2}\fancyscript{F}\tilde{V}_{1}^{3}\left( \tilde{V}_{2}+7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+6\varPsi ^{\phi _0}_{1}\varPsi ^{\phi _0}_{2}\fancyscript{F}\tilde{V}_{1}^{2} \left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) .\\ P^2_{22}&= \left( \varPsi ^{\sigma _0^{2}}_{1}\right) ^{2}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+15\left( \varPsi ^{\sigma _0}_{2}\right) ^{2}\fancyscript{F}\tilde{V}_{1}^{3} \left( \tilde{V}_{2}+7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+6\varPsi ^{\sigma _0^{2}}_{1}\varPsi ^{\sigma _0^{2}}_{2} \fancyscript{F}\tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) .\\ P^2_{12}&= \varPsi ^{\phi _0}_{1}\varPsi ^{\sigma _0^{2}}_{1}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+15\varPsi ^{\phi _0}_{2}\varPsi ^{\sigma _0^{2}}_{2}\fancyscript{F}\tilde{V}_{1}^{3} \left( \tilde{V}_{2}+7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+3\varPsi ^{\phi _0}_{1}\varPsi ^{\sigma _0^{2}}_{2}\fancyscript{F} \tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \\&+3\varPsi ^{\sigma _0^{2}}_{1}\varPsi ^{\phi _0}_{2}\fancyscript{F}\tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) , \end{aligned}$$

with $\fancyscript{F}\!=\!\frac{1}{(\sigma _{\varepsilon }^2\!+\!\gamma _{0}^{2})^{2}\!-\!\gamma _{0}^{4}\phi _{0}^{2}}\tilde{V}_{1}^{1/2}\tilde{V}_{2}^{1/2},\, \tilde{V}_{1}^{-1}\!=\!\frac{2}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}\!+\!\left( \frac{\gamma _{0}^2\!+\!\sigma _{\varepsilon }^2}{(\sigma _{\varepsilon }^2+\gamma _{0}^2)^{2}\!-\!\gamma _{0}^4\phi _{0}^{2}}\right) \left( 1\!-\!\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\right) , \tilde{V}_{2}=\frac{(\gamma _{0}^2+\sigma _{\varepsilon }^2)^{2}-\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _{0}^2+\sigma _{\varepsilon }^2)}$, and:

$$\begin{aligned} \varPsi ^{\phi _0}_{1}&= \frac{1}{\sqrt{2\pi }}\frac{1}{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)^{3/2}}\left( \frac{(1+\phi _{0}^{2})\gamma _{0}^{2}}{(1-\phi _{0}^{2})}-\frac{3\phi _{0}^{2}\gamma _{0}^{4}}{(1-\phi _{0}^{2})(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}\right) .\\ \varPsi ^{\sigma _0^{2}}_{1}&= \frac{1}{\sqrt{2\pi }}\frac{1}{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)^{3/2}}\left( \frac{\phi _{0}}{(1-\phi _{0}^{2})}-\frac{3\phi _{0}\gamma _{0}^{2}}{2(1-\phi _{0}^{2})(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}\right) .\\ \varPsi ^{\phi _0}_{2}&= \frac{1}{\sqrt{2\pi }}\frac{1}{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)^{7/2}}\frac{\gamma _{0}^{4}\phi _{0}^{2}}{(1-\phi _{0}^{2})}.\\ \varPsi ^{\sigma _0^{2}}_{2}&= \frac{1}{\sqrt{2\pi }}\frac{1}{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)^{7/2}}\frac{\gamma _{0}^{2}\phi _{0}}{2(1-\phi _{0}^{2})} \end{aligned}$$

The covariance terms are given by:

$$\begin{aligned} \tilde{C}_{j-1}&= \frac{\phi _{0}^{2}}{2\pi \gamma _{0}^{2}}\\&\begin{pmatrix} (4\phi _{0}^4-4\phi _{0}^2+1) \tilde{c}_1(j) + \frac{2\phi _{0}^2(1-2\phi _{0}^2)}{\gamma _{0}^2}\tilde{c}_2(j)+\frac{\phi _{0}^4}{\gamma _{0}^4}\tilde{c}_3(j)&{} \frac{\phi _{0}(2\phi _{0}^2-1)}{2\gamma _{0}^2}\tilde{c}_1(j) + \frac{\phi _{0}(1-3\phi _{0}^2)}{2\gamma _{0}^4}\tilde{c}_2(j)+ \frac{\phi _{0}^3}{2\gamma _{0}^6} \tilde{c}_3(j)\\ \frac{\phi _{0}(2\phi _{0}^2-1)}{2\gamma _{0}^2}\tilde{c}_1(j) + \frac{\phi _{0}(1-3\phi _{0}^2)}{2\gamma _{0}^4}\tilde{c}_2(i)+ \frac{\phi _{0}^3}{2\gamma _{0}^6} \tilde{c}_3(j) &{} \frac{\phi _{0}^2}{4\gamma _{0}^4} \tilde{c}_1(j)-\frac{\phi _{0}^2}{2\gamma _{0}^6}\tilde{c}_2(j) + \frac{\phi _{0}^2}{4\gamma _{0}^8} \tilde{c}_3(j)\\ \end{pmatrix}, \end{aligned}$$

with:

$$\begin{aligned}&\tilde{c}_1(j)=\frac{1}{\gamma _{0}}(2-\phi _{0}^{2j})^{-1/2} V_j^{3/2}\left( \fancyscript{V} + \frac{3\phi _{0}^{2j}V_j}{(2-\phi _{0}^{2j})^2} \right) ,\\&\tilde{c}_2(j)= \frac{3}{\gamma _{0}}(2-\phi _{0}^{2j})^{-1/2} V_j^{5/2} \left( \fancyscript{V} + 5\frac{\phi _{0}^{2j}V_j}{(2-\phi _{0}^{2j})^2} \right) ,\\&\tilde{c}_3(j)\!=\!\frac{3(2\!-\!\phi _{0}^{2j})^{-1/2}}{\gamma _{0}}V_j^{5/2}\left[ \!3\fancyscript{V}^2+5V_j(4\fancyscript{V}+2)\frac{\phi _{0}^{2j}}{(2\!-\!\phi _{0}^{2j})^2}\!+\!35V_j^{2}\frac{\phi _{0}^{4j}}{(2\!-\!\phi _{0}^{2j})^4}\!\right] , \end{aligned}$$

where:

$$\begin{aligned} V_j = \frac{\gamma _{0}^2(1-\phi _{0}^{2j})(2-\phi _{0}^{2j})}{(2-\phi _{0}^{2j})^2-\phi _{0}^{2j}} \quad \text{ and}\quad \fancyscript{V} = \frac{\gamma _{0}^2(1-\phi _{0}^{2j})}{2-\phi _{0}^{2j}} \end{aligned}$$

Moreover $\lim _{j\rightarrow \infty }\varOmega _{j-1}(\theta _0)=0_{\fancyscript{M}_{2\times 2}}$.

Remark 6

In practice, for the computing of the covariance matrix $\varOmega _{j-1}(\theta )$ that appears in Corollary 1, we have truncated the infinite sum ($q_{trunc}=100$).

Proof

Calculus of $\nabla m$

For all $x \in \mathbb{R }$, the function $l_{\theta }(x)$ is two times differentiable w.r.t $\theta$ on the compact subset $\varTheta$. More precisely, note that since $\gamma ^2 = \sigma ^2/(1-\phi ^2)$, it follows from the definition of the subset $\varTheta$ that $(\gamma ^2-\sigma _{\varepsilon }^2)>0$. So that for all $\mathbf{y }_{i}$ in $\mathbb{R }^{2}$ the function $m_{\theta }(\mathbf{y }_{i}): \theta \in \varTheta \mapsto m_{\theta }(\mathbf{y }_{i})$ is differentiable and:

$$\begin{aligned} \nabla _{\theta }(m_{\theta }(\mathbf{y }_{i}))&= \left( \frac{{\partial }m_{\theta }(\mathbf{y }_{i})}{{\partial }\phi }, \frac{{\partial }m_{\theta }(\mathbf{y }_{i})}{{\partial }\sigma ^{2}}\right) ^{\prime }\\&= \left( \frac{{\partial }\left\| l_{\theta }\right\| _{2}^2}{{\partial }\phi }-2y_{i+1}u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi }}(y_{i}), \frac{{\partial }\left\| l_{\theta }\right\| _{2}^2}{{\partial }\sigma ^{2}}-2y_{i+1}u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}}(y_{i})\right) ^{\prime }, \end{aligned}$$

with:

$$\begin{aligned}&\frac{\partial }{\partial \phi }||l_{\theta }||_{2}^{2}=\frac{\phi \gamma (2-\phi ^2)}{4\sqrt{\pi }(1-\phi ^2)},\\&\frac{\partial }{\partial \sigma ^2}||l_{\theta }||_{2}^{2}=\frac{\phi ^2}{8\sqrt{\pi }(1-\phi ^2)}. \end{aligned}$$

And, the function $u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi }}(x)$ and $u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}}(x)$ are given in Eqs. (28)–(29). Therefore,

$$\begin{aligned} \nabla _{\theta }m_{\theta }(\mathbf{y }_{i})&= \begin{pmatrix}\left( \frac{\phi _{0}\gamma _{0}(2-\phi _{0}^2)}{4\sqrt{\pi }(1-\phi _{0}^2)}-2y_{i+1}\left( \varPsi ^{\phi _{0}}_{1}y_{i}+\varPsi ^{\phi _{0}}_{2}y_{i}^{3}\right) \exp \left( -\frac{y_{i}^{2}}{2(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}\right) \right) \\ \left( \frac{\phi _{0}^2}{8\sqrt{\pi }(1-\phi _{0}^2)}-2y_{i+1}\left( \varPsi ^{\sigma _{0}^{2}}_{1}y_{i}+\varPsi ^{\sigma _{0}^{2}}_{2}y_{i}^{3}\right) \exp \left( -\frac{y_{i}^{2}}{2(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}\right) \right) \end{pmatrix}\nonumber \\&\quad \text{ at} \text{ the} \text{ point} \theta _{0}. \end{aligned}$$

(30)

Calculus of $P_{1}$: Recall that we have:

$$\begin{aligned}&P_1=\mathbb{E }\left[ b_{\theta _{0}}(X_1) \left( \nabla _{\theta }l_{\theta }(X_1) \right) \right] \mathbb{E }\left[ b_{\phi _{0}}(X_1) \left( \nabla _{\theta }l_{\theta }(X_1) \right) \right] ^{\prime }\\&P_{2}=\mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\nabla _{\theta } l_{\theta }}(Y_1)\right) ^{2}\right] . \end{aligned}$$

And the moments $(\mu _{2k})_{k \in \mathbb N }$ of a centered Gaussian random variable with variance $\sigma ^2$ are given by:

$$\begin{aligned} \mu _{2k} = \left( \frac{(2k)!}{2^kk!}\right) \sigma ^{2k}. \end{aligned}$$

We define by $P(x)$ a polynomial function of ordinary degree. We are interested in the calculus of $\displaystyle \mathbb{E }\left[ P(X) g_{0,\gamma ^{2}}(X)\right] ,$ where $X \sim \fancyscript{N}(0,\gamma ^2)$. We have:

$$\begin{aligned} \mathbb{E }\left[ P(X) g_{0,\gamma ^2}(X)\right]&= \int P(x) \frac{1}{\sqrt{2\pi }\gamma } e^{-\frac{x^2}{2\gamma ^2}} \frac{1}{\sqrt{2\pi }\gamma } e^{-\frac{x^2}{2\gamma ^2}} dx\\&= \frac{1}{2\pi \gamma ^{2}} \int P(x) e^{-\frac{x^2}{\gamma ^2}} dx\\&= \frac{1}{2\sqrt{\pi }\gamma } \mathbb{E }\left[ P(\bar{X}) \right] , \end{aligned}$$

where $\displaystyle \bar{X} \sim \fancyscript{N}\left( 0,\frac{\gamma ^2}{2}\right)$.

Denote by $B_1$ the constant $\frac{1}{2\sqrt{\pi }\gamma _0}$. We obtain:

$$\begin{aligned} P_1&= \begin{pmatrix} \mathbb{E }\left[ b_{\phi _{0}}(X_1)\frac{{\partial }l_{\theta }}{{\partial }\phi }(\theta ,X_1) \right] ^2 &{}\mathbb{E }\left[ b_{\phi _{0}}(X_1)\frac{{\partial }l_{\theta }}{{\partial }\phi }(\theta ,X_1) \right] \mathbb{E }\left[ b_{\phi _{0}}(X_1)\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}(\theta ,X_1) \right] \\ \mathbb{E }\left[ b_{\phi _{0}}(X_1)\frac{{\partial }l_{\theta }}{{\partial }\phi }(\theta ,X_1) \right] \mathbb{E }\left[ b_{\phi _{0}}(X_1)\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}(\theta ,X_1) \right] &{}\mathbb{E }\left[ b_{\phi _{0}}(X_1)\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}(\theta ,X_1) \right] ^2\\ \end{pmatrix}\\&= B_1^2\phi _{0}^2\begin{pmatrix} \mathbb{E }\left[ H_{11}(\bar{X})\right] ^{2} &{} \mathbb{E }\left[ H_{12}(\bar{X}) \right] \mathbb{E }\left[ H_{21}(\bar{X}) \right] \\ \mathbb{E }\left[ H_{21}(\bar{X}) \right] \mathbb{E }\left[ H_{12}(\bar{X}) \right] &{} \mathbb{E }\left[ H_{22}(\bar{X}) \right] ^2\\ \end{pmatrix},\\ \end{aligned}$$

where $\displaystyle \bar{X} \sim \fancyscript{N}\bigl (0,\frac{\gamma _0^2}{2} \bigr )$. The polynomials $\left( H_{ij}(x)\right) _{1 \le i,j \le 2}$ are given by:

$$\begin{aligned}&H_{11}(x)= \left( a_1x^2 + a_2x^4 \right) \!, \\&H_{12}(x)= \left( b_1x^2 + b_2x^4 \right) \!, \\&H_{21}(x)= \left( a_1x^2 + a_2x^4 \right) \!,\\&H_{22}(x)= \left( b_1x^2 + b_2x^4 \right) \!. \end{aligned}$$

Lastly, by replacing the terms $B_1,\, a_1$, and $a_2$ by their expressions given in Eq. (24) at the point $\theta _{0}$, we obtain:

$$\begin{aligned} P_1&= \mathbb{E }\left[ \!b_{\phi _{0}}(X_1) \left( \nabla _{\theta }l_{\theta }(X_1) \right) \!\right] \mathbb{E }\left[ \!b_{\phi _{0}}(X_1) \left( \nabla _{\theta }l_{\theta }(X_1) \right) \!\right] ^{\prime } \\&= \begin{pmatrix} \frac{\phi _{0}^2\gamma _{0}^2(2-\phi _0^2)^2}{64\pi (1-\phi _{0}^2)^2} &{} \frac{\phi _{0}^3(2-\phi _{0}^2)}{128\pi (1-\phi _{0}^2)^2}\\ \frac{\phi _{0}^3(2-\phi _{0}^2)}{128\pi (1-\phi _{0}^2)^2} &{} \frac{\phi _{0}^4}{256\pi (1-\phi _{0}^2)^2\gamma _{0}^2} \end{pmatrix}. \end{aligned}$$

Calculus of $P_2$:

$$\begin{aligned} \mathbb{E }\left[ \!\left( Y_{2}u^{*}_{\nabla _{\theta } l_{\theta }}(Y_{1})\right) \left( Y_{2}u^{*}_{\nabla _{\theta } l_{\theta }}(Y_{1})\right) ^{\prime }\!\right]&\!=\!&\begin{pmatrix} \mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi } }(Y_{1})\right) ^{2}\right] &{} \mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi }}(Y_{1})\right) \left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}} }(Y_{1})\right) \right] \\ \mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}}(Y_{1})\right) \left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi }}(Y_{1})\right) \right] &{} \mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}}}(Y_{1})\right) ^{2}\right] \end{pmatrix}. \end{aligned}$$

We have:

$$\begin{aligned}&\left( 2\pi \frac{(\gamma _0^2-\sigma _{\varepsilon }^2)}{2} \right) ^{-1/2}\mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi } }(Y_{1})\right) ^{2}\right] \nonumber \\&\qquad =\mathbb{E }\left[ Y^{2}_{2}\left( \varPsi ^{\phi _{0}}_{1}Y_{1}+\varPsi ^{\phi _{0}}_{2}Y^{3}_{1}\right) ^{2} \times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] \nonumber \\&\qquad =\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\mathbb{E }\left[ Y^{2}_{2}Y^{2}_{1}\times g_{\left( 0, \frac{(\gamma ^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] \nonumber \\&\qquad \quad \!+\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\mathbb{E }\left[ \!Y^{2}_{2}Y^{6}_{1}\times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] \nonumber \\&\qquad \quad +2\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\mathbb{E }\left[ Y^{2}_{2}Y^{4}_{1} \times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] . \end{aligned}$$

(31)

The density of $\mathbf{Y }_{1}$ is $g_{(0, \fancyscript{J}_{\theta _0})}$. Then, $g_{(0,\fancyscript{J}_{\theta _0})}\times$ $\small { \exp \left( -\frac{y^{2}_{1}}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}\right) }$ is equal to:

$$\begin{aligned}&\frac{1}{2\pi }\frac{1}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2} \right) ^{1/2}}\exp \left( -\frac{1}{2} \frac{1}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2} \right) }\right. \\&\quad \left. \left( (\sigma _{\varepsilon }^2+\gamma _{0}^2)(y^{2}_{1}+y^{2}_{2})-2\phi _{0}\gamma _{0}^{2}y_{1}y_{2}\right) \right) \times \exp \left( -\frac{1}{2}\frac{2}{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)} y^{2}_{1}\right) \\&= \frac{1}{2\pi }\frac{1}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2} \right) ^{1/2}}\\&\quad \times \exp \left( -\frac{1}{2}y^{2}_{1}\left( \frac{2}{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}-\frac{(\gamma _{0}^{2}+\sigma _{\varepsilon }^2)}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2} \right) }\right) \right) \\&\quad \times \exp \left( -\frac{1}{2}y^{2}_{2}\left( \frac{(\gamma _{0}^{2}+\sigma _{\varepsilon }^2)}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2} \right) }\right) \right) \\&\quad \times \exp \left( -\frac{1}{2}y_{1}y_{2}\left( \frac{2\phi _{0}\gamma _{0}^{2}}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2} \right) }\right) \right) \\&= \frac{1}{2\pi }\frac{1}{\left( (\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^2 \right) ^{1/2}}\exp \left( -\frac{1}{2}\left( \tilde{V}_{2}^{-1}\left( y_{2}-\frac{\phi _{0} \gamma _{0}^{2}}{\gamma _0^2 + \sigma _{\varepsilon }^2}y_{1}\right) ^{2}\right) \right) \\&\times \exp \left( -\frac{1}{2}y_{1}^{2}\tilde{V}_{1}^{-1}\right) , \end{aligned}$$

with $\tilde{V}_{1}^{-1}=\frac{2}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}+\left( \frac{\gamma _{0}^2+\sigma _{\varepsilon }^2}{(\sigma _{\varepsilon }^2+\gamma _{0}^2)^{2}-\gamma _{0}^4\phi _{0}^{2}}\right) \left( 1-\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\right)$ and $\tilde{V}_{2}=\frac{(\gamma _{0}^2+\sigma _{\varepsilon }^2)^{2}-\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _{0}^2+\sigma _{\varepsilon }^2)}$.

Then, we obtain:

$$\begin{aligned} g_{(0,\fancyscript{J}_{\theta _0})}\times \exp \left( -\frac{y^{2}_{1}}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}\right)&= \frac{1}{((\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2})^{1/2}}\tilde{V}_{1}^{1/2}\tilde{V}_{2}^{1/2}\\&\quad g_{(\phi _{0} \gamma _{0}^{2}y_{1}/(\gamma _0^2 + \sigma _{\varepsilon }^2),\tilde{V}_{2})}(y_{2})g_{(0,\tilde{V}_{1})}(y_{1}). \end{aligned}$$

In the following, we set $\fancyscript{F}=\frac{1}{(\sigma _{\varepsilon }^2+\gamma _{0}^2 )^{2}-\gamma _{0}^4\phi _{0}^{2}}\tilde{V}_{1}^{1/2}\tilde{V}_{2}^{1/2}$. Now, we can compute the moments:

$$\begin{aligned}&\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\mathbb{E }\left[ Y^{2}_{2}Y^{2}_{1}\exp \left( -\frac{Y^{2}_{1}}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}\right) \right] \\&\quad =\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F} \int y_{1}^{2}g_{(0,\tilde{V}_{1})}(y_{1}) dy_{1} \int y_{2}^{2}g_{(\phi _{0} \gamma _{0}^{2}y_{1}/(\gamma _0^2 + \sigma _{\varepsilon }^2),\tilde{V}_{2})}(y_{2})dy_{2}\\&\quad \!=\!\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F}\int y_{1}^{2}g_{(0,V_{1})}(y_{1}) dy_{1} \mathbb{E }\left[ \!G^{2}\right] \text{ where} \text{ G} \sim \fancyscript{N}(\phi _{0} \gamma _{0}^{2}y_{1}/(\gamma _0^2 + \sigma _{\varepsilon }^2), \tilde{V}_{2})\\&\quad =\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F}\int \left( \tilde{V}_{2}y_{1}^{2}+\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}y_{1}^{4}\right) g_{(0,\tilde{V}_{1})}(y_{1})dy_{1}\\&\quad =\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F}\tilde{V}_{2}\tilde{V}_{1}+3\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F}\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}^{2}\\&\quad =\left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) . \end{aligned}$$

In a similar manner, we have:

$$\begin{aligned}&\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\mathbb{E }\left[ Y^{2}_{2}Y^{6}_{1}\exp \left( -\frac{Y^{2}_{1}}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}\right) \right] \\&\quad =\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\fancyscript{F}\int y_{1}^{6}g_{(0,V_{1})}(y_{1}) dy_{1} \mathbb{E }\left[ \!G^{2}\!\right] \text{ where} G \sim \fancyscript{N}(\phi _{0} \gamma _{0}^{2}y_{1}/(\gamma _0^2 + \sigma _{\varepsilon }^2), \tilde{V}_{2})\\&\quad =\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\fancyscript{F}\int \left( \tilde{V}_{2}y_{1}^{6}+\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}y_{1}^{8}\right) g_{(0,\tilde{V}_{1})}(y_{1})dy_{1}\\&\quad =15\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\fancyscript{F}\tilde{V}_{2}\tilde{V}_{1}^{3}+105\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\fancyscript{F}\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}^{4}\\&\quad =15\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\fancyscript{F}\tilde{V}_{1}^{3}\left( \tilde{V}_{2}+7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) , \end{aligned}$$

and

$$\begin{aligned}&2\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\mathbb{E }\left[ Y^{2}_{2}Y^{4}_{1}\exp \left( -\frac{Y^{2}_{1}}{(\gamma _{0}^2-\sigma _{\varepsilon }^2)}\right) \right] \\&\quad =\!2\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\fancyscript{F}\int y_{1}^{4}g_{(0,V_{1})}(y_{1}) dy_{1} \mathbb{E }\left[ G^{2}\right] \text{ where} G \!\sim \! \fancyscript{N}(\phi _{0} \gamma _{0}^{2}y_{1}/(\gamma _0^2 \!+\! \sigma _{\varepsilon }^2), \tilde{V}_{2})\\&\quad =2\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\fancyscript{F}\int \left( \tilde{V}_{2}y_{1}^{4}+\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}y_{1}^{6}\right) g_{(0,\tilde{V}_{1})}(y_{1})dy_{1}\\&\quad =6\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\fancyscript{F}\tilde{V}_{2}\tilde{V}_{1}^{2}+30\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\fancyscript{F} \frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}^{3}\\&\quad =6\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\fancyscript{F}\tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) . \end{aligned}$$

By replacing all the terms of Eq. (31) we obtain:

$$\begin{aligned} \mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi } }(Y_1)\right) ^{2}\right]&= \left( \varPsi ^{\phi _{0}}_{1}\right) ^{2}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2+\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \nonumber \\&+15\left( \varPsi ^{\phi _{0}}_{2}\right) ^{2}\fancyscript{F}\tilde{V}_{1}^{3} \left( \tilde{V}_{2}+7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \nonumber \\&+6\varPsi ^{\phi _{0}}_{1}\varPsi ^{\phi _{0}}_{2}\fancyscript{F}\tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) , \end{aligned}$$

(32)

and

$$\begin{aligned} \mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\sigma ^{2}} }(Y_{1})\right) ^{2}\right]&= \left( \varPsi ^{\sigma _0^{2}}_{1}\right) ^{2}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3 \frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \nonumber \\&+15\left( \varPsi ^{\sigma _0^{2}}_{2}\right) ^{2}\fancyscript{F}\tilde{V}_{1}^{3}\left( \tilde{V}_{2}+7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \nonumber \\&+6\varPsi ^{\sigma _0^{2}}_{1}\varPsi ^{\sigma _0^{2}}_{2}\fancyscript{F}\tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) , \end{aligned}$$

(33)

and

$$\begin{aligned}&\mathbb{E }\left[ Y^{2}_{2}\left( u^{*}_{\frac{{\partial }l_{\theta }}{{\partial }\phi } }(Y_{1})\right) \left( u^{*}_{\frac{{\partial }}{{\partial }\sigma ^{2}} l_{\theta }}(Y_{1})\right) \right] \nonumber \\&=\varPsi ^{\phi _{0}}_{1}\varPsi ^{\sigma _0^{2}}_{1}\mathbb{E }\left[ Y^{2}_{2}Y^{2}_{1}\times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] +\varPsi ^{\phi _{0}}_{2}\varPsi ^{\sigma _0^{2}}_{2}\mathbb{E }\left[ Y^{2}_{2}Y^{6}_{1}\times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] \nonumber \\&\quad +\varPsi ^{\phi _{0}}_{1}\varPsi ^{\sigma _0^{2}}_{2}\mathbb{E }\left[ Y^{2}_{2}Y^{4}_{1}\times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] +\varPsi ^{\phi _{0}}_{2}\varPsi ^{\sigma _0^{2}}_{1}\mathbb{E }\left[ Y^{2}_{2}Y^{4}_{1}\times g_{\left( 0, \frac{(\gamma _{0}^{2}-\sigma _{\varepsilon }^2)}{2}\right) }\right] \nonumber \\&=\varPsi ^{\phi _{0}}_{1}\varPsi ^{\sigma _0^{2}}_{1}\fancyscript{F}\tilde{V}_{1}\left( \tilde{V}_{2}+3\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 +\sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) +15\varPsi ^{\phi _{0}}_{2}\varPsi ^{\sigma _0^{2}}_{2}\fancyscript{F}\tilde{V}_{1}^{3} \nonumber \\&\quad \times \left( \tilde{V}_{2} +7\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) \nonumber \\&\quad +3\varPsi ^{\phi _{0}}_{1}\varPsi ^{\sigma _0^{2}}_{2} \fancyscript{F}\tilde{V}_{1}^{2}\left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) +3\varPsi ^{\phi _{0}}_{2}\varPsi ^{\sigma _0^{2}}_{1} \fancyscript{F}\tilde{V}_{1}^{2}\nonumber \\&\quad \times \left( \tilde{V}_{2}+5\frac{\phi _{0}^{2}\gamma _{0}^{4}}{(\gamma _0^2 + \sigma _{\varepsilon }^2)^2}\tilde{V}_{1}\right) . \end{aligned}$$

(34)

Calculus of ${\mathbb{C }ov}\left( \nabla _{\theta }m_{\theta }(Y_{1}),\nabla _{\theta }m_{\theta }(Y_{j}) \right)$: We want to compute:

$$\begin{aligned} {\mathbb{C }ov}\left( \nabla _{\theta }m_{\theta }(Y_{1}), \nabla _{\theta }m_{\theta }(Y_{j}) \right) =4\left[ \tilde{C}_{j-1}-P_1\right] . \end{aligned}$$

Since we have already computed the terms of the matrix $P_1$, it remains to compute the terms of the covariance matrix $\tilde{C}_{j-1}$ given by:

$$\begin{aligned} \tilde{C}_{j-1}=\mathbb{E }\left[ b_{\phi _{0}}(X_1)b_{\phi _{0}}(X_j)\left( \nabla _{\theta }l_{\theta }(X_1)\right) \left( \nabla _{\theta }l_{\theta }(X_j)\right) ^{\prime }\right] . \end{aligned}$$

For all $j>1$, the pair $(X_1,X_j)$ has a multivariate normal density $g_{(0, \fancyscript{W})}$ where $\fancyscript{W}$ is given by:

$$\begin{aligned} \fancyscript{W}=\gamma _{0}^2\begin{pmatrix} 1 &{} \phi _{0}^j\\ \phi _{0}^j &{} 1\\ \end{pmatrix} \quad \text{ and}\quad \fancyscript{W}^{-1}=\frac{1}{\gamma _{0}^2(1-\phi _{0}^{2j})}\begin{pmatrix} 1 &{} - \phi _{0}^j\\ - \phi _{0}^j &{} 1\\ \end{pmatrix}. \end{aligned}$$

The density of the couple $(X_1,X_j)$ is:

$$\begin{aligned} g_{(0, \fancyscript{W})}(x_1,x_j) =\frac{1}{2\pi } \det {(\fancyscript{W})}^{-1/2} \exp \left( -\frac{1}{2} (x_1,x_j)^{^{\prime }}\fancyscript{W}^{-1}(x_1,x_j)\right) . \end{aligned}$$

We start by computing:

$$\begin{aligned} g_{(0, \fancyscript{W})}(x_1,x_j) \times \exp \left( -\frac{1}{2\gamma _{0}^2}\left( x_1^2+x_j^2\right) \right) . \end{aligned}$$

We have:

$$\begin{aligned}&g_{(0, \fancyscript{W})}(x_1,x_j) \times \exp \left( -\frac{1}{2\gamma _{0}^2}(x_1^2+x_j^2) \right) \\&\quad = \frac{1}{2\pi }\det {(\fancyscript{W})}^{-1/2} \exp \left( -\frac{1}{2(1-\phi _{0}^{2j})\gamma _{0}^2}\left( x_1^2(1-\phi _{0}^{2j})+x_j^2(1-\phi _{0}^{2j})+x_1^2-2\phi _{0}^jx_1x_j+x_j^2\right) \right) ,\\&\quad =\frac{1}{2\pi }\det {(\fancyscript{W})}^{-1/2} \exp \left( -\frac{1}{2(1-\phi _{0}^{2j})\gamma _{0}^2}\left[ \left( x_1^2(1-\phi _{0}^{2j})+x_1^2-2\phi _{0}^jx_1x_j\right) \right. \right. \\&\qquad +\left. \left. \left( x_j^2(1-\phi _{0}^{2j})+x_j^2\right) \right] \right) ,\\&\quad =\frac{1}{2\pi }\det {(\fancyscript{W})}^{-1/2} \exp \left( -\frac{1}{2(1-\phi _{0}^{2j})\gamma _{0}^2}\left[ (2-\phi _{0}^{2j})\left( x_1^2-2\frac{\phi _{0}^j}{(2-\phi _{0}^{2j})}x_1x_j\right) \right. \right. \\&\qquad +\left. \left. \left( x_j^2(1-\phi _{0}^{2j})+x_j^2\right) \right] \right) ,\\&\quad =\frac{1}{2\pi }\det {(\fancyscript{W})}^{-1/2} \exp \left( -\frac{(2-\phi _{0}^{2j})}{2(1-\phi _{0}^{2j})\gamma _{0}^2}\left( x_1 - \frac{\phi _{0}^j x_j}{(2-\phi _{0}^{2j})}\right) ^2 \right) \\&\qquad \times \exp \left( -\frac{(2-\phi _{0}^{2j})}{2(1-\phi _{0}^{2j})\gamma _{0}^2}x_j^2\left( 1-\frac{\phi _{0}^{2j}}{(2-\phi _{0}^{2j})^2}\right) \right) . \end{aligned}$$

For all $j>1$, we define:

$$\begin{aligned} V_j=\frac{\gamma _{0}^2(1-\phi _{0}^{2j})(2-\phi _{0}^{2j})}{(2-\phi _{0}^{2j})^2-\phi _{0}^{2j}} \quad \text{ and}\quad \fancyscript{V} = \frac{\gamma _{0}^2(1-\phi _{0}^{2j})}{(2-\phi _{0}^{2j})}. \end{aligned}$$

We can rewrite:

$$\begin{aligned}&g_{(0, \fancyscript{W})}(x_1,x_j) \times \exp \left( -\frac{1}{2\gamma _{0}^2}(x_1^2+x_j^2) \right) \\&=\frac{V_j^{1/2}}{\gamma _{0}} \frac{1}{\sqrt{2\pi }V_j^{1/2}} \exp \left( -\frac{1}{2V_j} x_j^2 \right) \times \frac{1}{(2-\phi _{0}^{2j})^{1/2}\sqrt{2\pi }\fancyscript{V}^{1/2}}\exp \left( -\frac{1}{2 \fancyscript{V}}\left( x_1 - \frac{\phi _{0}^ix_j}{(2-\phi _{0}^{2j})}\right) ^2\right) . \end{aligned}$$

So, by Fubini’s Theorem, we obtain:

$$\begin{aligned}&\mathbb{E }\left[ X_1^2 X_j^2 \exp \left( -\frac{1}{2\gamma _{0}^2}\left( X_1^2+X_j^2\right) \right) \right] \\&=\frac{1}{\gamma _{0}}V_j^{1/2}\int x_j^2\frac{1}{\sqrt{2\pi }V_j^{1/2}} \exp \left( -\frac{1}{2V_j} x_j^2 \right) \int x_1^2 \frac{(2-\phi _{0}^{2j})^{-1/2}}{\sqrt{2\pi }\fancyscript{V}^{1/2}}\exp \left( -\frac{1}{2\fancyscript{V}}\left( x_1 - \frac{\phi _{0}^jx_j}{(2-\phi _{0}^{2j})}\right) ^2\right) dx_1dx_j,\\&=\frac{1}{\gamma _{0}}V_j^{1/2}\int x_j^2\frac{1}{\sqrt{2\pi }V_j^{1/2}} \exp \left( -\frac{1}{2V_j} x_j^2 \right) (2-\phi _{0}^{2j})^{-1/2} \mathbb{E }[G^2]dx_j, \end{aligned}$$

where $\displaystyle G \sim \fancyscript{N}\left( \frac{\phi _{0}^jx_j}{(2-\phi _{0}^{2j})},\fancyscript{V} \right)$. Thus, $\mathbb{E }[G^2] = \fancyscript{V} + \left( \frac{\phi _{0}^jx_j}{(2-\phi _{0}^{2j})}\right) ^2$. We obtain:

$$\begin{aligned}&\mathbb{E }\left[ X_1^2 X_j^2 \exp \left( -\frac{1}{2\gamma _{0}^2}\left( X_1^2+X_j^2\right) \right) \right] \nonumber \\&=(2-\phi _0^{2j})^{-1/2}\frac{V_j^{1/2}}{\gamma _{0}}(2-\phi _{0}^{2j})^{-1/2}\int x_j^2\left( \fancyscript{V} + \left( \frac{\phi _{0}^jx_j}{(2-\phi _{0}^{2j})}\right) ^2\right) \frac{1}{\sqrt{2\pi }V_j^{1/2}} \exp \left( -\frac{1}{2V_j} x_j^2 \right) dx_j\nonumber \\&=\frac{V_j^{1/2}}{\gamma _{0}} (2-\phi _{0}^{2j})^{-1/2}\left( \fancyscript{V} \mathbb{E }[G_j^2] + \frac{\phi _{0}^{2j}}{(2-\phi _{0}^{2j})^2 \gamma _0}\mathbb{E }[G_j^4]\right) \nonumber \\&=\frac{V_j^{3/2}}{\gamma _{0}}(2-\phi _{0}^{2j})^{-1/2} \left( \fancyscript{V} + \frac{3\phi _{0}^{2j}V_j}{(2-\phi _{0}^{2j})^2} \right) \nonumber \\&=\tilde{c}_1(j), \end{aligned}$$

(35)

where $G_j \sim \fancyscript{N}\left( 0,V_j \right)$. Additionally, we have:

$$\begin{aligned}&\mathbb{E }\left[ X_1^2 X_j^4 \exp \left( -\frac{1}{2\gamma _{0}^2}\left( X_1^2+X_j^2\right) \right) \right] \nonumber \\&=\frac{ V_j^{1/2}}{\gamma _{0}}(2-\phi _{0}^{2j})^{-1/2}\fancyscript{V} \mathbb{E }[G_j^4] + V_j^{1/2} \frac{(2-\phi _{0}^{2j})^{-1/2}\phi _{0}^{2j}}{(2-\phi _{0}^{2j})^2}\mathbb{E }[G_j^6],\nonumber \\&=\frac{3V_j^{5/2}}{\gamma _{0}}(2-\phi _{0}^{2j})^{-1/2} \left( \fancyscript{V} + 5\frac{\phi _{0}^{2j}V_j}{(2-\phi _{0}^{2j})^2} \right) ,\nonumber \\&=\tilde{c}_2(j). \end{aligned}$$

(36)

Now, we are interested in $\mathbb{E }\left[ X_1^4 X_j^4 \exp \left( -\frac{1}{2\gamma _{0}^2}(X_1^2+X_j^2) \right) \right]$. In a similar manner, we obtain:

$$\begin{aligned}&\mathbb{E }\left[ X_1^4 X_j^4 \exp \left( -\frac{1}{2\gamma _{0}^2}\left( X_1^2+X_j^2 \right) \right) \right] \nonumber \\&=\frac{V_j^{1/2}}{\gamma _{0}}\int x_j^4\frac{1}{\sqrt{2\pi }V_j^{1/2}} \exp \left( -\frac{1}{2V_j} x_j^2 \right) (2-\phi _{0}^{2j})^{-1/2} \mathbb{E }[G^4]dx_j, \end{aligned}$$

(37)

where $\small {G \sim \fancyscript{N}\left( \frac{\phi _{0}^jx_j}{(2-\phi _{0}^{2j})},\fancyscript{V} \right) }$. We use the fact that the moments of a random variable $X\sim \fancyscript{N}(\mu ,v)$ are:

$$\begin{aligned} \mathbb{E }\left[ X^n\right]&= (n-1)v\mathbb{E }\left[ X^{n-2}\right] +\mu \mathbb{E }\left[ X^{n-1}\right] \\ \mathbb{E }[G^4]&= 3 \fancyscript{V} \mathbb{E }[G^2]+ \left( \frac{\phi _{0}^jx_j}{(2-\phi _{0}^{2j})}\mathbb{E }[G^3]\right) \\&= 3 \fancyscript{V}^{2}+(4\fancyscript{V}+2)\frac{\phi _{0}^{2j}x_j^2}{(2-\phi _{0}^{2j})^2}+\frac{\phi _{0}^{4j}x_j^4}{(2-\phi _{0}^{2j})^4}. \end{aligned}$$

By replacing $\mathbb{E }[G^4]$ in Eq. (37), we have:

$$\begin{aligned}&\mathbb{E }\left[ X_1^4 X_j^4 \exp \left( -\frac{1}{2\gamma _{0}^2}\left( X_1^2+X_j^2 \right) \right) \right] \nonumber \\&=\frac{3(2-\phi _{0}^{2j})^{-1/2}}{\gamma _{0}}V_j^{5/2}\left[ 3\fancyscript{V}^2+5V_j(4\fancyscript{V}+2)\frac{\phi _{0}^{2j}}{(2-\phi _{0}^{2j})^2}+35V_j^{2}\frac{\phi _{0}^{4j}}{(2-\phi _{0}^{2j})^4}\right] ,\nonumber \\&=\tilde{c}_3(j). \end{aligned}$$

(38)

For all $j>1$, the matrix $\tilde{C}_{j-1}$ is given by:

$$\begin{aligned} \tilde{C}_{j-1}=\frac{\phi _{0}^2}{2\pi \gamma _{0}^2} \begin{pmatrix} a_1^2 \tilde{c}_1(j)+2a_1a_2\tilde{c}_2(j)+a_2^2\tilde{c}_3(j)&{} a_1b_1\tilde{c}_1(j) + a_1b_2+a_2b_1\tilde{c}_2(j)+ a_2b_2 \tilde{c}_3(j)\\ a_1b_1 \tilde{c}_1(j) + a_1b_2+a_2b_1\tilde{c}_2(j)+ a_2b_2 \tilde{c}_3(j) &{} b_1^2\tilde{c}_1(j)+2b_1b_2 \tilde{c}_2(j) + b_2^2\tilde{c}_3(j)\\ \end{pmatrix}, \end{aligned}$$

where the coefficients $\tilde{c}_1(j),\, \tilde{c}_2(j)$, and $\tilde{c}_3(j)$ are given by (35), (36) and (38).

Finally, by replacing the terms $a_1,\, a_2,\, b_1$ and $b_2$, the matrix $\tilde{C}_{j-1}$ is equal to:

$$\begin{aligned} \tilde{C}_{j-1}\!=\!A \begin{pmatrix} (4\phi _{0}^4-4\phi _{0}^2+1) \tilde{c}_1(j) + \frac{2\phi _{0}^2(1-2\phi _{0}^2)}{\gamma _{0}^2}\tilde{c}_2(j)+\frac{\phi _{0}^4}{\gamma _{0}^4}\tilde{c}_3(j)&{} \frac{\phi _{0}(2\phi _{0}^2-1)}{2\gamma _{0}^2}\tilde{c}_1(j) + \frac{\phi _{0}(1-3\phi _{0}^2)}{2\gamma _{0}^4}\tilde{c}_2(j)+ \frac{\phi _{0}^3}{2\gamma _{0}^6} \tilde{c}_3(j)\\ \frac{\phi _{0}(2\phi _{0}^2-1)}{2\gamma _{0}^2}\tilde{c}_1(j) + \frac{\phi _{0}(1-3\phi _{0}^2)}{2\gamma _{0}^4}\tilde{c}_2(j)+ \frac{\phi _{0}^3}{2\gamma _{0}^6} \tilde{c}_3(j) &{} \frac{\phi _{0}^2}{4\gamma _{0}^4} \tilde{c}_1(j)+\frac{-\phi _{0}^2}{2\gamma _{0}^6}\tilde{c}_2(j) + \frac{\phi _{0}^2}{4\gamma _{0}^8} \tilde{c}_3(j)\\ \end{pmatrix}, \end{aligned}$$

where $A=\frac{\phi _{0}^2}{2\pi \gamma _{0}^2(1-\phi _0^2)^2}$.

Asymptotic behaviour of the covariance matrix $\varOmega _{j-1}(\theta _0)$: By the stationary assumption $|\phi _{0}|<1$, the limits of the following terms are:

$$\begin{aligned} \lim _{j\rightarrow \infty }V_{j}=\frac{\gamma _{0}^2}{2} \quad \text{ and}\quad \lim _{j \rightarrow \infty }\fancyscript{V}=\frac{\gamma _{0}^2}{2}, \end{aligned}$$

and

$$\begin{aligned} \lim _{j\rightarrow \infty }\tilde{c}_{1}(j)=\frac{\gamma _{0}^4}{8}, \lim _{j\rightarrow \infty }\tilde{c}_{2}(j)=\frac{3\gamma _{0}^6}{16}, \lim _{j\rightarrow \infty }\tilde{c}_{3}(j)=\frac{9\gamma _{0}^8}{32}. \end{aligned}$$

Therefore,

$$\begin{aligned}&\lim _{j\rightarrow \infty }\tilde{C}_{j-1}=\begin{pmatrix} \frac{\phi _{0}^2\gamma _{0}^2(2-\phi _{0}^2)^2}{64\pi (1-\phi _{0}^2)^2} &{} \frac{\phi _{0}^3(2-\phi _{0}^2)}{128\pi (1-\phi _{0}^2)^2}\\ \frac{\phi _{0}^3(2-\phi _{0}^2)}{128\pi (1-\phi _{0}^2)^2} &{} \frac{\phi _{0}^4}{256\pi (1-\phi _{0}^2)^2\gamma _{0}^2} \end{pmatrix}=P_1. \end{aligned}$$

We obtain:

$$\begin{aligned} \lim _{j\rightarrow \infty }\mathrm{Cov}\left( \nabla _{\theta }m_{\theta _0}(Y_{1}),\nabla _{\theta }m_{\theta _0}(Y_{j})\right)&= 4 \lim _{j\rightarrow \infty }(\tilde{C}_{j-1}-P_1)\\&= 0_{\fancyscript{M}_{2\times 2}}. \end{aligned}$$

We conclude that the covariance between the two vectors $\nabla _{\theta }m_{\theta _0}(Y_{1}), \nabla _{\theta }m_{\theta _0}(Y_{j})$ vanishes when the lag between the two observations $Y_{1}$ and $Y_{j}$ goes to the infinity.

Calculus of $V_{\theta _{0}}$: The Hessian matrix $V_{\theta _0}$ is given in Eq. (27).

1.2 C.2 The SV model

1.2.1 C.2.1 Contrast function

The $\mathbb L _2$-norm and the Fourier transform of the function $l_{\theta }$ are the same as the Gaussian AR(1) model. The only difference is the law of the measurement noise which is a log-chi-square for the log-transform SV model.

Consider the random variable $\varepsilon =\beta \log (X^2)-\tilde{\fancyscript{E}}$ where $\tilde{\fancyscript{E}}=\beta \mathbb{E }[\log (X^{2})]$ such that $\varepsilon$ is centered. The random variable $X$ is a standard Gaussian random. The Fourier transform of $\varepsilon$ is given by:

$$\begin{aligned} \mathbb{E }\left[ \exp \left( i\varepsilon y\right) \right]&= \exp \left( -i\tilde{\fancyscript{E}}y\right) \mathbb{E }\left[ X^{2i\beta y}\right] \\&= \exp \left( -i\tilde{\fancyscript{E}}y\right) \frac{1}{\sqrt{2\pi }}\int \limits _{-\infty }^{+\infty } x^{2i\beta y}\exp \left( -\frac{x^2}{2}\right) dx \end{aligned}$$

By a change of variable $z=\frac{x^2}{2}$, one has:

$$\begin{aligned} \mathbb{E }\left[ \exp \left( i \varepsilon y\right) \right]&= \exp \left( -i\tilde{\fancyscript{E}}y\right) \frac{2^{i\beta y}}{\sqrt{\pi }}\underbrace{\int _{0}^{+\infty } z^{i\beta y-\frac{1}{2}}e^{-z}dz}_{\varGamma \left( \frac{1}{2}+i\beta y\right) }\\&= \exp \left( -i\tilde{\fancyscript{E}}y\right) \frac{2^{i\beta y}}{\sqrt{\pi }}\varGamma \left( \frac{1}{2}+i\beta y\right) , \end{aligned}$$

and the expression (14) of the contrast function follows with $u_{l_{\theta }}(y)=\frac{1}{2\sqrt{\pi }} \left( \frac{-i\phi y\gamma ^2\exp \left( \frac{-y^2}{2}\gamma ^2\right) }{\exp \left( -i\tilde{\fancyscript{E}}y\right) 2^{i\beta y}\varGamma \left( \frac{1}{2}+i\beta y\right) }\right)$.

1.2.2 C.2.2 Checking assumption of Theorem 1

Regularity conditions: The proof is essentially the same as for the Gaussian case since the functions $l_{\theta }(x)$ and $\mathbf P m_{\theta }$ are the same. We need only to check the assumptions (C) and (T). These assumptions are satisfied since Fan (1991) showed that the noises $\varepsilon _i$ have a Fourier transform $f^{*}_{\varepsilon }$ which satisfies:

$$\begin{aligned} |f^{*}_{\varepsilon }(x)|=\sqrt{2}\exp \left( -\frac{\pi }{2} |x|\right) \left( 1+O\left( \frac{1}{|x|}\right) \right) , \quad |x|\rightarrow \infty , \end{aligned}$$

which means that $f_{\varepsilon }$ is super-smooth in its terminology. Furthermore, by the compactness of the parameter space $\varTheta$ and as the functions $l^{*}_{\theta }$, and for $j,k \in \left\{ 1, 2\right\}$, the functions $(\frac{{\partial }l_{\theta }}{{\partial }\theta _j })^* \, (\frac{{\partial }^{2} l_{\theta }}{{\partial }\theta _j {\partial }\theta _k})^{*}$, have the following form $C_1(\theta )P(x) \exp \left( -C_2(\theta )x^{2}\right)$ where $C_1(\theta )$ and $C_2(\theta )$ are two constants well defined in the parameter space $\varTheta$ with $C_2(\theta )>0$, we obtain:

$$\begin{aligned} \left\{ \begin{array}{ll} \mathbb{E }\left( \left| Y_{2}u^*_{\nabla _{\theta }l_{\theta }}(Y_{1}) \right| ^{2+\delta }\right) <\infty \qquad \qquad \text{ for} \text{ some} \delta >0,\\ \mathbb{E }\left( \sup _{\theta \in \fancyscript{U}}\left\| Y_{2}u^{*}_{\nabla _{\theta }^{2}l_{\theta }}(Y_{1})\right\| \right) <\infty \qquad \text{ for} \text{ some} \text{ neighbourhood} \fancyscript{U} \text{ of} \theta _0 . \end{array} \right. \end{aligned}$$

1.2.3 C.2.3 Expression of the covariance matrix

As, the functions $l_{\theta }(x)$ and $\mathbf P m_{\theta }$ are the same for the two models, the expressions of the matrix $V_{\theta _0}$ and $\varOmega _{j}(\theta _0)$ are given in Lemma 3. We need only to use an estimator of $P_{2}=\mathbb{E }[Y^{2}_{2}(u^{*}_{\nabla l_{\theta }}(Y_{1}))^2]$ since we can just approximate $u^{*}_{\nabla l_{\theta }}(y)$. A natural and consistent estimator of $P_2$ is given by:

$$\begin{aligned} \widehat{P}_{2}=\frac{1}{n}\sum _{i=1}^{n-1}\left( Y^{2}_{i+1}(u^{*}_{\nabla l_{\theta }}(Y_i))^2\right) \!, \end{aligned}$$

(39)

Remark 7

In some models, the covariance matrix $\varOmega _{j}(\hat{\theta }_n)$ cannot be explicitly computable. We refer the reader to Hayashi (2000) chapter 6 Section 6.6 p.408 for this case.

Appendix D: EM algorithm

We first refer to Dempster et al. (1977) for general details on the EM algorithm. The EM algorithm is an iterative procedure for maximizing the log-likelihood $l(\theta )=\log (f_{\theta }(Y_{1:n}))$. Suppose that after the $k^{th}$ iteration, the estimate for $\theta$ is given by $\theta _k$. Since the objective is to maximize $l(\theta )$, we want to compute an updated $\theta$ such that:

$$\begin{aligned} l(\theta )>l(\theta _k) \end{aligned}$$

Hidden variables can be introduced for making the ML estimation tractable. Denote the hidden random variables $U_{1:n}$ and a given realization by $u_{1:n}$. The total probability $f_{\theta }(Y_{1:n})$ can be written as:

$$\begin{aligned} f_{\theta }(Y_{1:n})=\sum _{u_{1:n}}^{}f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) \end{aligned}$$

Hence,

$$\begin{aligned}&l(\theta )-l(\theta _k)\nonumber \\&\quad =\log (f_{\theta }(Y_{1:n}))-\log (f_{\theta _k}(Y_{1:n}))\nonumber \\&\quad =\log \left( \sum _{u_{1:n}}^{}f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n})\!\right) \!-\!\log (f_{\theta _k}(Y_{1:n}))\nonumber \\&\quad =\log \left( \sum _{u_{1:n}}^{}f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n})\frac{f_{\theta _k}(u_{1:n}\vert Y_{1:n})}{f_{\theta _k}(u_{1:n}\vert Y_{1:n})}\right) -\log (f_{\theta _k}(Y_{1:n}))\nonumber \\&\quad =\log \left( \! \sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})\frac{f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) }{f_{\theta _k}(u_{1:n}\vert Y_{1:n})}\!\right) \!-\!\log (f_{\theta _k}(Y_{1:n}))\qquad \qquad \end{aligned}$$

(40)

$$\begin{aligned}&\quad \ge \sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})\log \left( \! \!\frac{f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) }{f_{\theta _k}(u_{1:n}\vert Y_{1:n})}\!\right) \!-\!\log (f_{\theta _k}(Y_{1:n}))\qquad \qquad \end{aligned}$$

(41)

$$\begin{aligned}&\quad =\sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})\log \left( \frac{f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) }{f_{\theta _k}(u_{1:n}\vert Y_{1:n})}\right) \nonumber \\&\qquad -\log (f_{\theta _k}(Y_{1:n}))\sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n}) \\&\quad =\sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})\log \left( \frac{f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) }{f_{\theta _k}(u_{1:n}\vert Y_{1:n})f_{\theta _k}(Y_{1:n})}\right) \nonumber \\&\quad =\varDelta (\theta , \theta _k).\nonumber \end{aligned}$$

(42)

In going from Eq. (40) to (41) we use the Jensen inequality: $\log \sum _{i=1}^{n} \lambda _i x_i \ge \sum _{i=1}^{n} \lambda _i \log (x_i )$ for constants $\lambda _i \ge 0$ with $\sum _{i=1}^{n} \lambda _i=1$. And in going from Eq. (41) to (42) we use the fact that $\sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})=1$. Hence,

$$\begin{aligned} l(\theta )\ge l(\theta _k)+\varDelta (\theta , \theta _k)=\fancyscript{L}(\theta , \theta _k) \quad \text{ and}\quad \varDelta (\theta , \theta _k) =0 \text{ for} \theta =\theta _k \end{aligned}$$

The function $\fancyscript{L}(\theta , \theta _k)$ is bounded by the log-likelihood function $l(\theta )$ and they are equal when $\theta =\theta _k$. Consequently, any $\theta$ which increases $\fancyscript{L}(\theta , \theta _k)$ will increases $l(\theta )$. The EM algorithm selects $\theta$ such that $\fancyscript{L}(\theta , \theta _k)$ is maximized. We denote this updated value $\theta _{k+1}$. Thus,

$$\begin{aligned} \theta _{k+1}&= \arg \max _{\theta }\left\{ l(\theta _k)+ \sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})\log \left( \frac{f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) }{f_{\theta _k}(u_{1:n}\vert Y_{1:n})f_{\theta _k}(Y_{1:n})}\right) \right\} \nonumber \\&= \arg \max _{\theta }\left\{ \sum _{u_{1:n}}^{} f_{\theta _k}(u_{1:n}\vert Y_{1:n})\log f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n}) \right\} \nonumber \\&\text{ if} \text{ we} \text{ drop} \text{ the} \text{ terms} \text{ which} \text{ don't} \text{ depend} \text{ on} \theta .\nonumber \\&= \arg \max _{\theta } \left\{ \mathbb{E }[\log f_{\theta }(Y_{1:n}\vert u_{1:n})f_{\theta }(u_{1:n})]\right\} \nonumber \\&\text{ where} \text{ the} \text{ expectation} \text{ is} \text{ according} \text{ to} f_{\theta _k}(u_{1:n}\vert Y_{1:n}). \end{aligned}$$

(43)

1.1 D.1 Simulated expectation maximization estimator

Here, we describe the SIEMLE proposed by Kim et al. (1994) for the SV model, these authors retain the linear log-transform model given in (13). However, instead of approximating the log-chi-square distribution of $\varepsilon _i$ with a Gaussian distribution, they approximate $\varepsilon _i$ by a mixture of seven Gaussian. The distribution of the noise is given by:

$$\begin{aligned} f_{\varepsilon _i}(x)&\approx \sum _{j=1}^{7}q_j \times g_{(m_j,v_j^2)}(x),\\&\approx \sum _{j=1}^{7}q_jf_{\varepsilon _i|s_i=j}(x) \end{aligned}$$

where $g_{(m,v)}(x)$ denotes the Gaussian distribution of $\varepsilon _i$ with mean $m$ and variance $v$, and $f_{\varepsilon _i|s_i=j}(x)$ is a Gaussian distribution conditional to an indicator variable $s_i$ at time $i$ and the variables $q_j, j=1\ldots , 7$ are the given weights attached to each component and such that $\sum _{j=1}^{7}q_j=1$. Note that, most importantly, given the indicator variable $s_i$ at each time $i$, the log-transform model is Gaussian. That is:

$$\begin{aligned} f_{\theta }(Y_i|s_i=j,X_i)\sim g_{(X_i+m_j, v_j^2)}. \end{aligned}$$

Then, conditionally to the indicator variable $s_i$, the SV model becomes a Gaussian state-space model and the Kalman filter can be used in the SIEMLE in order to compute the log-likelihood function given by:

$$\begin{aligned} \log f_{\theta }(Y_{1:n}|s_{1:n})= -\frac{n}{2}\log (2\pi )-\frac{1}{2}\sum _{i=1}^{n}\log F_i-\frac{1}{2}\sum _{i=1}^{n}\frac{\nu _{i}^{2}}{F_i}, \end{aligned}$$

with $\nu _i=(Y_i-\hat{Y}_i^{-}-m_{s_i})$ and $F_i=\mathbb{V }_{\theta }[\nu _i]=P_i^{-}+v^{2}_{s_i}$. The quantities $\hat{Y}_i^{-}=\mathbb{E }_{\theta }[Y_i| Y_{1:i-1}]$ and $P_i^{-}=\mathbb{V }_{\theta }[(X_i-\hat{X}_{i}^{-})^{2}]$ are computed by the Kalman filter.

Hence, if we consider that the missing data $u_{1:n}$ for the EM correspond to the indicator variables $s_{1:n}$, then according to Eq. (43) and since $f(s_{1:n})$ do not depend on $\theta$, the Maximization step is:

$$\begin{aligned} \theta _{k+1}=\arg \max _{\theta } \left\{ \mathbb{E }[\log f_{\theta }(Y_{1:n}|s_{1:n})]\right\} =\arg \max _{\theta }Q(\theta ,\theta _k) \end{aligned}$$

where the expectation is according to $f_{\theta _k}(s_{1:n}\vert Y_{1:n})$. Nevertheless, for the SV model, the problem with the EM algorithm is that the density $f_{\theta }(s_{1:n}|Y_{1:n})$ is unknown. The main idea consists in introducing a Gibbs algorithm to obtain $\tilde{M}$ draws $s^{(1)}_{1:n},\ldots , s^{(\tilde{M})}_{1:n}$ from the law $f_{\theta }(s_{1:n}|Y_{1:n})$. Hence, the objective function $Q(\theta ,\theta _{k})$ is approximated by:

$$\begin{aligned} \tilde{Q}(\theta ,\theta _{k})=\frac{1}{\tilde{M}}\sum _{l=1}^{\tilde{M}}\log f_{\theta }\left( Y_{1:n}|s_{1:n}^{(l)}\right) \end{aligned}$$

Then, the simulated EM algorithm for the SV model is as follows: Let $C>0$ be a threshold to stop the algorithm and $\theta _{k}$ a given arbitrary value of the parameter. While $|\theta _{k}-\theta _{k-1}|>C,$

• 1. Apply the Gibbs sampler as follows: The Gibbs Sampler: Choose arbitrary starting values $X_{1:n}^{(0)}$, and let $l=0$.

  • (a) Sample $s_{1:n}^{(l+1)}\sim f_{\theta _{k}}(s_{1:n}|Y_{1:n},X_{1:n}^{(l)})$.

  • (b) Sample $X_{1:n}^{(l+1)}\sim f_{\theta _{k}}(X_{1:n} |Y_{1:n} ,s_{1:n}^{(l+1)})$.

  • (c) Set $l=l+1$ and goto (a).

• 2. $\theta _{k+1}=\arg \max _{\theta } \tilde{Q}(\theta ,\theta _{k})$.

Step (a): to sample the vector $s_{1:n}$ from its full conditional density, we sample each $s_i$ independently. We have:

$$\begin{aligned} f_{\theta _{k}}(s_{1:n}|Y_{1:n},X_{1:n})=\prod _{r=1}^{n}f_{\theta _{k}}(s_{r}|Y_{r},X_{r})\propto \prod _{r=1}^{n}f_{\theta _{k}}(Y_{r}|s_{r},X_{r})f(s_r), \end{aligned}$$

and $f_{\theta _{k}}(Y_{r}|s_{r}=j,X_{r})\propto g_{(X_{r}+m_{j}, v_{j}^2)}$ for $j=1\ldots ,7.$ And the step (b) of the Gibbs sampler is conducted by the Kalman filter since the model is Gaussian.