Generalized Linear Spectral Models for Locally Stationary Processes

Abstract

A class of parametric models for locally stationary processes is introduced. The class depends on a power parameter that applies to the time-varying spectrum so that it can be locally represented by a (finite low dimensional) Fourier polynomial. The coefficients of the polynomial have an interpretation as time-varying autocovariances, whose dynamics are determined by a linear combination of smooth transition functions, depending on some static parameters. Frequency domain estimation is based on the generalized Whittle likelihood and the pre-periodogram, while model selection is performed through information criteria. Change points are identified via a sequence of score tests. Consistency and asymptotic normality are proved for the parametric estimators considered in the paper, under weak assumptions on the time-varying parameters.

Appendices

13.6 Proofs

Proof of Theorem 13.1

The proofs of consistency and asymptotic normality of the estimator $\hat{\theta }_n$ are based on results by [14] on the empirical spectral process

$$E_n(\phi ) = \sqrt{n}(F_n(\phi ) - F(\phi ))$$

evaluated at $\phi = f_\theta (\cdot ,\omega )^{-1}$ and at $\phi = \frac{\partial }{\partial \theta } f_\theta (\cdot ,\omega )^{-1}$, respectively, where

$$F(\phi ) = \int _0^1 \int _{-\pi }^\pi \phi (u,\omega )f_\theta (u,\omega )\text{ d }\omega $$

is a spectral measure and

$$F_n(\phi ) = \frac{1}{n}\sum _{t=1}^n \int _{-\pi }^\pi \phi \left( \frac{t}{n},\omega \right) J_n\left( \frac{t}{n},\omega \right) \text{ d }\omega $$

is an empirical spectral measure based on the pre-periodogram, defined in Eq. (13.17).

Under the assumptions of Theorem 13.1, almost sure consistency of $\hat{\theta }_n$ requires the uniform convergence of $\ell _n(\theta )$ to $\ell (\theta )$, defined in Eqs. (13.19) and (13.20), respectively,

$$\begin{aligned} \sup _{\theta \in \Theta } |\ell _n(\theta )-\ell (\theta )|\rightarrow 0, \;\;\; n\rightarrow \infty . \end{aligned}$$

Note that ([14, Example 3.1])

$$\begin{aligned} \ell _n(\theta )-\ell (\theta ) =&\int _{0}^1\int _{-\pi }^\pi \left\{ \log f_\theta \left( u,\omega \right) + \frac{f(u,\omega )}{f_{\theta }\left( u,\omega \right) }\right\} \text{ d }\omega \\&- \frac{1}{n}\sum _{t=1}^n\int _{-\pi }^\pi \left\{ \log f_\theta \left( \frac{t}{n},\omega \right) + \frac{J_n\left( \frac{t}{n},\omega \right) }{f_{\theta }\left( \frac{t}{n},\omega \right) } \right\} \text{ d }\omega \\ =&\int _{0}^1\int _{-\pi }^\pi \frac{1}{f_{\theta }\left( u,\omega \right) }f(u,\omega ) \text{ d }\omega \\&- \frac{1}{n}\sum _{t=1}^n\int _{-\pi }^\pi \frac{1}{f_{\theta }\left( \frac{t}{n},\omega \right) } J_n\left( \frac{t}{n},\omega \right) \text{ d }\omega + R_{\log }(f_\theta ) \\&= F(1/f_\theta ) - F_n(1/f_\theta ) + R_{\log }(f_\theta ), \end{aligned}$$

where $F,F_n$ are defined above and

$$\begin{aligned} R_{\log }(f_\theta ) = \int _{-\pi }^\pi \left[ \int _{0}^1 \log f_\theta \left( u,\omega \right) \text{ d }u - \frac{1}{n}\sum _{t=1}^n \log f_\theta \left( \frac{t}{n},\omega \right) \right] \text{ d }\omega . \end{aligned}$$

(13.25)

Hence, if $\sup _{\theta \in \Theta }|R_{\log }(f_\theta )|$ is small, the convergence of $\ell _n(\theta )-\ell (\theta )$ may be controlled by $E_n(1/f_\theta )$, since, by Assumptions 13.1 and 13.2, we obtain

$$\begin{aligned} \sup _{\theta \in \Theta } |\ell _n(\theta )-\ell (\theta )| =&\sup _{\theta \in \Theta }\bigg | \frac{1}{\sqrt{n}} E_n\Big (\frac{1}{f_\theta } \Big ) + R_{\log }(f_\theta ) \bigg | \\ \le&\sup _{\theta \in \Theta }\bigg | \frac{1}{\sqrt{n}} E_n\Big (\frac{1}{f_\theta } \Big )\bigg | + \sup _{\theta \in \Theta } | R_{\log }(f_\theta )|. \end{aligned}$$

Note that by Assumptions 13.1 and 13.2, we obtain that the empirical spectral process $E_n(1/f_\theta )$ has bounded variation in both the components of $f_\theta (u, \omega )$, see also the discussion in [14]. Then, the uniform convergence of $\ell _n(\theta )-\ell (\theta )$ could be easily obtained by Glivenko–Cantelli-type convergence results, see, for example, [14, Theorem 2.12], which ensures that we have

$$\begin{aligned} \sup _{\theta \in \Theta }\bigg | \frac{1}{\sqrt{n}} E_n\Big (\frac{1}{f_\theta } \Big )\bigg | \xrightarrow {P} 0. \end{aligned}$$

To show that this result is valid also in our case, it suffices to verify [15, Assumption 2.4 (c)].

First, we note that by construction the reciprocal $1/(f_\theta (u,\omega ))$ is an element of $\mathcal {F}_\lambda$ (when $\lambda =-1$) and the functions in $\mathcal {F}_\lambda$ are not indexed by n. Moreover, the implied spectral density $f_\theta (u,\omega )\in \mathcal {F}_\lambda$ is bounded away from zero.

Let us consider

$$\begin{aligned} f_\theta (u, \omega )^\lambda&= \sigma ^2_\lambda (u)|\beta (u, \omega )|^2 \\&= \sigma ^2_\lambda (u) \bigg ( \sum _{j=0}^{K} \beta _\lambda ^2(u, j) + 2 \sum _{k=1}^{K} \sum _{j=0}^{K-k} \beta _\lambda (u, j)\beta _\lambda (u, j+k) \cos (\omega k) \bigg ). \end{aligned}$$

By Kolmogorov’s formula ([7, Theorem 5.8.1]) we have

$$\begin{aligned} \int _{-\pi }^\pi \log f_{\theta }(u, \omega ) \textrm{d}\omega = 2\pi \lambda \log \Big ( \frac{\sigma ^2_\lambda (u)}{2\pi } \Big ). \end{aligned}$$

We define the class of functions

$$\begin{aligned} \mathcal {S}_{\lambda } = \bigg \{ \sigma ^2_\lambda (u):&[0, 1] \mapsto \mathbb {R}_+, \, \\&\text {increasing with}\, 0< L \le \inf _{u} \sigma ^2_\lambda (u) \le \sup _{u} \sigma ^2_\lambda (u) \le B < \infty , \, \lambda \in \mathbb {R} \bigg \}, \end{aligned}$$

and note that for uniformly bounded monotonic functions, the metric entropy is bounded

$$\begin{aligned} H(\eta , \mathcal {S}_\lambda , \rho _2) \le C_2 B \epsilon ^{-1}, \end{aligned}$$

for some $\epsilon > 0$, where $C_2$ is a constant, see [19]. Moreover, $\exists \,\, 0< M_{\star } \le 1 \le M^{\star } < \infty \,, \, M_{\star } \le f_\theta (u,\omega ) \le M^{\star } \,\,, \forall \,\, u,\omega \,\, \text {and} \,\, f_\theta (u,\omega ) \in \mathcal {F}_\lambda$. However, we do not need the lower bound $M_{\star }$ and then we may set $M_{\star } = 1$.

In addition, to ensure that

$$\begin{aligned} \bigg | 1 + \sum _{k=1}^{K} \beta _\lambda (u, k)z^k \bigg | \ne 0, \, \forall 0< |z| \le 1+\delta , \, \delta >0, \, \lambda \in \mathbb {R}, \end{aligned}$$

we choose

$$\begin{aligned} \mathcal {B}_{\lambda } = \bigg \{ \boldsymbol{\beta }_\lambda (u) =&(\beta _\lambda (u, 1), \dots , \beta _\lambda (u, K))^\prime : [0, 1]^K \mapsto \mathbb {R}^K, \, \\&\sup _{u} \bigg | 1 + \sum _{k=1}^{K} \beta _\lambda (u, k)z^k \bigg | \ne 0, \, \forall 0< |z| \le 1+\delta , \, \delta >0, \, \lambda \in \mathbb {R} \bigg \}, \end{aligned}$$

such that $\mathcal {B}_{\lambda } \in \mathcal {W}_{2}^{\alpha }$, that is, the Sobolev class of functions with $m\in \mathbb {N}$ smoothness parameter, such that the first $m \ge 1$ derivatives exist with finite $L_2$-norm, see [20]. For this class of functions, the metric entropy can be bounded by

$$\begin{aligned} H(\eta , \mathcal {B}_\lambda , \rho _2) \le A \epsilon ^{-1/m}, \end{aligned}$$

for some $A>0$ and $\forall \epsilon > 0$. It follows that the metric entropy of the class of the time-varying sdf is bounded by

$$\begin{aligned} H(\eta , \mathcal {F}_\lambda , \rho _2) \le A_2 B \epsilon ^{-(1+1/m)}. \end{aligned}$$

As a result, all the conditions of [13][Assumption 2.4 (c)] are easily satisfied, and hence we apply [13, Lemma A.2] to conclude that

$$\begin{aligned} \sup _{\theta \in \Theta } | R_{\log }(f_\theta )| = O\bigg ( \frac{v_{\Sigma }}{n} \bigg ), \end{aligned}$$

which implies $\sup _{\theta \in \Theta }|R_{\log }(f_\theta )| \rightarrow 0$ as $n \rightarrow \infty$.

Moreover, by [13, Theorem 2.6], we obtain the convergence of $\rho _2 (1/\hat{f}_\theta - 1/\hat{f}_\theta )$ by setting $\gamma = (1+1/m)$, so that

$$\begin{aligned} \rho _2\left( \frac{1}{\hat{f}_{\theta }}, \frac{1}{{f}_{\theta }} \right) = O_P \left( n^{- \frac{2-(1+1/m)}{4(1+1/m)}} (\log n)^{\frac{(1+1/m)-1}{2(1+1/m)}} \right) , m > 1. \end{aligned}$$

In conclusion, by compactness of $\Theta$ and the uniqueness of $\theta _0$ implied by Assumptions 13.3 and 13.4, we obtain the claimed convergence $\hat{\theta }_n \rightarrow _p \theta _0$, which concludes the proof.   $\square$

13.7 Proof of Theorem 13.2

Before presenting the proof of Theorem 13.2, we need to check the smoothness conditions given in [9, Assumption 2.1].

First, we note that

$$\sigma ^2_\lambda (u) = \exp \{ 2 \Psi (u)^\prime \theta _\sigma \},$$

where the $(q+1)$-vector $\Psi (u)$ is defined as

$$\Psi (u) = (1, \Psi (u, \gamma _1, \tau _1), \dots , \Psi (u, \gamma _q, \tau _q))^\prime ,$$

with $\Psi (u, \gamma _i, \tau _i)= [1+\exp \{ - \gamma _i(u-\tau _i) \}]^{-1}$ for $i = 1, \dots , q$. Moreover $\theta _\sigma = (\theta _{00}, \dots , \theta _{0q})^\prime$ and define $\boldsymbol{\gamma }=(\gamma _1, \dots , \gamma _q)^\prime$.

Therefore, the first derivatives of $\sigma ^2_\lambda (u)$ with respect to $\theta _\sigma$ and $\boldsymbol{\gamma }$ are

$$\begin{aligned} \nabla \sigma _\lambda ^2(u) = \left[ \begin{array}{c} \frac{\partial \sigma _\lambda ^2(u)}{\partial \theta _\sigma } \\ \frac{\partial \sigma _\lambda ^2(u)}{\partial \boldsymbol{\gamma }} \end{array} \right] , \end{aligned}$$

where

$$\begin{aligned} \frac{\partial \sigma _\lambda ^2(u)}{\partial \theta _\sigma } = 2\exp \{ 2 \Psi (u)^\prime \theta _\sigma \}\Psi (u), \end{aligned}$$

and

$$\begin{aligned} \frac{\partial \sigma _\lambda ^2(u)}{\partial \boldsymbol{\gamma }} = 2\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} \Big [\theta _\sigma \odot \Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u} - \boldsymbol{\tau })\Big ], \end{aligned}$$

with $\boldsymbol{1}_{q+1}$ being a $(q+1)$-vector of ones, $\boldsymbol{u} = (0, u, \dots , u)^\prime \in [0, 1]^{q+1}$ and $\boldsymbol{\tau } = (0, \tau _1, \dots , \tau _{q})^\prime$.

The second derivatives of $\sigma ^2_\lambda (u)$ with respect to $\theta _\sigma$ and $\boldsymbol{\gamma }$ are

$$\begin{aligned} \nabla ^2 \sigma _\lambda ^2(u) = \left[ \begin{array}{cc} \frac{\partial ^2 \sigma _\lambda ^2(u)}{\partial \theta _\sigma \partial \theta _\sigma ^\prime } &{} \frac{\partial ^2 \sigma _\lambda ^2(u)}{\partial \theta _\sigma \partial \boldsymbol{\gamma }^\prime } \\ \frac{\partial ^2 \sigma _\lambda ^2(u)}{\partial \boldsymbol{\gamma } \partial \theta _\sigma ^\prime } &{}\frac{\partial \sigma _\lambda ^2(u)}{\partial \boldsymbol{\gamma } \partial \boldsymbol{\gamma }^\prime } \end{array} \right] , \end{aligned}$$

where

$$\begin{aligned} \frac{\partial ^2 \sigma _\lambda ^2(u)}{\partial \theta _\sigma \partial \theta _\sigma ^\prime } = 4\exp \{ 2 \Psi (u)^\prime \theta _\sigma \}\Psi (u)\Psi (u)^\prime , \end{aligned}$$

$$\begin{aligned} \frac{\partial \sigma _\lambda ^2(u)}{\partial \boldsymbol{\gamma } \partial \boldsymbol{\gamma }^\prime } =&4\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} \\&\,\,\times \Big [\theta _\sigma \theta _\sigma ^\prime \odot \text {diag}\Big (\Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })\Big )^2 \Big ]\\&+ 2\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} \\&\,\,\times \Big [ \text {diag}\Big (\theta _\sigma \odot \Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{1}_{q+1} - 2\Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })^2 \Big ) \Big ], \end{aligned}$$

and

$$\begin{aligned} \frac{\partial ^2 \sigma _\lambda ^2(u)}{\partial \theta _\sigma \partial \boldsymbol{\gamma }^\prime } =&4\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} \\&\,\,\times \Big [\Psi (u) \theta _\sigma ^\prime \odot \text {diag}\Big (\Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })\Big )^2 \Big ]\\&+ 2\exp \{ 2 \Psi (u)^\prime \theta _\sigma \}\\&\,\,\times \Big [ \text {diag}\Big (\Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })\Big ) \Big ]. \end{aligned}$$

Furthermore, we require the first and second derivatives of $\beta _\lambda (u, \omega )$ with respect to $\theta _\sigma$ and $\boldsymbol{\gamma }$. However, we recall that for estimation purposes we adopt the parametrization $\beta _\lambda (u, k)$ for $k=1,\dots ,K$ and each $\beta _\lambda (u, k)$ from the last iteration of the Durbin–Levinson recursion. As a result, we have $\beta _\lambda (u, k) = \zeta _\lambda (u, k)$, where $\zeta _\lambda (u, k)$ denotes the generalized partial autocorrelation coefficients, and we set, for $k=1, \dots , K$,

$$\begin{aligned} \zeta _\lambda (u, k) = \frac{\exp \{2 \Psi (u)^\prime \theta _\sigma \} - 1}{\exp \{2 \Psi (u)^\prime \theta _\sigma \} + 1}. \end{aligned}$$

Thus, it suffices to derive $\zeta _\lambda (u, k)$ with respect to $\theta _\sigma$ and $\boldsymbol{\gamma }$. Then we have

$$\begin{aligned} \nabla \zeta _\lambda (u, k) = \left[ \begin{array}{c} \frac{\partial \zeta _\lambda (u, k)}{\partial \theta _\sigma } \\ \frac{\partial \zeta _\lambda (u, k)}{\partial \boldsymbol{\gamma }} \end{array} \right] , \end{aligned}$$

where

$$\begin{aligned} \frac{\partial \zeta _\lambda (u, k)}{\partial \theta _\sigma } = \frac{4 \exp \{ 2 \Psi (u)^\prime \theta _\sigma \} }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^2} \Psi (u) \end{aligned}$$

and

$$\begin{aligned} \frac{\partial \zeta _\lambda (u, k)}{\partial \boldsymbol{\gamma }} = \frac{4 \exp \{ 2 \Psi (u)^\prime \theta _\sigma \} }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^2} \Big [\theta _\sigma \odot \Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u} - \boldsymbol{\tau })\Big ]. \end{aligned}$$

The second derivatives of $\zeta _\lambda (u, k)$ with respect to $\theta _\sigma$ and $\boldsymbol{\gamma }$ are

$$\begin{aligned} \nabla ^2 \zeta _\lambda (u, k) = \left[ \begin{array}{cc} \frac{\partial ^2 \zeta _\lambda (u, k)}{\partial \theta _\sigma \partial \theta _\sigma ^\prime } &{} \frac{\partial ^2 \zeta _\lambda (u, k)}{\partial \theta _\sigma \partial \boldsymbol{\gamma }^\prime } \\ \frac{\partial ^2 \zeta _\lambda (u, k)}{\partial \boldsymbol{\gamma } \partial \theta _\sigma ^\prime } &{}\frac{\partial \zeta _\lambda (u, k)}{\partial \boldsymbol{\gamma } \partial \boldsymbol{\gamma }^\prime } \end{array} \right] , \end{aligned}$$

where

$$\begin{aligned} \frac{\partial ^2 \zeta _\lambda (u, k)}{\partial \theta _\sigma \partial \theta _\sigma ^\prime } = \frac{8 \exp \{ 2 \Psi (u)^\prime \theta _\sigma \} (1-\exp \{ 2 \Psi (u)^\prime \theta _\sigma \}) }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^3} \Psi (u) \Psi (u)^\prime , \end{aligned}$$

$$\begin{aligned} \frac{\partial \zeta _\lambda (u, k)}{\partial \boldsymbol{\gamma } \partial \boldsymbol{\gamma }^\prime } =&\frac{8 \exp \{ 2 \Psi (u)^\prime \theta _\sigma \} (1-\exp \{ 2 \Psi (u)^\prime \theta _\sigma \}) }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^3}\\&\times \Big [\Psi (u) \Psi (u)^\prime \odot \text {diag}\Big (\Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })\Big )^2 \Big ]\\&+ \frac{4 \exp \{ 2 \Psi (u)^\prime \theta _\sigma \} }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^2}\\&\times \Big [ \text {diag}\Big (\theta _\sigma \odot \Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{1}_{q+1} - 2\Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })^2 \Big ) \Big ] , \end{aligned}$$

and finally

$$\begin{aligned} \frac{\partial ^2 \zeta _\lambda (u, k)}{\partial \theta _\sigma \partial \boldsymbol{\gamma }^\prime } =&\frac{16 \exp \{ 4 \Psi (u)^\prime \theta _\sigma \} }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^3}\\&\times \Big [\Psi (u) \theta _\sigma ^\prime \odot \text {diag}\Big (\Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })\Big ) \Big ]\\&+ \frac{4 \exp \{ 2 \Psi (u)^\prime \theta _\sigma \} }{(\exp \{ 2 \Psi (u)^\prime \theta _\sigma \} + 1)^2}\\&\times \Big [ \text {diag}\Big (\Psi (u) \odot (\boldsymbol{1}_{q+1} - \Psi (u)) \odot (\boldsymbol{u}-\boldsymbol{\tau })\Big ) \Big ] . \end{aligned}$$

Then, we conclude that all the derivatives derived above reveal that the first and second derivatives $\nabla \sigma _\lambda ^2(u)$, $\nabla ^2 \sigma _\lambda ^2(u)$, $\nabla \beta _\lambda (u, \omega )$, and $\nabla ^2 \beta _\lambda (u, \omega )$ satisfy all the smoothness conditions given in [9, Assumption 2.1]. Therefore, we can prove the asymptotic normality of our estimator in (13.18) by checking the remaining relevant conditions.

Our estimator solves the likelihood equations $\nabla \ell _n(\hat{\theta }_n ) = 0$. Hence the mean value theorem gives

$$\begin{aligned} \nabla \ell _n(\hat{\theta }_n ) - \nabla \ell _n(\theta _0) = \nabla ^2 \ell _n(\theta _n^\star ) (\hat{\theta }_n - \theta _0), \end{aligned}$$

where $|\theta _n^\star - \theta _0| \le |\hat{\theta }_n - \theta _0|$ and, since $\hat{\theta }_n \in \Theta$, $\nabla \ell _n(\hat{\theta }_n ) = 0$. As we have proved in Theorem 1, Assumptions 13.1–13.4 ensure the consistency of $\hat{\theta }_n$, and thus a central limit theorem for $\sqrt{n}(\hat{\theta }_n - \theta _0)$ could be obtained by following analogous arguments discussed in the proof of [9, Theorem 2.4]. Then, the result follows by proving that

• 1. $\sqrt{n}\nabla \ell _n(\theta _0) \rightarrow _d N(\boldsymbol{0}, \Gamma )$,

• 2. $\nabla ^2 \ell _n(\theta _n^\star ) - \nabla ^2 \ell _n(\theta _0) \rightarrow _p 0$, and

• 3. $\nabla ^2 \ell _n(\theta _0) \rightarrow _p \mathbb {E}[\nabla ^2 \ell _n(\theta _0)]$ where $\mathbb {E}[\nabla ^2 \ell _n(\theta _0)] = V$.

However, as in [9, Remark 2.6], if the model is correctly specified we have $\Gamma =V$, where

$$\begin{aligned} V= \int _{0}^{1} \int _{-\pi }^{\pi } \nabla \log f_\theta (u, \omega ) \nabla \log f_\theta (u, \omega )^\prime \textrm{d}u \textrm{d}\omega . \end{aligned}$$

As far as convergence in distribution of the score vector in the item Assumption 13.7, we first prove the equicontinuity of $\nabla \ell _n(\theta )$. We use similar arguments as those used in the proof of Theorem 13.1. From Eq. (13.21), one has

$$\begin{aligned} \nabla \ell (\theta ) - \nabla \ell _n(\theta )&= \int _{-\pi }^{\pi } \Big [ \int _{0}^{1} \frac{f(u, \omega )}{f^2_\theta (u, \omega )} \nabla f_\theta (u, \omega ) \textrm{d}u \\&\,\, - \frac{1}{n} \sum _{t=1}^{n} \frac{J_n(\frac{t}{n}, \omega )}{f^2_\theta (\frac{t}{n}, \omega )} \nabla f_\theta \Big (\frac{t}{n}, \omega \Big ) \Big ] \textrm{d}\omega \\&\,\, + \int _{-\pi }^{\pi } \Big [ \int _{0}^{1} \frac{\nabla f_\theta (u, \omega )}{f_\theta (u, \omega )} \textrm{d}u - \frac{1}{n} \sum _{t=1}^{n} \frac{ \nabla f_\theta (\frac{t}{n}, \omega )}{f_\theta (\frac{t}{n}, \omega )} \Big ] \textrm{d}\omega . \end{aligned}$$

By Leibniz’s rule, it is easy to see that the preceding equation is equivalent to

$$\begin{aligned} \nabla \ell (\theta ) - \nabla \ell _n(\theta )&= \frac{1}{\sqrt{n}} E_n \Big ( \nabla \frac{1}{f_\theta } \Big )\\&\, + \int _{-\pi }^{\pi } \Big [ \int _{0}^{1} \nabla \log f_\theta (u, \omega ) \textrm{d}u - \frac{1}{n} \sum _{t=1}^{n} \nabla \log f_\theta \Big (\frac{t}{n}, \omega \Big ) \Big ] \textrm{d}\omega \\&= E_n \Big ( \nabla \frac{1}{f_\theta } \Big ) \\&\,+ \nabla \int _{-\pi }^{\pi } \Big [ \int _{0}^{1} \log f_\theta (u, \omega ) \textrm{d}u - \frac{1}{n} \sum _{t=1}^{n} \log f_\theta \Big (\frac{t}{n}, \omega \Big ) \Big ] \textrm{d}\omega \\&= E_n \Big ( \nabla \frac{1}{f_\theta } \Big ) + \nabla R_{\log } (f_\theta ), \end{aligned}$$

where $R_{\log } (f_\theta )$, defined in (13.25), was proved to be asymptotically negligible. The same bounds obtained in the proof of Theorem 13.1 can be applied to the class of functions

$$\begin{aligned} \mathcal {F}^\nabla _\lambda = \Big \{ \nabla f_{\theta }(u, \, \cdot \,), u \in [0, 1], \theta \in \Theta , \lambda \in \mathbb {R} \Big \}, \end{aligned}$$

such that, by using Assumptions 13.1 and 13.2, we get Glivenko–Cantelli-type convergence result

$$\begin{aligned} \sup _{\theta \in \theta } \bigg \Vert \frac{1}{\sqrt{n}} E_n \Big (\nabla \frac{1}{f_{\theta }}\Big ) \bigg \Vert \rightarrow _p 0, \end{aligned}$$

which implies the equicontinuity in probability of $\nabla \ell _n(\theta )$. Thus,

$$\begin{aligned} \sup _{\theta \in \Theta }| \nabla \ell (\theta ) - \nabla \ell _n(\theta ) | \rightarrow _p 0. \end{aligned}$$

Clearly, since by Assumptions 13.3 and 13.4 the parameter space $\Theta$ is compact with a unique minimum at $\theta _0\in \Theta$, we have $\nabla \ell (\theta _0) = 0$, and so, evaluating Eq. (13.21) at the true value $\theta _0$ is equivalent to write $\nabla \ell _n(\theta _0)$ as an empirical process of the form

$$\begin{aligned} \sqrt{n}\nabla \ell _n(\theta _0) = E_n \Big (\nabla \frac{1}{f_{\theta _0}}\Big ). \end{aligned}$$

Note also that, under the assumption that the model is correctly specified, it holds that $\mathbb {E}[\nabla \ell _n(\theta _0) ] = 0$, and therefore, we obtain

$$\begin{aligned} n \mathbb {V}[ \nabla \ell _n(\theta )] = n \mathbb {E} \Big [ \int _{-\pi }^{\pi } \frac{1}{n^2} \sum _{t=1}^{n} \frac{J^2_n(\frac{t}{n}, \omega )}{f^2_\theta (\frac{t}{n}, \omega )} \nabla \log f_\theta \Big (\frac{t}{n}, \omega \Big ) \nabla \log f_\theta \Big (\frac{t}{n}, \omega \Big )^\prime \textrm{d}\omega \Big ] \rightarrow \Gamma , \end{aligned}$$

since all the required Assumptions in [9, Lemma A.5] are fulfilled.

Now we focus on the item Assumption 13.7, which is satisfied if $\nabla ^2 \ell _n(\theta )$ is equicontinuous. From Eq. (13.22) and the results obtained in the proof of condition Assumption 13.7, one can express $\nabla ^2 \ell _n(\theta )$ as

$$\begin{aligned} \nabla ^2 \ell _n(\theta )&= \frac{1}{\sqrt{n}} E_n \Big (\nabla ^2 \frac{1}{f_{\theta }}\Big ) + \frac{1}{n} \sum _{t=1}^{n} \int _{-\pi }^{\pi } \frac{ 1 }{f^2_\theta (\frac{t}{n}, \omega )} \nabla f_\theta \Big (\frac{t}{n}, \omega \Big ) \nabla f_\theta \Big (\frac{t}{n}, \omega \Big )^\prime \textrm{d}\omega \nonumber \\&= \frac{1}{\sqrt{n}} E_n \Big (\nabla ^2 \frac{1}{f_{\theta }}\Big ) + \frac{1}{n} \sum _{t=1}^{n} \int _{-\pi }^{\pi } \nabla \log f_\theta \Big (\frac{t}{n}, \omega \Big ) \nabla \log f_\theta \Big (\frac{t}{n}, \omega \Big )^\prime \textrm{d}\omega , \end{aligned}$$

(13.26)

where

$$\begin{aligned} \nabla ^2 f_\theta (u, \omega )^{-1} = \frac{2}{f^3_\theta (u, \omega )} \nabla f_\theta (u, \omega ) \nabla f_\theta (u, \omega )^\prime - \frac{1}{f^2_\theta (u, \omega )} \nabla ^2 f_\theta (u, \omega ). \end{aligned}$$

Therefore, by similar arguments as above, under Assumptions 13.1 and 13.2, we can show the uniform convergence

$$\begin{aligned} \sup _{\theta \in \theta } \bigg \Vert \frac{1}{\sqrt{n}} E_n \Big (\nabla ^2 \frac{1}{f_{\theta }}\Big ) \bigg \Vert \rightarrow _p 0, \end{aligned}$$

which further implies the equicontinuity in probability of $\nabla ^2 \ell _n(\theta )$. Indeed, the second term in (13.26) converges to its expectation as $n \rightarrow \infty$, and, hence,

$$\begin{aligned} \sup _{\theta \in \Theta }| \nabla ^2 \ell (\theta ) - \nabla ^2 \ell _n(\theta ) | \rightarrow _p 0. \end{aligned}$$

The proof of the item Assumption 13.7 follows directly, since, under the maintained Assumptions, Theorem 13.1 implies that the second term of Eq. (13.26) tends to V in probability for $\theta \rightarrow \theta _0$. To conclude, all the relevant conditions stated in [9, Theorem 2.4] are fulfilled and then the asymptotic normality follows, see also [11, Theorem 3.1].   $\square$