Self-exciting hysteretic binomial autoregressive processes

Abstract

This paper introduces an observation-driven integer-valued time series model, in which the underlying generating stochastic process is binomially distributed conditional on past information in the form of a hysteretic autoregressive structure. The basic probabilistic and statistical properties of the model are discussed. Conditional least squares, weighted conditional least squares, and maximum likelihood estimators are obtained together with their asymptotic properties. A search algorithm for the two boundary parameters, and the corresponding strong consistency of the estimators, are also provided. Finally, some numerical results on the estimators and a real-data example are presented.

Appendices

Appendix A: Derivations of moments

By taking the expectation of (2.5), we directly get (2.7). For the unconditional variance (2.8), we know that $\sigma _{X}^{2}=Var(X_t)=E(Var(X_{t}|X_{t-1},R_{t-1}))+Var(E(X_{t}|X_{t-1},R_{t-1}))$. Consider first

$$\begin{aligned} \begin{aligned} E(Var(X_{t}|X_{t-1},R_{t-1}))&=E(R_t(\rho _1(1-\rho _1)(1-2\pi _1)X_{t-1}\\&\quad +N(1-\rho _1)\pi _1(1-(1-\rho _1)\pi _1)))\\&\quad +E((1-R_t)(\rho _2(1-\rho _2)(1-2\pi _2)X_{t-1}\\&\quad +N(1-\rho _2)\pi _2(1-(1-\rho _2)\pi _2)))\\&\quad =\rho _1(1-\rho _1)(1-2\pi _1)\mu _{RX}\\&\quad +pN(1-\rho _1)\pi _1(1-(1-\rho _1)\pi _1)\\&\quad +\rho _2(1-\rho _2)(1-2\pi _2)(\mu _{X}-\mu _{RX})\\&\quad +(1-p)N(1-\rho _2)\pi _2(1-(1-\rho _2)\pi _2), \end{aligned} \end{aligned}$$

and (note that $E(R_t(1-R_t)\cdot Y)=0$)

$$\begin{aligned} Var(E(X_{t}|X_{t-1},R_{t-1}))&=Var(R_{t}(\rho _{1}X_{t-1}+N(1-\rho _{1}) \pi _{1})\\&\quad +(1-R_{t})(\rho _{2}X_{t-1}+N(1-\rho _{2})\pi _{2}))\\&\quad =Var(R_{t}(\rho _{1}X_{t-1}+N(1-\rho _{1})\pi _{1}))\\&\quad +Var((1-R_{t})(\rho _{2}X_{t-1}+N(1-\rho _{2})\pi _{2}))\\&\quad +2Cov(R_{t}(\rho _{1}X_{t-1}+N(1-\rho _{1})\pi _{1}),\\&\quad (1-R_{t})(\rho _{2}X_{t-1}+N(1-\rho _{2})\pi _{2}))\\&\quad =Var(R_{t}(\rho _{1}X_{t-1}]+Var[N(1-\rho _{1})\pi _{1}))\\&\quad +2Cov(R_{t}(\rho _{1}X_{t-1},N(1-\rho _{1})\pi _{1}))\\&\quad +Var((1-R_{t})(\rho _{2}X_{t-1}))+Var(N(1-\rho _{2})\pi _{2})\\&\quad +2Cov((1-R_{t})(\rho _{2}X_{t-1}),N(1-\rho _{2})\pi _{2})+0\\&\quad -2E(R_{t}(\rho _{1}X_{t-1}+N(1-\rho _{1})\pi _{1}))\cdot E((1-R_{t})(\rho _{2}X_{t-1}\\&\quad +N(1-\rho _{2})\pi _{2})) \\&\quad =\rho _1^{2}Var(R_tX_{t-1})+(1-\rho _1)^{2}\pi _1^{2}N^{2}p(1-p)\\&\quad +2\rho _1(1-\rho _1)\pi _1NCov(R_tX_{t-1},R_t)\\&\quad +\rho _2^{2}Var((1-R_t)X_{t-1})+(1-\rho _2)^{2}\pi _2^{2}N^{2}p(1-p)\\&\quad +2\rho _2(1-\rho _2)\pi _2NCov((1-R_t)X_{t-1},(1-R_{t}))\\&\quad -2\rho _1\rho _2\mu _{RX}(\mu _{X}-\mu _{RX})-2\rho _1(1-\rho _2)\pi _2(1-p)N\mu _{RX}\\&\quad -2\rho _2(1-\rho _1)\pi _1pN(\mu _{X}-\mu _{RX})\\&\quad -2p(1-p)(1-\rho _1)\pi _1(1-\rho _2)\pi _2N^{2}\\&\quad =\rho _{1}^{2}(\mu _{RX,2}-\mu _{RX}^{2})+(1-\rho _1)^{2}\pi _{1}^{2}N^{2}p(1-p)\\&\quad +2\rho _1(1-\rho _1)\pi _1N(1-p)\mu _{RX}\\&\quad +\rho _{2}^{2}(\sigma _{X}^{2}+2\mu _{X}\mu _{RX}-\mu _{RX,2}-\mu _{RX}^{2})\\&\quad +(1-\rho _2)^{2}\pi _{2}^{2}N^{2}p(1-p)\\&\quad +2\rho _2(1-\rho _2)\pi _2Np(\mu _X-\mu _{RX})\\&\quad -2\rho _1\rho _2\mu _{RX}(\mu _X-\mu _{RX})\\&\quad -2\rho _1(1-\rho _2)\pi _2(1-p)N\mu _{RX}\\&\quad -2\rho _2(1-\rho _1)\pi _1NP(\mu _X-\mu _{RX})\\&\quad -2p(1-p)(1-\rho _1)\pi _1(1-\rho _2)\pi _2N^{2}. \end{aligned}$$

Then, inserting the above two formulas into $\sigma _{X}^{2}=E(Var(X_{t}|X_{t-1},{R_{t-1}}))+Var(E(X_{t}|X_{t-1},{R_{t-1}}))$, we get

$$\begin{aligned} (1-\rho _{2}^{2})\sigma _{X}^{2}&=\rho _1(1-\rho _1)(1-2\pi _2)\mu _{RX}+pN(1-\rho _1)\pi _1(1-(1-\rho _1)\pi _1)\\&\quad +\rho _2(1-\rho _2)(1-2\pi _2)(\mu _{X}-\mu _{RX})\\&\quad +(1-p)N(1-\rho _2)\pi _2(1-(1-\rho _2)\pi _2)\\&\quad +\rho _{1}^{2}(\mu _{RX,2}-\mu _{RX}^{2})+(1-\rho _1)^{2}\pi _{1}^{2}N^{2}p(1-p)\\&\quad +2\rho _1(1-\rho _1)\pi _1N(1-p)\mu _{RX}\\&\quad +\rho _{2}^{2}(2\mu _{X}\mu _{RX}-\mu _{RX,2}-\mu _{RX}^{2})+(1-\rho _2)^{2}\pi _{2}^{2}N^{2}p(1-p)\\&\quad +2\rho _2(1-\rho _2)\pi _2Np(\mu _X-\mu _{RX})-2\rho _1\rho _2\mu _{RX}(\mu _X-\mu _{RX})\\&\quad -2\rho _1(1-\rho _2)\pi _2(1-p)N\mu _{RX}-2\rho _2(1-\rho _1)\pi _1NP(\mu _X-\mu _{RX})\\&\quad -2p(1-p)(1-\rho _1)\pi _1(1-\rho _2)\pi _2N^{2}\\&\quad =\rho _{2}(1-\rho _{2})(1-2\pi _{2})\mu _{X}-2Np\rho _{2}((1-\rho _{1})\pi _{1}-(1-\rho _{2})\pi _{2})\mu _{X}\\&\quad -2\rho _{2}(\rho _{1}-\rho _{2})\mu _{X}\mu _{RX}+(\rho _{1}^{2}-\rho _{2}^{2})\mu _{RX,2} -(\rho _{1}-\rho _{2})^{2}\mu _{RX}^{2}\\&\quad +2N(\rho _{1}-p(\rho _{1}-\rho _{2}))((1-\rho _{1})\pi _{1}-(1-\rho _{2})\pi _{2})\mu _{RX}\\&\quad +(\rho _{1}(1-\rho _{1})(1-2\pi _{1})-\rho _{2}(1-\rho _{2})(1-2\pi _{2}))\mu _{RX}\\&\quad +NP(1-\rho _{1})\pi _{1}(1-(1-\rho _{1})\pi _{1})\\&\quad +N(1-p)(1-\rho _{2})\pi _{2})(1-(1-\rho _{2})\pi _{2})\\&\quad +N^{2}p(1-p)((1-\rho _{1})\pi _{1}-(1-\rho _{2})\pi _{2})^{2}. \end{aligned}$$

We go on to prove (2.9). By the law of total covariance, we obtain

$$\begin{aligned} \gamma (k)&=Cov(X_{t},X_{t-k})=Cov(E(X_{t}|X_{t-1},\cdots ),E(X_{t-k}|X_{t-1},\cdots ))+0\\&\quad =Cov((R_{t}(\rho _1-\rho _2)+\rho _2)X_{t-1}+N(1-\rho _2)\pi _{2}\\&\quad +R_{t}(N(\pi _{1}-\pi _{2}-\pi _{1}\rho _{1}+\pi _{2}\rho _{2})),X_{t-k})\\&\quad =\rho _2Cov(X_{t-1},X_{t-k})+(\rho _1-\rho _2)Cov(R_{t}X_{t-1},X_{t-k})\\&\quad +N(\pi _{1}-\pi _{2}-\pi _{1}\rho _{1}+\pi _{2}\rho _{2})Cov(R_{t},X_{t-k})\\&\quad =\rho _2^{2}Cov(X_{t-2},X_{t-k})+\rho _2(\rho _1-\rho _2)Cov(R_{t-1}X_{t-2},X_{t-k})\\&\quad +N\rho _2(\pi _{1}-\pi _{2}-\pi _{1}\rho _{1}+\pi _{2}\rho _{2})Cov(R_{t-1},X_{t-k})\\&\quad +(\rho _1-\rho _2)Cov(R_{t}X_{t-1},X_{t-k})+N(\pi _{1}-\pi _{2}-\pi _{1}\rho _{1}\\&\quad +\pi _{2}\rho _{2})Cov(R_{t},X_{t-k})\\&\quad =\cdots \nonumber \\&\quad =\rho _2^{k}\sigma _{X}^{2}+(\rho _1-\rho _2)\sum _{s=1}^{k}\rho _2^{s-1}Cov(R_{t-s+1}X_{t-s},X_{t-k})\\&\quad +N(\pi _{1}-\pi _{2}-\pi _{1}\rho _{1}+\pi _{2}\rho _{2})\sum _{s=1}^{k}\rho _2^{s-1}Cov(R_{t-s+1},X_{t-k}). \end{aligned}$$

Appendix B: Proofs of theorems

Proof of Proposition 2.1

It is easy to see that $\{\varvec{Y}_t \}_{t\in {\mathbb {Z}}}$ is a Markov chain on ${\mathbb {S}}:=\left\{ 0, 1, \cdots , N\right\} \times \{0,1\}$, i.e., ${\mathbb {S}}=\{(x,r)|x\in \{0, 1, \cdots , N\},r\in \{0,1\}\}$. Note that there is an indicator function ${\mathbb {I}}\big (r_t = h(x_{t-1}, r_{t-1}) \big )$ in (2.4), which possibly switches the value of the one-step transition probability to zero. Thus, we consider the two-step transition probabilities

$$\begin{aligned}{} & {} P(\varvec{Y}_t=\varvec{y}_t|\varvec{Y}_{t-2}=\varvec{y}_{t-2}) =\sum _{\varvec{y}_{t-1}\in {\mathbb {S}}}P(\varvec{Y}_t=\varvec{y}_t|\varvec{Y}_{t-1}=\varvec{y}_{t-1}) \\{} & {} \quad \cdot P(\varvec{Y}_{t-1}=\varvec{y}_{t-1}|\varvec{Y}_{t-2}=\varvec{y}_{t-2}), \end{aligned}$$

and continue to prove that $P(\varvec{Y}_t=\varvec{y}_t|\varvec{Y}_{t-2}=\varvec{y}_{t-2})>0$. In fact, we only need to prove that there is one state $({\tilde{x}},{\tilde{r}})\in {\mathbb {S}}$ such that

$$\begin{aligned}{} & {} P(\varvec{Y}_t =\varvec{y}_t|X_{t-1}={\tilde{x}},R_{t-1}={\tilde{r}}) \cdot P(X_{t-1}={\tilde{x}},\nonumber \\{} & {} \quad R_{t-1}={\tilde{r}}|\varvec{Y}_{t-2}=\varvec{y}_{t-2})>0. \end{aligned}$$

(B.1)

First, note that for any $(x_{t-2},r_{t-2})\in {\mathbb {S}}$, if we choose ${\tilde{r}}=h(x_{t-2},r_{t-2})$, then ${\mathbb {I}}\big (r_t = h(x_{t-1}, r_{t-1}) \big )=1$ and, thus, implies that $P(X_{t-1}={\tilde{x}},R_{t-1}={\tilde{r}}|\varvec{Y}_{t-2}=\varvec{y}_{t-2})>0$ for any ${\tilde{x}}\in \{0,1,\cdots ,N\}$. Secondly, we choose ${\tilde{x}}=0$ if $R_t=1$, and ${\tilde{x}}=N$ if $R_t=0$. Then, $P(\varvec{Y}_t=\varvec{y}_t|X_{t-1}={\tilde{x}},R_{t-1}={\tilde{r}})>0$ holds. Therefore, (B.1) holds true, which implies that the two-step transition probabilities $P(\varvec{Y}_t=\varvec{y}_t|\varvec{Y}_{t-2}=\varvec{y}_{t-2})$ are always positive. As a consequence, $\{\varvec{Y}_t \}_{t\in {\mathbb {Z}}}$ is a primitive and thus irreducible and aperiodic Markov chain (Weiß 2018, Appendix B.2.2). Furthermore, the state space ${\mathbb {S}}$ contains an only finite number of paired elements, implying that $\{\varvec{Y}_t \}_{t\in {\mathbb {Z}}}$ is positive recurrent. Hence, $\{\varvec{Y}_t \}_{t\in {\mathbb {Z}}}$ is an ergodic Markov chain. Finally, Theorem 1.3 in Karlin and Taylor (1975) guarantees the existence of the stationary distribution for $\{\varvec{Y}_t \}_{t\in {\mathbb {Z}}}$. $\square$

Proof of Theorem 3.3

It follows by (2.4) that the CML estimation discussed in Sect. 3.3 can be equivalently embedded in a bivariate Markov chain $\{\varvec{Y}_t\}$, which brings great convenience to the proof of asymptotic normality. To prove Theorem 3.3 in such a framework, we need to verify that Condition 5.1 in Billingsley (1961) holds. Denote by $P_{\varvec{x}|\varvec{y}}(\varvec{\theta }):=P(\varvec{Y}_t=\varvec{x}|\varvec{Y}_{t-1}=\varvec{y})$ the transition probability of $\{\varvec{Y}_t\}$. Condition 5.1 of Billingsley (1961) is satisfied provided that:

• 1. The set D of $(\varvec{x},\varvec{y})$ such that $P_{\varvec{x}|\varvec{y}}(\varvec{\theta })>0$ is independent of $\varvec{\theta }$.

• 2. Each $P_{\varvec{x}|\varvec{y}}(\varvec{\theta })$ has continuous partial derivatives of third order throughout $\Theta$.

• 3. The $d\times r$ matrix

  $$\begin{aligned} \left( \frac{\partial P_{\varvec{x}|\varvec{y}}(\varvec{\theta })}{\partial \theta _{u}}\right) _{(\varvec{x},\varvec{y})\in D,~u=1,\cdots ,r,} \end{aligned}$$

  (B.2)

  has rank r throughout $\Theta$, where $d:=|D|$ and $r:=\dim (\Theta )$.

• 4. For each $\varvec{\theta }\in \Theta$, there is only one ergodic set and there are no transient states.

  Conditions 1 and 2 are easily implied by (2.4). For fixed $r_{L}, r_{U}$ $(0< r_{L} \le r_{U }< N-1)$, we can select an r-dimensional square matrix of rank r from the $d\times r$ dimensional matrix (B.2), then Condition 3 is also true. Since the state space of the SEHBAR(1) process is a finite-range set, and $P_{\varvec{x}|\varvec{y}}(\varvec{\theta })>0$, then Condition 4 holds. Thus, Conditions 1 to 4 are all satified, which implies that Condition 5.1 in Billingsley (1961) holds. Then, Theorems 2.1 and 2.2 of Billingsley (1961) guarantee that the CML-estimators $\hat{\varvec{\theta }}_{CML}$ are strongly consistent and asymptotically normal.

Proof of Theorem 3.4

Let $H_{t}(\varvec{\lambda })=-U_{t}(\varvec{\lambda })$. The proof shall be done in three steps.

Step 1: We show that $E(U_{t}(\varvec{\lambda }))$ is continuous in $\varvec{\lambda }$, hence $E(H_{t}(\varvec{\lambda }))$ is also continuous in $\varvec{\lambda }$.

First, let us denote $I_{1,t}:={\mathbb {I}}(R_{t}=1),~I_{2,t}:={\mathbb {I}}(R_{t}=0)$, then $U_{t}(\varvec{\lambda })=(X_{t}-\sum _{i=1}^2(\rho _{i}X_{t-1}+N(1-\rho _{i})\pi _{i})I_{i,t})^{2}$. For any $\varvec{\lambda }\in \Theta \times CR$, let $V_{\eta }(\varvec{\lambda })=B(\varvec{\lambda },\eta )$ be an open ball centred at $\varvec{\lambda }$ with radius $\eta$ ($\eta <1$). Next, we show the following property:

$$\begin{aligned} E\left( \mathop {\sup }\limits _{\lambda ^{'}\in V_{\eta }(\varvec{\lambda })}|U_{t}(\varvec{\lambda })-U_{t}(\varvec{\lambda ^{'}})|\right) \rightarrow 0,~as~\eta \rightarrow 0. \end{aligned}$$

To see this, observe that

$$\begin{aligned} |U_{t}(\varvec{\lambda })-U_{t}(\varvec{\lambda ^{'}})|&=\left| \left( X_{t}-\sum \limits _{i=1}^2 (\rho _{i}X_{t-1}+N(1-\rho _{i})\pi _{i})I_{i,t}\right) ^{2}\right. \\&\quad -\left. \left( X_{t}-\sum \limits _{i=1}^2 (\rho _{i}^{'}X_{t-1}+N(1-\rho _{i}^{'})\pi _{i}^{'})I_{i,t}\right) ^{2}\right| \\&\quad =\left| 2X_{t}+\sum \limits _{i=1}^2N(\pi _{i}\rho _{i}+\pi _{i}^{'}\rho _{i}^{'})I_{i,t}\right. \\&\quad -\left. \left( \sum \limits _{i=1}^2((\rho _{i}+\rho _{i}^{'})X_{t-1}+N(\pi _{i}+\pi _{i}^{'}))I_{i,t}\right) \right| \\&\quad \times \left| \sum \limits _{i=1}^2((\rho _{i}^{'}-\rho _{i})X_{t-1}+N(\rho _{i}^{'}-r_{i}) -N(\pi _{i}^{'}\rho _{i}^{'}-\pi _{i}\rho _{i}))I_{i,t}\right| \\ \le&(2X_{t}+2N+2X_{t-1}+2N)\times (\eta (X_{t-1}+N)+\eta N)\\&\quad =2\eta [X_{t}(X_{t-1}+2N)+(X_{t-1}+2N)^{2}]. \end{aligned}$$

Then,

$$\begin{aligned}{} & {} E\left( \mathop {\sup }\limits _{\lambda ^{'}\in V_{\eta }(\varvec{\lambda })}|U_{t}(\varvec{\lambda })-U_{t}(\varvec{\lambda ^{'}})|\right) \le 2\eta E[X_{t}(X_{t-1}+2N)\\{} & {} \quad +(X_{t-1}+2N)^{2}]\rightarrow 0, \eta \rightarrow 0. \end{aligned}$$

Step 2: We prove that $E_{\varvec{\lambda }_{0}}[H_{t}(\varvec{\lambda })-H_{t}(\varvec{\lambda }_{0})] < 0$, or equivalently, $E_{\varvec{\lambda }_{0}}[U_{t}(\varvec{\lambda })-U_{t}(\varvec{\lambda }_{0})] > 0$ for any $\varvec{\lambda }\ne \varvec{\lambda }_{0}$, where $\varvec{\lambda }_{0}$ is the true value of $\varvec{\lambda }$. It follows that

$$\begin{aligned} E_{\varvec{\lambda }_{0}}[U_{t}(\varvec{\lambda })]&=E_{\varvec{\lambda }_{0}}[(X_{t} -g(\varvec{\lambda },X_{t-1},R_{t-1}))^{2}]\nonumber \\&\quad =E_{\varvec{\lambda }_{0}}[(X_{t}-g(\varvec{\lambda _{0}},X_{t-1},R_{t-1}) +g(\varvec{\lambda _{0}},X_{t-1},R_{t-1})\nonumber \\&\quad -g(\varvec{\lambda },X_{t-1},R_{t-1}))^{2}]\nonumber \\&\quad =E_{\varvec{\lambda }_{0}}[(X_{t}-g(\varvec{\lambda _{0}},X_{t-1},R_{t-1}))^{2}+2(X_{t} -g(\varvec{\lambda _{0}},X_{t-1},R_{t-1}))\nonumber \\&\quad (g(\varvec{\lambda _{0}},X_{t-1},R_{t-1})\nonumber \\&\quad -g(\varvec{\lambda },X_{t-1},R_{t-1}))+(g(\varvec{\lambda _{0}},X_{t-1},R_{t-1}) -g(\varvec{\lambda },X_{t-1},R_{t-1}))^{2}]\nonumber \\&\quad =E_{\varvec{\lambda }_{0}}[U_{t}(\varvec{\lambda }_{0})]+I+II, \end{aligned}$$

(B.3)

where

$$\begin{aligned} I&=2E_{\varvec{\lambda }_{0}}[(X_{t}-g(\varvec{\lambda _{0}},X_{t-1},R_{t-1})) (g(\varvec{\lambda _{0}},X_{t-1},R_{t-1})-g(\varvec{\lambda },X_{t-1},R_{t-1}))]\nonumber \\&\quad =2E_{\varvec{\lambda }_{0}}[E_{\varvec{\lambda }_{0}}((X_{t} -g(\varvec{\lambda _{0}},X_{t-1},R_{t-1}))(g(\varvec{\lambda _{0}},X_{t-1},R_{t-1})\nonumber \\&\quad -g(\varvec{\lambda },X_{t-1},R_{t-1}))|X_{t-1},R_{t-1})]\nonumber \\&\quad =2E_{\varvec{\lambda }_{0}}(0)=0, \end{aligned}$$

(B.4)

and where

$$\begin{aligned} II=E_{\varvec{\lambda }_{0}}[(g(\varvec{\lambda _{0}},X_{t-1},R_{t-1})-g(\varvec{\lambda },X_{t-1},R_{t-1}))^{2}] > 0. \end{aligned}$$

(B.5)

Thus, by (B.3), (B.4), and (B.5), we have $E_{\varvec{\lambda }_{0}}[U_{t}(\varvec{\lambda })-U_{t}(\varvec{\lambda }_{0})] > 0$.

Step 3: Now, we are ready to prove the consistency for $\hat{\varvec{\lambda }}_{CLS}$.

Consider an arbitrary (small) open neighbourhood of $\varvec{\lambda }_{0}$, say V, then for any $\varvec{\lambda } \in V^{c}\cap \Theta$, we have $E[H_{t}(\varvec{\lambda })] < E[H_{t}(\varvec{\lambda }_{0})]$. Since $V^{c}\cap \Theta$ is compact and $E[H_{t}(\varvec{\lambda })]$ is continuous in $\varvec{\lambda }$, we have $\kappa =E[H_{t}(\varvec{\lambda }_{0})]-\sup _{\varvec{\lambda } \in V^{c}\cap \Theta } E[H_{t}(\varvec{\lambda })]>0$. For any $\varvec{\lambda } \in V^{c}\cap \Theta$, there exists $\eta _{\varvec{\lambda }}>0$ such that $E[\sup _{\tilde{\varvec{\lambda }} \in V_{\eta _{\varvec{\lambda }}}(\varvec{\lambda })}H_{t}(\tilde{\varvec{\lambda }})] < E[H_{t}(\varvec{\lambda })]+\frac{\kappa }{6}$. Also by the compactness of $V^{c}\cap \Theta$, there exists a finite open cover of $V^{c}\cap \Theta$, say, $\{V_{\eta _{\varvec{\lambda }_{j}}}(\varvec{\lambda }_{j}), j = 1,\cdots ,m \}$.

For any $\lambda \in V^{c}\cap \Theta$, $n\gg 0$ and $j=1,\cdots ,m$, we have

$$\begin{aligned} \mathop {\sup }\limits _{\varvec{\lambda } \in V_{\eta _{\varvec{\lambda }_{j}}}(\varvec{\lambda }_{j})}\frac{1}{n}\sum \limits _{t=1}^n H_{t}(\lambda ) \le&\frac{1}{n}\sum \limits _{t=1}^n \mathop {\sup }\limits _{\varvec{\lambda } \in V_{\eta _{\varvec{\lambda }_{j}}}(\varvec{\lambda }_{j})}H_{t}(\varvec{\lambda })+\frac{\kappa }{6}\\ \le&E\left( \mathop {\sup }\limits _{\varvec{\lambda } \in V_{\eta _ {\varvec{\lambda }_{j}}}(\varvec{\lambda }_{j})}H_{t}(\varvec{\lambda })\right) +\frac{\kappa }{3}\\ \le&E\left[ H_{t}(\varvec{\lambda }_{j})\right] +\frac{\kappa }{2}\\ \le&\mathop {\sup }\limits _{\varvec{\lambda } \in V^{c}\cap \Theta } E\left[ H_{t}(\varvec{\lambda })\right] +\frac{\kappa }{2}\\ \le&E\left[ H_{t}(\varvec{\lambda }_{0})\right] -\frac{\kappa }{3}. \end{aligned}$$

On the other hand,

$$\begin{aligned} \mathop {\sup }\limits _{\varvec{\lambda } \in V} \frac{1}{n}\sum \limits _{t=1}^n H_{t}(\varvec{\lambda }) \ge \frac{1}{n}\sum \limits _{t=1}^n H_{t}(\varvec{\lambda }_{0})\ge \frac{1}{n}\sum \limits _{t=1}^nH_{t}(\varvec{\lambda }_{0})-\frac{\kappa }{6}\ge E\left[ H_{t}(\varvec{\lambda }_{0})\right] -\frac{\kappa }{3}. \end{aligned}$$

Therefore, for any (small) neighbourhood of $\varvec{\lambda }_{0}$, say V, for $n\gg 0$, we have almost surely

$$\begin{aligned} \mathop {\sup }\limits _{\varvec{\lambda } \in V_{\eta _{\varvec{\lambda }_{j}}}(\varvec{\lambda }_{j})}\frac{1}{n}\sum \limits _{t=1}^n H_{t}(\varvec{\lambda })\le \mathop {\sup }\limits _{\varvec{\lambda } \in V} \frac{1}{n}\sum \limits _{t=1}^n H_{t}(\varvec{\lambda }), \end{aligned}$$

which implies $\hat{\varvec{\lambda }}_{CLS} \in V$. $\square$

Appendix C: A general result on Markov chains

We state a general result of an ergodic Markov chain in the following proposition.

Proposition C.1

Let $\{X_t\}$ be an ergodic Markov chain on state space S with stationary distribution $\varvec{\pi }=(\pi _1,\pi _2,\cdots )$. Let $\{Y_t\}$ be a Markov chain that has the same transition probabilities with $\{X_t\}$. Then, for any $m \ge 1$,

• (i) $\lim _{t \rightarrow \infty } P(Y_t=i_0, Y_{t+1}=i_1,\cdots ,Y_{t+m}=i_m)=P(X_0=i_0, X_{1}=i_1,\cdots ,X_{m}=i_m)$;

• (ii) for sufficiently large t, the distribution of $(Y_{t},Y_{t+1},\cdots ,Y_{t+m})$ and $(X_{t},X_{t+1},\cdots ,X_{t+m})$ are approximately the same.

Proof

• (i) Denote by $p_{j|i}$ and $p_{j|i}(n)$ the one-step- and n-step-ahead transition probability, respectively, from state i to j. As $\sum _{i\in S} P(Y_0=i)=1$ and $\lim _{t\rightarrow \infty } p_{j|i}(t)=\pi _j$, we have

  $$\begin{aligned} \lim _{t\rightarrow \infty }P(Y_t=j)=\lim _{t\rightarrow \infty }\sum _{i\in S}P(Y_0=i)p_{j|i}(t) =\sum _{i\in S}P(Y_0=i)\pi _j=\pi _j. \end{aligned}$$

  Therefore,

  $$\begin{aligned} \lim _{t\rightarrow \infty }P(Y_t=i_0,Y_{t+1}=i_i,\cdots ,Y_{t+m}=i_m)&=\lim _{t\rightarrow \infty }P(Y_t=i_0)p_{ i_1| i_0}p_{ i_2| i_1}\cdots p_{ i_m| i_{m-1}}\\&=p_{i_0}p_{ i_1| i_0}p_{ i_2| i_1}\cdots p_{ i_m| i_{m-1}}\\&=P(X_0=i_0,X_1=i_1,\cdots ,X_m=i_m). \end{aligned}$$

• (ii) Since $\{X_n\}$ is stationary, $(X_t,X_{t+1},\cdots ,X_{t+m})$ and $(X_0,X_{1},\cdots ,X_{m})$ have the same distribution. Thus,

• (iii) follows from (i). $\square$