Particle-based online estimation of tangent filters with application to parameter estimation in nonlinear state-space models

Abstract

This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished with a number of convergence results, including a central limit theorem with an asymptotic variance that can be shown to be uniformly bounded in time. Using the proposed filter derivative estimator, we design a recursive maximum likelihood algorithm updating the parameters according the gradient of the one-step predictor log-likelihood. The efficiency of this online parameter estimation scheme is illustrated in a simulation study.

Appendices

Proofs

Define for all $t \in \mathbb {N}$ and $\theta \in \Theta$,

$$\begin{aligned} \mathbf {L}_{t;\theta } : \textsf {X} \times \mathcal {X} \ni (x, A) \mapsto g_{t;\theta }(x) \mathbf {Q}_{\theta }(x, A). \end{aligned}$$

(Note that our definition of $\mathbf {L}_{t}$ differs from that used by Olsson and Westerborn (2017), in which the order of $g_{t;\theta }$ and $\mathbf {Q}_{\theta }$ is swapped.) With this notation, by the filtering recursion (4)–(5),

$$\begin{aligned} \pi _{t+1;\theta } = \frac{\pi _{t;\theta } \mathbf {L}_{t;\theta }}{\pi _{t;\theta } \mathbf {L}_{t;\theta }\mathbb {1}_{\textsf {X}}}, \end{aligned}$$

(20)

with, as previously, . This condensed form of the filtering recursion will be used in Sect. A.3.

In the coming analysis, the following decomposition will be instrumental. For all $t \in \mathbb {N}$,

(21)

1.1 Proof of Theorem 1

We apply the decomposition (21). Note that

(22)

where

(23)

Now, since and both belong to $\textsf {F}(\mathcal {X})$, Proposition 1 provides constants $c_t > 0$ and $\tilde{c}_t > 0$ such that for all $\varepsilon > 0$,

(24)

To deal with the second part of the decomposition (21), we use the same technique. First, by applying Proposition 1 with $f \equiv 1/g_{t;\theta }$ and $\tilde{f} \equiv 0$, we obtain constants $a_t > 0$ and $\tilde{a}_t > 0$ such that for all $\varepsilon > 0$,

(25)

Similarly, using Proposition 1 with and $\tilde{f} \equiv f_t / g_{t;\theta }$ provides constants $b_t > 0$ and $\tilde{b}_t > 0$ such that for all $\varepsilon > 0$,

(26)

Combining (24), (25), and (26) yields, for all $\varepsilon > 0$,

from which the statement of the theorem follows. $\square$

The following result is obtained by inspection of the proof of Olsson and Westerborn (2017, Theorem 1(i)).

Proposition 1

Let Assumption 1 hold. Then, for all $t \in \mathbb {N}$, all $\theta \in \Theta$, all additive state functionals , all measurable functions $f_t$and $\tilde{f}_t$ such that $f_t g_{t;\theta } \in \textsf {F}(\mathcal {X})$ and $\tilde{f}_t g_{t;\theta } \in \textsf {F}(\mathcal {X})$, and all $\tilde{N}\in \mathbb {N}$, there exist constants $c_t > 0$ and $\tilde{c}_t > 0$ (possibly depending on $\theta$, $h_{t}$$f_t$, $\tilde{f}_t$, and $\tilde{N}$) such that for all $\varepsilon > 0$,

where are produced using the PaRIS algorithm.

1.2 Proof of corollary 1

The $\mathbb {P}$-a.s. convergence of to is implied straightforwardly by the exponential convergence rate in Theorem 1. Indeed, note that

now, by Theorem 1,

where the right-hand side tends to zero when n tends to infinity. This completes the proof. $\square$

1.3 Proof of Theorem 2

By combining (21) and (22),

where in this case

are defined in (23). By Proposition 2, since and ,

where Z is standard normally distributed and

with $\sigma _{t;\theta }(h_{t;\theta })$ being defined in (14). Now, Proposition 1 and Proposition 2 yield

and

(with 0 denoting the zero function), respectively, implying, by Slutsky’s theorem,

Finally, we complete the proof by noting that the term in (14) coincides with the asymptotic variance provided by Del Moral et al. (2015, Theorem 3.2). $\square$

Proposition 2

Assumption 1 hold. Then for all $t \in \mathbb {N}$, all $\theta \in \Theta$, all additive state functionals , all measurable functions $f_t \in \textsf {F}(\mathcal {X})$ and $\tilde{f}_t \in \textsf {F}(\mathcal {X})$, and all $\tilde{N}\in \mathbb {N}$, as $N\rightarrow \infty$,

where Z is a standard Gaussian random variable and

$$\begin{aligned} \sigma _{t;\theta }^2 \langle f_t, \tilde{f}_t \rangle (h_{t}) {:=}\tilde{\sigma }_{t;\theta }^2 \langle f_t, \tilde{f}_t \rangle (h_{t}) + \sum _{s = 0}^{t-1} \sum _{\ell = 0}^{s} \tilde{N}^{\ell - (s+1)} \varsigma _{s, \ell , t; \theta } \langle f_t \rangle (h_{t}), \end{aligned}$$

(27)

with

and

Proof of Proposition 2

Assume first that . Then, by Lemma 1 and Slutsky’s theorem, as $\Omega _{t} / N\overset{\mathbb {P}}{\longrightarrow }\pi _{t;\theta } g_{t;\theta }$ by Proposition 1,

where again Z has standard Gaussian distribution, is given in Lemma 1, and we have set and and used, first, that and and, second, that

Now, by iterating (20) we conclude that for all $(s, t) \in \mathbb {N}^2$,

$$\begin{aligned} \pi _{t;\theta }= & {} \frac{\pi _{t - 1;\theta } \mathbf {L}_{t - 1;\theta }}{\pi _{t - 1;\theta } \mathbf {L}_{t - 1;\theta } \mathbb {1}_{\textsf {X}}} = \frac{\pi _{t - 2;\theta } \mathbf {L}_{t - 2;\theta } \mathbf {L}_{t - 1;\theta }}{(\pi _{t - 2;\theta } \mathbf {L}_{t - 2;\theta } \mathbb {1}_{\textsf {X}}) (\pi _{t - 1;\theta } \mathbf {L}_{t - 1;\theta } \mathbb {1}_{\textsf {X}})}\\= & {} \frac{\pi _{t - 2;\theta } \mathbf {L}_{t - 2;\theta } \mathbf {L}_{t - 1;\theta }}{\pi _{t - 2;\theta } \mathbf {L}_{t - 2;\theta } \mathbf {L}_{t - 1;\theta } \mathbb {1}_{\textsf {X}}} \\= & {} \frac{\pi _{s + 1;\theta } \mathbf {L}_{s + 1;\theta } \ldots \mathbf {L}_{t - 1;\theta }}{\pi _{s + 1;\theta } \mathbf {L}_{s + 1;\theta } \ldots \mathbf {L}_{t - 1;\theta } \mathbb {1}_{\textsf {X}}} \end{aligned}$$

and, consequently,

$$\begin{aligned} \pi _{t;\theta } g_{t;\theta } = \frac{\pi _{s + 1;\theta } \mathbf {L}_{s + 1;\theta } \ldots \mathbf {L}_{t - 1;\theta } g_{t;\theta }}{\pi _{s + 1;\theta } \mathbf {L}_{s + 1;\theta } \ldots \mathbf {L}_{t - 1;\theta } \mathbb {1}_{\textsf {X}}}. \end{aligned}$$

(28)

Finally, by (28) it holds that

where $\Gamma ^2_{t;\theta } \langle f_t, \tilde{f}_t \rangle (h_{t})$ is defined in (29) and $\sigma ^2_{t;\theta } \langle f_t, \tilde{f}_t \rangle (h_{t})$ is defined in (27). Finally, in the general case, the previous holds true when $\tilde{f}_t$ is replaced by , which completes the proof.

The following lemma is obtained by inspection of the proof of Olsson and Westerborn (2017, Theorem 3).

Lemma 1

Assumption 1 hold. Then for all $t \in \mathbb {N}$, all $\theta \in \Theta$, all additive state functionals , all measurable functions $f_t$and $\tilde{f}_t$ such that $f_t g_{t;\theta } \in \textsf {F}(\mathcal {X})$ and $\tilde{f}_t g_{t;\theta } \in \textsf {F}(\mathcal {X})$, and all $\tilde{N}\in \mathbb {N}$, as $N\rightarrow \infty$,

where Z is a standard normal distribution and

$$\begin{aligned} \Gamma _{t;\theta }^2 \langle f_t, \tilde{f}_t \rangle (h_{t}) {:=}\tilde{\Gamma }_{t;\theta }^2 \langle f_t, \tilde{f}_t \rangle (h_{t}) + \sum _{s = 0}^{t-1} \sum _{\ell = 0}^{s} \tilde{N}^{\ell - (s+1)} \gamma _{s, \ell , t; \theta } \langle f_t \rangle (h_{t}), \end{aligned}$$

(29)

with

and

1.4 Proof of Theorem 3

As noted above, the first term of the asymptotic variance coincides with the asymptotic variance obtained by Del Moral et al. (2015, Theorem 3.2). The same work provides a constant $c \in \mathbb {R}_+$ such that , and we may hence focus on bounding second term of the asymptotic variance.

For this purpose, note that for all $s \le t - 1$ and $x_{s + 1} \in \textsf {X}$,

(30)

By applying the forgetting of the filter, or, more particularly, Douc et al. (2011, Lemma 10), to (30) we obtain

Note that in the previous bound, the exponential contraction follows from the fact that the objective function $f_t$ is centred around its predicted mean. The latter is a consequence of the fact that the tangent filter is, as a covariance, centred itself (recall the identities (10) and (11) and the decomposition (21)). In addition, from the proof of Olsson and Westerborn (2017, Theorem 8) we extract, using Assumption 3,

and under Assumption 2, for all $x \in \textsf {X}$,

Combining the previous bounds gives

where

Finally, summing up yields

which completes the proof. $\square$

Kernels

Given two measurable spaces $(\textsf {X},\mathcal {X})$ and $(\textsf {Y},\mathcal {Y})$, an unnormalised transition kernel $\mathbf {K}$ between these spaces induces two integral operators, one acting on functions and the other on measures. Specifically, we define the function

and the measure

$$\begin{aligned} \nu \mathbf {K} : \mathcal {Y} \ni \textsf {A} \mapsto \int \mathbf {K}(x, \textsf {A}) \, \nu (\text {d}x) \quad (\nu \in \textsf {M}(\mathcal {X})) \end{aligned}$$

whenever these quantities are well defined. Moreover, let $\mathbf {L}$ be another unnormalised transition kernel from $(\textsf {Y},\mathcal {Y})$ to the measurable space $(\textsf {Z},\mathcal {Z})$; then two different products of $\mathbf {K}$ and $\mathbf {L}$ can be defined, namely

$$\begin{aligned} \mathbf {K} \mathbf {L} : \textsf {X} \times \mathcal {Z} \ni (x, \textsf {A}) \mapsto \int \mathbf {K}(x, \text {d}y) \, \mathbf {L}(y, \textsf {A}) \end{aligned}$$

and

whenever these are well defined. These products form new transition kernels from $(\textsf {X},\mathcal {X})$ to $(\textsf {Z},\mathcal {Z})$ and from $(\textsf {X},\mathcal {X})$ to , respectively. Also the -product of a kernel $\mathbf {K}$ and a measure $\nu \in \textsf {M}(\mathcal {X})$ is defined as the new measure

Finally, for any kernel $\mathbf {K}$ and any bounded measurable function h we write $\mathbf {K}^2 h {:=}(\mathbf {K} h)^2$ and $\mathbf {K} h^2 {:=}\mathbf {K}(h^2)$. Similar notation will be used for measures.