Fundamental Frequency and its Harmonics Model: A Robust Method of Estimation

Abstract

In this paper we have proposed a novel robust method of estimation of the unknown parameters of a fundamental frequency and its harmonics model. Although the least squares estimators (LSEs) or the periodogram type estimators are the most efficient estimators, it is well known that they are not robust. In presence of outliers the LSEs are known to be not efficient. In presence of outliers, robust estimators like least absolute deviation estimators (LADEs) or Huber’s M-estimators (HMEs) may be used. But implementation of the LADEs or HMEs are quite challenging, particularly if the number of component is large. Finding initial guesses in the higher dimensions is always a non-trivial issue. Moreover, theoretical properties of the robust estimators can be established under stronger assumptions than what are needed for the LSEs. In this paper we have proposed novel weighted least squares estimators (WLSEs) which are more robust compared to the LSEs or periodogram estimators in presence of outliers. The proposed WLSEs can be implemented very conveniently in practice. It involves in solving only one non-linear equation. We have established the theoretical properties of the proposed WLSEs. Extensive simulations suggest that in presence of outliers, the WLSEs behave better than the LSEs, periodogram estimators, LADEs and HMEs. The performance of the WLSEs depend on the weight function, and we have discussed how to choose the weight function. We have analyzed one synthetic data set to show how the proposed method can be used in practice.

Appendices

Appendix A

To prove Theorem 1, we need the following lemmas.

Lemma 1

Let $\{X(n)\}$ be a sequence of independent and identically distributed random variables with mean 0 and finite variance, then for $k = 0,1,\ldots$,

$$\begin{aligned} \lim _{N \rightarrow \infty } \sup _{\theta } \left| \frac{1}{N^{k+1}} \sum _{n=1}^N X(n) n^k \cos (n \theta ) \right| = 0 \ \ \ a.s. \end{aligned}$$

The same result holds when $\cos (t\theta )$ is replaced by $\sin (t \theta )$.

Proof

$k = 0$, the result is available in Kundu and Mitra [8]. For general k, the result follows from the fact $\displaystyle \frac{n}{N} \le 1$, for $1 \le n \le N$. $\square$

Lemma 2

Let $\{X(n)\}$ be a sequence of independent and identically distributed random variables with mean 0 and finite variance, and the weight function w(t) has the form (8), then

$$\begin{aligned} \lim _{N \rightarrow \infty } \sup _{\theta } \left| \frac{1}{N} \sum _{n=1}^N X(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| = 0 \ \ \ a.s. \end{aligned}$$

The same result holds when $\cos (t\theta )$ is replaced by $\sin (t \theta )$.

Proof

Using Lemma 1, Lemma 2 can be easily obtained. $\square$

Lemma 3

Let ${{\varvec{\Theta }}}$ be the same as before, and let us define the set

$$\begin{aligned} S_{\delta ,M}= & {} \{{{\varvec{\Theta }}}:|\lambda -\lambda ^0|> \delta \ \ \hbox {or} \ \ |A_k^0-A_k|> \delta \ \ \hbox {or} \ \ |B_k^0-B_k| > \delta , \\{} & {} \hbox {for any} \,k = 1,\ldots , p \ \ \hbox {and}\, |A_k| \le M, |B_k| \le M,\, \hbox {for all}\,\, k = 1,\ldots , p\}. \end{aligned}$$

If for any $\delta > 0$ and for some $M < \infty$,

$$\begin{aligned} \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in S_{\delta ,M}} \frac{1}{N} \{Q({{\varvec{\Theta }}}) - Q({{\varvec{\Theta }}}^0) \} > 0, \end{aligned}$$

(1)

where $Q({{\varvec{\Theta }}})$ is same as defined in (10), then ${\widehat{\Theta }}$ is a strongly consistent estimator of ${{\varvec{\Theta }}}^0$.

Proof

It mainly follows by contradiction, along the same line as the proof of Lemma 1 of Wu [20]. Hence, the details are avoided. $\square$

Lemma 4

For any given $\delta > 0$ and for some $M < \infty$,

$$\begin{aligned} \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in S_{\delta ,M}} \frac{1}{N} \{Q({{\varvec{\Theta }}}) - Q({{\varvec{\Theta }}}^0) \} > 0. \end{aligned}$$

Proof

Consider

$$\begin{aligned} \frac{1}{N} Q({{\varvec{\Theta }}})= & {} \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) (y(n) - \mu _n({{\varvec{\Theta }}}))^2 \\= & {} \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}) + X(n))^2 \\= & {} \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}))^2 + \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) X^2(n) + \\{} & {} \frac{2}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}))X(n) \end{aligned}$$

Hence, using Lemma 2, and the condition of the weight function w(t), we obtain

$$\begin{aligned} \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in S_{\delta ,M}} \frac{1}{N} \{Q({{\varvec{\Theta }}}) - Q({{\varvec{\Theta }}}^0) \} \ge \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in S_{\delta ,M}} \frac{\gamma }{N} \sum _{n=1}^N (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}))^2. \end{aligned}$$

Consider the following sets for $k = 1, \ldots , p$:

$$\begin{aligned} \Gamma _{1k}= & {} \{{{\varvec{\Theta }}}: |A_j| \le M, |B_j| \le M, |A_k-A_k^0|> \delta \} \\ \Gamma _{2k}= & {} \{{{\varvec{\Theta }}}: |A_j| \le M, |B_j| \le M, |B_k-B_k^0|> \delta \} \\ \Gamma _0= & {} \{{{\varvec{\Theta }}}: |A_j| \le M, |B_j| \le M, |\lambda -\lambda ^0| > \delta \}, \end{aligned}$$

and $\displaystyle \Gamma _1 = \cup _{k=1}^p \Gamma _{1k}$, $\displaystyle \Gamma _2 = \cup _{k=1}^p \Gamma _{2k}$. Since

$$\begin{aligned}{} & {} \{{{\varvec{\Theta }}}: {{\varvec{\Theta }}} \in S_{\delta ,M}\} \subset \Gamma _1 \cup \Gamma _2 \cup \Gamma _0,\\{} & {} \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in S_{\delta ,M}} \frac{\gamma }{N} \sum _{n=1}^N (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}))^2 \\{} & {} \quad \ge \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in \Gamma _1 \cup \Gamma _2 \cup \Gamma _0} \frac{\gamma }{N} \sum _{n=1}^N (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}))^2. \end{aligned}$$

Observe that

$$\begin{aligned} \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in \Gamma _{1k}} \frac{\gamma }{N} \sum _{n=1}^N (\mu _n({{\varvec{\Theta }}}^0) - \mu _n({{\varvec{\Theta }}}))^2= & {} \liminf _{N \rightarrow \infty } \frac{\gamma |A_k-A_k^0|^2}{N} \sum _{n=1}^N \cos ^2(\lambda ^0 n) \\= & {} \frac{\gamma |A_k-A_k^0|^2}{2} > 0. \end{aligned}$$

Similarly, it can be shown for other sets also, hence the result follows. $\square$

Proof of Theorem 1:

Using Lemmas 3 and 4, it immediately follows. $\square$

Proof of Theorem 2:

To prove this result, let us consider the following $2p+1$ vector $Q'({{\varvec{\Theta }}})$, where

$$\begin{aligned} Q'({{\varvec{\Theta }}}) = \left( \frac{\partial Q({{\varvec{\Theta }}})}{\partial A_1}, \frac{\partial Q({{\varvec{\Theta }}})}{\partial B_1}, \ldots , \frac{\partial Q({{\varvec{\Theta }}})}{\partial A_p}, \frac{\partial Q({{\varvec{\Theta }}})}{\partial B_p}, \frac{\partial Q({{\varvec{\Theta }}})}{\partial \lambda } \right) , \end{aligned}$$

$\displaystyle Q''({{\varvec{\Theta }}})$ is a $(2p+1)\times (2p+1)$ matrix contains the double derivative of $Q({{\varvec{\Theta }}})$. Now using the Taylor series expansion

$$\begin{aligned} Q'(\widehat{{\varvec{\Theta }}}) - Q'({{\varvec{\Theta }}}^0) = (\widehat{{\varvec{\Theta }}} - {{\varvec{\Theta }}}^0)Q''(\overline{{\varvec{\Theta }}}), \end{aligned}$$

(2)

here $\overline{{\varvec{\Theta }}}$ is a point on the line joining $\widehat{{\varvec{\Theta }}}$ and ${{\varvec{\Theta }}}^0$. Since $Q'(\widehat{{\varvec{\Theta }}}) = {{\varvec{0}}}$, hence (2) can be written as

$$\begin{aligned} - Q'({{\varvec{\Theta }}}^0){{\varvec{D}}}^{-1} [{{\varvec{D}}}^{-1} Q''(\overline{{\varvec{\Theta }}}) {{\varvec{D}}}^{-1} ]^{-1} = (\widehat{{\varvec{\Theta }}} - {\Theta }^0){{\varvec{D}}}. \end{aligned}$$

Now using Central limit theorem and (13) to (19), it follows that

$$\begin{aligned} Q'({{\varvec{\Theta }}}^0){{\varvec{D}}}^{-1} {\mathop {\longrightarrow }\limits ^{d}} N_{2p+1}({{\varvec{0}}}, 2 \sigma ^2 {{\varvec{\Sigma }}}), \end{aligned}$$

and using (13) to (19), it can be shown that

$$\begin{aligned} \lim _{N \rightarrow \infty } {{\varvec{D}}}^{-1} Q''(\overline{{\varvec{\Theta }}}) {{\varvec{D}}}^{-1} = \lim _{N \rightarrow \infty } {{\varvec{D}}}^{-1} Q''({{\varvec{\Theta }}}^0) {{\varvec{D}}}^{-1} = {{\varvec{\Gamma }}}, \end{aligned}$$

hence, the result follows.

Appendix B

To prove Theorem 3, we need the following Lemma.

Lemma 5

Let $\{X(n)\}$ be a sequence of independent and identically distributed random variables with mean 0 and finite variance, and the weight function w(t) satisfies Assumption 1, then

$$\begin{aligned} \lim _{N \rightarrow \infty } \sup _{\theta } \left| \frac{1}{N} \sum _{n=1}^N X(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| = 0 \ \ \ a.s. \end{aligned}$$

The same result holds when $\cos (t\theta )$ is replaced by $\sin (t \theta )$.

Proof

Let $Z(n) = X(n) I_{[|X(n)| \le \sqrt{n}]}$, where $I_{[|X(n)| \le \sqrt{n}]} = 1$, if $|X(n)| \le \sqrt{n}$, and 0, otherwise. Thus

$$\begin{aligned} \sum _{n=1}^{\infty } P(X(n) \ne Z(n))= & {} \sum _{n=1}^{\infty } P(|X(n)| > \sqrt{n}) \\= & {} \sum _{n=1}^{\infty } \left( 1 - F(\sqrt{n} \right) \ \ \ (\hbox {here }\,F(\cdot )\, \hbox {is the distribution function of}\, |X(n)|) \\\le & {} 1 + \int _0^{\infty } (1-F(\sqrt{x}) dx \\= & {} 1 + 2 \int _0^{\infty } y(1-F(y))dy \\= & {} 1 + 2 \int _0^{\infty } \int _y^{\infty } dF(z) \ dy \\= & {} 1 + 2 \int _0^{\infty } \int _0^z y dy \ dF(z) \\= & {} 1 + \int _0^{\infty } z^2 dF(z) < \infty . \end{aligned}$$

Hence, $\{X(n)\}$ and $\{Z(n)\}$ are equivalent sequences. Thus it is enough to show that

$$\begin{aligned} \lim _{N \rightarrow \infty } \sup _{\theta } \left| \frac{1}{N} \sum _{n=1}^N Z(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| = 0 \ \ \ a.s. \end{aligned}$$

Let $U(n) = Z(n) - E(Z(n))$. Note that if $G(\cdot )$ denotes the distribution function of X(1), then

$$\begin{aligned}{} & {} \sup _{\theta } \left| \frac{1}{N} \sum _{n=1}^N E[Z(n) w \left( \frac{n}{N} \right) \cos (n \theta )] \right| \\{} & {} \le \frac{1}{N} \sum _{n=1}^N \left| E [Z(n)] \right| = \frac{1}{N} \sum _{n=1}^N \left| \int _{|x| \le \sqrt{n}}x dG(x) \right| \longrightarrow 0. \end{aligned}$$

Hence, the result is proved if we can show that

$$\begin{aligned} \lim _{N \rightarrow \infty } \sup _{\theta } \left| \frac{1}{N} \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| = 0 \ \ \ a.s. \end{aligned}$$

For any fixed $\theta$ and $\epsilon > 0$, and for $\displaystyle 0< h < \frac{1}{4\sqrt{N}}$, we have

$$\begin{aligned}{} & {} P \left\{ \left| \frac{1}{N} \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| \ge \epsilon \right\} \\\le & {} e^{-h N\epsilon } E \ \hbox {exp} \left\{ h \left| \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| \right\} \\\le & {} e^{-h N\epsilon } \prod _{n=1}^N E \ \hbox {exp}\left| h U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| . \end{aligned}$$

Note that $\displaystyle h U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \le \frac{1}{4}$, for all $n = 1, \ldots , N$, and on using the fact that $e^{|x|} \le 2 e^x$ and $e^x \le (1+x+x^2)$, then for $\displaystyle |x| \le \frac{1}{4}$

$$\begin{aligned}{} & {} e^{-h N\epsilon } \prod _{n=1}^N E \ \hbox {exp}\left| h U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| \\ {}\le & {} 2 e^{-h N \epsilon } \prod _{n=1}^N E \ \hbox {exp} \left( h U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right) \\\le & {} 2 e^{-h N \epsilon } \prod _{n=1}^N (1+h^2 \sigma ^2) \le 2 \hbox {exp} \ \left( -hN \epsilon + Nh^2 \sigma ^2 \right) . \end{aligned}$$

Choose $\displaystyle h = \frac{1}{4\sqrt{N}}$, then for large N,

$$\begin{aligned} P \left\{ \left| \frac{1}{N} \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| \ge \epsilon \right\}\le & {} 2 \ \hbox {exp} \ \left( -\frac{\sqrt{N} \epsilon }{4} + \frac{\sigma ^2}{16} \right) \\ {}\le & {} C \hbox {exp} \ \left( -\frac{\sqrt{N} \epsilon }{4} \right) . \end{aligned}$$

For some constant $C > 0$. Let $k = N^2$, and choose $\theta _1, \ldots \theta _k$, such that for each $\theta \in [0,\pi ]$, there exists a $\theta _j$, such that $\displaystyle |\theta _j-\theta | \le \frac{\pi }{N^2}$. Hence

$$\begin{aligned} \left| \frac{1}{N} \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) (\cos (n \theta ) - \cos (n \theta _j) \right| \le \frac{1}{N} \sum _{n=1}^N \sqrt{n} n \frac{\pi }{N^2} \le \frac{\pi }{\sqrt{N}} \longrightarrow 0. \end{aligned}$$

Therefore, for large N, we have

$$\begin{aligned} P \left\{ \sup _{\theta } \left| \frac{1}{N} \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| \ge 2 \epsilon \right\}\le & {} \left\{ \max _{j \le N^2} \left| \frac{1}{N} \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta _j) \right| \ge \epsilon \right\} \\\le & {} 2CN^2 \hbox {exp} \ \left( -\frac{\sqrt{N} \epsilon }{4} \right) . \end{aligned}$$

Since, $\displaystyle \sum _{N=1}^{\infty } N^2 \hbox {exp} \ \left( -\frac{\sqrt{N} \epsilon }{4} \right) < \infty$, hence, by Borel-Cantelli lemma

$$\begin{aligned} \sup _{\theta } \left| \sum _{n=1}^N U(n) w \left( \frac{n}{N} \right) \cos (n \theta ) \right| \longrightarrow 0, \ \ a.s. \end{aligned}$$

Similarly, it can be shown when the $\cos (n\theta )$ is replaced by $\sin (n \theta )$.

Lemma 6

For any given $\delta > 0$ and for some $M < \infty$, if the weight function w(t) satisfies Assumption 1, then

$$\begin{aligned} \liminf _{N \rightarrow \infty } \inf _{{{\varvec{\Theta }}} \in S_{\delta ,M}} \frac{1}{N} \{Q({{\varvec{\Theta }}}) - Q({{\varvec{\Theta }}}^0) \} > 0. \end{aligned}$$

Proof

Note that based on Lemma 5, and following the proof of Lemma 4, it can be obtained. $\square$

Proof of Theorem 3:

Since we have a similar version of Lemma 3, where the weight function w(t) satisfies Assumption 1, and using Lemma 5, Theorem 3 follows. $\square$

We need the following Lemma to prove Theorem 4.

Lemma 7

Suppose $0< \theta < \pi$ and w(t) satisfies Assumption 1, then

$$\begin{aligned} \lim _{N^{k+1} \rightarrow \infty } \frac{1}{N} \sum _{n=1}^N n^k w \left( \frac{n}{N} \right) \sin ^2(\theta t)= & {} \lim _{N^{k+1} \rightarrow \infty } \frac{1}{N} \sum _{n=1}^N n^k w \left( \frac{n}{N} \right) \cos ^2(\theta t) \\ {}= & {} \frac{1}{2} \int _0^1 t^k w(t) dt, \ \ \hbox {for } k = 0,1, \ldots . \end{aligned}$$

Proof of Lemma 7:

We will show the result for $k = 0$, for general k, it follows along the same line.

$$\begin{aligned} \lim _{N \rightarrow \infty } \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) \cos ^2(\theta t) = \frac{1}{2} \int _0^1 w(t) dt. \end{aligned}$$

For $\epsilon > 0$, there exists a polynomial $p_{\epsilon }(t)$, such that $|w(t) - p_{\epsilon }(t)| \le \epsilon$, for all $t \in [0,1]$. Hence

$$\begin{aligned} \int _0^1 w(t) dt - \epsilon \le \int _0^1 p_{\epsilon }(t) dt \le \int _0^1 w(t) dt + \epsilon . \end{aligned}$$

(3)

Further,

$$\begin{aligned}{} & {} \frac{1}{N} \sum _{n=1}^N p_{\epsilon } \left( \frac{n}{N} \right) \cos ^2(n \theta ) - \frac{\epsilon }{N} \sum _{n=1}^N \cos ^2(n \theta ) \le \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) \cos ^2(n \theta ) \le \nonumber \\{} & {} \frac{1}{N} \sum _{n=1}^N p_{\epsilon } \left( \frac{n}{N} \right) \cos ^2(n \theta ) + \frac{\epsilon }{N} \sum _{n=1}^N \cos ^2(n \theta ). \end{aligned}$$

(4)

Suppose

$$\begin{aligned} p_{\epsilon }(t) = a_0 + a_1 t + \cdots + a_k t^k \ \ \Rightarrow \ \ \int _0^1 p_{\epsilon }(t) dt = a_0 + \frac{a_1}{2} + \cdots + \frac{a_k}{k+1}. \end{aligned}$$

Now using (14),

$$\begin{aligned} \frac{1}{N} \sum _{n=1}^N p_{\epsilon } \left( \frac{n}{N} \right) \cos ^2(t \theta )= & {} \frac{1}{N} \sum _{n=1}^N \left\{ a_0 + \frac{a_1 n}{N} + \cdots + \frac{a_k n^k}{N^k} \right\} \cos ^2(n \theta ) \\\longrightarrow & {} \frac{1}{2} \left[ a_0 + \frac{a_1}{2} + \cdots + \frac{a_k}{k+1} \right] = \frac{1}{2} \int _0^1 p_{\epsilon }(t) dt. \end{aligned}$$

Taking $N \rightarrow \infty$ in (4), we obtain

$$\begin{aligned} \frac{1}{2} \int _0^1 p_{\epsilon }(t) dt - \frac{\epsilon }{2} \le \lim _{N \rightarrow \infty } \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) \cos ^2(n \theta ) \le \frac{1}{2} \int _0^1 p_{\epsilon }(t) dt + \frac{\epsilon }{2}, \end{aligned}$$

and using (3), it follows

$$\begin{aligned} \frac{1}{2} \int _0^1 w(t) dt - \frac{3\epsilon }{2} \le \lim _{N \rightarrow \infty } \frac{1}{N} \sum _{n=1}^N w \left( \frac{n}{N} \right) \cos ^2(n \theta ) \le \frac{1}{2} \int _0^1 w(t) dt + \frac{3\epsilon }{2}. \end{aligned}$$

Since $\epsilon$ is arbitrary, the result follows. $\square$

Theorem 4

Following the same line as the proof of Theorem 2, it can be proved. $\square$