Improving the efficiency of traditional DTW accelerators

Abstract

Dynamic time warping (DTW) is the most popular approach for evaluating the similarity of time series, but its computation is costly. Therefore, simple functions lower bounding DTW distances have been designed, accelerating searches by quickly pruning sequences that could not possibly be best matches. The tighter the bounds, the more they prune and the better the performance. Designing new functions that are even tighter is difficult because their computation is likely to become complex, canceling the benefits of their pruning. It is possible, however, to design simple functions with a higher pruning power by relaxing the no false dismissal assumption, resulting in approximate lower bound functions. This paper describes how very popular approaches accelerating DTW such as $\text {LB}\_\text {Keogh}{}$ and $\text {LB}\_\text {PAA}{}$ can be made more efficient via approximations. The accuracy of approximations can be tuned, ranging from no false dismissal to potential losses when aggressively set for great response time savings. At very large scale, indexing time series is mandatory. This paper also describes how approximate lower bound functions can be used with iSAX. Furthermore, it shows that a $k$-means-based quantization step for iSAX gives significant performance gains.

Appendices

Appendices

1.1 Proofs and mathematical definitions for upper bounds

We prove here that UB_Keogh is upper bounding DTW when the latter is restricted to a Sakoe–Chiba band. We also introduce upper bounds related to $\text {LB}\_\text {PAA}{}$ and $i\text {SAX}\_\text {MinDist}{}$ for which we omit the proofs as they follow the exact same principles.

Definition 1

Let UB_Keogh be:

$$\begin{aligned} \text {UB}\_\text {Keogh}{}(Q,C) = \sum _{i=1}^{n} {\left\{ \begin{array}{ll} (c_i - L_i) &{} \text { if } c_i > U_i\\ (U_i - c_i) &{} \text { if } c_i < L_i\\ \max (U_i - c_i, c_i - L_i) &{} \text { otherwise} \end{array}\right. } \end{aligned}$$

(11)

Lemma 1

For any two sequences $Q$ and $C$ of length $n$, the following inequality stands:

$$\begin{aligned} L_1(Q,C) \ge \text {DTW}{}(Q,C) \end{aligned}$$

where the considered DTW is constrained to a Sakoe–Chiba band of width $r$.

Proof

Let $Q$ and $C$ be two sequences of length $n$. Manhattan distance $L_1$ corresponds to the alignment that follows the diagonal path. Hence, this distance is associated with one of the possible paths considered by the DTWalgorithm and is therefore greater than the cost of the minimal path, that is the value returned by DTW, which concludes the proof for Lemma 1. $\square$

Proposition 1

For any two sequences $Q$ and $C$ of length $n$, the following inequality stands:

$$\begin{aligned} \text {UB}\_\text {Keogh}{}(Q,C) \ge \text {DTW}{}(Q,C) \end{aligned}$$

where the considered DTW is constrained to a Sakoe–Chiba band of width $r$.

Proof

It is important to notice that each term in the sum that occurs in the definition of UB_Keogh is related to exactly one term in the computation of the $L_1$ distance. The only difference is that for UB_Keogh, the $i$th term corresponds to the distance between the $i$th point in the candidate sequence and its furthest corresponding point in the envelope of the query, while for $L_1$, the same term is equal to the distance between the $i$th point in the candidate sequence and one of its possible corresponding points in the envelope of the query. The latter distance is then, by definition, smaller than the former, and the following inequality is then straightforward, coming from Lemma 1:

$$\begin{aligned} \text {UB}\_\text {Keogh}{}(Q,C) \ge L_1(Q,C) \ge \text {DTW}{}(Q,C). \end{aligned}$$

$\square$

Definition 2

Let us define UB_PAA as:

$$\begin{aligned} \text {UB}\_\text {PAA}{}(Q,C)&= \frac{n}{N} \cdot {} \left( \sum _{i=1}^{N}\max (\hat{U}_i - \bar{c_i}, \bar{c_i} - \hat{L}_i)\right) \nonumber \\&+ \frac{n}{N} \cdot {} \left( \max (C)-\min (C)\right) . \end{aligned}$$

(12)

Lemma 2

For any two sequences $Q$ and $C$ of length $n$, the following inequality stands:

$$\begin{aligned} \text {UB}\_\text {PAA}{}(Q,C) \ge \text {UB}\_\text {Keogh}{}(Q,C). \end{aligned}$$

Proposition 2

For any two sequences $Q$ and $C$ of length $n$, the following inequality stands:

$$\begin{aligned} \text {UB}\_\text {PAA}{}(Q,C) \ge \text {DTW}{}(Q,C) \end{aligned}$$

where the considered DTW is constrained to a Sakoe–Chiba band of width $r$.

Proof

It is straightforward that proving Lemma 2 is sufficient to prove, using Proposition 1, that Proposition 2 holds.

Let $Q$ and $C$ be two sequences of length $n$ and $W$ be the minimum cost path used for DTW computation with a Sakoe–Chiba band of with $r$ between $Q$ and $C$. So as to prove:

$$\begin{aligned} \sum _{i=1}^{n}\max (U_i - c_i, c_i - L_i) \le \frac{n}{N} \cdot {} \left( \sum _{i=1}^{N}\max (\hat{U}_i - \bar{c_i}, \bar{c_i} - \hat{L}_i) + \max (C)-\min (C)\right) \end{aligned}$$

it is sufficient to prove that, for all $i \in \{ 1,\ldots ,N \}$:

$$\begin{aligned} \sum _{j=\frac{n}{N}(i-1)+1}^{\frac{n}{N}i}\max (U_j - c_j, c_j - L_j) \le \frac{n}{N} \cdot {} \left( \max (\hat{U}_i - \bar{c_i}, \bar{c_i} - \hat{L}_i) + (\max (C)-\min (C)\right) \end{aligned}$$

Let $i \in \{ 1,\ldots ,N \}$. If we denote, for all $j \in \{ \frac{n}{N}(i-1)+1,\ldots ,\frac{n}{N}i \}$, $c_j = \bar{c_i} \!+\! \Delta c_j$, we get :

$$\begin{aligned}&\sum _{j=\frac{n}{N}(i-1)+1}^{\frac{n}{N}i}\max (U_j - c_j, c_j - L_j)\\&\quad \qquad = \sum _{j=\frac{n}{N}(i-1)+1}^{\frac{n}{N}i}\max (U_j - (\bar{c_i} + \Delta c_j), \bar{c_i} + \Delta c_j - L_j) \\&\quad \qquad \le \sum _{j=\frac{n}{N}(i-1)+1}^{\frac{n}{N}i}\max (\hat{U_i} - (\bar{c_i} + \Delta c_j), \bar{c_i} + \Delta c_j - \hat{L_i}) \\&\quad \qquad \le \sum _{j=\frac{n}{N}(i-1)+1}^{\frac{n}{N}i}\max (\hat{U_i} - \bar{c_i}, \bar{c_i} - \hat{L_i}) + |\Delta c_j| \\&\quad \qquad \le \frac{n}{N}\left( \max (\hat{U_i} - \bar{c_i}, \bar{c_i} - \hat{L_i}) + \max (C) - \min (C)\right) \end{aligned}$$

which concludes the proof. $\square$

Definition 3

Let us define iSAX_MaxDist as:

$$\begin{aligned} { i}\text {SAX}\_\text {MaxDist}{}(Q,R) = \sqrt{\frac{n}{N} \sum _{i=1}^{N} X_i} \end{aligned}$$

(13)

where

$$\begin{aligned} \forall i \le N, X_i = {\left\{ \begin{array}{ll} (\bar{q_i} - B_i)^2 &{} \text { if } \bar{q_i} > H_i\\ (H_i - \bar{q_i})^2 &{} \text { if } \bar{q_i} < B_i\\ \max (H_i - \bar{q_i}, \bar{q_i} - B_i)^2 &{} \text { otherwise} \end{array}\right. }. \end{aligned}$$

(14)

1.2 Balancing $k$-means

When $k=2$, balancing $k$-means does not require any iterative process as proposed in Tavenard et al. [17]. It is possible to derive elevation $h$ that the most populated clusters’ centroid will get in order for both clusters to finally get equal populations without resorting to any iterative process.

Let us assume, without loss of generality, that $k$-means produced two centroids $\mathbf {C_1}$ and $\mathbf {C_2}$ and that cluster $C_1$ is more populated than $C_2$. Using notations introduced in Fig. 9, intersection between the line $(\mathbf {C_1}, \mathbf {C_2})$ and the boundary between classes $C_1$ and $C_2$ is then $\mathbf {C_0}$, middle of the line segment $[\mathbf {C_1}, \mathbf {C_2}]$. We aim at evaluating elevation $h$ such that this point moves to $\mathbf {C'_0}$ that is the median of projected data points. It is straightforward that if one builds a new boundary that is parallel to the original one and passes through $\mathbf {C'_0}$, both clusters will be equally populated. After solving the related system of equations, one gets:

$$\begin{aligned} h = \sqrt{2(x_2-x_1) \left( \frac{x_1+x_2}{2}-x'_0\right) ,} \end{aligned}$$

(15)

where $x_1$ and $x_2$ are known from the $k$-means and $x'_0$ is the median of projected data points.

Fig. 9

Balancing $k$-means for the $k=2$ case