Faster and more accurate classification of time series by exploiting a novel dynamic time warping averaging algorithm

Abstract

A concerted research effort over the past two decades has heralded significant improvements in both the efficiency and effectiveness of time series classification. The consensus that has emerged in the community is that the best solution is a surprisingly simple one. In virtually all domains, the most accurate classifier is the nearest neighbor algorithm with dynamic time warping as the distance measure. The time complexity of dynamic time warping means that successful deployments on resource-constrained devices remain elusive. Moreover, the recent explosion of interest in wearable computing devices, which typically have limited computational resources, has greatly increased the need for very efficient classification algorithms. A classic technique to obtain the benefits of the nearest neighbor algorithm, without inheriting its undesirable time and space complexity, is to use the nearest centroid algorithm. Unfortunately, the unique properties of (most) time series data mean that the centroid typically does not resemble any of the instances, an unintuitive and underappreciated fact. In this paper we demonstrate that we can exploit a recent result by Petitjean et al. to allow meaningful averaging of “warped” time series, which then allows us to create super-efficient nearest “centroid” classifiers that are at least as accurate as their more computationally challenged nearest neighbor relatives. We demonstrate empirically the utility of our approach by comparing it to all the appropriate strawmen algorithms on the ubiquitous UCR Benchmarks and with a case study in supporting insect classification on resource-constrained sensors.

Appendices

Appendix 1: proof of convergence of DBA

We want to prove that, at each iteration, DBA provides a better average sequence $\overline{T}$, i.e., it has a lower sum of squares (Eq. 2). DTW guarantees to find the minimum alignment between two sequences, which proves optimality for the first step of DBA (Table 1—Algorithm 2—lines 1–8). Proving convergence thus requires showing that for a given multiple alignment M, the computed $\overline{T}$ is optimal.

Let $M=DTW\_multiple\_alignment\left( {\overline{T} ,{\mathbf {D}}} \right)$ (Table 1—Algorithm 3) and $M_{\ell } =M\left[ \ell \right]$. We start by rewriting the objective function (sum of squares—SS):

$$\begin{aligned} \hbox {SS}\left( {\overline{T} ,{\mathbf {D}}} \right) =\mathop \sum \limits _{i=0}^N \hbox {DTW}^{2}\left( {\overline{T} ,T_i } \right) =\mathop \sum \limits _{\ell =1}^L \mathop \sum \limits _{e\in M_{\ell } } \left( {\overline{T} \left( \ell \right) -e} \right) ^{2} \end{aligned}$$

(4)

where e is an element of a sequence of ${\mathbf {D}}$ that has been “linked” to the $\ell \mathrm{th}$ element of $\overline{T}$ by Dynamic Time Warping. Given that this function has no maximum, it is minimized when its partial derivative is 0:

$$\begin{aligned}&\displaystyle \frac{\partial \hbox {SS}\left( {\overline{T} ,{\mathbf {D}}} \right) }{\partial \overline{T} \left( \ell \right) }&=0\nonumber \\&\displaystyle \Rightarrow \quad \mathop \sum \limits _{e\in M_{\ell } } 2\times \left( {\overline{T} \left( \ell \right) -e} \right)&= 0\nonumber \\&\displaystyle \Rightarrow \quad \overline{T} \left( \ell \right)&=\frac{1}{\left| {M_{\ell } } \right| }\mathop \sum \limits _{e\in M_{\ell } } e \end{aligned}$$

(5)

This leads to $\hbox {SS}\left( {\overline{T} ,{\mathbf {D}}} \right)$ being minimized when every element $\ell$ of $\overline{T}$ is positioned as the mean of $\left| {M_{\ell }} \right|$. $\square$

Appendix 2: quantitative evaluation of DBA

See Table 6.

Table 6 Comparison of intra-class sum of squares for dynamic time warping (as per Eq. 2)

Appendix 3: representative samples of the full set of results available at [33]

Figure 9a presents the results on the electrocardiograms time series dataset (ECG 200) which show the electrical potential between two points on the surface of the body caused by a beating heart [27]. In this dataset, the proposed condensing methods that make use of the average (KMeans and AHC) outperform all other methods. Similarly, as in our example for insect surveillance (Fig. 8), a better overall accuracy can be reached while using a subset of prototypes instead of using the entire training set. The technique based on AHC reaches an error rate of 14 % with only 16 prototypes per class, while the full 1-NN algorithm requires more than 50 prototypes per class to obtain a 23 % error rate.

Fig. 9

(Best viewed in color) the error rate (with standard deviation) of various data condensing techniques for every output training size from 1 per class to 100 per class. The curves are slightly smoothed for visual clarity; the raw data spreadsheets are available at [33] (color figure online)

Figure 9b presents the results on the Gun/NoGun motion capture time series dataset. Here again, our average-based condensing techniques dominate state-of-the-art methods. It is interesting to observe the important reduction in the error rate with 2 to 5 items per class. This can be explained by the multimodality of the two classes of the dataset, which has been created from recording of movements of people with different heights.

Figure 9c presents the results on the uWaveGestureLibrary(Z) time series dataset which contains over 4000 samples of accelerometer readings for gesture recognition. This example shows that one prototype per class makes it possible to “explain” most of the variance in the classes of the dataset. This is another critical example, because gesture recognition systems not only have to be reliable, but also often must perform the recognition very quickly. With one prototype per class on this dataset that is composed of more than 100 training time series for each class, our condensing technique offers a 100-fold speedup, with a loss in the recovery of only 5 %. This starkly contrasts with a condensing using the best non-average-based method (K-medoids), for which the error rate increases by 14 % for the same speedup.