CID: an efficient complexity-invariant distance for time series

Abstract

The ubiquity of time series data across almost all human endeavors has produced a great interest in time series data mining in the last decade. While dozens of classification algorithms have been applied to time series, recent empirical evidence strongly suggests that simple nearest neighbor classification is exceptionally difficult to beat. The choice of distance measure used by the nearest neighbor algorithm is important, and depends on the invariances required by the domain. For example, motion capture data typically requires invariance to warping, and cardiology data requires invariance to the baseline (the mean value). Similarly, recent work suggests that for time series clustering, the choice of clustering algorithm is much less important than the choice of distance measure used.In this work we make a somewhat surprising claim. There is an invariance that the community seems to have missed, complexity invariance. Intuitively, the problem is that in many domains the different classes may have different complexities, and pairs of complex objects, even those which subjectively may seem very similar to the human eye, tend to be further apart under current distance measures than pairs of simple objects. This fact introduces errors in nearest neighbor classification, where some complex objects may be incorrectly assigned to a simpler class. Similarly, for clustering this effect can introduce errors by “suggesting” to the clustering algorithm that subjectively similar, but complex objects belong in a sparser and larger diameter cluster than is truly warranted.We introduce the first complexity-invariant distance measure for time series, and show that it generally produces significant improvements in classification and clustering accuracy. We further show that this improvement does not compromise efficiency, since we can lower bound the measure and use a modification of triangular inequality, thus making use of most existing indexing and data mining algorithms. We evaluate our ideas with the largest and most comprehensive set of time series mining experiments ever attempted in a single work, and show that complexity-invariant distance measures can produce improvements in classification and clustering in the vast majority of cases.

Appendix

1.1 A $\rho$-relaxed triangular inequality proof

In this section we prove that CID obeys the $\rho$-relaxed triangular inequality:

$$\begin{aligned} D_{CID}(A,B) \le \rho (D_{CID}(A,C) + D_{CID}(C,B)) \end{aligned}$$

We start our proof by stating the triangular inequality of Euclidean distance:

$$\begin{aligned} ED(A,B) \le ED(A,C) + ED(C,A) \end{aligned}$$

Remember that the complexity correction factor $CF$ is a quantity greater than or equal to one; therefore, we can multiply both sides of the inequality by $CF(A,B)$:

$$\begin{aligned} ED(A,B)CF(A,B) \le CF(A,B)(ED(A,C)+ED(C,A)) \end{aligned}$$

The left-hand side of the inequality is our definition of CID; hence:

$$\begin{aligned} D_{CID}(A,B) \le \rho (ED(A,C)+ED(C,A)) \end{aligned}$$

with $\rho = CF(A,B)$.

Finally, we can again use the fact that $CF$ is greater than or equal to one to change Euclidean distances to CID on the right-hand side of the equation:

$$\begin{aligned} D_{CID}(A,B) \le \rho (ED(A,C)CF(A,C)+ED(C,A)CF(C,A)) \end{aligned}$$

And, therefore:

$$\begin{aligned} D_{CID}(A,B) \le \rho (D_{CID}(A,C)+D_{CID}(C,A)) \end{aligned}$$