Approximate Nearest Neighbor for Curves: Simple, Efficient, and Deterministic

Abstract

In the $(1+{\varepsilon },r)$-approximate near-neighbor problem for curves (ANNC) under some similarity measure $\delta$, the goal is to construct a data structure for a given set $\mathcal {C}$ of curves that supports approximate near-neighbor queries: Given a query curve Q, if there exists a curve $C\in \mathcal {C}$ such that $\delta (Q,C)\le r$, then return a curve $C'\in \mathcal {C}$ with $\delta (Q,C')\le (1+{\varepsilon })r$. There exists an efficient reduction from the $(1+{\varepsilon })$-approximate nearest-neighbor problem to ANNC, where in the former problem the answer to a query is a curve $C\in \mathcal {C}$ with $\delta (Q,C)\le (1+{\varepsilon })\cdot \delta (Q,C^*)$, where $C^*$ is the curve of $\mathcal {C}$ most similar to Q. Given a set $\mathcal {C}$ of n curves, each consisting of m points in d dimensions, we construct a data structure for ANNC that uses $n\cdot O(\frac{1}{{\varepsilon }})^{md}$ storage space and has O(md) query time (for a query curve of length m), where the similarity measure between two curves is their discrete Fréchet or dynamic time warping distance. Our method is simple to implement, deterministic, and results in an exponential improvement in both query time and storage space compared to all previous bounds. Further, we also consider the asymmetric version of ANNC, where the length of the query curves is $k \ll m$, and obtain essentially the same storage and query bounds as above, except that m is replaced by k. Finally, we apply our method to a version of approximate range counting for curves and achieve similar bounds.

Appendices

Appendix: A Deterministic Construction Using a Prefix Tree

When implementing the dictionary $\mathcal {D}$ as a hash table, the construction of the data structure is randomized and thus in the worst case we might get higher prepeocessing time. To avoid this, we can implement $\mathcal {D}$ as a prefix tree.

1.1 Appendix: A.1 Discrete Fréchet Distance

In this section we describe the implementation of $\mathcal {D}$ as a prefix tree in the case of ANNC under DFD.

We can construct a prefix tree $\mathcal {T}$ for the curves in $\mathcal {I}$, where any path in $\mathcal {T}$ from the root to a leaf corresponds to a curve that is stored in it. For each $1\le i\le n$ and curve ${\overline{Q}}\in \mathcal {I}_i$, if ${\overline{Q}}\notin \mathcal {T}$, insert ${\overline{Q}}$ into $\mathcal {T}$, and set $C({\overline{Q}})\leftarrow C_i$.

Each node $v\in \mathcal {T}$ corresponds to a grid point from $\mathcal {G}$. Denote the set of v’s children by N(v). We store with v a multilevel search tree on N(v), with a level for each coordinate. The points in $\mathcal {G}$ are the grid points contained in nm balls of radius $(1+{\varepsilon })r$. Thus when projecting these points to a single dimension, the number of 1-dimensional points is at most $nm\cdot \frac{\sqrt{d}(1+{\varepsilon })2r}{{\varepsilon }r}=O(\frac{nm\sqrt{d}}{{\varepsilon }})$. So in each level of the search tree on N(v) we have $O(\frac{nm\sqrt{d}}{{\varepsilon }})$ 1-dimensional points, so the query time is $O(d\log (\frac{nmd}{{\varepsilon }}))$.

Inserting a curve of length m to the tree $\mathcal {T}$ takes $O(md\log (\frac{nmd}{{\varepsilon }}))$ time. Since $\mathcal {T}$ is a compact representation of $|\mathcal {I}|=n\cdot O(\frac{1}{{\varepsilon }})^{dm}$ curves of length m, the number of nodes in $\mathcal {T}$ is $m\cdot |\mathcal {I}|=nm\cdot O(\frac{1}{{\varepsilon }})^{dm}$. Each node $v\in \mathcal {T}$ contains a search tree for its children of size $O(d\cdot |N(v)|)$, and $\sum _{v\in \mathcal {T}}|N(v)|=nm\cdot O(\frac{1}{{\varepsilon }})^{dm}$ so the total space complexity is $O(nmd)\cdot O(\frac{1}{{\varepsilon }})^{md}=n\cdot O(\frac{1}{{\varepsilon }})^{md}$. Constructing $\mathcal {T}$ takes $O(|\mathcal {I}|\cdot md\log (\frac{nmd}{{\varepsilon }}))=n\log (\frac{nmd}{{\varepsilon }})\cdot O(\frac{1}{{\varepsilon }})^{md}$ time.

Theorem 23

There exists a data structure for the $(1+{\varepsilon },r)$-ANNC under DFD, with $n\cdot O(\frac{1}{{\varepsilon }})^{dm}$ space, $n\cdot \log (\frac{n}{{\varepsilon }})\cdot O(\frac{1}{{\varepsilon }})^{md}$ preprocessing time, and $O(md\log (\frac{nmd}{{\varepsilon }}))$ query time.

Similarly, for the asymmetric case we obtain the following theorem.

Theorem 24

There exists a data structure for the asymmetric $(1+{\varepsilon },r)$-ANNC under DFD, with $n\cdot O(\frac{1}{{\varepsilon }})^{dk}$ space, $nm\log (\frac{n}{{\varepsilon }})\cdot \left( O(d\log m)+O(\frac{1}{{\varepsilon }})^{kd}\right)$ preprocessing time, and $O(kd\log (\frac{nkd}{{\varepsilon }}))$ query time.

1.2 Appendix: A.2 $\ell _{p,2}$-Distance

For the case of ANNC under $\ell _{p,2}$-distance, the total number of curves stored in the tree $\mathcal {T}$ is roughly the same as in the case of DFD. We only need to show that for a given node v of the tree $\mathcal {T}$, the upper bound on the size and query time of the search tree associated with it are similar.

The grid points corresponding to the nodes in N(v) are from n sets of m balls with radius $(1+{\varepsilon })$. When projecting the grid points in one of the balls to a single dimension, the number of 1-dimensional points is at most $\frac{m^{1/p}\sqrt{d}}{{\varepsilon }}\cdot (1+{\varepsilon })$, so the total number of projected points is at most $\frac{nm^{1+\frac{1}{p}}\sqrt{d}}{{\varepsilon }}\cdot (1+{\varepsilon })$.

Thus in each level of the search tree of v we have $O(\frac{nm^2\sqrt{d}}{{\varepsilon }})$ 1-dimensional points, so the query time is $O(d\log (\frac{nmd}{{\varepsilon }}))$, and inserting a curve of length m into the tree $\mathcal {T}$ takes $O(md\log (\frac{nmd}{{\varepsilon }}))$ time. Note that the size of the search tree of v remains $O(d\cdot |N(v)|)$.

We conclude that the total space complexity is $O(\frac{nm^2\sqrt{d}}{{\varepsilon }})\cdot O(\frac{1}{{\varepsilon }})^{m(d+1)}=n\cdot O(\frac{1}{{\varepsilon }})^{m(d+1)}$, constructing $\mathcal {T}$ takes $O(|\mathcal {I}|\cdot md\log (nmd/{\varepsilon }))=n\log (\frac{n}{{\varepsilon }})\cdot O(\frac{1}{{\varepsilon }})^{m(d+1)}$ time, and the total query time is $O(md\log (\frac{nmd}{{\varepsilon }}))$.

Theorem 25

There exists a data structure for the $(1+{\varepsilon },r)$-ANNC under $\ell _{p,2}$-distance, with $n\cdot O(\frac{1}{{\varepsilon }})^{m(d+1)}$ space, $n\cdot \log (\frac{n}{{\varepsilon }})\cdot O(\frac{1}{{\varepsilon }})^{m(d+1)}$ preprocessing time, and $O(md\log (\frac{nmd}{{\varepsilon }}))$ query time.

Appendix: B Dealing with Query Curves and Input Curves of Varying Size

Notice that if an input curve $C_i$ has length $t<m$, then the size of the set of candidates $\mathcal {I}_i$ (and $\mathcal {I}'_i$ in the asymmetric case) can only decrease.

In addition, our assumption that all query curves are of length exactly k can be easily removed by constructing k data structures $\mathcal {D}_1,\dots ,\mathcal {D}_k$, where $\mathcal {D}_i$ is our data structure constructed for query curves of length i (instead of k), for $1 \le i \le k$. Clearly, the query time does not change. The storage space is multiplied by k, so for the case of DFD we have storage space $nk\cdot O(\frac{1}{{\varepsilon }})^{kd}$, but $k<2^{kd}$, so the storage space remains $n\cdot O(\frac{1}{{\varepsilon }})^{kd}$. Similarly, for the case of $\ell _{p,2}$-distance we obtain storage space of $n\cdot O(\frac{1}{{\varepsilon }})^{k(d+1)}\cdot \left( \frac{m}{k}\right) ^{kd/p}$.

Appendix: C One-Way Alignments

Claim 26

Let A, B, C be three curves, and let $\tau _1$, $\tau _2$ be two one-way alignments such that $\tau _1$ matches C to A and $\tau _2$ matches C to B. Then $d_{p,2}(A,B)\le \sigma _{p,2}(\tau _1(C,A))+\sigma _{p,2}(\tau _2(C,B))$.

Proof

Denote by $k_A,k_B,k_C$ the lengths of the curves A, B, C respectively. Consider the following algorithm that constructs an alignment $\tau$. For every $1\le x\le k_C$, denote by $i_x,j_x$ the unique indexes such that $(x,i_x)\in \tau _1$ and $(x,j_x)\in \tau _2$. Add the pair $(i_x,j_x)$ to $\tau$ if it is not already there.

First, we need to show that $\tau =\langle (i_1,j_1),\dots ,(i_t,j_t)\rangle$ is a valid alignment. Clearly, $(i_1,j_1)=(1,1)$ because $(1,1)\in \tau _1$ and $(1,1)\in \tau _2$. Similarly, $(i_t,j_t)=(k_A,k_B)$ because $(k_C,k_A)\in \tau _1$ and $(k_C,k_B)\in \tau _2$.

For any $1\le s<t$, consider the two consecutive pairs $(i_s,j_s),(i_{s+1},j_{s+1})\in \tau$. Let $x_1$ be an index such that $(x_1,i_s)\in \tau _1$ and $(x_1,j_s)\in \tau _2$, and $x_2$ an index such that $(x_2,i_{s+1})\in \tau _1$ and $(x_2,j_{s+1})\in \tau _2$. Since $\tau _1,\tau _2$ are one-way alignments, we have $x_1\ne x_2$. Moreover, since the algorithm added $(i_s,j_s)$ to $\tau$ before $(i_{s+1},j_{s+1})$, we have $x_1<x_2$. This implies that $i_{s+1}\ge i_s$ and $j_{s+1}\ge j_s$. Assume by contradiction that $i_{s+1} > i_s+1$, and let x be the index such that $(x,i_s+1)\in \tau _1$, then $x_1<x<x_2$ and thus the algorithm adds a pair $(i_s+1,j)$ for some index j after $(i_s,j_s)$ and before $(i_{s+1},j_{s+1})$, a contradiction. So we have $i_s\le i_{s+1} \le i_s+1$, and by symmetric arguments, $j_s\le j_{s+1} \le j_s+1$, and therefore $\tau$ is valid.

Using the triangle inequality for the $\ell _p$ norm, we get that

$$\begin{aligned} d_{p,2}(A,B)\le \sigma _{p,2}(\tau (A,B))&=\Big (\sum _{(i,j)\in \tau }\Vert a_{i}-b_{j}\Vert _{2}^{p}\Big )^{1/p}\\&\le \Big (\sum _{x=1}^{k_{C}}\Vert a_{i_{x}}-b_{j_{x}}\Vert _{2}^{p}\Big )^{1/p}\\&\le \Big (\sum _{x=1}^{k_{C}}\Vert a_{i_{x}}-c_{x}\Vert _{2}^{p}\Big )^{1/p}+\Big (\sum _{x=1}^{k_{C}}\Vert c_{x}-b_{j_{x}}\Vert _{2}^{p}\Big )^{1/p}\\&=\sigma _{p,2}(\tau _{1}(C,A))+\sigma _{p,2}(\tau _{2}(C,B))~. \end{aligned}$$

$\square$