A distributed framework for large-scale semantic trajectory similarity join

Abstract

The similarity join is a common yet expensive operator for large-scale semantic trajectories analytics. In this paper, we propose DFST, an efficient framework for semantic trajectory similarity join in distributed systems. We devise ITS index and summary index, which consider textual, temporal, and spatial domains, and theoretically demonstrate that they can effectively prune pairs of dissimilar trajectories. Moreover, DFST can support most existing similarity functions to quantify the spatial similarity between semantic trajectories. We have conducted extensive experiments on real world datasets, and experimental results show that DFST achieves a 13.6% improvement of join performance compared to existing semantic trajectory similarity join methods.

Appendices

Appendix A: Other distance measures

A.1 Dynamic Time Warping (DTW)

It computes the minimum cumulative distance when two trajectories match [4].

Definition 5

(DTW) Given two trajectories $\mathcal {T}=\left \lbrace o^{\mathcal {T}}_1,\ldots ,o^{\mathcal {T}}_m\right \rbrace$ and $tr=\left \lbrace o^{tr}_1,\ldots ,o^{tr}_n\right \rbrace$, DTW [4] is computed as below.

$$ DTW(\mathcal{T},tr)=\begin{cases} {\sum}_{i=1}^{m}dist(o^{\mathcal{T}}_{i}.p,o^{tr}_{1}.p)&if\ n=1\\ {\sum}_{j=1}^{n}dist(o^{\mathcal{T}}_{1}.p,o^{tr}_{j}.p)&if\ m=1\\ dist(o^{\mathcal{T}}_{m}.p,o^{tr}_{n}.p)+\min\Big(DTW(\mathcal{T}^{m-1},tr^{n-1}),\\ DTW(\mathcal{T}^{m-1},tr),DTW(\mathcal{T},tr^{n-1})\Big)& otherwise \end{cases} $$

(16)

where $\mathcal {T}^{m-1}$ is the prefix trajectory of $\mathcal {T}$ by removing the last point.

According to (6) and Definition 5, we can conclude that $DTW(\mathcal {T},tr)$ is not less than Fréchet$(\mathcal {T},tr)$ constant. Given two trajectories $\mathcal {T}_1$ and $\mathcal {T}_3$ in Fig. 17, $DTW(\mathcal {T}_1,\mathcal {T}_3)=6.41 >$ Fréchet$(\mathcal {T}_1,\mathcal {T}_3)=1.41$. To support DTW, DFST doesn’t need to update ε by accumulating distances from it when querying the partitions. Similarly, we can still utilize partition/node distance lower bound pruning and summary pruning.

Fig. 17

Example Trajectories

A.2 Edit Distance on Real Sequences(EDR)

Definition 6 (EDR)

Given two trajectories $\mathcal {T}$ and $\mathcal {Q}$, and a matching threshold δ ≥ 0, EDR_δ [23] is:

$$ EDR_{\delta}(\mathcal{T},\mathcal{Q})= \begin{cases} n&if\ m=0\\ m&if\ n=0\\ \min\Big(EDR_{\delta}(\mathcal{T}^{2,m},q^{2,n})+subcost(t_{1},q_{1}),\\ \text{EDR}_{\delta}(\mathcal{T}^{2,m},q)+1,EDR_{\delta}(\mathcal{T},q^{2,n})+1\Big)&otherwise \end{cases} $$

(17)

where $\mathcal {T}^{2,m}$ stands for trajectory $\mathcal {T}$ with its first point removed, and subcost(t,q) = 0 if dist(t,q) ≤ δ; 1 otherwise.

Given two trajectories $\mathcal {T}_1$ and $\mathcal {T}_3$ in Fig. 17, let δ = 1, we have $EDR_{\delta }(\mathcal {T}_1,\mathcal {T}_3)=2$. To support EDR, for the MBR of each partition, we compute the distance. If it exceeds δ, subcost(t,q) is always equal to 1 and $EDR_{\delta }(\mathcal {T},tr)=\max \limits (m,n)$, we safely prune this partition.

A.3 Longest Common Subsequence Distance (LCSS)

Definition 7 (LCSS)

Given two trajectories $\mathcal {T}$ and $\mathcal {Q}$ with lengths m and n, and a matching threshold δ, LCSS_δ is defined as below [23]:

$$ LCSS_{\delta}(\mathcal{T},\mathcal{Q})=\begin{cases} n&if\ m=0\\ m&if\ n=0\\ 1+LCSS_{\delta}(\mathcal{T}^{m-1},\mathcal{Q}^{n-1})&if\ dist(t_{m},q_{n})\leq \delta\\ \max\Big(LCSS_{\delta}(\mathcal{T}^{m-1},\mathcal{Q}),\\ LCSS_{\delta}(\mathcal{T},\mathcal{Q}^{n-1})\Big) &otherwise \end{cases} $$

(18)

where $\mathcal {T}^{m-1}$ is the prefix trajectory of $\mathcal {T}$ with the last point removed.

Given two trajectories $\mathcal {T}_1$ and $\mathcal {T}_3$ in Fig. 17, let δ = 1, we have $LCSS_{\delta }(\mathcal {T}_1,\mathcal {T}_3)=5$. Similar to EDR, for each partition’s MBR, we compute the distance to the query trajectory tr. According Definition 7, if it is beyond δ, $LCSS_{\delta }(\mathcal {T},tr)$ is always equal to 0, we also safely prune this partition.

Appendix B: Comparison with other distance measures

We evaluated DFST’s performance with different distance measures, including Fréchet, DTW, EDR and LCSS (ε = 0.0001), in Fig. 18. We could observe that: (1) Fréchet was slower than DTW with the same threshold, because DTW sums the values up through the whole path from (1, 1) to (m,n) while Fréchet chooses the maximum value; (2) LCSS is as fast as EDR because the time complexity of both LCSS and EDR is O(mn).

Fig. 18

Other distance measures