Fast Exact Algorithm to Solve Continuous Similarity Search for Evolving Queries
We study the continuous similarity search problem for evolving queries which has recently been formulated. Given a data stream and a database composed of n sets of items, the purpose of this problem is to maintain the top-k most similar sets to the query which evolves over time and consists of the latest W items in the data stream. For this problem, the previous exact algorithm adopts a pruning strategy which, at the present time T, decides the candidates of the top-k most similar sets from past similarity values and computes the similarity values only for them. This paper proposes a new exact algorithm which shortens the execution time by computing the similarity values only for sets whose similarity values at T can change from time \(T-1\). We identify such sets very fast with frequency-based inverted lists (FIL). Moreover, we derive the similarity values at T in O(1) time by updating the previous values computed at time \(T-1\). Experimentally, our exact algorithm runs faster than the previous exact algorithm by one order of magnitude.
KeywordsData stream Evolving query Set similarity search Inverted lists
This work was supported by JSPS KAKENHI Grant Number JP15K00148, 2016.
- 2.Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: Proceedings of ACM SCG 2004, pp. 253–262. ACM (2004)Google Scholar
- 9.Yang, D., Shastri, A., Rundensteiner, E.A., Ward, M.O.: An optimal strategy for monitoring top-k queries in streaming windows. In: Proceedings of the 14th International Conference on Extending Database Technology, pp. 57–68. ACM (2011)Google Scholar