Experimental Analysis of a Fast Intersection Algorithm for Sorted Sequences
This work presents an experimental comparison of intersection algorithms for sorted sequences, including the recent algorithm of Baeza-Yates. This algorithm performs on average less comparisons than the total number of elements of both inputs (n and m respectively) when n=αm (α > 1). We can find applications of this algorithm on query processing in Web search engines, where large intersections, or differences, must be performed fast. In this work we concentrate in studying the behavior of the algorithm in practice, using for the experiments test data that is close to the actual conditions of its applications. We compare the efficiency of the algorithm with other intersection algorithm and we study different optimizations, showing that the algorithm is more efficient than the alternatives in most cases, especially when one of the sequences is much larger than the other.
KeywordsQuery Processing Hybrid Algorithm Binary Search Original Algorithm Invert Index
Unable to display preview. Download preview PDF.
- 2.Baeza-Yates, R.A.: Efficient Text Serching. PhD thesis, Dept. of Computer Science, University of Waterloo, Also as Research Report CS-89-17 (May 1989)Google Scholar
- 3.Baeza-Yates, R., Bradford, P.G., Culberson, J.C., Rawlins, G.J.E.: The Complexity of Multiple Searching, unpublished manuscript (1993)Google Scholar
- 4.Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, p. 513. ACM Press/Addison-Wesley (1999)Google Scholar
- 6.Baeza-Yates, R.: Query Usage Mining in Search Engines. In: Scime, A. (ed.) Web Mining: Applications and Techniques, Idea Group, USA (2004)Google Scholar
- 7.Baeza-Yates, R., Hurtado, C., Mendoza, M., Dupret, G.: Modeling User Search Behavior, LA-WEB 2005. IEEE CS Press, Los Alamitos (2005)Google Scholar
- 8.Barbay, J., Kenyon, C.: Adaptive Intersection and t-Threshold Problems. In: Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, pp. 390–399 (January 2002)Google Scholar
- 9.Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. In: 7th WWW Conference, Brisbane, Australia (April 1998)Google Scholar
- 10.Demaine, E.D., López-Ortiz, A., Munro, J.I.: Adaptive set intersections, unions, and differences. In: Proceedings of the 11th Annual ACM-SIAM Symposium, on Discrete Algorithms, San Francisco, California, pp. 743–752 (January 2000)Google Scholar
- 11.Demaine, E.D., López-Ortiz, A., Munro, J.I.: Experiments on Adaptive Set Intersections for Text Retrieval Systems. In: Proceedings of the 3rd Workshop on Algorithm Engineering and Experiments. LNCS. Springer, Washington, DC (2001)Google Scholar
- 12.Paul, D., Kurt, M., Rajeev, R., Christian, U.: Lower Bounds for Set Intersection Queries. In: Proceedings of the 4th Annual Symposium on Discrete Algorithms, pp. 194–201 (1993)Google Scholar
- 15.Rawlins Gregory, J.E.: Compared to What?: An Introduction to the Analysis of Algorithms. Computer Science Press/W. H. Freeman (1992)Google Scholar