Experimental Analysis of a Fast Intersection Algorithm for Sorted Sequences

  • Ricardo Baeza-Yates
  • Alejandro Salinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3772)

Abstract

This work presents an experimental comparison of intersection algorithms for sorted sequences, including the recent algorithm of Baeza-Yates. This algorithm performs on average less comparisons than the total number of elements of both inputs (n and m respectively) when n=αm (α > 1). We can find applications of this algorithm on query processing in Web search engines, where large intersections, or differences, must be performed fast. In this work we concentrate in studying the behavior of the algorithm in practice, using for the experiments test data that is close to the actual conditions of its applications. We compare the efficiency of the algorithm with other intersection algorithm and we study different optimizations, showing that the algorithm is more efficient than the alternatives in most cases, especially when one of the sequences is much larger than the other.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baeza-Yates, R.: A Fast Set Intersection Algorithm for Sorted Sequences. In: Sahinalp, S.C., Muthukrishnan, S.M., Dogrusoz, U. (eds.) CPM 2004. LNCS, vol. 3109, pp. 400–408. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Baeza-Yates, R.A.: Efficient Text Serching. PhD thesis, Dept. of Computer Science, University of Waterloo, Also as Research Report CS-89-17 (May 1989)Google Scholar
  3. 3.
    Baeza-Yates, R., Bradford, P.G., Culberson, J.C., Rawlins, G.J.E.: The Complexity of Multiple Searching, unpublished manuscript (1993)Google Scholar
  4. 4.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, p. 513. ACM Press/Addison-Wesley (1999)Google Scholar
  5. 5.
    Baeza-Yates, R., Sainte-Jean, F.: A Three Level Search Engine Index bases in Query Log Distribution. In: Nascimento, M.A., de Moura, E.S., Oliveira, A.L. (eds.) SPIRE 2003. LNCS, vol. 2857, pp. 56–65. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  6. 6.
    Baeza-Yates, R.: Query Usage Mining in Search Engines. In: Scime, A. (ed.) Web Mining: Applications and Techniques, Idea Group, USA (2004)Google Scholar
  7. 7.
    Baeza-Yates, R., Hurtado, C., Mendoza, M., Dupret, G.: Modeling User Search Behavior, LA-WEB 2005. IEEE CS Press, Los Alamitos (2005)Google Scholar
  8. 8.
    Barbay, J., Kenyon, C.: Adaptive Intersection and t-Threshold Problems. In: Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, pp. 390–399 (January 2002)Google Scholar
  9. 9.
    Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. In: 7th WWW Conference, Brisbane, Australia (April 1998)Google Scholar
  10. 10.
    Demaine, E.D., López-Ortiz, A., Munro, J.I.: Adaptive set intersections, unions, and differences. In: Proceedings of the 11th Annual ACM-SIAM Symposium, on Discrete Algorithms, San Francisco, California, pp. 743–752 (January 2000)Google Scholar
  11. 11.
    Demaine, E.D., López-Ortiz, A., Munro, J.I.: Experiments on Adaptive Set Intersections for Text Retrieval Systems. In: Proceedings of the 3rd Workshop on Algorithm Engineering and Experiments. LNCS. Springer, Washington, DC (2001)Google Scholar
  12. 12.
    Paul, D., Kurt, M., Rajeev, R., Christian, U.: Lower Bounds for Set Intersection Queries. In: Proceedings of the 4th Annual Symposium on Discrete Algorithms, pp. 194–201 (1993)Google Scholar
  13. 13.
    David, D., Richard, L.: On the Complexity of Computations Under Varying Sets of Primitives. Journal of Computer and Systems Sciences 18, 86–91 (1979)MATHCrossRefGoogle Scholar
  14. 14.
    Hwang, F.K., Lin, S.: A Simple algorithm for merging two disjoint linearly ordered lists. SIAM J. on Computing 1, 31–39 (1972)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Rawlins Gregory, J.E.: Compared to What?: An Introduction to the Analysis of Algorithms. Computer Science Press/W. H. Freeman (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Ricardo Baeza-Yates
    • 1
  • Alejandro Salinger
    • 1
  1. 1.Center for Web Research, Department of Computer ScienceUniversity of ChileSantiagoChile

Personalised recommendations