Advertisement

A Fast Set Intersection Algorithm for Sorted Sequences

  • Ricardo Baeza-Yates
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3109)

Abstract

This paper introduces a simple intersection algorithm for two sorted sequences that is fast on average. It is related to the multiple searching problem and to merging. We present the worst and average case analysis, showing that in the former, the complexity nicely adapts to the smallest list size. In the later case, it performs less comparisons than the total number of elements on both inputs when n = αm (α > 1). Finally, we show its application to fast query processing in Web search engines, where large intersections, or differences, must be performed fast.

Keywords

Set operations merging multiple search Web search engines inverted indices 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baeza-Yates, R.A.: Efficient Text Searching. PhD thesis, Dept. of Computer Science, University of Waterloo, Also as Research Report CS-89-17 (May 1989)Google Scholar
  2. 2.
    Baeza-Yates, R., Bradford, P.G., Culberson, J.C., Rawlins, G.J.E.: The Complexity of Multiple Searching (1993)(unpublished manuscript)Google Scholar
  3. 3.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, p. 513. ACM Press/Addison-Wesley (1999)Google Scholar
  4. 4.
    Baeza-Yates, R., Saint-Jean, F.: A Three Level Search Engine Index based in Query Log Distribution. In: Nascimento, M.A., de Moura, E.S., Oliveira, A.L. (eds.) SPIRE 2003. LNCS, vol. 2857, pp. 56–65. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Baeza-Yates, R.: Query Usage Mining in Search Engines. In: Scime, A. (ed.) Web Mining: Applications and Techniques, Idea Group (2004)Google Scholar
  6. 6.
    Barbay, J., Kenyon, C.: Adaptive Intersection and t-Threshold Problems. In: Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, January 2002, pp. 390–399 (2002)Google Scholar
  7. 7.
    Brin, S., Page, L.: The anatomy of a large-scale hypertextualWeb search engine. In: 7th WWW Conference, Brisbane, Australia (April 1998)Google Scholar
  8. 8.
    Demaine, E.D., López-Ortiz, A., Munro, J.I.: Adaptive set intersections, unions, and differences. In: Proceedings of the 11th Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, January 2000, pp. 743–752 (2000)Google Scholar
  9. 9.
    Demaine, E.D., López-Ortiz, A., Munro, J.I.: Experiments on Adaptive set intersections for text retrieval systems. In: Mount, D.M., Stein, C. (eds.) ALENEX 2002. LNCS, vol. 2409, Springer, Heidelberg (2002)Google Scholar
  10. 10.
    Paul, D., Kurt, M., Rajeev, R., Christian, U.: Lower Bounds for Set Intersection Queries. In: Proceedings of the 4th Annual Symposium on Discrete Algorithms, pp. 194–201 (1993)Google Scholar
  11. 11.
    David, D., Richard, L.: On the Complexity of Computations Under Varying Sets of Primitives. Journal of Computer and Systems Sciences 18, 86–91 (1979)zbMATHCrossRefGoogle Scholar
  12. 12.
    de la Vega, W.F., Kannan, S., Santha, M.: Two probabilistic results on merging. SIAM J. on Computing 22(2), 261–271 (1993)zbMATHCrossRefGoogle Scholar
  13. 13.
    de la Vega, W.F., Frieze, A.M., Santha, M.: Average case analysis of the merging algorithm of Hwang and Lin. Algorithmica 22(4), 483–489 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Hwang, F.K., Lin, S.: A Simple algorithm for merging two disjoint linearly ordered lists. SIAM J. on Computing 1, 31–39 (1972)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Rawlins, Gregory, J.E.: Compared to What?: An Introduction the the Analysis of Algorithms. Computer Science Press/W. H. Freeman (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Ricardo Baeza-Yates
    • 1
  1. 1.Center for Web Research, Departamento de Ciencias de la ComputaciónUniversidad de ChileCasilla, SantiagoChile

Personalised recommendations