Counting Distinct Elements in a Data Stream

  • Ziv Bar-Yossef
  • T. S. Jayram
  • Ravi Kumar
  • D. Sivakumar
  • Luca Trevisan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2483)

Abstract

We present three algorithms to count the number of distinct elements in a data stream to within a factor of 1 ±ε. Our algorithms improve upon known algorithms for this problem, and offer a spectrum of time/space tradeoffs.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [AMS99]
    N. Alon, Y. Matias, and M. Szegedy. The space complexity of approximating the frequency moments. Journal of Computer and System Sciences, 58(1):137–147, 1999.MATHCrossRefMathSciNetGoogle Scholar
  2. [BKS02]
    Z. Bar-Yossef, R. Kumar, and D. Sivakumar. Reductions in streaming algorithms, with an application to counting triangles in graphs. In Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 623–632, 2002.Google Scholar
  3. [Coh97]
    E. Cohen. Size-estimation framework with applications to transitive closure and reachability. Journal of Computer and System Sciences, 55(3):441–453, 1997.MATHCrossRefMathSciNetGoogle Scholar
  4. [CW77]
    L. Carter and M. Wegman. Universal classes of hash functions. In Proceedings of the 9th ACM Annual Symposium on Theory of Computing, pages 106–112, 1977. Journal version in Journal of Computer and System Sciences, 18(2) 143-154, 1979.Google Scholar
  5. [FM85]
    P. Flajolet and G. N. Martin. Probabilistic counting algorithms for data base applications. Journal of Computer and System Sciences, 31:182–209, 1985.MATHCrossRefMathSciNetGoogle Scholar
  6. [GT01]
    P. Gibbons and S. Tirthapura. Estimating simple functions on the union of data streams. In Proceedings of the 13th ACM Symposium on Parallel Algorithms and Architectures, pages 281–291, 2001.Google Scholar
  7. [HNSS96]
    P.J. Haas, J.F. Naughton, S. Seshadri, and A.N. Swami. Selectivity and cost estimation for joins based on random sampling. Journal of Computer and System Sciences, 52(3), 1996.Google Scholar
  8. [WC79]
    M. Wegman and L. Carter. New classes and applications of hash functions. In Proceedings of the 20th IEEE Annual Symposium on Foundations of Computer Science, pages 175–182, 1979. Journal version titled “New Hash Functions and Their Use in Authentication and Set Equality” in Journal of Computer and System Sciences, 22(3): 265-279, 1981.Google Scholar
  9. [WVT90]
    K.-Y. Whang, B. T. Vander-Zanden, and H. M. Taylor. A linear-time probabilistic counting algorithm for database applications. ACM Transactions on Database Systems, 15(2):208–229, 1990.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Ziv Bar-Yossef
    • 1
  • T. S. Jayram
    • 2
  • Ravi Kumar
    • 2
  • D. Sivakumar
    • 2
  • Luca Trevisan
    • 3
  1. 1.Computer Science DivisionUniv. of California at BerkeleyBerkeley
  2. 2.IBM Almaden Research CenterSan Jose
  3. 3.Computer Science DivisionUniv. of California at BerkeleyBerkeley

Personalised recommendations