Skip to main content

Counting Distinct Elements in a Data Stream

Part of the Lecture Notes in Computer Science book series (LNCS,volume 2483)

Abstract

We present three algorithms to count the number of distinct elements in a data stream to within a factor of 1 ±ε. Our algorithms improve upon known algorithms for this problem, and offer a spectrum of time/space tradeoffs.

Keywords

  • Data Stream
  • Hash Function
  • Error Probability
  • Distinct Element
  • Computer Science Division

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Part of this work was done while the author was visiting IBM Almaden Research Center. Supported by NSF Grant CCR-9820897.

Work supported by a Sloan Research Fellowship and an NSF Career Award.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/3-540-45726-7_1
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   74.99
Price excludes VAT (USA)
  • ISBN: 978-3-540-45726-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Alon, Y. Matias, and M. Szegedy. The space complexity of approximating the frequency moments. Journal of Computer and System Sciences, 58(1):137–147, 1999.

    MATH  CrossRef  MathSciNet  Google Scholar 

  2. Z. Bar-Yossef, R. Kumar, and D. Sivakumar. Reductions in streaming algorithms, with an application to counting triangles in graphs. In Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 623–632, 2002.

    Google Scholar 

  3. E. Cohen. Size-estimation framework with applications to transitive closure and reachability. Journal of Computer and System Sciences, 55(3):441–453, 1997.

    MATH  CrossRef  MathSciNet  Google Scholar 

  4. L. Carter and M. Wegman. Universal classes of hash functions. In Proceedings of the 9th ACM Annual Symposium on Theory of Computing, pages 106–112, 1977. Journal version in Journal of Computer and System Sciences, 18(2) 143-154, 1979.

    Google Scholar 

  5. P. Flajolet and G. N. Martin. Probabilistic counting algorithms for data base applications. Journal of Computer and System Sciences, 31:182–209, 1985.

    MATH  CrossRef  MathSciNet  Google Scholar 

  6. P. Gibbons and S. Tirthapura. Estimating simple functions on the union of data streams. In Proceedings of the 13th ACM Symposium on Parallel Algorithms and Architectures, pages 281–291, 2001.

    Google Scholar 

  7. P.J. Haas, J.F. Naughton, S. Seshadri, and A.N. Swami. Selectivity and cost estimation for joins based on random sampling. Journal of Computer and System Sciences, 52(3), 1996.

    Google Scholar 

  8. M. Wegman and L. Carter. New classes and applications of hash functions. In Proceedings of the 20th IEEE Annual Symposium on Foundations of Computer Science, pages 175–182, 1979. Journal version titled “New Hash Functions and Their Use in Authentication and Set Equality” in Journal of Computer and System Sciences, 22(3): 265-279, 1981.

    Google Scholar 

  9. K.-Y. Whang, B. T. Vander-Zanden, and H. M. Taylor. A linear-time probabilistic counting algorithm for database applications. ACM Transactions on Database Systems, 15(2):208–229, 1990.

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bar-Yossef, Z., Jayram, T.S., Kumar, R., Sivakumar, D., Trevisan, L. (2002). Counting Distinct Elements in a Data Stream. In: Rolim, J.D.P., Vadhan, S. (eds) Randomization and Approximation Techniques in Computer Science. RANDOM 2002. Lecture Notes in Computer Science, vol 2483. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45726-7_1

Download citation

  • DOI: https://doi.org/10.1007/3-540-45726-7_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-44147-2

  • Online ISBN: 978-3-540-45726-8

  • eBook Packages: Springer Book Archive