Error-Adaptive and Time-Aware Maintenance of Frequency Counts over Data Streams

  • Hongyan Liu
  • Ying Lu
  • Jiawei Han
  • Jun He
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4016)


Maintaining frequency counts for items over data stream has a wide range of applications such as web advertisement fraud detection. Study of this problem has attracted great attention from both researchers and practitioners. Many algorithms have been proposed. In this paper, we propose a new method, error-adaptive pruning method, to maintain frequency more accurately. We also propose a method called fractionization to record time information together with the frequency information. Using these two methods, we design three algorithms for finding frequent items and top-k frequent items. Experimental results show these methods are effective in terms of improving the maintenance accuracy.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Manku, G.S., Motwani, R.: Approximate Frequency Counts over Data Streams. In: Proc. of 28th Intl. Conf. on Very Large Data Bases, pp. 346–357 (2002)Google Scholar
  2. 2.
    Metwally, A., Agrawal, D., El Abbadi, A.: Efficient. Computation of Frequent and Top-k Elements in Data Streams. In: Proceedings of the 10th ICDT International Conference on Database Theory, pp. 398–412 (2005)Google Scholar
  3. 3.
    Cormode, G., Muthukrishnan, S.: What’s Hot and What’s Not: Tracking Most Frequent Items Dynamically. In: Proc. of 22nd ACM Symposium on Principles of Database Systems (PODS), pp. 296–306 (2003)Google Scholar
  4. 4.
    Demaine, E., Lopez-Ortiz, A., Munro, J.: Frequency Estimation of Internet Packet Streams with Limited Space. In: Proc. of 10th Annual European Symposium on Algorithms (2002)Google Scholar
  5. 5.
    Jin, C., Qian, W., Sha, C., Yu, J., Zhou, A.: Dynamically Maintaining Frequent Items Over a Data Stream. In: Proc. of CIKM (2003)Google Scholar
  6. 6.
    Yu, J., Chong, Z., Lu, H., Zhou, A.: False Positive or False Negative: Mining Frequent Item Sets from High Speed Transactional Data Streams. In: Proc. of 30th VLDB, pp. 204–215 (2004)Google Scholar
  7. 7.
    Charikar, M., Chen, K., Farach-Colton, M.: Finding frequent items in data streams. In: Widmayer, P., Triguero, F., Morales, R., Hennessy, M., Eidenbenz, S., Conejo, R. (eds.) ICALP 2002. LNCS, vol. 2380, pp. 693–703. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Knuth, D.E.: The Art of Programming. Addison-Wesley, Reading (1973)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hongyan Liu
    • 1
  • Ying Lu
    • 2
  • Jiawei Han
    • 2
  • Jun He
    • 3
  1. 1.Tsinghua UniversityChina
  2. 2.University of Illinois, UrbanaChampaignUSA
  3. 3.Renmin University of ChinaChina

Personalised recommendations