Skip to main content
SpringerLink
Log in

Adaptive sampling algorithm for detection of superpoints

  • Published:
Science in China Series F: Information Sciences Aims and scope Submit manuscript

Abstract

The superpoints are the sources (or the destinations) that connect with a great deal of destinations (or sources) during a measurement time interval, so detecting the superpoints in real time is very important to network security and management. Previous algorithms are not able to control the usage of the memory and to deliver the desired accuracy, so it is hard to detect the superpoints on a high speed link in real time. In this paper, we propose an adaptive sampling algorithm to detect the superpoints in real time, which uses a flow sample and hold module to reduce the detection of the non-superpoints and to improve the measurement accuracy of the superpoints. We also design a data stream structure to maintain the flow records, which compensates for the flow Hash collisions statistically. An adaptive process based on different sampling probabilities is used to maintain the recorded IP addresses in the limited memory. This algorithm is compared with the other algorithms by analyzing the real network trace data. Experiment results and mathematic analysis show that this algorithm has the advantages of both the limited memory requirement and high measurement accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yang F, Duan H X, Li X. Modeling and analyzing of the interaction between worms and antiworms during network worm propagation. Sci China Ser F-Inf Sci, 2005, 48(1): 91–106

    Article  Google Scholar 

  2. Moore D, Paxson V, Savage S, et al. Inside the slammer worm. Secur Privacy Mag, 2003, 1(4): 33–39

    Article  Google Scholar 

  3. Abhishek K, Xu Jun, Li Li, et al. Space code bloom filter for efficient traffic flow measurement. In: Proceedings of ACM/USENIX Internet Measurement Conference. Miami: ACM Press, October 2003: 167–172

    Google Scholar 

  4. Abhishek K, Minho S, Xu Jun, et al. Data streaming algorithms for efficient and accurate estimation of flow size distribution. New York: ACM Press, June 2004. 177–188

    Google Scholar 

  5. Estan C, Varghese G, Fisk M, et al. Bitmap algorithms for counting active flows on high speed links. In: Internet Measurement Conference. Miami: ACM Press, 2003. 153–166

    Chapter  Google Scholar 

  6. Estan C, Varghese G. New directions in traffic measurement and accounting. In Sigcomm. Pittsburgh: ACM Press, 2002. 323–336

    Google Scholar 

  7. Frederic R, Sebastia S, Josep Y. Shared state sampling. In: Internet Measurement Conference. Rio de Janeriro: ACM Press, 2006. 1–14

    Google Scholar 

  8. Bruno R, Towsley D, Ye Tao, et al. Fisher information of sampled packets: an application to flow size estimation. In: Internet Measurement Conference. Rio de Janeriro: ACM Press, 2006. 15–26

    Google Scholar 

  9. Duffield N, Lund C, Thorup M. Charging from sampled network usage. In: ACM Sigcomm Internet Measurement Workshop. San Francisco: ACM Press, 2001. 245–256

    Chapter  Google Scholar 

  10. Duffield N, Lund C, Thorup M. Estimating flow distributions from sampled flow statistics. In: Sigcomm. Karlsruhe: ACM Press, 2003. 325–336

    Google Scholar 

  11. Roesch M. Snort-lightweight intrusion detection for network. In: Proc USENIX Systems Administration Conference. Seattle: USENIX Assoc, 1999. 229–238

    Google Scholar 

  12. Plonka D. Flowscan: a network traffic flow reporting and visualization tool. In: USENIX LISA, New Orleans: USENIX Assoc, 2000. 305–317

    Google Scholar 

  13. Venkataraman S, Song D, Gibbons P, et al. New streaming algorithms for fast detection of superspreaders. In: Proc NDSS. San Diego: Internet Society, 2005

    Google Scholar 

  14. Zhao Q, Kumar A, Xu J. Joint data streaming and sampling techniques for detection of super sources and destinations. In: IMC Berkeley: ACM Press, 2005. 77–90

    Google Scholar 

  15. Kamiyama N, Mori T, Kawahara R. Simple and adaptive identification of superspreaders by flow sampling. In: Infocom. Anchorage: IEEE press, 2007. 2481–2485

    Google Scholar 

  16. Estan C, Keys K, David M, et al. Building a better netflow. In: Sigcomm. Portland: ACM Press, 2004. 245–256

    Chapter  Google Scholar 

  17. Keys K, David M, Estan C, et al. A robust system for accurate real-time summaries of internet traffic. ACM Sigmetrics. Banff: ACM Press, 2005. 85–96

    Google Scholar 

  18. Cheng G, Gong J. A resource-efficient flow monitoring system. IEEE Commun Lett, 2007, 11(6): 558–560

    Article  Google Scholar 

  19. Cohen E, Duffield N, Kaplan H. Algorithm and estimators for accurate summarization of internet traffic. In: IMC. San Diego: ACM Press, 2007. 265–278

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Lab of Jiangsu Computer Network, School of Computer Science and Engineering, Southeast University, Nanjing, 210096, China

    Guang Cheng, Jian Gong, Wei Ding, Hua Wu & ShiQiang Qiang

Authors
  1. Guang Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hua Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. ShiQiang Qiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guang Cheng.

Additional information

Supported by the National Basic Research Program of China (Grant No. 2003cb314804)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheng, G., Gong, J., Ding, W. et al. Adaptive sampling algorithm for detection of superpoints. Sci. China Ser. F-Inf. Sci. 51, 1804–1821 (2008). https://doi.org/10.1007/s11432-008-0158-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-008-0158-2

Keywords

Search

Navigation