Skip to main content

Towards Efficient Flow Sampling Technique for Anomaly Detection

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNCCN,volume 7189)

Abstract

With increasing amount of network traffic, sampling techniques have become widely employed allowing monitoring and analysis of high-speed network links. Despite of all benefits, sampling methods negatively influence the accuracy of anomaly detection techniques and other subsequent processing. In this paper, we present an adaptive, feature-aware sampling technique that reduces the loss of information bounded with the sampling process, thus minimizing the decrease of anomaly detection efficiency.

To verify the optimality of our proposed technique, we build a model of the ideal sampling algorithm and define general metrics allowing us to compute the distortion of traffic feature distribution for various types of sampling algorithms. We compare our technique with random flow sampling and reveal their impact on several anomaly detection methods by using real network traffic data. The presented ideas can be applied on high-speed network links to refine the input data by suppressing highly-redundant information.

Keywords

  • sampling
  • anomaly detection
  • NetFlow
  • intrusion detection

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ali, S., Haq, I.U., Rizvi, S., Rasheed, N., Sarfraz, U., Khayam, S.A., Mirza, F.: On mitigating sampling-induced accuracy loss in traffic anomaly detection systems. SIGCOMM Comput. Commun. Rev. 40, 4–16 (2010)

    CrossRef  Google Scholar 

  2. Androulidakis, G., Chatzigiannakis, V., Papavassiliou, S.: Network anomaly detection and classification via opportunistic sampling. Netwrk. Mag. of Global Internetwkg. 23, 6–12 (2009)

    Google Scholar 

  3. Androulidakis, G., Papavassiliou, S.: Improving network anomaly detection via selective flow-based sampling. Communications, IET 2(3), 399–409 (2008)

    CrossRef  Google Scholar 

  4. Choi, B.-Y., Zhang, Z.-L.: Adaptive random sampling for traffic volume measurement. Telecommunication Systems 34, 71–80 (2007), doi:10.1007/s11235-006-9023-z

    CrossRef  Google Scholar 

  5. Duffield, N.: Sampling for passive internet measurement: A review. Statistical Science 19, 472–498 (2004)

    CrossRef  MathSciNet  MATH  Google Scholar 

  6. Duffield, N., Lund, C., Thorup, M.: Properties and prediction of flow statistics from sampled packet streams. In: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurment, New York, NY, USA, pp. 159–171 (2002)

    Google Scholar 

  7. Duffield, N., Lund, C., Thorup, M.: Estimating flow distributions from sampled flow statistics. IEEE/ACM Trans. Netw. 13, 933–946 (2005)

    CrossRef  Google Scholar 

  8. Ertoz, L., Eilertson, E., Lazarevic, A., Tan, P.-N., Kumar, V., Srivastava, J., Dokas, P.: Minds - minnesota intrusion detection system. In: Next Generation Data Mining. MIT Press (2004)

    Google Scholar 

  9. Estan, C., Keys, K., Moore, D., Varghese, G.: Building a better netflow. SIGCOMM Comput. Commun. Rev. 34, 245–256 (2004)

    CrossRef  Google Scholar 

  10. Estan, C., Varghese, G.: New directions in traffic measurement and accounting. In: Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2002, pp. 323–336. ACM, New York (2002)

    CrossRef  Google Scholar 

  11. Hohn, N., Veitch, D.: Inverting sampled traffic. IEEE/ACM Transactions on Networking 14(1), 68–80 (2006)

    CrossRef  Google Scholar 

  12. Lakhina, A., Crovella, M., Diot, C.: Diagnosis Network-Wide Traffic Anomalies. In: ACM SIGCOMM 2004, pp. 219–230. ACM Press, New York (2004)

    Google Scholar 

  13. Lakhina, A., Crovella, M., Diot, C.: Mining Anomalies using Traffic Feature Distributions. In: ACM SIGCOMM, Philadelphia, PA, pp. 217–228. ACM Press, New York (2005)

    Google Scholar 

  14. Mai, J., Chuah, C.-N., Sridharan, A., Ye, T., Zang, H.: Is sampled data sufficient for anomaly detection? In: Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement, IMC 2006, pp. 165–176. ACM, New York (2006)

    Google Scholar 

  15. Rehak, M., Pechoucek, M., Grill, M., Stiborek, J., Bartos, K., Celeda, P.: Adaptive multiagent system for network traffic monitoring. IEEE Intelligent Systems 24(3), 16–25 (2009)

    CrossRef  Google Scholar 

  16. Sridharan, A., Ye, T., Bhattacharyya, S.: Connectionless port scan detection on the backbone, Phoenix, AZ, USA (2006)

    Google Scholar 

  17. Xu, K., Zhang, Z.-L., Bhattacharrya, S.: Reducing Unwanted Traffic in a Backbone Network. In: USENIX Workshop on Steps to Reduce Unwanted Traffic in the Internet (SRUTI), Boston, MA (July 2005)

    Google Scholar 

  18. Yang, L., Michailidis, G.: Sampled based estimation of network traffic flow characteristics. In: 26th IEEE International Conference on Computer Communications, INFOCOM 2007, pp. 1775–1783. IEEE (May 2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 IFIP International Federation for Information Processing

About this paper

Cite this paper

Bartos, K., Rehak, M. (2012). Towards Efficient Flow Sampling Technique for Anomaly Detection. In: Pescapè, A., Salgarelli, L., Dimitropoulos, X. (eds) Traffic Monitoring and Analysis. TMA 2012. Lecture Notes in Computer Science, vol 7189. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28534-9_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28534-9_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28533-2

  • Online ISBN: 978-3-642-28534-9

  • eBook Packages: Computer ScienceComputer Science (R0)