Towards Efficient Flow Sampling Technique for Anomaly Detection

  • Karel Bartos
  • Martin Rehak
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7189)

Abstract

With increasing amount of network traffic, sampling techniques have become widely employed allowing monitoring and analysis of high-speed network links. Despite of all benefits, sampling methods negatively influence the accuracy of anomaly detection techniques and other subsequent processing. In this paper, we present an adaptive, feature-aware sampling technique that reduces the loss of information bounded with the sampling process, thus minimizing the decrease of anomaly detection efficiency.

To verify the optimality of our proposed technique, we build a model of the ideal sampling algorithm and define general metrics allowing us to compute the distortion of traffic feature distribution for various types of sampling algorithms. We compare our technique with random flow sampling and reveal their impact on several anomaly detection methods by using real network traffic data. The presented ideas can be applied on high-speed network links to refine the input data by suppressing highly-redundant information.

Keywords

sampling anomaly detection NetFlow intrusion detection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ali, S., Haq, I.U., Rizvi, S., Rasheed, N., Sarfraz, U., Khayam, S.A., Mirza, F.: On mitigating sampling-induced accuracy loss in traffic anomaly detection systems. SIGCOMM Comput. Commun. Rev. 40, 4–16 (2010)CrossRefGoogle Scholar
  2. 2.
    Androulidakis, G., Chatzigiannakis, V., Papavassiliou, S.: Network anomaly detection and classification via opportunistic sampling. Netwrk. Mag. of Global Internetwkg. 23, 6–12 (2009)Google Scholar
  3. 3.
    Androulidakis, G., Papavassiliou, S.: Improving network anomaly detection via selective flow-based sampling. Communications, IET 2(3), 399–409 (2008)CrossRefGoogle Scholar
  4. 4.
    Choi, B.-Y., Zhang, Z.-L.: Adaptive random sampling for traffic volume measurement. Telecommunication Systems 34, 71–80 (2007), doi:10.1007/s11235-006-9023-zCrossRefGoogle Scholar
  5. 5.
    Duffield, N.: Sampling for passive internet measurement: A review. Statistical Science 19, 472–498 (2004)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Duffield, N., Lund, C., Thorup, M.: Properties and prediction of flow statistics from sampled packet streams. In: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurment, New York, NY, USA, pp. 159–171 (2002)Google Scholar
  7. 7.
    Duffield, N., Lund, C., Thorup, M.: Estimating flow distributions from sampled flow statistics. IEEE/ACM Trans. Netw. 13, 933–946 (2005)CrossRefGoogle Scholar
  8. 8.
    Ertoz, L., Eilertson, E., Lazarevic, A., Tan, P.-N., Kumar, V., Srivastava, J., Dokas, P.: Minds - minnesota intrusion detection system. In: Next Generation Data Mining. MIT Press (2004)Google Scholar
  9. 9.
    Estan, C., Keys, K., Moore, D., Varghese, G.: Building a better netflow. SIGCOMM Comput. Commun. Rev. 34, 245–256 (2004)CrossRefGoogle Scholar
  10. 10.
    Estan, C., Varghese, G.: New directions in traffic measurement and accounting. In: Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2002, pp. 323–336. ACM, New York (2002)CrossRefGoogle Scholar
  11. 11.
    Hohn, N., Veitch, D.: Inverting sampled traffic. IEEE/ACM Transactions on Networking 14(1), 68–80 (2006)CrossRefGoogle Scholar
  12. 12.
    Lakhina, A., Crovella, M., Diot, C.: Diagnosis Network-Wide Traffic Anomalies. In: ACM SIGCOMM 2004, pp. 219–230. ACM Press, New York (2004)Google Scholar
  13. 13.
    Lakhina, A., Crovella, M., Diot, C.: Mining Anomalies using Traffic Feature Distributions. In: ACM SIGCOMM, Philadelphia, PA, pp. 217–228. ACM Press, New York (2005)Google Scholar
  14. 14.
    Mai, J., Chuah, C.-N., Sridharan, A., Ye, T., Zang, H.: Is sampled data sufficient for anomaly detection? In: Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement, IMC 2006, pp. 165–176. ACM, New York (2006)Google Scholar
  15. 15.
    Rehak, M., Pechoucek, M., Grill, M., Stiborek, J., Bartos, K., Celeda, P.: Adaptive multiagent system for network traffic monitoring. IEEE Intelligent Systems 24(3), 16–25 (2009)CrossRefGoogle Scholar
  16. 16.
    Sridharan, A., Ye, T., Bhattacharyya, S.: Connectionless port scan detection on the backbone, Phoenix, AZ, USA (2006)Google Scholar
  17. 17.
    Xu, K., Zhang, Z.-L., Bhattacharrya, S.: Reducing Unwanted Traffic in a Backbone Network. In: USENIX Workshop on Steps to Reduce Unwanted Traffic in the Internet (SRUTI), Boston, MA (July 2005)Google Scholar
  18. 18.
    Yang, L., Michailidis, G.: Sampled based estimation of network traffic flow characteristics. In: 26th IEEE International Conference on Computer Communications, INFOCOM 2007, pp. 1775–1783. IEEE (May 2007)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Karel Bartos
    • 1
  • Martin Rehak
    • 1
  1. 1.Faculty of Electrical EngineeringCzech Technical UniversityPragueCzech Republic

Personalised recommendations