Enhancing Network Intrusion Detection with Integrated Sampling and Filtering

  • Jose M. Gonzalez
  • Vern Paxson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4219)


The structure of many standalone network intrusion detection systems (NIDSs) centers around a chain of analysis that begins with packets captured by a packet filter, where the filter describes the protocols (TCP/UDP port numbers) and sometimes hosts or subnets to include or exclude from the analysis. In this work we argue for augmenting such analysis with an additional, separately filtered stream of packets. This “Secondary Path” supplements the “Main Path” by integrating sampling and richer forms of filtering into a NIDS’s analysis.

We discuss an implementation of a secondary path for the Bro intrusion detection system and enhancements we developed to the Berkeley Packet Filter to work in concert with the secondary path. Such an additional packet stream provides benefits in terms of both efficiency and ease of expression, which we illustrate by applying it to three forms of NIDS analysis: tracking very large individual connections, finding “heavy hitter” traffic streams, and implementing backdoor detectors (developed in previous work) with particular ease.


Hash Function Intrusion Detection Intrusion Detection System Main Path Event Handler 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Communications of the ACM 13(7), 422–426 (1970)zbMATHCrossRefGoogle Scholar
  2. 2.
    Carter, J.L., Wegman, M.N.: Universal classes of hash functions. Journal of Computer and Systems Sciences 18 (April 1979)Google Scholar
  3. 3.
    Crosby, S., Wallach, D.: Denial of service via algorithmic complexity attacks. In: Proceedings of the 12th USENIX Security Symposium, pp. 29–44 (August 2003)Google Scholar
  4. 4.
    Dreger, H., Feldmann, A., Mai, M., Paxson, V., Sommer, R.: Dynamic application-layer protocol analysis for network intrusion detection. Technical report (in submission, 2006)Google Scholar
  5. 5.
    Dreger, H., Feldmann, A., Paxson, V., Sommer, R.: Operational experiences with high-volume network intrusion detection. In: Proceedings of CCS (2004)Google Scholar
  6. 6.
    Duffield, N., Lund, C., Thorup, M.: Properties and prediction of flow statistics from sampled packet streams. In: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurement, pp. 159–171. ACM Press, New York (2002)CrossRefGoogle Scholar
  7. 7.
    Duffield, N., Lund, C., Thorup, M.: Estimating flow distributions from sampled flow statistics. In: Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 325–336. ACM Press, New York (2003)CrossRefGoogle Scholar
  8. 8.
    Estan, C., Savage, S., Varghese, G.: Automatically inferring patterns of resource consumption in network traffic. In: Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 137–148. ACM Press, New York (2003)CrossRefGoogle Scholar
  9. 9.
    Estan, C., Varghese, G.: New directions in traffic measurement and accounting. In: Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 323–336. ACM Press, New York (2002)CrossRefGoogle Scholar
  10. 10.
    Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., Berners-Lee, T.: RFC 2616: Hypertext transfer protocol – HTTP/1.1, June 1999. Status: INFORMATIONAL (1999)Google Scholar
  11. 11.
    Gonzalez, J.M.: Efficient Filtering Support for High-Speed Network Intrusion Detection. PhD thesis, University of California, Berkeley (2005)Google Scholar
  12. 12.
    Ioannidis, S., Anagnostakis, K., Ioannidis, J., Keromytis, A.: xpf: packet filtering for lowcost network monitoring. In: Proceedings of the IEEE Workshop on High-Performance Switching and Routing (HPSR), pp. 121–126 (2002)Google Scholar
  13. 13.
    Karagiannis, T., Broido, A., Faloutsos, M., Claffy, K.C.: Transport layer identification of p2p traffic. In: IMC 2004: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, pp. 121–134 (2004)Google Scholar
  14. 14.
    Kreibich, C., Warfield, A., Crowcroft, J., Hand, S., Pratt, I.: Using packet symmetry to curtail malicious traffic. In: Proceedings of the Fourth Workshop on Hot Topics in Networks (HotNets-IV), ACM SIGCOMM (to appear, 2005)Google Scholar
  15. 15.
    Lee, W., Cabrera, J.B.D., Thomas, A., Balwalli, N., Saluja, S., Zhang, Y.: Performance adaptation in real-time intrusion detection systems. In: Wespi, A., Vigna, G., Deri, L. (eds.) RAID 2002. LNCS, vol. 2516, pp. 252–273. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  16. 16.
    McCanne, S., Jacobson, V.: The BSD packet filter: A new architecture for user-level packet capture. In: USENIX Winter, pp. 259–270 (1993)Google Scholar
  17. 17.
    Park, S.K., Miller, K.W.: Random number generators: good ones are hard to find. Communications of the ACM 31(10), 1192–1201 (1988)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Paxson, V.: Bro: A system for detecting network intruders in real-time. In: Proceedings of the 7th USENIX Security Symposium (1998)Google Scholar
  19. 19.
    Ptacek, T.H., Newsham, T.N.: Insertion, evasion, and denial of service: Eluding network intrusion detection. Technical report, Secure Networks, Inc., Calgary, Alberta, Canada (1998)Google Scholar
  20. 20.
    Rivest, R.: RFC 1321: The MD5 message-digest algorithm, Status: INFORMATIONAL (April 1992)Google Scholar
  21. 21.
    Roesch, M.: Snort: Lightweight intrusion detection for networks. In: Proceedings of the 13th USENIX Conference on System Administration, pp. 229–238. USENIX Association (1999)Google Scholar
  22. 22.
    Schneier, B.: Applied Cryptography: Protocols, Algorithms, and Source Code in C. John Wiley & Sons, Inc., New York (1995)Google Scholar
  23. 23.
    Shannon, C., Moore, D., Claffy, K.C.: Beyond folklore: Observations on fragmented traffic. IEEE/ACM Transactions on Networking 10(6), 709–720 (2002)CrossRefGoogle Scholar
  24. 24.
    van der Merwe, J., Caceres, R., Chu, Y., Sreenan, C.: mmdump: a tool for monitoring internet multimedia traffic. In SIGCOMM Computer Communications Review 30, 48–59 (2000)CrossRefGoogle Scholar
  25. 25.
    Yuhara, M., Bershad, B.N., Maeda, C., Moss, J.E.B.: Efficient packet demultiplexing for multiple endpoints and large messages. In: USENIX Winter, pp. 153–165 (1994)Google Scholar
  26. 26.
    Zhang, Y., Paxson, V.: Detecting backdoors. In: Proceedings of the 9th USENIX Security Symposium, pp. 157–170 (August 2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jose M. Gonzalez
    • 1
  • Vern Paxson
    • 1
  1. 1.International Computer Science InstituteBerkeleyUSA

Personalised recommendations