The NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware

  • Matthias Vallentin
  • Robin Sommer
  • Jason Lee
  • Craig Leres
  • Vern Paxson
  • Brian Tierney
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4637)


In this work we present a NIDS cluster as a scalable solution for realizing high-performance, stateful network intrusion detection on commodity hardware. The design addresses three challenges: (i)  distributing traffic evenly across an extensible set of analysis nodes in a fashion that minimizes the communication required for coordination, (ii)  adapting the NIDS’s operation to support coordinating its low-level analysis rather than just aggregating alerts; and (iii)  validating that the cluster produces sound results. Prototypes of our NIDS cluster now operate at the Lawrence Berkeley National Laboratory and the University of California at Berkeley. In both environments the clusters greatly enhance the power of the network security monitoring.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blanc, M., Oudot, L., Glaume, V.: Global Intrusion Detection: Prelude Hybrid IDS. Technical report (2003)Google Scholar
  2. 2.
    Dreger, H.: Operational Network Intrusion Detection: Resource-Analysis Tradeoffs. PhD thesis, TU München (2007)Google Scholar
  3. 3.
    Dreger, H., Feldmann, A., Mai, M., Paxson, V., Sommer, R.: Dynamic Application-Layer Protocol Analysis for Network Intrusion Detection. In: Proc. USENIX Security Symposium (2006)Google Scholar
  4. 4.
    Dreger, H., Feldmann, A., Paxson, V., Sommer, R.: Operational Experiences with High-Volume Network Intrusion Detection. In: Proc. ACM Conference on Computer and Communications Security, ACM Press, New York (2004)Google Scholar
  5. 5.
    Fox, A., Gribble, S.D., Chawathe, Y., Brewer, E.A., Gauthier, P.: Cluster-Based Scalable Network Services. In: Proc. Symposium on Operating Systems Principles (1997)Google Scholar
  6. 6.
    Intrusion Detection Message Exchange Format,
  7. 7.
    Kohler, E., Morris, R., Chen, B., Jannotti, J., Kaashoek, F.: The Click Modular Router. ACM Transactions on Computer Systems 18(3) (August 2000)Google Scholar
  8. 8.
    Kruegel, C., Valeur, F., Vigna, G., Kemmerer, R.A.: Stateful Intrusion Detection for High-Speed Networks. In: Proc. IEEE Symposium on Research on Security and Privacy, IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
  9. 9.
    Paxson, V.: Bro: A System for Detecting Network Intruders in Real-Time. Computer Networks 31(23–24), 2435–2463 (1999)CrossRefGoogle Scholar
  10. 10.
    Paxson, V., Asanovic, K., Dharmapurikar, S., Lockwood, J., Pang, R., Sommer, R., Weaver, N.: Rethinking Hardware Support for Network Analysis and Intrusion Prevention. In: Proc. USENIX Hot Security (2006)Google Scholar
  11. 11.
    Porras, P.A., Neumann, P.G.: EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances. In: Proc. National Information Systems Security Conference (1997)Google Scholar
  12. 12.
    Puketza, N.J., Zhang, K., Chung, M., Mukherjee, B., Olsson, R.A.: A Methodology for Testing Intrusion Detection Systems. IEEE Transactions on Software Engineering 22(10), 719–729 (1996)CrossRefGoogle Scholar
  13. 13.
    Roesch, M.: Snort: Lightweight Intrusion Detection for Networks. In: Proc. Systems Administration Conference (1999)Google Scholar
  14. 14.
    Schaelicke, L., Freeland, C.: Characterizing Sources and Remedies for Packet Loss in Network Intrusion Detection. In: Proc. IEEE Symposium on Workload Characterization, IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  15. 15.
    Schaelicke, L., Slabach, T., Moore, B., Freeland, C.: Characterizing the Performance of Network Intrusion Detection Sensors. In: Proc. Symposium on Recent Advances in Intrusion Detection (2003)Google Scholar
  16. 16.
    Schaelicke, L., Wheeler, K., Freeland, C.: SPANIDS: A Scalable Network Intrusion Detection Loadbalancer. In: Proc. Computing Frontiers Conference (2005)Google Scholar
  17. 17.
    Sommer, R., Paxson, V.: Exploiting Independent State For Network Intrusion Detection. In: Proc. Computer Security Applications Conference (2005)Google Scholar
  18. 18.
    Vallentin, M.: Transparent Load-Balancing for Network Intrusion Detection Systems. Bachelor’s Thesis, TU München (2006)Google Scholar
  19. 19.
    Vigna, G., Eckmann, S.T., Kemmerer, R.A.: The STAT Tool Suite. In: Proc. DARPA Information Survivability Conference and Exposition (2000)Google Scholar
  20. 20.
    Vigna, G., Kemmerer, R.A.: NetSTAT: A Network-based Intrusion Detection System. Journal of Computer Security 7(1), 37–71 (1999)Google Scholar
  21. 21.
    Vigna, G., Kemmerer, R.A., Blix, P.: Designing a Web of Highly-Configurable Intrusion Detection Sensors. In: Proc. Symposium on Recent Advances in Intrusion Detection (2001)Google Scholar
  22. 22.
    Weaver, N., Paxson, V., Gonzalez, J.M.: The Shunt: An FPGA-Based Accelerator for Network Intrusion Prevention. In: Proc. ACM Symposium on Field Programmable Gate Arrays, February 2007, ACM Press, New York (2007)Google Scholar
  23. 23.
    Zhang, Y., Paxson, V.: Detecting Stepping Stones. In: Proc. USENIX Security Symposium (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Matthias Vallentin
    • 3
  • Robin Sommer
    • 2
    • 1
  • Jason Lee
    • 2
  • Craig Leres
    • 2
  • Vern Paxson
    • 1
    • 2
  • Brian Tierney
    • 2
  1. 1.International Computer Science Institute 
  2. 2.Lawrence Berkeley National Laboratory 
  3. 3.TU München 

Personalised recommendations