A Framework for the Application of Association Rule Mining in Large Intrusion Detection Infrastructures

  • James J. Treinen
  • Ramakrishna Thurimella
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4219)


The high number of false positive alarms that are generated in large intrusion detection infrastructures makes it difficult for operations staff to separate false alerts from real attacks. One means of reducing this problem is the use of meta alarms, or rules, which identify known attack patterns in alarm streams. The obvious risk with this approach is that the rule base may not be complete with respect to every true attack profile, especially those which are new. Currently, new rules are discovered manually, a process which is both costly and error prone. We present a novel approach using association rule mining to shorten the time that elapses from the appearance of a new attack profile in the data to its definition as a rule in the production monitoring infrastructure.


Association Rules Data Mining Intrusion Detection Graph Algorithms 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules Between Sets of Items in Large Databases. In: Proceedings of the ACM SIGMOD Conference on Management of Data, pp. 207–216 (1993)Google Scholar
  2. 2.
    Ali, K., Manganaris, S., Srikant, R.: Partial Classification Using Association Rules. In: Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, pp. 115–118 (1997)Google Scholar
  3. 3.
    Apap, F., Honig, A., Hershkop, S., Eskin, E., Stolfo, S.J.: Detecting malicious software by monitoring anomalous windows registry accesses. In: Wespi, A., Vigna, G., Deri, L. (eds.) RAID 2002. LNCS, vol. 2516, pp. 36–53. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Arcsight Corporation. Arcsight ESM Product Brief (2005),
  5. 5.
    Arcsight Corporation. Arcsight Pattern Discovery Product Brief (2005),
  6. 6.
    Barbara, D., Couto, J., Jajodia, S., Wu, N.: ADAM: A Testbed for Exploring the Use of Data Mining in Intrusion Detection. SIGMOD Record 30(4), 15–24 (2001)CrossRefGoogle Scholar
  7. 7.
    Cisco Systems. Network Security Database (2005),
  8. 8.
    Debar, H., Wespi, A.: Aggregation and correlation of intrusion-detection alerts. In: Lee, W., Mé, L., Wespi, A. (eds.) RAID 2001. LNCS, vol. 2212, pp. 85–103. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  9. 9.
    Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: The KDD Process for Extracting Useful Knowledge From Volumes of Data. Communications of the ACM, 27–34 (1996)Google Scholar
  10. 10.
    Guan, Y., Ghorbani, A., Belacel, N.: Y-Means: A Clustering Method for Intrusion Detection. In: Proceedings of Canadian Conference on Electrical and Computer Engineering (2003)Google Scholar
  11. 11.
    Han, J., Cai, Y., Cercone, N.: Knowledge Discovery in Databases: An Attribute-Oriented Approach. In: Proceedings of the 18th International Conference on Very Large Data Bases, pp. 547–559 (1992)Google Scholar
  12. 12.
    Han, J., Cai, Y., Cercone, N.: Data-Driven Discovery of Quantitative Rules in Relational Databases. IEEE Transactions on Knowledge and Data Engineering 5, 29–40 (1993)CrossRefGoogle Scholar
  13. 13.
    Honig, A., Howard, A., Eskin, E., Stolfo, S.: Adaptive Model Generation: An Architecture for the Deployment of Data Mining-based Intrusion Detection Systems. In: Barbara, D., Sushil, J. (eds.) Applications of Data Mining in Computer Security, pp. 153–194. Kluwer Academic Publishers, Boston (2002)Google Scholar
  14. 14.
    Hosel, V., Walcher, S.: Clustering Techniques: A Brief Survey (2000),
  15. 15.
    IBM Corporation: DB2 Intelligent Miner for Modeling, New York (2005)Google Scholar
  16. 16.
    IBM Corporation: IBM DB2 Intelligent Miner Modeling Administration and Programming Guide v8.2. Second Edition. New York (2004)Google Scholar
  17. 17.
    Julisch, K.: Mining Alarm Clusters to Improve Alarm Handling Efficiency. In: Proceedings of the 17th Annual Computer Security Applications Conference, pp. 12–21 (2001)Google Scholar
  18. 18.
    Julisch, K.: Data Mining for Intrusion Detection A Critical Review. In: Barbara, D., Sushil, J. (eds.) Applications of Data Mining in Computer Security, pp. 33–62. Kluwer Academic Publishers, Boston (2002)Google Scholar
  19. 19.
    Julisch, K., Dacier, M.: Mining Intrusion Detection Alarms for Actionable Knowledge. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 366–375 (2002)Google Scholar
  20. 20.
    Julisch, K.: Clustering Intrusion Detection Alarms to Support Root Cause Analysis. ACM Transactions on Information and System Security 6(4), 443–471 (2003)CrossRefGoogle Scholar
  21. 21.
    Julisch, K.: Using Root Cause Analysis to Handle Intrusion Detection Alarms. PhD Thesis. Universität Dortmund (2003)Google Scholar
  22. 22.
    Lee, W., Stolfo, S.: Data Mining Approaches for Intrusion Detection. In: Proceedings of the 7th USENIX Security Symposium, pp. 79–94 (1998)Google Scholar
  23. 23.
    Lee, W., Stolfo, W., Mok, K.: Mining Audit Data to Build Intrusion Detection Models. In: Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 66–72 (1998)Google Scholar
  24. 24.
    Lee, W., Stolfo, S., Kui, M.: A Data Mining Framework for Building Intrusion Detection Models. In: IEEE Symposium on Security and Privacy, pp. 120–132 (1999)Google Scholar
  25. 25.
    Lee, W., Stolfo, S., Chan, P., Eskin, E., Fan, W., Miller, M., Hershkop, S., Zhang, J.: Real Time Data Mining-based Intrusion Detection. In: Proceedings of the 2nd DARPA Information Survivability Conference and Exposition (2001)Google Scholar
  26. 26.
    Lippmann, R., Haines, J., Fried, D., Korba, J., Das, K.: The 1999 DARPA Off-Line Intrusion Detection Evaluation. Computer Networks 34, 579–595 (2000)CrossRefGoogle Scholar
  27. 27.
    Manganaris, S., Christensen, M., Zerkle, D., Hermiz, K.: A Data Mining Analysis of RTID Alarms. In: Proceedings of Recent Advances in Intrusion Detection, Second International Workshop (1999)Google Scholar
  28. 28.
    Mchugh, J.: Testing Intrusion Detection Systems: A Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory. ACM Transactions on Information and System Security 3(4), 262–294 (2000)CrossRefGoogle Scholar
  29. 29.
    McLure, S., Scambray, J., Kurtz, G.: Hacking Exposed Fifth Edition: Network Security Secrets & Solutions: McGraw-Hill/Osborne (2005)Google Scholar
  30. 30.
    Nauta, K., Lieble, F.: Offline Network Intrusion Detection: Mining TCPDUMP Data to Identify Suspicious Activity. In: Proceedings of the AFCEA Federal Database Colloquium (1999)Google Scholar
  31. 31.
    Ning, P., Cui, Y., Reeves, D., Xu, D.: Techniques and Tools for Analyzing Intrusion Alerts. ACM Transaction on Information and System Security 7(2), 274–318 (2004)CrossRefGoogle Scholar
  32. 32.
    Noel, S., Wijesekera, D., Youman, C.: Modern Intrusion Detection, Data Mining, and Degrees of Attack Guilt. In: Barbara, D., Sushil, J. (eds.) Applications of Data Mining in Computer Security, pp. 1–31. Kluwer Academic Publishers, Boston (2002)Google Scholar
  33. 33.
    Portnoy, L., Eskin, E., Stolfo, S.: Intrusion Detection with Unlabeled Data Using Clustering. In: Proceedings of ACM CSS Workshop on Data Mining Applied to Security (2001)Google Scholar
  34. 34.
    Schultz, M., Eskin, E., Zadok, E., Stolfo, S.: Data Mining Methods for Detection of New Malicious Executables. In: Proceedings of IEEE Symposium on Security and Privacy (2001)Google Scholar
  35. 35.
    Stolfo, S., Lee, W., Chan, P., Fan, W., Eskin, E.: Data Mining-based Intrusion Detectors: An Overview of the Columbia IDS Project. SIGMOD Record 30(4), 5–14 (2001)CrossRefGoogle Scholar
  36. 36.
    Valdes, A., Skinner, K.: Probabilistic alert correlation. In: Lee, W., Mé, L., Wespi, A. (eds.) RAID 2001. LNCS, vol. 2212, pp. 54–68. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  37. 37.
    Yang, D., Hu, C., Chen, Y.: A Framework of Cooperating Intrusion Detection Based on Clustering Analysis and Expert System. In: Proceedings of the 3rd international conference on Information Security (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • James J. Treinen
    • 1
  • Ramakrishna Thurimella
    • 2
  1. 1.IBM Global ServicesBoulderUSA
  2. 2.University of DenverDenverUSA

Personalised recommendations