Effectiveness Evaluation of Data Mining Based IDS

  • Agustín Orfila
  • Javier Carbó
  • Arturo Ribagorda
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4065)


Data mining has been widely applied to the problem of Intrusion Detection in computer networks. However, the misconception of the underlying problem has led to out of context results. This paper shows that factors such as the probability of intrusion and the costs of responding to detected intrusions must be taken into account in order to compare the effectiveness of machine learning algorithms over the intrusion detection domain. Furthermore, we show the advantages of combining different detection techniques. Results regarding the well known 1999 KDD dataset are shown.


Receiver Operating Characteristic False Alarm Rate Intrusion Detection Machine Learning Algorithm Intrusion Detection System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ISO: Information technology - security techniques - it intrusion detection frameworks. Technical report (2002) ISO/IEC TR 15947Google Scholar
  2. 2.
    Maloof, M.A.: Machine Learning and Data Mining for Computer Security: Methods and Applications (Advanced Information and Knowledge Processing). Springer, New York (2005)Google Scholar
  3. 3.
    Sy, B.K.: Signature-based approach for intrusion detection. In: [27], pp. 526–536Google Scholar
  4. 4.
    Giacinto, G., Perdisci, R., Roli, F.: Alarm clustering for intrusion detection systems in computer networks. In: [27], pp. 184–193Google Scholar
  5. 5.
    Lippmann, R., Fried, D., Graf, I., Haines, J., Kendall, K., McClung, D., Weber, D., Webster, S., Wyschogrod, D., Cunningham, R., Zissman, M.: Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation. In: Proceedings of the DARPA Information Survivability Conference and Exposition, Los Alamitos, California, USA. IEEE Computer Society Press, Los Alamitos (2000)Google Scholar
  6. 6.
    Lippmann, R., Haines, J.W., Fried, D.J., Korba, J., Das, K.: The 1999 darpa off-line intrusion detection evaluation. Computer Networks 34(4), 579–595 (2000)CrossRefGoogle Scholar
  7. 7.
    McHugh, J.: Testing intrusion detection systems: a critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory. ACM Transactions on Information and System Security 3(4), 262–294 (2000)CrossRefGoogle Scholar
  8. 8.
    Mahoney, M.V., Chan, P.K.: An analysis of the 1999 darpa/Lincoln laboratory evaluation data for network anomaly detection. In: Vigna, G., Krügel, C., Jonsson, E. (eds.) RAID 2003. LNCS, vol. 2820, pp. 220–237. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Mell, P., Hu, V., Lippman, R., Haines, J., Zissman, M.: An overview of issues in testing intrusion detection, National Institute of Standards and Technologies. Internal report 7007 (2003)Google Scholar
  10. 10.
    Elkan, C.: Results of the KDD 1999 classifier learning contest (1999)Google Scholar
  11. 11.
    Drummond, C., Holte, R.C.: Explicitly representing expected cost: an alternative to roc representation. In: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2000, pp. 198–207. ACM Press, New York (2000)CrossRefGoogle Scholar
  12. 12.
    Gaffney, J.E., Ulvila, J.W.: Evaluation of intrusion detectors: A decision theory approach. In: Proceedings of the IEEE Symposium on Security and Privacy, SP 2001, Washington, DC, USA, pp. 50–61. IEEE Computer Society, Los Alamitos (2001)CrossRefGoogle Scholar
  13. 13.
    Ulvila, J.W., Gaffney, J.E.: Evaluation of intrusion detection systems. Journal of Research of the National Institute of Standards and Technology 108(6), 453–473 (2003)Google Scholar
  14. 14.
    Ulvila, J.W., Gaffney, J.E.: A decision analysis method for evaluating computer intrusion detection systems. Decision Analysis 1(1), 35–50 (2004)CrossRefGoogle Scholar
  15. 15.
    Orfila, A., Carbó, J., Ribagorda, A.: Intrusion detection effectiveness improvement by a multi-agent system. International Journal of Computer Science & Applications 2(1), 1–6 (2005)Google Scholar
  16. 16.
    Swets, J.A., Dawes, R., Monahan, J.: Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest 1(1), 1–26 (2000)CrossRefGoogle Scholar
  17. 17.
    Sen, A.: Choice functions and revealed preferences. Review of Economic Studies 38, 307–317 (1971)zbMATHCrossRefGoogle Scholar
  18. 18.
    Katz, R.W., Murphy, A.H.: Economic Value of Weather and Climate Forecasts. Cambridge University Press, UK (1997)CrossRefGoogle Scholar
  19. 19.
    Orfila, A., Carbó, J., Ribagorda, A.: Fuzzy logic on decision model for ids. In: Proceedings of the Twelveth IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2003, St. Louis, Missouri, USA, vol. 2, pp. 1237–1242 (2003)Google Scholar
  20. 20.
    Axelsson, S.: The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security, TISSEC 3(3), 186–205 (2000)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Athanasiades, N., Abler, R., Levine, J.G., Owen, H.L., Riley, G.F.: Intrusion detection testing and benchmarking methodologies. In: Proceedings of the International Information Assurance Workshop, IWIA 2003, Maryland, USA, pp. 63–72 (2003)Google Scholar
  22. 22.
    Giacinto, G., Roli, F., Didaci, L.: Fusion of multiple classifiers for intrusion detection in computer networks. Pattern Recognition Letters 24(12), 1795–1803 (2003)CrossRefGoogle Scholar
  23. 23.
    Sabhnani, M., Serpen, G.: Kdd feature set complaint heuristic rules for r2l attack detection. In: Security and Management, pp. 310–316 (2003)Google Scholar
  24. 24.
    Laskov, P., Düssel, P., Schäfer, C., Rieck, K.: Learning intrusion detection: Supervised or unsupervised? In: Roli, F., Vitulano, S. (eds.) ICIAP 2005. LNCS, vol. 3617, pp. 50–57. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  25. 25.
    Lee, W., Stolfo, S.J.: A framework for constructing features and models for intrusion detection systems. ACM Transactions on Information and System Security 3(4), 227–261 (2000)CrossRefGoogle Scholar
  26. 26.
    Witten, I.H., Frank, E.: Data mining: practical machine learning tools and techniques. Morgan Kaufmann Publishers Inc., San Francisco (2005)zbMATHGoogle Scholar
  27. 27.
    Perner, P., Imiya, A. (eds.): MLDM 2005. LNCS, vol. 3587. Springer, Heidelberg (2005)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Agustín Orfila
    • 1
  • Javier Carbó
    • 1
  • Arturo Ribagorda
    • 1
  1. 1.Computer Science DepartmentCarlos III University of MadridLeganésSpain

Personalised recommendations