Abstract
Network intrusion detection systems (NIDSs) are an important and essential defense mechanism against network attacks. However, during their detection, a large number of NIDS false alarms could be generated, which is a major challenging problem for these systems. To mitigate this issue, machine-learning based false alarm filters have been developed to refine false alarms, but it is very laborious and difficult for security experts to provide many labeled examples to train a classifier. In this paper, we therefore attempt to investigate the performance of active learning, which can make the optimal use of the given datasets, in this particular field of NIDS false alarm reduction. After analyzing the relationship between the process of false alarm reduction and the process of intrusion detection, we design a simple but efficient pool-based active learning algorithm in a false alarm filter and evaluate its performance by comparing it with several traditional supervised machine learning algorithms. The experimental results show that the designed pool-based active learner can generally achieve a better outcome than a traditional machine learning algorithm, and that the designed scheme can approximatively reduce the required number of labeled alarms by half.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alharby, A., Imai, H.: IDS False Alarm Reduction Using Continuous and Discontinuous Patterns. In: Ioannidis, J., Keromytis, A.D., Yung, M. (eds.) ACNS 2005. LNCS, vol. 3531, pp. 192–205. Springer, Heidelberg (2005)
Almgren, M., Jonsson, E.: Using Active Learning in Intrusion Detection. In: Proceedings of the 17th IEEE Computer Security Foundations Workshop (CSFW), pp. 88–98 (2004)
Axelsson, S.: The Base-rate Fallacy and the Difficulty of Intrusion Detection. ACM Transactions on Information and System Security, 186–205 (August 2000)
DARPA: KDD Cup 1999 Data, http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html
Ghosh, A.K., Wanken, J., Charron, F.: Detecting Anomalous and Unknown Intrusions Against Programs. In: Proceedings of the 1998 Annual Computer Security Applications Conference (ACSAC), pp. 259–267 (1998)
Görnitz, N., Kloft, M., Rieck, K., Brefeld, U.: Active Learning for Network Intrusion Detection. In: Proceedings of the 2nd ACM Workshop on Security and Artificial Intelligence (AISec), pp. 47–54 (2009)
Law, K.H., Kwok, L.F.: IDS False Alarm Filtering Using KNN Classifier. In: Lim, C.H., Yung, M. (eds.) WISA 2004. LNCS, vol. 3325, pp. 114–121. Springer, Heidelberg (2004)
Lee, W., Stolfo, S.J., Mok, K.W.: A Data Mining Framework for Building Intrusion Detection Models. In: Proc. of the 1999 IEEE Symposium on Security and Privacy, pp. 120–132 (1999)
Li, Y., Guo, L.: An Active Learning based TCM-KNN Algorithm for Supervised Network Intrusion Detection. Computers and Security 26(7-8), 459–467 (2007)
Lippmann, R.P., et al.: Evaluating Intrusion Detection Systems: the 1998 DARPA off-line Intrusion Detection Evaluation. In: Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX), pp. 12–26 (2000)
McCallum, A., Nigam, K.: Employing EM and Pool-Based Active Learning for Text Classification. In: Proceedings of the 15th International Conference on Machine Learning (ICML), pp. 350–358 (1998)
McHugh, J.: Testing Intrusion Detection Systems: a Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory. ACM Transactions on Information System Security, 262–294 (2000)
Meng, Y., Kwok, L.-f.: Adaptive False Alarm Filter Using Machine Learning in Intrusion Detection. In: Wang, Y., Li, T. (eds.) Practical Applications of Intelligent Systems. AISC, vol. 124, pp. 573–584. Springer, Heidelberg (2011)
Meng, Y., Li, W.: Constructing Context-based Non-Critical Alarm Filter in Intrusion Detection. In: Proceedings of the 7th International Conference on Internet Monitoring and Protection (ICIMP), pp. 75–81 (2012)
Meng, Y., Li, W., Kwok, L.-f.: Intelligent Alarm Filter Using Knowledge-based Alert Verification in Network Intrusion Detection. In: Chen, L., Felfernig, A., Liu, J., Raś, Z.W. (eds.) ISMIS 2012. LNCS, vol. 7661, pp. 115–124. Springer, Heidelberg (2012)
Pietraszek, T.: Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 102–124. Springer, Heidelberg (2004)
Roesch, M.: Snort: Lightweight Intrusion Detection for Networks. In: Proceedings of the 13th Large Installation System Administration Conference (LISA), pp. 229–238 (1999)
Scarfone, K., Mell, P.: Guide to Intrusion Detection and Prevention Systems (IDPS), pp. 800–894. NIST Special Publication (2007), http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf
Seliya, N., Khoshgoftaar, T.M.: Active Learning with Neural Networks for Intrusion Detection. In: Proceedings of the 2010 IEEE International Conference on Information Reuse and Integration (IRI), pp. 49–54 (2010)
Snort. (May 2012), http://www.snort.org/
Sommer, R., Paxson, V.: Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. In: Proceedings of the 2010 IEEE Symposium on Security and Privacy, pp. 305–316 (2010)
Stokes, J.W., Platt, J.C.: ALADIN: Active Learning of Anomalies to Detect Intrusion. Technique Report. Microsoft Network Security Redmond, WA 98052 USA (2008)
Symantec Corp., Internet Security Threat Report, vol. 16 (July 2012), http://www.symantec.com/business/threatreport/index.jsp
Valdes, A., Anderson, D.: Statistical Methods for Computer Usage Anomaly Detection Using NIDES. Technical Report, SRI International (January 1995)
Vigna, G., Kemmerer, R.A.: NetSTAT: a Network-based Intrusion Detection Approach. In: Proceedings of the 1998 Annual Computer Security Applications Conference (ACSAC), pp. 25–34. IEEE Press, New York (1998)
Wireshark, (May 2012), http://www.wireshark.org
Zhou, Z.-H., Chen, K.-J., Dai, H.-B.: Enhancing Relevance Feedback in Image Retrieval using Unlabeled Data. ACM Transactions on Information Systems 24(2), 219–244 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Meng, Y., Kwok, LF. (2013). Enhancing False Alarm Reduction Using Pool-Based Active Learning in Network Intrusion Detection. In: Deng, R.H., Feng, T. (eds) Information Security Practice and Experience. ISPEC 2013. Lecture Notes in Computer Science, vol 7863. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38033-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-38033-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38032-7
Online ISBN: 978-3-642-38033-4
eBook Packages: Computer ScienceComputer Science (R0)