Abstract
Cyber security issues are always worthy of attention. Intrusion detection system (IDS) is one of approaches used to protect computer system and identify potential attack. However, existing methods are limited by high-dimensional computational complexity in rare or unknown attack detection task. To improve the ability of detecting anomaly intrusions, a hybrid intrusion detection framework is proposed in this paper. The proposed RuleRCD algorithm first uses fuzzy association rules based on Apriori and K-Means algorithm for normal pattern and major attack detection. The other data are then fed to an active learning-based rare category detection algorithm to identify its attack pattern. This paper also introduces an interactive visualization system, which integrates experts’ decision into intrusion detection workflow. The method improves the effectiveness and interpretability of detection process. KDD-99 dataset is used to evaluate the proposed framework. The result shows that the approach outperforms some methods, especially in terms of the rare attack.
Graphical abstract
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aggus LR, Morais DI, Baker RF (2000) Evaluating intrution detection systems: the 1998 darpa offline intrusion detection evaluation. Computer Netw 34(4):579–595
Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: Proceedings of the 20th International Conference on Very Large Data Bases
Brahmi H, Brahmi I, Yahia SB (2012) OMC-IDS: at the cross-roads of OLAP mining and intrusion detection. In: Tan P, Chawla S, Ho CK, Bailey J (eds) Advances in Knowledge Discovery and Data Mining - 16th Pacific-Asia Conference, PAKDD 2012, Kuala Lumpur, Malaysia, May 29 - June 1, 2012, Proceedings, Part II, Springer, Lecture Notes in Computer Science, vol 7302, pp 13–24
Breiman (2001) Random forests. MACH LEARN 2001,45(1)(-):5–32
Breunig MM, Kriegel HP, Ng RT, Sander J (2000) Lof: Identifying density-based local outliers. In: Acm Sigmod International Conference on Management of Data
Buczak A, Guven E (2017) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor 18(2):1153–1176
Cao N, Shi C, Lin S, Lu J, Lin YR, Lin CY (2016) Targetvue: Visual analysis of anomalous user behaviors in online communication systems. IEEE Transactions V Computer Graphics 22(1):280–9
Florez G, Bridges SA, Vaughn RB (2002) An improved algorithm for fuzzy data mining for intrusion detection. In: Annual meeting of the North American fuzzy information processing society proceedings. NAFIPS-FLINT 2002 (Cat. No. 02TH8622). IEEE, pp 457–462
Hettich S, Bay SD (1999) The uci kdd archive http://kdd.ics.uci.edu, irvine, CA: University of California, Department of Information and Computer Science
Huang H, He Q, Chiew K, Qian F, Ma L (2013) CLOVER: a faster prior-free approach to rare-category detection. Knowl Inf Syst 35(3):713–736
Huang H, Chiew K, Gao Y, He Q, Li Q (2014) Rare category exploration. Expert Syst Appl 41:4197–4210. https://doi.org/10.1016/j.eswa.2013.12.039
Kim HKDHAGMLJDYKHPHK (2019) Iot network intrusion dataset. https://doi.org/10.21227/q70p-q449, https://dx.doi.org/10.21227/q70p-q449
Kuok CM, Fu A, Wong MH (1998) Mining fuzzy association rules in databases. SIGMOD Rec 27(1):41–46
Landstorfer J, Herrmann I, Stange JE, Drk M, Wettach R (2014) Weaving a carpet from log entries: A network security visualization built with co-creation. In: Proc. IEEE Conference on Visual Analytics Science and Technology (VAST)
Li J, Hong S, Topor R (2002) Mining the optimal class association rule set. Knowl-Based Syst 15(7):399–405
Li Y, Xia J, Zhang S, Yan J, Ai X, Dai K (2012) An efficient intrusion detection system based on support vector machines and gradually feature removal method. Expert Syst Appl 39(1):424–430
Lin H, Gao S, Gotz D, Du F, He J, Cao N (2017) Rclens: Interactive rare category exploration and identification. IEEE Trans Visual Comput Graphics 24(7):2223–2237
Liu B, Hsu W, Ma Y, Ma B (1998) Integrating classification and association rule mining. Knowl Discov Data Min 80–86
Liu Z, Chiew K, He Q, Huang H, Huang B (2014) Prior-free rare category detection: More effective and efficient solutions. Expert Syst Appl Int J 41(17):7691–7706
Panda M, Patra MR (2007) Network intrusion detection using naive bayes. Int J Comput Sci Netw Secur 7(12):258–263
Ren P, Yan G, Li Z, Yan C, Watson B (2005) Idgraphs: Intrusion detection and analysis using histographs. In: IEEE workshop on visualization for computer security, (VizSEC 05). IEEE, pp 39–46
Song J, Takakura H, Okabe Y (2006) Description of kyoto university benchmark data[J]. https://www.takakura.com/Kyoto_data/BenchmarkData-Description-v5.pdf. Accessed 15 Mar 2016
Srikant R, Agrawal R (1999) Mining quantitative association rules in large relation tables. ACM SIGMOD Rec 25(2):1–12
Tajbakhsh A, Rahmati M, Mirzaei A (2009) Intrusion detection using fuzzy association rules. Appl Soft Comput 9(2):462–469
zgür A, Erdem H (2016) A review of kdd99 dataset usage in intrusion detection and machine learning between 2010 and 2015
Zhang J, Zulkernine M, Haque A (2008) Random-forests-based network intrusion detection systems. IEEE Transactions Syst Man Cybern Part C 38(5):649–659
Zhao J, Cao N, Wen Z, Song Y, Lin Y, Collins C (2014) #fluxflow: visual analysis of anomalous information spreading on social media. IEEE Transactions V Computer Graphic 20(12):1773–1782
Acknowledgements
Supported by the National Natural Science Foundation of China under Grant No. 61100053, and SJTU-HUAWEI TECH Cybersecurity Innovation Lab. The authors also wish to thank the experts from the cooperative company and the cyber security team in our university who helped with the system evaluation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, Y., Liu, H., Dong, X. et al. HyIDSVis: hybrid intrusion detection visualization analysis based on rare category and association rules. J Vis 25, 175–190 (2022). https://doi.org/10.1007/s12650-021-00789-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-021-00789-5