Skip to main content

HyIDSVis: hybrid intrusion detection visualization analysis based on rare category and association rules

Abstract

Cyber security issues are always worthy of attention. Intrusion detection system (IDS) is one of approaches used to protect computer system and identify potential attack. However, existing methods are limited by high-dimensional computational complexity in rare or unknown attack detection task. To improve the ability of detecting anomaly intrusions, a hybrid intrusion detection framework is proposed in this paper. The proposed RuleRCD algorithm first uses fuzzy association rules based on Apriori and K-Means algorithm for normal pattern and major attack detection. The other data are then fed to an active learning-based rare category detection algorithm to identify its attack pattern. This paper also introduces an interactive visualization system, which integrates experts’ decision into intrusion detection workflow. The method improves the effectiveness and interpretability of detection process. KDD-99 dataset is used to evaluate the proposed framework. The result shows that the approach outperforms some methods, especially in terms of the rare attack.

Graphical abstract

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Aggus LR, Morais DI, Baker RF (2000) Evaluating intrution detection systems: the 1998 darpa offline intrusion detection evaluation. Computer Netw 34(4):579–595

    Article  Google Scholar 

  2. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: Proceedings of the 20th International Conference on Very Large Data Bases

  3. Brahmi H, Brahmi I, Yahia SB (2012) OMC-IDS: at the cross-roads of OLAP mining and intrusion detection. In: Tan P, Chawla S, Ho CK, Bailey J (eds) Advances in Knowledge Discovery and Data Mining - 16th Pacific-Asia Conference, PAKDD 2012, Kuala Lumpur, Malaysia, May 29 - June 1, 2012, Proceedings, Part II, Springer, Lecture Notes in Computer Science, vol 7302, pp 13–24

  4. Breiman (2001) Random forests. MACH LEARN 2001,45(1)(-):5–32

  5. Breunig MM, Kriegel HP, Ng RT, Sander J (2000) Lof: Identifying density-based local outliers. In: Acm Sigmod International Conference on Management of Data

  6. Buczak A, Guven E (2017) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor 18(2):1153–1176

    Article  Google Scholar 

  7. Cao N, Shi C, Lin S, Lu J, Lin YR, Lin CY (2016) Targetvue: Visual analysis of anomalous user behaviors in online communication systems. IEEE Transactions V Computer Graphics 22(1):280–9

    Article  Google Scholar 

  8. Florez G, Bridges SA, Vaughn RB (2002) An improved algorithm for fuzzy data mining for intrusion detection. In: Annual meeting of the North American fuzzy information processing society proceedings. NAFIPS-FLINT 2002 (Cat. No. 02TH8622). IEEE, pp 457–462

  9. Hettich S, Bay SD (1999) The uci kdd archive http://kdd.ics.uci.edu, irvine, CA: University of California, Department of Information and Computer Science

  10. Huang H, He Q, Chiew K, Qian F, Ma L (2013) CLOVER: a faster prior-free approach to rare-category detection. Knowl Inf Syst 35(3):713–736

  11. Huang H, Chiew K, Gao Y, He Q, Li Q (2014) Rare category exploration. Expert Syst Appl 41:4197–4210. https://doi.org/10.1016/j.eswa.2013.12.039

    Article  Google Scholar 

  12. Kim HKDHAGMLJDYKHPHK (2019) Iot network intrusion dataset. https://doi.org/10.21227/q70p-q449, https://dx.doi.org/10.21227/q70p-q449

  13. Kuok CM, Fu A, Wong MH (1998) Mining fuzzy association rules in databases. SIGMOD Rec 27(1):41–46

    Article  Google Scholar 

  14. Landstorfer J, Herrmann I, Stange JE, Drk M, Wettach R (2014) Weaving a carpet from log entries: A network security visualization built with co-creation. In: Proc. IEEE Conference on Visual Analytics Science and Technology (VAST)

  15. Li J, Hong S, Topor R (2002) Mining the optimal class association rule set. Knowl-Based Syst 15(7):399–405

    Article  Google Scholar 

  16. Li Y, Xia J, Zhang S, Yan J, Ai X, Dai K (2012) An efficient intrusion detection system based on support vector machines and gradually feature removal method. Expert Syst Appl 39(1):424–430

    Article  Google Scholar 

  17. Lin H, Gao S, Gotz D, Du F, He J, Cao N (2017) Rclens: Interactive rare category exploration and identification. IEEE Trans Visual Comput Graphics 24(7):2223–2237

  18. Liu B, Hsu W, Ma Y, Ma B (1998) Integrating classification and association rule mining. Knowl Discov Data Min 80–86

  19. Liu Z, Chiew K, He Q, Huang H, Huang B (2014) Prior-free rare category detection: More effective and efficient solutions. Expert Syst Appl Int J 41(17):7691–7706

    Article  Google Scholar 

  20. Panda M, Patra MR (2007) Network intrusion detection using naive bayes. Int J Comput Sci Netw Secur 7(12):258–263

  21. Ren P, Yan G, Li Z, Yan C, Watson B (2005) Idgraphs: Intrusion detection and analysis using histographs. In: IEEE workshop on visualization for computer security, (VizSEC 05). IEEE, pp 39–46

  22. Song J, Takakura H, Okabe Y (2006) Description of kyoto university benchmark data[J]. https://www.takakura.com/Kyoto_data/BenchmarkData-Description-v5.pdf. Accessed 15 Mar 2016

  23. Srikant R, Agrawal R (1999) Mining quantitative association rules in large relation tables. ACM SIGMOD Rec 25(2):1–12

  24. Tajbakhsh A, Rahmati M, Mirzaei A (2009) Intrusion detection using fuzzy association rules. Appl Soft Comput 9(2):462–469

    Article  Google Scholar 

  25. zgür A, Erdem H (2016) A review of kdd99 dataset usage in intrusion detection and machine learning between 2010 and 2015

  26. Zhang J, Zulkernine M, Haque A (2008) Random-forests-based network intrusion detection systems. IEEE Transactions Syst Man Cybern Part C 38(5):649–659

    Article  Google Scholar 

  27. Zhao J, Cao N, Wen Z, Song Y, Lin Y, Collins C (2014) #fluxflow: visual analysis of anomalous information spreading on social media. IEEE Transactions V Computer Graphic 20(12):1773–1782

    Article  Google Scholar 

Download references

Acknowledgements

Supported by the National Natural Science Foundation of China under Grant No. 61100053, and SJTU-HUAWEI TECH Cybersecurity Innovation Lab. The authors also wish to thank the experts from the cooperative company and the cyber security team in our university who helped with the system evaluation.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Xiaoju Dong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Liu, H., Dong, X. et al. HyIDSVis: hybrid intrusion detection visualization analysis based on rare category and association rules. J Vis (2021). https://doi.org/10.1007/s12650-021-00789-5

Download citation

Keywords

  • Visualization system
  • Cyber intrusion detection
  • Rare category detection
  • Association rules