Soft Computing

, Volume 22, Issue 13, pp 4185–4195 | Cite as

Improved clustering algorithm based on high-speed network data stream

  • Chunyong Yin
  • Lian Xia
  • Sun Zhang
  • Ruxia Sun
  • Jin Wang
Focus

Abstract

With the continuous development of network technology, the attack has become the biggest threat to the stable operation of the network. Intrusion detection technology is a proactive safety protection measure which provides real-time monitoring of internal attacks, external attacks, and misuse. Traditional intrusion detection system is short of adaptability due to the complication and scale of the network. The main problem is that the real-time performance of the network is poor and the reliability is not high. This paper designs the intrusion detection mechanism combined with data stream clustering algorithm and intrusion detection system to solve the problem in processing a large amount of high-speed data streams. The performance of processing data streams is improved through the clustering algorithm based on density and the sliding window and the experiments show that the intrusion detection efficiency is higher than DenStream algorithm.

Keywords

High-speed network Intrusion detection Data stream Clustering algorithm 

Notes

Acknowledgements

This work was funded by the National Natural Science Foundation of China (61373134, 61402234). It was also supported by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD), Jiangsu Key Laboratory of Meteorological Observation and Information Processing (KDXS1105) and Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology (CICAEET). Prof. Ruxia Sun is the corresponding author. We declare that we do not have any conflicts of interest to this work.

Compliance with ethical standards

Conflict of interest:

Chunyong Yin declares that he has no conflict of interest. Lian Xia declares that she has no conflict of interest. Sun Zhang declares that he has no conflict of interest. Ruxia Sun declares that she has no conflict of interest. Jin Wang declares that he has no conflict of interest.

Ethical approval:

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. Barbara D, Daniel J Couto et al (2001) ADAM: a testbed for exploring the use of data mining in intrusion detection. Acm Sigmod Rec 30(4):15–24CrossRefGoogle Scholar
  2. Bu S, Yu R, Liu P et al (2011) Distributed combined authentication and intrusion detection with data fusion in high-security mobile Ad Hoc networks. IEEE Trans Veh Technol 60(3):1025–1036CrossRefGoogle Scholar
  3. Cai Y, Xie K, Ma X (2004) An improved DBSCAN algorithm which is insensitive to input parameters. Acta Sci Nat Univ Pekin 40(3):480–486Google Scholar
  4. Chen Z, Zhang H, Hatcher W, Nguyen J Yu W (2016) A streaming-based network monitoring and threat detection system. In: IEEE 14th international conference on software engineering research, management and applications (SERA), pp 31–37Google Scholar
  5. Cho Y, Moon S (2015) Recommender system using periodicity analysis via mining sequential patterns with time-series and FRAT analysis. J Converg (JoC) 6(1):9–17Google Scholar
  6. Dash T (2017) A study on intrusion detection using neural networks trained with evolutionary algorithms. Soft Comput 21(10):2687–2700CrossRefGoogle Scholar
  7. Gao H, Zhu D, Wang X (2011) A parallel clustering ensemble algorithm for intrusion detection system. Ninth Int Symp Distrib Comput Appl Bus Eng Sci 2011:450–453Google Scholar
  8. Gaur M, Pant B (2015) Trusted and secure clustering in mobile pervasive environment. Hum–cent Comput Inform Sci (HCIS) 5:32Google Scholar
  9. Gu B, Sheng VS, Tay K, Romano W, Li S (2014) Incremental support vector learning for ordinal regression. IEEE T Neur Net Lear 26(7):1403–1416MathSciNetCrossRefGoogle Scholar
  10. Gu B, Sheng VS, Wang Z, Ho D, Osman S, Li S (2015) Incremental learning for \(\nu \)-support vector regression. Neural Netw 67:140–150CrossRefGoogle Scholar
  11. Keegan N, Ji S, Chaudhary A, Concolato C, Yu B, Jeong D (2016) A survey of cloud-based network intrusion detection analysis. Hum–cent Comput Inform Sci (HCIS) 6:19Google Scholar
  12. Kranen P, Assent I, Baldauf C, Seidl T (2009) Self-adaptive anytime stream clustering. In: The 9th IEEE international conference on data mininig (ICDM), pp 249–258Google Scholar
  13. Kranen P, Assent I, Baldauf C, Seidl T (2011) The clu-stree: Indexing micro-clusters for anytime stream mining. Knowl Inform Syst 29(2):249–272CrossRefGoogle Scholar
  14. Lee W, Stolfo S, Mok K (1998) Mining audit data to build intrusion detection models. In: International conference on knowledge discovery & data mining, pp 66–72Google Scholar
  15. Liang C, Zhang Y, Shi P et al (2012) Learning very fast decision tree from uncertain data streams with positive and unlabeled samples. Inform Sci 213(23):50–67MathSciNetCrossRefGoogle Scholar
  16. Liu X, Xu H, Dong Y, Wang Y, Qian J (2005) Discovering frequent pattern in the data stream. Comput Res Dev 2005(12):2192–2198CrossRefGoogle Scholar
  17. Rajput R, Mishra A, Kumar S (2014) Optimize intrusion prevention and minimization of threats for stream data classification. Fourth Int Conf Commun Syst Netw Technol 2014:408–413Google Scholar
  18. Wang D, Sun J (2010) Research on data mining technology. Stat Decis 2010(7):161–162Google Scholar
  19. Wang W, Guyet T, Quiniou R et al (2014) Autonomic intrusion detection: adaptively detecting anomalies over unlabeled audit data streams in computer networks. Knowl-Based Syst 70:103–117CrossRefGoogle Scholar
  20. Xia L, Jing J (2009) SA-DBSCAN: a self-adaptive density-based clustering algorithm. J Grad Sc Chin Acad Sci 26(4):530–538MathSciNetGoogle Scholar
  21. Xu K, Kliger M, Hero A III (2014) Adaptive evolutionary clustering. Data Min Knowl Discov 28(2):304–336MathSciNetCrossRefMATHGoogle Scholar
  22. Yang D, Rundensteiner E, Ward M (2012) Shared execution strategy for neighbor-based pattern mining requests over streaming windows. ACM Trans Database Syst 37(1):5CrossRefGoogle Scholar
  23. Yin C, Ma L, Feng L (2016) A feature selection method for improved clonal algorithm towards intrusion detection. Int J Pattern Recog Artif Intell 30(5):1–14CrossRefGoogle Scholar
  24. Yin C, Zhang S, Xi J (2016) An improved anonymity model for big data security based on clustering algorithm. Concurr Comput Pract Exp 2016:1–13Google Scholar
  25. Yin C, Feng L, Ma L, Kim J, Wang J (2016) An effective feature selection and data-stream classification model HDP. J Internet Technol 17(4):695–702Google Scholar
  26. Yu Y, Guo S, Huang H (2007) Abnormal intrusion detection based on data stream. Comput Sci 34(5):66–71CrossRefGoogle Scholar
  27. Zhen J, Hu M, Yun X (2006) Large scale network anomaly discovery based on data flow method. J Commun 27(2):1–8Google Scholar
  28. Zhu C, Dun X, Zhu L (2011) A study on the application of data stream clustering mining through a sliding and damped window to intrusion detection. Fourth Int Conf Inform Comput 2011:22–26Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.School of Computer and Software, Jiangsu Engineering Center of Network MonitoringJiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science & TechnologyNanjingChina
  2. 2.College of Information EngineeringYangzhou UniversityYangzhouChina

Personalised recommendations