Advertisement

Cluster Computing

, Volume 22, Supplement 1, pp 1729–1738 | Cite as

Anomaly detection model based on data stream clustering

  • Chunyong Yin
  • Sun Zhang
  • Zhichao Yin
  • Jin WangEmail author
Article

Abstract

Intrusion detection provides important protection for network security and anomaly detection as a type of intrusion detection, which can recognize the pattern of normal behaviors and label the behaviors which departure from normal pattern as anomaly behaviors. The updating of network equipment and broadband speed makes the data mining object change from static data sets to dynamic data streams. We think that the traditional methods based on data set do not satisfy the needs of dynamic network environment. The network data stream is temporal and cannot be treated as static data set. The concept and distribution of data objects is variety in different time stamps and the changing is unpredictable. Therefore, we propose an improved data stream clustering algorithm and design the anomaly detection model according to the improved algorithm. The established model can be modified with the changing of data stream and detect anomaly behaviors in time.

Keywords

Intrusion detection Anomaly detection Data stream Clustering 

Notes

Acknowledgements

This work was funded by the National Natural Science Foundation of China (61373134, 61402234). It was also supported by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD), Jiangsu Key Laboratory of Meteorological Observation and Information Processing (KDXS1105) and Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology (CICAEET). Prof. Jin Wang is the corresponding author. We declare that we do not have any conflicts of interest to this work.

Funding This study was funded by the National Natural Science Foundation of China (Grant Number 61373134, 61402234).

Compliance with ethical standards

Conflict of interest

Chunyong Yin, Sun Zhang, Zhichao Yin and Jin Wang declares that they have no conflict of interest. Ethical approval This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. 1.
    Lee, W., Stolfo, S., Mok, K.: Mining audit data to build intrusion detection models. In: International Conference on Knowledge Discovery & Data Mining, pp. 66–72 (1998)Google Scholar
  2. 2.
    Keegan, N., Ji, S.Y., Chaudhary, A., Concolato, C., Yu, B., Jeong, D.: A survey of cloud-based network intrusion detection analysis. Hum. Cent. Comput. Inf. Sci. 6(1), 19–35 (2016)CrossRefGoogle Scholar
  3. 3.
    Gu, B., Sheng, V.S., Tay, K.Y., Romano, W., Li, S.: Incremental support vector learning for ordinal regression. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1403–1416 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Gu, B., Sheng, V.S., Wang, Z., Ho, D., Osman, S., Li, S.: Incremental learning for \(\nu \)-support vector regression. Neural Netw. 67, 140–150 (2015)CrossRefzbMATHGoogle Scholar
  5. 5.
    Li, W., Li, X., Yao, M., Jiang, J., Jin, Q.: Personalized fitting recommendation based on support vector regression. Hum. Cent. Comput. Inf. Sci. 5(1), 21–32 (2015)CrossRefGoogle Scholar
  6. 6.
    Gaur, M., Pant, B.: Trusted and secure clustering in mobile pervasive environment. Hum.-Cent. Comput. Inf. Sci. 5(1), 1–17 (2015)CrossRefGoogle Scholar
  7. 7.
    Li, L., Ye, J., Deng, F., Xiong, S., Zhong, L.: A comparison study of clustering algorithms for microblog posts. Clust. Comput. 19(3), 1333–1345 (2016)CrossRefGoogle Scholar
  8. 8.
    Chen, C.-C., Fu, X., Chang, C.-Y.: A terms mining and clustering technique for surveying network and content analysis of academic groups exploration. Clust. Comput. 20(1), 43–52 (2017)CrossRefGoogle Scholar
  9. 9.
    Jang, J., Lee, Y., Lee, S., Shin, D., Kim, D., Rim, H.: A novel density-based clustering method using word embedding features for dialogue intention recognition. Clust. Comput. 19(4), 2315–2326 (2016)CrossRefGoogle Scholar
  10. 10.
    Yin, C., Zhang, S.: Parallel implementing improved k-means applied for image retrieval and anomaly detection. Multimed. Tools Appl. 2016, 1–17 (2016)Google Scholar
  11. 11.
    Yin, C., Zhang, S., Kim, K.: Mobile anomaly detection based on improved self-organizing maps. Mob. Inf. Syst. 2017, 1–9 (2017)Google Scholar
  12. 12.
    Yin, C., Zhang, S., Xi, J., Wang, J.: An improved anonymity model for big data security based on clustering algorithm. Concurr. Comput.-Pract. Exp. 29(7), 1–13 (2016)Google Scholar
  13. 13.
    Silva, J., Faria, E., Barros, R., Hruschka, E.: Data stream clustering: a survey. ACM. Comput. Surv. 46(1), 125–134 (2013)CrossRefzbMATHGoogle Scholar
  14. 14.
    Oh, S., Kang, S., Byun, Y., Jeong, T., Lee, W.: Anomaly intrusion detection based on clustering a data stream. In: ACIS International Conference on Software Engineering Research, Management and Applications, vol. 4176, pp. 220–227 (2005)Google Scholar
  15. 15.
    Guha, S., Meyerson, A., Mishra, N., Motwani, R.: Clustering data streams: theory and practice. IEEE Trans. Knowl. Data Eng. 15(3), 515–528 (2003)CrossRefGoogle Scholar
  16. 16.
    Aggarwal, C., Yu, P., Han, J., Wang, J.: A framework for clustering evolving data streams. In: International Conference on Very Large Data Bases, vol. 29, pp. 81–92 (2003)Google Scholar
  17. 17.
    Cao, F., Ester, M., Qian, W., Zhou, A.: Density-based clustering over an evolving data stream with noise. In: SIAM International Conference on dData Mining, pp. 328–339 (2006)Google Scholar
  18. 18.
    Udommanetanakit, K., Rakthanmanon, T., Waiyamai, K.: E-Stream: evolution-based technique for stream clustering. In: Third International Conference on Advanced Data Mining and Applications, pp. 605–615 (2007)Google Scholar
  19. 19.
    Chen, Y., Tu, L.: Density-based clustering for real-time stream data. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 133–142 (2007)Google Scholar
  20. 20.
    Axelsson, S.: The base-rate fallacy and the difficulty of intrusion detection. ACM Trans. Inf. Syst. Sec. 3(3), 186–205 (2000)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Chunyong Yin
    • 1
  • Sun Zhang
    • 1
  • Zhichao Yin
    • 2
  • Jin Wang
    • 3
    Email author
  1. 1.School of Computer and Software, Jiangsu Engineering Center of Network Monitoring, Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment TechnologyNanjing University of Information Science and TechnologyNanjingChina
  2. 2.No.1 Middle SchoolNanjingChina
  3. 3.College of Information EngineeringYangzhou UniversityYangzhouChina

Personalised recommendations