Abstract
Deep learning promotes the fields of image processing, machine translation and natural language processing etc. It also can be used in network anomaly detection. In practice, it is not hard to obtain normal instances. However, it is always difficult to label anomalous instances. Semi-supervised learning can be utilized to resolve this problem. In this paper, we make a comprehensive study of semi-supervised deep learning techniques for network anomaly detection. Three kinds of deep learning techniques including GAN (Generative Adversarial networks), Auto-encoder and LSTM (Long Short-Term Memory) are studied on the latest network traffic dataset of CICIDS2017. Five deep architectures based on semi-supervised learning are designed, including BiGAN, regular GAN, WGAN, Auto-encoder and LSTM. Seven schemes of semi-supervised deep learning for anomaly detection are proposed according to different functions of anomaly score. Grid search is utilized to find the threshold of anomaly detection. Two traditional schemes of machine learning are also adopted to compare performance. There are altogether nine schemes of anomaly detection for CICIDS2017. From results of the experiment for network anomaly detection, it can be found that Auto-encoder outperforms LSTM and the three kinds of GAN. BiGAN and LSTM are both better than WGAN and regular GAN. All the seven schemes of semi-supervised deep learning for anomaly detection outperform the two traditional schemes. The work and results in this paper are meaningful on the application of semi-supervised deep learning for network anomaly detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kotsiantis, S.B., Zaharakis, I., Pintelas, P.: Supervised machine learning: a review of classification techniques. Emerg. Artif. Intell. Appl. Comput. Eng. 160, 3–24 (2007)
Hodeghatta, U.R., Nayak: Unsupervised machine learning. In: Business Analytics Using R - A Practical Approach, pp. 233–255. Apress, Berkeley (2017)
Adeli, E., Thung, K.H., An, L., et al.: Semi-supervised discriminative classification robust to sample-outliers and feature-noises. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 515–522 (2019)
Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)
Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. Comput. Sci. 5(1), 36 (2015)
Chandar, A.P.S., Lauly, S., Larochelle, H., et al.: An autoencoder approach to learning bilingual word representations In: International Conference on Neural Information Processing Systems (2014)
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets In: International Conference on Neural Information Processing Systems (2014)
Sharafaldin, I., Lashkari, A.H., Ghorbani, A.A.: Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: 4th International Conference on Information Systems Security and Privacy (ICISSP), Portugal, January 2018
Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks. Comput. Sci. (2015)
Donahue, J., Krähenbühl, P., Darrell, T.: Adversarial feature learning. arXiv preprint arXiv:1605.09782 (2016)
Goodfellow, I.J., et al.: Generative adversarial nets. In: International Conference on Neural Information Processing Systems (2014)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)
Zhang, J., Wang, H., Yang, H.: Dimension reduction method of high resolution range profile based on Autoencoder. J. Pla Univ. Sci. Technol. (2016)
Sakurada, M., Yairi, T.: Anomaly detection using autoencoders with nonlinear dimensionality reduction. In: Mlsda Workshop on Machine Learning for Sensory Data Analysis (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jason Brownlee Blog. https://machinelearningmastery.com/convert-time-series-upervised-learning-problem-python/. Accessed 25 June 2019
Zenati, H., Foo, C.S., Lecouat, B., et al.: Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)
UNB. https://www.unb.ca/cic/datasets/index.html. Accessed 25 June 2019
Acknowledgement
This work is supported by the National Natural Science Foundation of China (No. 61901454), and the Foundation of key Laboratory of Space Utilization, Technology and Engineering Center for Space utilization Chinese Academy of Sciences (No. CSU-QZKT-2018-08).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, Y., Guo, L., Li, Y., Xu, L., Wang, Y. (2020). Semi-supervised Deep Learning for Network Anomaly Detection. In: Wen, S., Zomaya, A., Yang, L.T. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11945. Springer, Cham. https://doi.org/10.1007/978-3-030-38961-1_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-38961-1_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38960-4
Online ISBN: 978-3-030-38961-1
eBook Packages: Computer ScienceComputer Science (R0)