Skip to main content
SpringerLink
Log in

A survey of deep learning-based network anomaly detection

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

A great deal of attention has been given to deep learning over the past several years, and new deep learning techniques are emerging with improved functionality. Many computer and network applications actively utilize such deep learning algorithms and report enhanced performance through them. In this study, we present an overview of deep learning methodologies, including restricted Bolzmann machine-based deep belief network, deep neural network, and recurrent neural network, as well as the machine learning techniques relevant to network anomaly detection. In addition, this article introduces the latest work that employed deep learning techniques with the focus on network anomaly detection through the extensive literature survey. We also discuss our local experiments showing the feasibility of the deep learning approach to network traffic analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Semente: 2016 Internet Security Threat Report (ISTR), vol. 21, p. 8, April 2016

  2. Gartner Provides Three Immediate Actions to Take as WannaCry Ransomware Spreads. http://www.gartner.com/newsroom/id/3715918

  3. Li, Y., Ma, R., Jiao, R.: Hybrid malicious code detection method based on deep learning. Int. J. Secur. Appl. 9(5), 205–216 (2014)

    Google Scholar 

  4. Salama, M.A., Eid, H.F., Ramadan, R.A., Darwish, A., Hassanien, A.E.: Hybrid intelligent intrusion detection scheme. Soft Comput. Ind. Appl. 96, 293–303 (2011)

    Google Scholar 

  5. Niyaz, Q., Sun, W., Javaid, A.Y., Alam, M.: A deep learning approach for network intrusion detection system. In: 9th EAI International Conference on Bio-Inspired Information and Communications Technologies, pp. 1–11, May 2016

  6. Ahmed, A.: Signature-based network inrusion detection system using JESS(SNIDJ). Graduate Project Technical Report, TAMUCC, pp. 2–6 (2004)

  7. Ning, P., Jajodia, S.: Intrusion detection techniques. The Internet Encyclopedia. doi:10.1002/047148296X.tie097

  8. Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R., Muharemagic, E.: Deep learning applications and challenges in big data analytics. J. Big Data 2(1), 1 (2015)

    Article  Google Scholar 

  9. Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends Signal Process. 7(3–4), 197–387 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD CUP 99 dataset. In: Proceedings of the 2009 IEEE Symposium on Computational Intelligence in Security and Defense Applications (CISDA 2009), pp. 53–58 (2009)

  11. Revathi, S., Malathi, A.: A detailed analysis on NSL-KDD dataset using various machine learning techniques for intrusion detection. Int. J. Eng. Res. Technol. 2(12), 1848–1853 (2013)

    Google Scholar 

  12. Vinchurkar, D.P., Reshamwala, A.: A review of intrusion detectiom system using neural network and machine learning technique. Int. J. Eng. Sci. Innov. Technol. 1(2), 54–63 (2012)

    Google Scholar 

  13. Das, S., Kalita, H.K.: Advanced dimensionality reduction method for big data. In: Research advances in the integration of big data and smart computing, information science reference (an imprint of IGI global), p. 200 (2016)

  14. Panwar, S.S., Raiwani, Y.P.: Data reduction techniques to analyze NSL-KDD Dataset. Int. J. Comput. Eng. Technol. 5(10), 21–31 (2014)

    Google Scholar 

  15. Jain, A.K.: Data clustering: 50 years beyond K-means. J. Pattern Recognit. Lett. 31(8), 651–666 (2010)

    Article  Google Scholar 

  16. John, G.H., Langley, P.: Static versus dynamic sampling for data mining, KDD 96. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 367–370 (1996)

  17. Motoda, H., Liu, H.: Feature selection, extraction, and construction. Commun. Inst. Inf. Comput. Mach. Taiwan 5(2), 67–72 (2002)

    Google Scholar 

  18. Elrawy, M.F., Abdelhamid, T.K., Mohamed, A.M.: IDS in telecommunication network using PCA. Int. J. Comput. Netw. Commun. 5(4), 147–157 (2013)

    Article  Google Scholar 

  19. Datti, R., Lakhina, S.: Performance comparison of features reduction techniques for intrusion detection system. Int. J. Comput. Sci. Technol. 3(1), 332–335 (2012)

    Google Scholar 

  20. Bajaj, K., Arora, A.: Dimension reduction in intrusion detection features using discriminative machine learning approach. Int. J. Comput. Sci. Issues 10(4), 324–328 (2013)

    Google Scholar 

  21. Ibraheem, N.B., Jawhar, M.M.T., Osman, H.M.: Principle components analysis and multi-layer perceptron based intrusion detection system. In: Fifth Scientific Conference Information Technology, vol. 10(1), pp. 127–135 (2013)

  22. Chae, H., Jo, B., Choi, S., Park, T.: Feature selection for intrusion detection using NSL-KDD. In: Proceedings of the 12th WSEAS International Conference on Information Security and Privacy, pp. 184–187, November 2013

  23. Namratha, M., Prajwala, T.R.: A comprehensive overview of clustering algorithms in pattern recognition. IOSR J. Comput. Eng. 4(6), 23–30 (2012)

    Article  Google Scholar 

  24. Koturwar, P., Girase, S., Mukhopadhyay, D.: A survey of classification techniques in the area of big data. Int. J. Adv. Found. Res. Comput. 1(11), 1–7 (2014)

    Google Scholar 

  25. Caruana, R., Niculescu-Mizil, A.: An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 161–168, June 2006

  26. Lin, F., Cohen, W.W.: Semi-supervised classification of network data using very few labels. In: Proceedings of the 2010 International Conference on Advances in Social Networks and Mining, pp. 192–198, August 2010

  27. Deng, L., Yu, D.: Deep learning methods and applications. Found. Trends Signal Process., 7(3–4), 199–201, 217 (2014)

  28. Hinton, G.E.: Boltzmann machine. Scholarpedia 2(5), 1668 (2007)

    Article  Google Scholar 

  29. Fischer, A., Igel, C.: Training restricted Boltzmann machines: an introduction. Pattern Recognit. 47, 25–39 (2014)

    Article  MATH  Google Scholar 

  30. Alom, M.Z., Bontupalli, V., Taha, T.M.: Intrusion detection using deep belief networks. Int. J. Monit. Surveill. Technol. Res. 3(2), 35–56 (2015)

    Google Scholar 

  31. Kim, S.K., McMahon, P.L., Olulotun, K.: A large-scale architecture for restricted Boltzmann machines. In: Proceedings of the 2010 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines, pp. 201–208, May 2010

  32. Kang, M., Kang, J.: Intrusion detection system using deep neural network for in-vehicle network security. PLoS ONE 11(6), e0155781 (2016). doi:10.1371/journal.pone.0155781e0155781

    Article  Google Scholar 

  33. Hinton, G.E.: A practical guide to training restricted Boltzmann machines. UTML Technical Report 2010-003, University of Toronto, August 2010

  34. Yamashita, T., Tanaka, M., Yoshida, E., Yamauchi, Y., Fujiyoshii, H.: To be Bernoulli or to be Gaussian, for a restricted boltzmann machine. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 1520–1525. IEEE (2014)

  35. Sze, V., Chen, Y.-H., Yang, T.-J., Emer, J.: Efficient processing of deep neural networks: a tutorial and survey. arXiv preprint, arXiv:1703.09039 (2017)

  36. Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. Kayack, H.G., Zincir-Heywood, A.N., Heywood, M.I.: Selecting features for intrusion detection: a feature relevance analysis on KDD 99 intrusion detection datasets. In: Proceedings of the 3rd Annual Conference on Privacy Security and Trust, October 2005

  38. Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the kdd cup 99 data set. In: CISDA 2009. IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1–6. IEEE (2009)

  39. Tao, X., Kong, D., Wei, Y., Wang, Y.: A big network traffic data fusion approach based on fisher and deep auto-encoder. Information 7(2), 20 (2016)

    Article  Google Scholar 

  40. Kim, J., Kim, J., Thu, H.L.T., Kim, H.: Long short term memory recurrent neural network classifier for intrusion detection. In: 2016 International Conference on Platform Technology and Service (PlatCon), pp. 1–5, Feb 2016

  41. Tang, T.A., Mhamdi, L., McLernon, D., Zaidi, S.A.R., Ghogho, M.: Deep learning approach for network intrusion detection in software defined networking. In: 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM), pp. 258–263. IEEE (2016)

  42. Baek, S., Kwon, D., Kim, J., Suh, S., Kim, H., Kim, I.: Unsupervised labeling for supervised anomaly detection in enterprise and cloud networks. In: The 4th IEEE International Conference on Cyber Security and Cloud Computing (IEEE CSCloud 2017), July 2017

  43. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. arXiv preprint, arXiv:1703.05921 (2017)

  44. Xue, Y., Xu, T., Zhang, H., Long, R., Huang, X.: Segan: adversarial network with multi-scale \( l_1 \) loss for medical image segmentation. arXiv preprint, arXiv:1706.01805 (2017)

  45. Goodfellow, I.: Nips 2016 tutorial: generative adversarial networks. arXiv preprint, arXiv:1701.00160 (2016)

Download references

Acknowledgements

The authors are grateful to Ritesh Malaiya for his assistance for experimenting. This work was supported in part by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (No. 2016-0-00078, Cloud-based Security Intelligence Technology Development for the Customized Security Service Provisioning).

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Systems, Texas A&M University, Commerce, Commerce, USA

    Donghwoon Kwon, Jinoh Kim & Sang C. Suh

  2. Information Security Research Division, Electronics & Telecommunications Research Institute, Daejeon, Korea

    Hyunjoo Kim & Ikkyun Kim

  3. Department of Convergence Security, Kyonggi University, Suwon, Korea

    Kuinam J. Kim

Authors
  1. Donghwoon Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyunjoo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinoh Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sang C. Suh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ikkyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kuinam J. Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hyunjoo Kim or Kuinam J. Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kwon, D., Kim, H., Kim, J. et al. A survey of deep learning-based network anomaly detection. Cluster Comput 22 (Suppl 1), 949–961 (2019). https://doi.org/10.1007/s10586-017-1117-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-1117-8

Keywords

Search

Navigation