Skip to main content
SpringerLink
Log in

Improving threat detection in networks using deep learning

  • Published:
Annals of Telecommunications Aims and scope Submit manuscript

Abstract

Detecting threats on the Internet is a key factor in maintaining data and information security. An intrusion detection system tries to prevent such attacks from occurring through the analysis of patterns and behavior of the data stream in the network. This paper presents a large data stream detection and analysis distributed platform, through the use of machine learning to dimensionality reduction. The system is evaluated based on three criteria: the accuracy, the number of false positives, and number of false negatives. Each classifier presented better accuracy when using 5 and 13 features, having fewer false positives and false negatives, allowing the detection of threats in real-time over a large volume of data, with greater precision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Leu F, Tsai K, Hsiao Y, Yang C (2017) An internal intrusion detection and protection system by using data mining and forensic techniques. IEEE Syst J 11(2):427–438. https://doi.org/10.1109/JSYST.2015.2418434

    Article  Google Scholar 

  2. Symantec: internet security threat report (2019), vol 24. https://www.symantec.com/content/dam/symantec/docs/reports/istr-24-2019-en.pdf

  3. Tan Z, et al. (2014) Enhancing big data security with collaborative intrusion detection. IEEE Cloud Comput 1(3):27–33. https://doi.org/10.1109/MCC.2014.53

    Article  Google Scholar 

  4. Lazarevic A, Kumar V, Srivastava J (2005) Intrusion detection: a survey. In: Kumar V, Srivastava J, Lazarevic A (eds) Managing cyber threats. Massive computing, vol 5. Springer, Boston, https://doi.org/10.1007/0-387-24230-9_2

  5. Harbi N, Bahri E (2013) Real detection intrusion using supervised and unsupervised learning. In: 2013 International conference on soft computing and pattern recognition (SoCPaR). Hanoi, pp 321–326. https://doi.org/10.1109/SOCPAR.2013.7054151

  6. Lopez MA, Lobato AGP, Duarte OCMB, Pujolle G (2018) An evaluation of a virtual network function for real-time threat detection using stream processing. In: 2018 Fourth international conference on mobile and secure services (MobiSecServ). Miami Beach, pp 1–5. https://doi.org/10.1109/MOBISECSERV.2018.8311440

  7. Vinayakumar R, Alazab M, Soman KP, Poornachandran P, Al-Nemrat A, Venkatraman S (2019) Deep learning approach for intelligent intrusion detection system. IEEE Access 7:41525–41550. https://doi.org/10.1109/ACCESS.2019.2895334

    Article  Google Scholar 

  8. Information, C. S. U. of California. Kddcup 1999 data (1999) http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html

  9. Sahu SK, Sarangi S, Jena SK (2014) A detail analysis on intrusion detection datasets. In: 2014 IEEE international advance computing conference (IACC). Gurgaon, pp 1348–1353. https://doi.org/10.1109/IAdCC.2014.6779523

  10. Van NT, Thinh TN, Sach LT (2017) An anomaly-based network intrusion detection system using deep learning. In: 2017 International conference on system science and engineering (ICSSE). Ho Chi Minh City, pp 210–214. https://doi.org/10.1109/ICSSE.2017.8030867

  11. Kim K, Aminanto ME (2017) Deep learning in intrusion detection perspective: overview and further challenges. In: 2017 International workshop on big data and information security (IWBIS). Jakarta, pp 5–10. https://doi.org/10.1109/IWBIS.2017.8275095

  12. Alom MZ, Taha TM (2017) Network intrusion detection for cyber security using unsupervised deep learning approaches. In: 2017 IEEE National aerospace and electronics conference (NAECON). Dayton, pp 63–69. https://doi.org/10.1109/NAECON.2017.8268746

  13. Shone N, Ngoc TN, Phai VD, Shi Q (2018) A deep learning approach to network intrusion detection. IEEE Trans Emerg Topics Comput Intell 2(1):41–50. https://doi.org/10.1109/TETCI.2017.2772792

    Article  Google Scholar 

  14. Wang Z (2018) Deep learning-based intrusion detection with adversaries. IEEE Access 6:38367–38384. https://doi.org/10.1109/ACCESS.2018.2854599

    Article  Google Scholar 

  15. Papamartzivanos D, Gómez Mármol F, Kambourakis G (2019) Introducing deep learning self-adaptive misuse network intrusion detection systems. IEEE Access 7:13546–13560. https://doi.org/10.1109/ACCESS.2019.2893871

    Article  Google Scholar 

  16. Al-Qatf M, Lasheng Y, Al-Habib M, Al-Sabahi K (2018) Deep learning approach combining sparse autoencoder with SVM for network intrusion detection. IEEE Access 6:52843–52856. https://doi.org/10.1109/ACCESS.2018.2869577

    Article  Google Scholar 

  17. Schuartz FC, Fonseca MSP, Munaretto A (2019) Distributed system for threat detection in networks using machine learning. In: 1st Blockchain, robotics and AI for networking security conference - BRAINS (2019)

  18. Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127. https://doi.org/10.1561/2200000006

    Article  MathSciNet  MATH  Google Scholar 

  19. Chen Z, Yeo CK, Lee BS, Lau CT (2018) Autoencoder-based network anomaly detection. In: 2018 Wireless telecommunications symposium (WTS). Phoenix, pp 1–5. https://doi.org/10.1109/WTS.2018.8363930

  20. Lee W, Stolfo SJ, Mok KW (1999) Mining in a data-flow environment: experience in network intrusion detection. In: Proceedings of the fifth ACM SIGKDD international conference on knowledge discovery and data mining (KDD ’99). ACM, New York, pp 114–124, https://doi.org/10.1145/312129.312212

  21. Tavallaee M, Bagheri E, Lu W, Ghorbani AA (2009) A detailed analysis of the KDD CUP 99 data set. In: 2009 IEEE symposium on computational intelligence for security and defense applications, Ottawa, pp 1–6, https://doi.org/10.1109/CISDA.2009.5356528

  22. Paliwal S, Gupta R (2012) Article: denial-of-service, probing & remote to user (R2L) attack detection using genetic algorithm. Int J Comput Appl 60(19):57–62

    Google Scholar 

  23. Farid DM, Rahman MZ (2008) Learning intrusion detection based on adaptive Bayesian algorithm. In: 2008 11th International conference on computer and information technology. Khulna, pp 652–656. https://doi.org/10.1109/ICCITECHN.2008.4803036

  24. Foundation AS (2015) Apache storm. http://storm.apache.org

  25. Witten IH, Frank E, Hall MA (2011) Chapter 10 - introduction to Weka. In: The Morgan Kaufmann series in data management systems, data mining: practical machine learning tools and techniques. 3rd edn. Morgan Kaufmann, pp 403–406, https://doi.org/10.1016/B978-0-12-374856-0.00010-9

  26. Buczak AL, Guven E (2016) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor 18(2):1153–1176. https://doi.org/10.1109/COMST.2015.2494502. Secondquarter

    Article  Google Scholar 

  27. Bhargava N, Sharma G, Bhargava R, Mathuria M (2013) Decision tree analysis on j48 algorithm for datamining. Proc Int J Adv Res Comput Sci Softw Eng 3:6. ISSN: 2277 128X

    Google Scholar 

  28. Sun J (2010) Application of data mining for decision tree model of multi-variety discrete production and manufacture. In: 2010 Third international symposium on intelligent information technology and security informatics, Jinggangshan, pp 724–728, https://doi.org/10.1109/IITSI.2010.13

  29. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco. ISBN:1-55860-238-0

    Google Scholar 

  30. Aggarwal CC (2014) Data classification: algorithms and applications, 1st edn. Chapman & Hall/CRC. ISBN:1466586745 9781466586741

  31. John GH, Langley P (1995) Estimating continuous distributions in Bayesian classifiers. In: Besnard P, Hanks S (eds) Proceedings of the eleventh conference on uncertainty in artificial intelligence (UAI’95). ISBN:1-55860-385-9. Morgan Kaufmann Publishers Inc., San Francisco, pp 338–345

  32. Kohavi R (1995) The power of decision tables. In: Lavrač N, Wrobel S (eds) Proceedings of the 8th European conference on machine learning (ECML’95). Springer, Berlin, pp 174–189, https://doi.org/10.1007/3-540-59286-5_57

Download references

Author information

Authors and Affiliations

  1. Universidade Tecnólogica Federal do Paraná, Curitiba, PR, Brazil

    Fábio César Schuartz, Mauro Fonseca & Anelise Munaretto

Authors
  1. Fábio César Schuartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mauro Fonseca
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anelise Munaretto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fábio César Schuartz.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schuartz, F.C., Fonseca, M. & Munaretto, A. Improving threat detection in networks using deep learning. Ann. Telecommun. 75, 133–142 (2020). https://doi.org/10.1007/s12243-019-00743-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12243-019-00743-5

Keywords

Search

Navigation