Skip to main content
SpringerLink
Log in

Embedding residuals in graph-based solutions: the E-ResSAGE and E-ResGAT algorithms. A case study in intrusion detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Neural network architectures have been used to address multiple real-world problems with high success. Their extension to graph-structured data started recently to be explored. Graph-neural network (GNN) achieved state-of-the-art performance in multiple problems. In highly imbalanced application domains, such as network intrusion problems, GNN was used to model the network topology. However, in this scenario, the class imbalance problem still affects the performance. Another graph-based solution, the graph attention network (GAT) has also been applied to multiple predictive tasks. Although being a promising avenue, graph-based solutions are still under-explored in imbalanced scenarios. This paper proposes two novel graph-based algorithms, the E-ResSAGE and E-ResGAT algorithms, which build on top of the established GraphSAGE and GAT algorithms, respectively. The key idea is to integrate residual learning into the GNN leveraging the available graph information. Residual connections are added as a strategy to deal with the high class imbalance, aiming at retaining the original information and improving the minority classes’ performance. A case study on intrusion detection is provided. Extensive experiments on four recent intrusion detection datasets show the excellent performance of our proposed approaches, especially when predicting minority classes. We demonstrate that embedding residuals in graph-based algorithms presents a strong advantage when learning under imbalanced domains.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Fig. 2
Algorithm 3
Fig. 3
Fig. 4

Similar content being viewed by others

Availability of data and materials

The link to each dataset is listed in Section 4.1. The code for pre-processing steps is included in the Github repository https://github.com/George730/E-ResGAT.

Code availability

The code for the two proposed models is also included in the Github repository https://github.com/George730/E-ResGAT.

Notes

  1. By replacing addition with concatenation in residual learning, we will learn two different weight matrices for original and residual features, instead of one. This change allows for more model complexity and may deal with more diverse datasets.

  2. https://www.unb.ca/cic/datasets/darknet2020.html

  3. https://research.unsw.edu.au/projects/toniot-datasets

  4. https://research.unsw.edu.au/projects/unsw-nb15-dataset

  5. https://www.unb.ca/cic/datasets/ids-2018.html

References

  1. Alsaedi A, Moustafa N, Tari Z et al (2020) Ton_iot telemetry dataset: A new generation dataset of iot and iiot for data-driven intrusion detection systems. IEEE Access 8:165130–165150

    Article  Google Scholar 

  2. Altaf T, Wang X, Ni W et al (2023) NE-GConv: A lightweight node edge graph convolutional network for intrusion detection. Comput Secur 130(103):285

    Google Scholar 

  3. Cheng Q, Wu C, Zhou S (2021) Discovering attack scenarios via intrusion alert correlation using graph convolutional networks. IEEE Commun Lett 25(5):1564–1567

    Article  Google Scholar 

  4. Dai H, Dai B, Song L (2016) Discriminative embeddings of latent variable models for structured data. In: International conference on machine learning. PMLR, pp 2702–2711

  5. Duan G, Lv H, Wang H et al (2022) Application of a dynamic line graph neural network for intrusion detection with semisupervised learning. IEEE Trans Inf Forensics Secur 18:699–714

    Article  Google Scholar 

  6. Habibi Lashkari A, Kaur G, Rahali A (2020) Didarknet: A contemporary approach to detect and characterize the darknet traffic using deep image learning. In: 2020 the 10th international conference on communication and network security, pp 1–13

  7. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Advances in neural information processing systems 30

  8. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  9. Johnson JM, Khoshgoftaar TM (2020) The effects of data sampling with deep learning and highly imbalanced big data. Inf Syst Front 22(5):1113–1131

    Article  Google Scholar 

  10. Lee J, Park K (2021) Gan-based imbalanced data intrusion detection system. Pers Ubiquit Comput 25(1):121–128

    Article  Google Scholar 

  11. Lehot PG (1974) An optimal algorithm to detect a line graph and output its root graph. J ACM (JACM) 21(4):569–575

    Article  MathSciNet  Google Scholar 

  12. Liu X, Ding J, Jin W et al (2021) Graph neural networks with adaptive residual. Adv Neural Inf Process Syst 34:9720–9733

    Google Scholar 

  13. Maas AL, Hannun AY, Ng AY, et al (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proc. icml, Citeseer, p 3

  14. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(11)

  15. Moustafa N, Slay J (2015) Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In: 2015 military communications and information systems conference (MilCIS). IEEE, pp 1–6

  16. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Icml

  17. Nguyen H, Kashef R (2023) TS-IDS: Traffic-aware self-supervised learning for IoT Network Intrusion Detection. Knowl-Based Syst 279(110):966

    Google Scholar 

  18. Paszke A, Gross S, Massa F et al (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037

    Google Scholar 

  19. Scarselli F, Gori M, Tsoi AC et al (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80

    Article  Google Scholar 

  20. Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108–116

    Google Scholar 

  21. Shuaiyi L, Wang K, Zhang L et al (2022) Global-local integration for GNN-based anomalous device state detection in industrial control systems. Expert Syst Appl 209(118):345

    Google Scholar 

  22. Tang J, Gao X, Hu W (2021) Rgln: Robust residual graph learning networks via similarity-preserving mapping on graphs. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 2940–2944

  23. Clevert D-A, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network Learning by Exponential Linear Units (ELUs). In: Bengio Y, LeCun Y (eds) 4th International conference on learning representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings

  24. Gao Mengnan, Wu Lifa, Li Qi, Chen Wei (2023) Anomaly traffic detection in IoT security using graph neural networks. J Inf Secur Appl 76(103):532

  25. Jiang Weiwei (2022) Graph-based Deep Learning for Communication Networks: A Survey. Comput Commun 185:40–54

    Article  Google Scholar 

  26. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the 3rd international conference for learning representations (iclr’15). San Diego 500

  27. Lo WW, Layeghy S, Sarhan M, Gallagher M, Portmann M (2022) E-GraphSAGE: a graph neural network based intrusion detection system for IoT. NOMS 2022-2022 IEEE/IFIP network operations and management symposium, pp 1–9

  28. Veličković P, Cucurull G, Casannova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. Stat 1050(20):10–48550

  29. Wan Y, Liu Y, Wang D et al (2021) Glad-paw: Graph-based log anomaly detection by position aware wighted graph attention network. In: PAKDD (1). Springer, pp 66–77

  30. Wang S, Liu W, Wu J et al (2016) Training deep neural networks on imbalanced data sets. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 4368–4374

  31. Xiao Q, Liu J, Wng Q et al (2020) Towards network anomaly detection using graph embedding. In: International conference on computational science. Springer, pp 156–169

  32. Yang C, Zhou Z, Wen H et al (2020) Mstnn: A graph learning based method for origin-destination traffic prediction. In: ICC 2020-2020 IEEE interntional conference on communications (ICC). IEEE, pp 1–6

  33. Yu F, Koltun V, Funkhouser T (2017) Dilated residual networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 472–480

  34. Zhou J, Xu Z, Rush AM et al (2020) Automating botnet detection with graph neural network. Preprint arXiv:2003.06344

Download references

Funding

This project received research support from Mitacs Global Research Internship program (amount: CAD $2,856), from the Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. LTI - School of Computer Science, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, 15213, USA

    Liyan Chang

  2. EECS - Faculty of Engineering, University of Ottawa, 800 King Edward Ave, SITE, Ottawa, Ontario, K1N 6N5, Canada

    Paula Branco

Authors
  1. Liyan Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paula Branco
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Liyan Chang. Liyan Chang and Paula Branco co-wrote and revised the draft of the manuscript and both authors read and approved the final manuscript.

Corresponding author

Correspondence to Paula Branco.

Ethics declarations

Conflict of interest/Competing interests

The authors do not have any conflict or competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, L., Branco, P. Embedding residuals in graph-based solutions: the E-ResSAGE and E-ResGAT algorithms. A case study in intrusion detection. Appl Intell (2024). https://doi.org/10.1007/s10489-024-05404-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05404-2

Keywords

Search

Navigation