Skip to main content
SpringerLink
Log in

A novel hierarchical attention-based triplet network with unsupervised domain adaptation for network intrusion detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Network Intrusion Detection Systems(NIDSs) are crucial for resisting cyber threats. However, NIDSs equipped with supervised learning models do not generalize well to unknown attacks because the training samples for previously unseen new intrusions are usually not available in advance. Thus, a new framework based on a Hierarchical Attention-based Triplet network with Unsupervised Domain Adaptation(HAT-UDA) is proposed for this purpose. Concretely, a joint loss is introduced to force HAT-UDA to learn compact and discriminative embeddings for benign network traffic while being far from the representations of known attacks. Then, a One-class Support Vector Machine(OCSVM) model is trained on top of the benign embeddings for the unknown attack detection task. Furthermore, we propose an unsupervised domain adaptation module in an adversarial manner to reduce the false positives of HAT-UDA when applied to new network scenarios. HAT-UDA provides a novel approach for building a robust NIDS from benign traffic and available (known) attacks. This is particularly meaningful since collecting samples for benign traffic and known attacks is much easier than obtaining instances for unseen new attacks. Extensive experiments show that HAT-UDA outperforms other state-of-the-art methods and significantly improves the detection rate of unknown attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Yang J, Chen X, Chena S, Jiang X, Tan X (2021) Conditional variational auto-encoder and extreme value theory aided two-stage learning approach for intelligent fine-grained known/unknown intrusion detection. IEEE Transactions on Information Forensics and Security

  2. Kan X, Fan Y, Fang Z, Cao L, Xiong NN, Yang D, Li X (2021) A novel iot network intrusion detection approach based on adaptive particle swarm optimization convolutional neural network. Inform Sci 568:147–162

    MathSciNet  Google Scholar 

  3. Khan IA, Pi D, Khan N, Khan ZU, Hussain Y, Nawaz A, Ali F (2021) A privacy-conserving framework based intrusion detection method for detecting and recognizing malicious behaviours in cyber-physical power networks. Appl Intell 51(10):7306–7321

    Google Scholar 

  4. Kravchik M, Shabtai A (2021) Efficient cyber attack detection in industrial control systems using lightweight neural networks and pca. IEEE Transactions on Dependable and Secure Computing

  5. Chou D, Jiang M (2021) A survey on data-driven network intrusion detection. ACM Comput Surv (CSUR) 54(9):1–36

    Google Scholar 

  6. Çavuşoğlu Ü (2019) A new hybrid approach for intrusion detection using machine learning methods. Appl Intell 49(7):2735–2761

    Google Scholar 

  7. Feng T, Dou M (2021) A weighted intrusion detection model of dynamic selection. Appl Intell 51(7):4860–4873

    Google Scholar 

  8. Kaja N, Shaout A, Ma D (2019) An intelligent intrusion detection system. Appl Intell 49 (9):3235–3247

    Google Scholar 

  9. Lei S, Xia C, Li Z, Li X, Wang T (2021) Hnn: a novel model to study the intrusion detection based on multi-feature correlation and temporal-spatial analysis. IEEE Transactions on Network Science and Engineering

  10. Al S, Dener M (2021) Stl-hdl: a new hybrid network intrusion detection system for imbalanced dataset on big data environment. Comput Secur 110:102435

    Google Scholar 

  11. Tian Q, Han D, Li K-C, Liu X, Duan L, Castiglione A (2020) An intrusion detection approach based on improved deep belief network. Appl Intell 50(10):3162–3178

    Google Scholar 

  12. Lin K, Xu X, Xiao F (2021) Mffusion: a multi-level features fusion model for malicious traffic detection based on deep learning. Computer Networks, 108658

  13. Wang Z, Liu Y, He D, Chan S (2021) Intrusion detection methods based on integrated deep learning model. Comput Secur 103:102177

    Google Scholar 

  14. Liu A, Wang Y, Li T (2021) Sfe-gacn: a novel unknown attack detection under insufficient data via intra categories generation in embedding space. Comput Secur 105:102262

    Google Scholar 

  15. Xu C, Shen J, Du X (2020) A method of few-shot network intrusion detection based on meta-learning framework. IEEE Trans Inform Foren Secur 15:3540–3552

    Google Scholar 

  16. Li X, Chen W, Zhang Q, Wu L (2020) Building auto-encoder intrusion detection system based on random forest feature selection. Comput Secur 95:101851

    Google Scholar 

  17. Binbusayyis A, Vaiyapuri T (2021) Unsupervised deep learning approach for network intrusion detection combining convolutional autoencoder and one-class svm. Appl Intell, 1–15

  18. Moustafa N, Keshk M, Choo K-KR, Lynar T, Camtepe S, Whitty M (2021) Dad: a distributed anomaly detection system using ensemble one-class statistical learning in edge networks. Futur Gener Comput Syst 118:240–251

    Google Scholar 

  19. Zhang Z, Liu Q, Qiu S, Zhou S, Zhang C (2020) Unknown attack detection based on zero-shot learning. IEEE Access 8:193981–193991

    Google Scholar 

  20. Fang Y, Li K, Zheng R, Liao S, Wang Y (2021) A communication-channel-based method for detecting deeply camouflaged malicious traffic. Comput Netw 197:108297

    Google Scholar 

  21. Shao G, Chen X, Zeng X, Wang L (2019) Deep learning hierarchical representation from heterogeneous flow-level communication data. IEEE Trans Inform Forens Secur 15:1525–1540

    Google Scholar 

  22. Ahmad R, Alsmadi I, Alhamdani W, Tawalbeh L (2022) A comprehensive deep learning benchmark for iot ids. Comput Secur 114:102588

    Google Scholar 

  23. Li Z, Rios ALG, Trajković L (2021) Machine learning for detecting anomalies and intrusions in communication networks. IEEE Journal on Selected Areas in Communications

  24. Elmasry W, Akbulut A, Zaim AH (2020) Evolving deep learning architectures for network intrusion detection using a double pso metaheuristic. Comput Netw 168:107042

    Google Scholar 

  25. Chen A, Fu Y, Zheng X et al (2022) An efficient network behavior anomaly detection using a hybrid dbn-lstm network. Computers & Security, 102600

  26. Zhao J, Liu X, Yan Q, Li B, Shao M, Peng H (2020) Multi-attributed heterogeneous graph convolutional network for bot detection. Inform Sci 537:380–393

    Google Scholar 

  27. Wang W, Du X, Shan D, Qin R, Wang N (2020) Cloud intrusion detection method based on stacked contractive auto-encoder and support vector machine. IEEE Transactions on Cloud Computing

  28. Xie J, Li S, Yun X, Zhang Y, Chang P (2020) Hstf-model: an http-based trojan detection model via the hierarchical spatio-temporal features of traffics. Comput Secur 96:101923

    Google Scholar 

  29. Kanna PR, Santhi P (2021) Unified deep learning approach for efficient intrusion detection system using integrated spatial–temporal features. Knowl-Based Syst 226:107132

    Google Scholar 

  30. Hassan MM, Gumaei A, Alsanad A, Alrubaian M, Fortino G (2020) A hybrid deep learning model for efficient intrusion detection in big data environment. Inform Sci 513:386–396

    Google Scholar 

  31. Shang L, Guo D, Ji Y, Li Q (2021) Discovering unknown advanced persistent threat using shared features mined by neural networks. Comput Netw 189:107937

    Google Scholar 

  32. Zhang J, Ling Y, Fu X, Yang X, Xiong G, Zhang R (2020) Model of the intrusion detection system based on the integration of spatial-temporal features. Comput Secur 89:101681

    Google Scholar 

  33. Bedi P, Gupta N, Jindal V (2021) I-siamids: an improved siam-ids for handling class imbalance in network-based intrusion detection systems. Appl Intell 51(2):1133–1151

    Google Scholar 

  34. Cui J, Zong L, Xie J, Tang M (2022) A novel multi-module integrated intrusion detection system for high-dimensional imbalanced data. Appl Intell, 1–17

  35. Luo Z, He K, Yu Z (2022) A robust unsupervised anomaly detection framework. Appl Intell 52(6):6022–6036

    Google Scholar 

  36. Odiathevar M, Seah WK, Frean M, Valera A (2021) An online offline framework for anomaly scoring and detecting new traffic in network streams. IEEE Transactions on Knowledge and Data Engineering

  37. Zhong Y, Chen W, Wang Z, Chen Y, Wang K, Li Y, Yin X, Shi X, Yang J, Li K (2020) Helad: a novel network anomaly detection model based on heterogeneous ensemble learning. Comput Netw 169:107049

    Google Scholar 

  38. Marteau P-F (2021) Random partitioning forest for point-wise and collective anomaly detection—application to network intrusion detection. IEEE Trans Inform Forens Secur 16:2157–2172

    Google Scholar 

  39. Camacho J, Macia-Fernandez G, Fuentes-García NM, Saccenti E (2019) Semi-supervised multivariate statistical network monitoring for learning security threats. IEEE Trans Inform Forens Secur 14 (8):2179–2189

    Google Scholar 

  40. Kamarudin MH, Maple C, Watson T, Safa NS (2017) A logitboost-based algorithm for detecting known and unknown web attacks. IEEE Access 5:26190–26200

    Google Scholar 

  41. Andresini G, Appice A, Malerba D (2021) Autoencoder-based deep metric learning for network intrusion detection. Inform Sci 569:706–727

    MathSciNet  Google Scholar 

  42. Abdelnabi S, Krombholz K, Fritz M (2020) Visualphishnet: zero-day phishing website detection by visual similarity. In: Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pp 1681–1698

  43. Zhou X, Liang W, Shimizu S, Ma J, Jin Q (2020) Siamese neural network based few-shot learning for anomaly detection in industrial cyber-physical systems. IEEE Trans Industr Inform 17(8):5790–5798

    Google Scholar 

  44. George A, Marcel S (2020) Learning one class representations for face presentation attack detection using multi-channel convolutional neural networks. IEEE Trans Inform Forens Secur 16:361–375

    Google Scholar 

  45. Zhang M, Cheng Q, Luo F, Ye L (2021) A triplet nonlocal neural network with dual-anchor triplet loss for high-resolution remote sensing image retrieval. IEEE J Select Topics Appl Earth Observ Rem Sens 14:2711–2723

    Google Scholar 

  46. Wang W, Jian S, Tan Y, Wu Q, Huang C (2022) Representation learning-based network intrusion detection system by capturing explicit and implicit feature interactions. Comput Secur 112:102537

    Google Scholar 

  47. Alazzam H, Sharieh A, Sabri KE (2022) A lightweight intelligent network intrusion detection system using ocsvm and pigeon inspired optimizer. Appl Intell 52(4):3527–3544

    Google Scholar 

  48. Wang G, Han H, Shan S, Chen X (2020) Unsupervised adversarial domain adaptation for cross-domain face presentation attack detection. IEEE Trans Inform Forens Secur 16:56–69

    Google Scholar 

  49. Feng P, Fu J, Ge Z, Wang H, Zhou Y, Zhou B, Wang Z (2022) Unsupervised semantic-aware adaptive feature fusion network for arrhythmia detection. Inform Sci 582:509–528

    MathSciNet  Google Scholar 

  50. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. Adv Neural Inform Process Syst, 30

  51. Shiravi A, Shiravi H, Tavallaee M, Ghorbani AA (2012) Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput Secur 31(3):357–374

    Google Scholar 

  52. Moustafa N, Slay J Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In: 2015 military communications and information systems conference (MilCIS), pp 1–6 (2015). IEEE

  53. Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108–116

    Google Scholar 

  54. Garcia S, Grill M, Stiborek J, Zunino A (2014) An empirical comparison of botnet detection methods. Comput Secur 45:100–123

    Google Scholar 

  55. Pontes CF, de Souza MM, Gondim JJ, Bishop M, Marotta MA (2021) A new method for flow-based network intrusion detection using the inverse potts model. IEEE Trans Netw Serv Manag 18(2):1125–1136

    Google Scholar 

  56. Li X, Hu Z, Xu M, Wang Y, Ma J (2021) Transfer learning based intrusion detection scheme for internet of vehicles. Inform Sci 547:119–135

    Google Scholar 

  57. Nicolau M, McDermott J, et al. (2018) Learning neural representations for network anomaly detection. IEEE Trans Cybern 49(8):3074–3087

    Google Scholar 

Download references

Acknowledgments

This work was supported by the 2020 Industrial Internet Innovation and Development Project-the Key Project of Intelligent Connected Vehicle Safety Inspection Platform (Tender No.TC200H01S) and the Opening Project of Shanghai Trusted Industrial Control Platform(TICPSH202003020-ZC).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Beihang University, Beijing, China

    Jinghong Lan, Xudong Liu & Bo Li

  2. Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, China

    Jinghong Lan, Xudong Liu & Bo Li

  3. School of Information Science and Engineering, Shandong Normal University, Jinan, China

    Jun Zhao

Authors
  1. Jinghong Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xudong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jinghong Lan: Conceptualization, Methodology, Validation, Software, Investigation, Data curation, Writing - original draft, Writing - review & editing. Xudong Liu: Supervision, Resources, Carrying out additional analyses, Writing - review & editing. Bo Li: Funding acquisition, Supervision, Project administration. Jun Zhao: Writing - review & editing.

Corresponding authors

Correspondence to Jinghong Lan, Xudong Liu or Bo Li.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lan, J., Liu, X., Li, B. et al. A novel hierarchical attention-based triplet network with unsupervised domain adaptation for network intrusion detection. Appl Intell 53, 11705–11726 (2023). https://doi.org/10.1007/s10489-022-04076-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04076-0

Keywords

Search

Navigation