Skip to main content
SpringerLink
Log in

Efficient and interpretable SRU combined with TabNet for network intrusion detection in the big data environment

  • Regular contribution
  • Published:
International Journal of Information Security Aims and scope Submit manuscript

Abstract

While digital application infrastructure services are becoming increasingly abundant and the scale of the network continues to expand, many new network vulnerabilities and attacks (such as DoS, Botnet, and MITM) have emerged in an endless stream. The timely and accurate detection of network anomalies is of extraordinary importance for the stability of the network. Previous works designed based on deep learning have faced difficulties in their adoption in practice due to the lack of interpretability. Recently, Recurrent Neural Networks perform a superior ability to analyze high-dimensional complex network flow. However, these methods have the problems of limited parallelizability and time-consuming training, so they cannot meet the particular requirements of intrusion detection. To solve the above issues, we propose an efficient and interpretable intrusion detection scheme based on simple recurrent networks (Tab-AttSRU) to identify abnormal network traffic patterns accurately. Concretely, to obtain high-quality interpretation, we utilize model-specific feature importance and a learnable mask of TabNet for soft selection. The sequential attention mechanism is used to select the decision-making features for necessary interpretability. To realize efficient parallel computing, we combine SRU with attention mechanism to capture latent connections between traffic at different times and implement it on Spark. The performance of proposed method is assessed on the benchmark UNSW-NB15 and a real-world dataset UKM-IDS20. Experimental results have demonstrated the efficiency and interpretability of proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability statement

The datasets generated during and/or analyzed during the current study are not publicly available due to privacy but are available from the corresponding author on reasonable request.

References

  1. Gamage, S., Samarabandu, J.: Deep learning methods in network intrusion detection: a survey and an objective comparison. J. Netw. Comput. Appl. 169, 102767 (2020). https://doi.org/10.1016/j.jnca.2020.102767

    Article  Google Scholar 

  2. Qassim, Qais, Jamil, Norziana, Daud, Maslina, Patel, Ahmed, Ja’affar, Norhamadi: A review of security assessment methodologies in industrial control systems. Inf. Comput. Secur. 27(1), 47–61 (2019). https://doi.org/10.1108/ICS-04-2018-0048

    Article  Google Scholar 

  3. Patel, A., Alhussian, H., Pedersen, J.M., Bounabat, B., Junior, J.C., Katsikas, S.: A nifty collaborative intrusion detection and prevention architecture for smart grid ecosystems. Comput. Secur. (COSE) 64(C), 92–109 (2019). https://doi.org/10.1016/j.cose.2016.07.002

    Article  Google Scholar 

  4. Yang, C.: Anomaly network traffic detection algorithm based on information entropy measurement under the cloud computing environment. Cluster Comput. 22, 8309–8317 (2019). https://doi.org/10.1007/s10586-018-1755-5

    Article  Google Scholar 

  5. Althobaiti, M.M., Kumar, K.P.M., Gupta, D., Kumar, S., Mansour, R.F.: An intelligent cognitive computing based intrusion detection for industrial cyber-physical systems. Measurement 186, 110145 (2021). https://doi.org/10.1016/j.measurement.2021.110145

    Article  Google Scholar 

  6. Patel, A., Taghavi, M., Bakhtiyari, K., Junior, J.C.: An intrusion detection and prevention system in cloud computing: a systematic review. J. Netw. Comput. Appl. 36(1), 25–41 (2013). https://doi.org/10.1016/j.jnca.2012.08.007

    Article  Google Scholar 

  7. Venturi, A., Zanasi, C.: On the feasibility of adversarial machine learning in malware and network intrusion detection. In: 2021 IEEE 20th International Symposium on Network Computing and Applications (NCA), pp. 1-8. IEEE (2021). https://doi.org/10.1109/NCA53618.2021.9685709

  8. Riyaz, B., Ganapathy, S.: A deep learning approach for effective intrusion detection in wireless networks using CNN. Soft Comput. 24, 17265–17278 (2020). https://doi.org/10.1007/s00500-020-05017-0

    Article  Google Scholar 

  9. Sohn, I.: Deep belief network based intrusion detection techniques: a survey. Expert Syst. Appl. 167, 114170 (2021). https://doi.org/10.1016/j.eswa.2020.114170

    Article  Google Scholar 

  10. Hu, W., Fu, Z., Guo, Z.: Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting. IEEE Trans. Image Process. 28(8), 4087–4100 (2019). https://doi.org/10.1109/TIP.2019.2906554

    Article  MathSciNet  MATH  Google Scholar 

  11. Deore, B., Bhosale, S.: Intrusion detection system based on RNN classifier for feature reduction. SN Comput. Sci. 3, 114 (2022). https://doi.org/10.1007/s42979-021-00991-0

    Article  Google Scholar 

  12. Liang, Y., Li, S., Yan, C., Li, M., Jiang, C.: Explaining the black-box model: a survey of local interpretation methods for deep neural networks. Neurocomputing 419, 168–182 (2021). https://doi.org/10.1016/j.neucom.2020.08.011

    Article  Google Scholar 

  13. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol. 8689. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

  14. Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5188–5196 (2015)

  15. Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. arXiv:1506.06579 (2015)

  16. Zhang, Q., Wu, Y. N., Zhu, S. C.: Interpretable convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8827–8836 (2018)

  17. Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6541–6549 (2017)

  18. Ribeiro, M.T., Singh, S., Guestrin, C.: “ Why should i trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016). https://doi.org/10.1145/2939672.2939778

  19. Guo, W., Mu, D., Xu, J., Su, P., Wang, G., Xing, X.: Lemna: explaining deep learning based security applications. In: proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 364–379 (2018). https://doi.org/10.1145/3243734.3243792

  20. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)

  21. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv:1312.4400 (2013)

  22. Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is object localization for free?-weakly-supervised learning with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 685–694 (2015)

  23. Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1713–1721 (2015)

  24. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)

  25. Yu, J., Ye, X., Li, H.: A high precision intrusion detection system for network security communication based on multi-scale convolutional neural network. Future Gener. Comput. Syst. 129, 399–406 (2022). https://doi.org/10.1016/j.future.2021.10.018

    Article  Google Scholar 

  26. Moizuddin, M.D., Jose, M.V.: A bio-inspired hybrid deep learning model for network intrusion detection. Knowl.-Based Syst. 238, 107894 (2022). https://doi.org/10.1016/j.knosys.2021.107894

    Article  Google Scholar 

  27. Yin, C., Zhu, Y., Fei, J., He, X.: A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access 5, 21954–21961 (2017). https://doi.org/10.1109/ACCESS.2017.2762418

    Article  Google Scholar 

  28. Albahar, M.A.: Recurrent neural network model based on a new regularization technique for real-time intrusion detection in SDN environments. Secur. Commun. Netw. (2019). https://doi.org/10.1155/2019/8939041

    Article  Google Scholar 

  29. Zhou, X., Hu, Y., Liang, W., Ma, J., Jin, Q.: Variational LSTM enhanced anomaly detection for industrial big data. IEEE Trans. Ind. Inform. 17(5), 3469–3477 (2020). https://doi.org/10.1109/TII.2020.3022432

    Article  Google Scholar 

  30. Singh, N.B., Singh, M.M., Sarkar, A., Mandal, J.K.: A novel wide and deep transfer learning stacked GRU framework for network intrusion detection. J. Inf. Secur. Appl. 61, 102899 (2021). https://doi.org/10.1016/j.jisa.2021.102899

    Article  Google Scholar 

  31. Arik, S.Ö., Pfister, T.: Tabnet: attentive interpretable tabular learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, No. 8, pp. 6679–6687 (2021). https://ojs.aaai.org/index.php/AAAI/article/view/16826

  32. Moustafa, N., Slay, J.: UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military Communications and Information Systems Conference (MilCIS), pp. 1–6 (2015). https://doi.org/10.1109/MilCIS.2015.7348942

  33. Mahfouz, A., Abuhussein, A., Venugopal, D., Shiva, S.: Ensemble classifiers for network intrusion detection using a novel network attack dataset. Future Internet 12(11), 180 (2020). https://doi.org/10.3390/fi12110180

  34. Al-Daweri, M.S., Abdullah, S., Ariffin, K.A.Z.: An adaptive method and a new dataset, UKM-IDS20, for the network intrusion detection system. Comput. Commun. 180, 57–76 (2021). https://doi.org/10.1016/j.comcom.2021.09.007

  35. Peng, K., Leung, V., Zheng, L., Wang, S., Huang, C., Lin, T.: Intrusion detection system based on decision tree over big data in fog environment. Wirel. Commun. Mob. Comput. (2018). https://doi.org/10.1155/2018/4680867

    Article  Google Scholar 

  36. Reddy, R.R., Ramadevi, Y., Sunitha, K.N.: Effective discriminant function for intrusion detection using SVM. In: 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1148–1153. IEEE (2016). https://doi.org/10.1109/ICACCI.2016.7732199

  37. Mughal, M.O., Kim, S.: Signal classification and jamming detection in wide-band radios using Naíve Bayes classifier. IEEE Commun. Lett. 22(7), 1398–1401 (2018). https://doi.org/10.1109/LCOMM.2018.2830769

    Article  Google Scholar 

  38. Liu, J., Gao, Y., Hu, F.: A fast network intrusion detection system using adaptive synthetic oversampling and LightGBM. Comput. Secur. 106, 102289 (2021). https://doi.org/10.1016/j.cose.2021.102289

    Article  Google Scholar 

  39. Yang, S.U.: Research on network behavior anomaly analysis based on bidirectional LSTM. In: 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 798–802. IEEE (2019). https://doi.org/10.1109/ITNEC.2019.8729475

  40. Roy, B., Cheung, H.: A deep learning approach for intrusion detection in internet of things using bi-directional long short-term memory recurrent neural network. In: 2018 28th International Telecommunication Networks and Applications Conference (ITNAC), pp. 1–6. IEEE (2018). https://doi.org/10.1109/ATNAC.2018.8615294

Download references

Funding

This work is supported by the National Natural Science Foundation of China (U1936213).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, China

    Yingchun Chen & Jinguo Li

  2. State Grid Shanghai Municipal Electric Power Company, Shanghai, China

    Naiwang Guo

Authors
  1. Yingchun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinguo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naiwang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinguo Li.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Li, J. & Guo, N. Efficient and interpretable SRU combined with TabNet for network intrusion detection in the big data environment. Int. J. Inf. Secur. 22, 679–689 (2023). https://doi.org/10.1007/s10207-022-00656-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10207-022-00656-w

Keywords

Search

Navigation