Abstract
Network-based technologies such as cloud computing, web services, and Internet of Things systems are becoming widely used due to their flexibility and preeminence. On the other hand, the exponential proliferation of network-based technologies exacerbated network security concerns. Intrusion takes an important share in the security concerns surrounding network-based technologies. Developing a robust intrusion detection system is crucial to solving the intrusion problem and ensuring the secure delivery of network-based technologies and services. In this paper, we propose a novel approach using deep reinforcement learning to detect intrusions to make network applications more secure, reliable, and efficient. As for the reinforcement learning approach, Deep Q-learning is used alongside a custom-built Gym environment that mimics network attacks and guides the learning process. The NSL-KDD dataset is used to create the reinforcement learning environment to train and evaluate the proposed model. The experimental results show that our proposed reinforcement learning approach outperforms other related solutions in the literature, achieving an accuracy that exceeds 93%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cloud infrastructure market share. https://www.statista.com/chart/ 7994/cloud-market-share/ (2021)
Cloud infrastructure market share. https://www.statista.com/chart/ 18819/worldwide-market-share-of-leading-cloud-infrastructure-service-providers/ (2021)
Armin, J., Thompson, B., Ariu, D., Giacinto, G., Roli, F., Kijewski, P.: 2020 cybercrime economic costs: no measure no solution. In: 2015 10th International Conference on Availability, Reliability and Security, pp. 701–710. IEEE (2015)
The hidden costs of cybercrime. https://www.csis.org/analysis/hidden-costs-cybercrime (2021)
Scarfone, K.A., Mell, P.M.: Sp 800-94, guide to intrusion detection and prevention systems (IDPS). Tech. Rep., Gaithersburg, MD, USA (2007)
Roesch, M., et al.: Snort: Lightweight intrusion detection for networks. In: Lisa, vol. 99, pp. 229–238 (1999)
Brockman, G., et al.: OpenAI Gym. arXiv preprint arXiv:1606.01540 (2016)
Sharafaldin, I., Gharib, A., Lashkari, A.H., Ghorbani, A.A.: Towards a reliable intrusion detection benchmark dataset. Softw. Netw. 2018(1), 177–200 (2018)
Liang, C., Shanmugam, B., Azam, S., Jonkman, M., De Boer, F., Narayansamy, G.: Intrusion detection system for internet of things based on a machine learning approach. In: 2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN), pp. 1–6. IEEE (2019)
Koduvely, H.: Anomaly detection through reinforcement learning (2018). https://doi.org/10.13140/RG.2.2.33673.29283
Caminero, G., Lopez-Martin, M., Carro, B.: Adversarial environment reinforcement learning algorithm for intrusion detection. Comput. Netw. 159, 96–109 (2019)
Lopez-Martin, M., Carro, B., Sanchez-Esguevillas, A.: Application of deep reinforcement learning to intrusion detection for supervised problems. Expert Syst. Appl. 141, 112963 (2020)
Suwannalai, E., Polprasert, C.: Network intrusion detection systems using adversarial reinforcement learning with deep Q-network. In: 2020 18th International Conference on ICT and Knowledge Engineering (ICT &KE), pp. 1–7. IEEE (2020)
Hsu, Y.F., Matsuoka, M.: A deep reinforcement learning approach for anomaly network intrusion detection system. In: 2020 IEEE 9th International Conference on Cloud Networking (CloudNet), pp. 1–6 (2020). https://doi.org/10.1109/CloudNet51028.2020.9335796
Ma, X., Shi, W.: AESMOTE: adversarial reinforcement learning with SMOTE for anomaly detection. IEEE Trans. Netw. Sci. Eng. 8, 943–956 (2020)
Hougen, D.F., Shah, S.N.H.: The evolution of reinforcement learning. In: 2019 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1457–1464. IEEE (2019)
Qiang, W., Zhongli, Z.: Reinforcement learning model, algorithms and its application. In: 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 1143–1146. IEEE (2011)
5 things you need to know about reinforcement learning. https://www.kdnuggets.com/2018/03/5-things-reinforcement-learning.html (2021)
Yildiz, B.: Reinforcement learning using fully connected, attention, and transformer models in knapsack problem solving. Concurrency and Computation: Practice and Experience (2021). https://doi.org/10.1002/cpe.6509
A hands-on introduction to deep Q-learning using openAI GYM in python. https://www.analyticsvidhya.com/blog/2019/04/introduction- deep-q-learning-python/ (2021)
NSL-KDD dataset. https://www.unb.ca/cic/datasets/nsl.html (2021)
Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD cup 99 data set. In: 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, pp. 1–6. IEEE (2009)
Vihinen, M.: How to evaluate performance of prediction methods? measures and their interpretation in variation effect analysis. In: BMC genomics, vol. 13, pp. 1–10. BioMed Central (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Saad, A.M.S.E., Yildiz, B. (2023). Reinforcement Learning for Intrusion Detection. In: García Márquez, F.P., Jamil, A., Eken, S., Hameed, A.A. (eds) Computational Intelligence, Data Analytics and Applications. ICCIDA 2022. Lecture Notes in Networks and Systems, vol 643. Springer, Cham. https://doi.org/10.1007/978-3-031-27099-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-27099-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27098-7
Online ISBN: 978-3-031-27099-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)