Abstract
Intrusion Detection Systems (IDS) have been investigated for many years and the field has matured. Nevertheless, there are still important challenges, e.g., how an IDS can detect new and complex distributed attacks. To tackle these problems, we propose a distributed Reinforcement Learning (RL) approach in a hierarchical architecture of network sensor agents. Each network sensor agent learns to interpret local state observations, and communicates them to a central agent higher up in the agent hierarchy. These central agents, in turn, learn to send signals up the hierarchy, based on the signals that they receive. Finally, the agent at the top of the hierarchy learns when to signal an intrusion alarm. We evaluate our approach in an abstract network domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A.G., Mahadevan, S.: Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems 13(4), 341–379 (2003)
Barford, P., Jha, S., Yegneswaran, V.: Fusion and Filtering in Distributed Intrusion Detection Systems. In: Proceedings of the 42nd Annual Allerton Conference on Communication, Control and Computing (September 2004)
Bass, T.: Intrusion Detection Systems and Multisensor Data Fusion. Communications of the ACM 43(4), 99–105 (2000)
Chang, T.H., Kaelbling, L.: All learning is local: Multi-agent learning in global reward games. In: Advances in NIPS, vol. 14 (2004)
Elfwing, S., Uchibe, E., Doya, K., Christensen, H.I.: Multi-agent reinforcement learning: using macro actions to learn a mating task. In: IROS 2004. Intelligent Robots and Systems (2004)
Jennings, N., Sycara, K., Wooldridge, M.: A roadmap of agents research and development. Autonomous Agents and Multi-Agent Systems 1, 7–38 (1998) In: [12]
Kapetanakis, S., Kudenko, D., Strens, M.: Learning to coordinate using commitment sequences in cooperative multi-agent systems. In: AISB 2003. Proceedings of the Third Symposium on Adaptive Agents and Multi-agent Systems, Society for the study of Artificial Intelligence and Simulation of Behaviour (2003)
Kostiadis, K., Hu, H.: KaBaGe-RL: Kanerva-based generalisation and reinforcement learning for possession football. In: IROS 2001. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2001)
Mirkovic, J., Reiher, P.: A taxonomy of DDoS attack and DDoS defense mechanisms. ACM SIGCOMM Computer Communication Review 34(2) (April 2004)
Moore, D., Shannon, C., Voelker, G.M., Savage, S.: Internet Quarantine: Requirements for Containing Self-Propagating Code. In: INFOCOM 2003. 22th Joint Conference of the IEEE Computer and Communications Societies, March 30- April 3, 2003, vol. 3, pp. 1901–1910 (2003)
Neumann, P.G., Porras, P.A.: Experience with EMERALD to DATE. In: 1st USENIX Workshop on Intrusion Detection and Network Monitoring, Santa Clara, California (April 11-12, 1999)
Panait, L., Luke, S.: Cooperative Multi-Agent Learning: The State of the Art. Autonomous Agents and Multi-Agent Systems 11(3), 387–434 (2005)
Porta, J., Celaya, E.: Reinforcement Learning for Agents with Many Sensors and Actuators Acting in Categorizable Environments. Journal of Artificial Intelligence Research 23, 79–122 (2005)
Powers, R., Shoham, Y.: New criteria and a new algorithm for learning in multi-agent systems. In: Advances in Neural Information Processing Systems (forthcoming), Rubinstein, A.: Modeling Bounded Rationality. MIT Press, Washington (1998)
Sen, S., Weiss, G.: Learning in Multiagent Systems. In: Weiss, G. (ed.) Multiagent Systems, A Modern Approach to Distributed Artificial Intelligence, pp. 259–298. MIT Press, Cambridge (1999)
Siaterlis, C., Maglaris, B.: Towards multisensor data fusion for DoS detection. In: Proceedings of the 2004 ACM Symposium on Applied Computing, pp. 439–446 (2004)
Stone, P., Sutton, R.S., Singh, S.: Reinforcement Learning for 3 vs. 2 Keepaway. In: Stone, P., Balch, T., Kreatzschmarr, G. (eds.) RoboCup-2000: Robot Soccer World Cup IV, Springer, Berlin (2001)
Sutton, R., Barto, A.: Reinforcement Learning, An Introduction. MIT Press, Cambridge (1998)
Wasniowski, R.A.: Multisensor Agent Based Intrusion Detection. Transactions on Engineering, Computing and Technology 5, 110–113 (2005)
Yegneswaran, V., Barford, P., Jha, S.: Global Intrusion Detection in the DOMINO Overlay System. In: Proceedings of the Network and Distributed System Security Symposium (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Servin, A., Kudenko, D. (2008). Multi-agent Reinforcement Learning for Intrusion Detection. In: Tuyls, K., Nowe, A., Guessoum, Z., Kudenko, D. (eds) Adaptive Agents and Multi-Agent Systems III. Adaptation and Multi-Agent Learning. AAMAS ALAMAS ALAMAS 2005 2007 2006. Lecture Notes in Computer Science(), vol 4865. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77949-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-77949-0_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77947-6
Online ISBN: 978-3-540-77949-0
eBook Packages: Computer ScienceComputer Science (R0)