# Game Theoretical Model for Adaptive Intrusion Detection System

## Abstract

We present a self-adaptation mechanism for network intrusion detection system based on the use of game-theoretical formalism. The key innovation of our method is a secure runtime definition and solution of the game and real-time use of game solutions for immediate system reconfiguration. Our approach is suited for realistic environments where we typically lack any ground truth information regarding traffic legitimacy/maliciousness and where the significant portion of system inputs may be shaped by the attacker in order to render the system ineffective. Therefore, we rely on the concept of challenge insertion: we inject a small sample of simulated attacks into the unknown traffic and use the system response to these attacks to define the game structure and utility functions. This approach is also advantageous from the security perspective, as the manipulation of the adaptive process by the attacker is far more difficult.

## Keywords

Utility Function Nash Equilibrium Pure Strategy Solution Concept Intrusion Detection System## Notes

### Acknowledgment

This material is based upon work supported by the ITC-A of the US Army under Contract No. W911NF-10-1-0070. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the ITC-A of the US Army. Also supported by Czech Ministry of Education grants 6840770038 and AMVIS-AnomalyNET. Also supported by MVČR Grant number VG2VS/242.

## References

- 1.Kayacik, H.G., Zincir-Heywood, A.N.: Mimicry attacks demystified: what can attackers do to evade detection? In: Annual Conference on Privacy, Security and Trust, pp. 213–223 (2008)Google Scholar
- 2.Rubinstein, B.I.P., Nelson, B., Huang, L., Joseph, A.D., Lau, S., Taft, N., Tygar, J.D.: Evading anomaly detection through variance injection attacks on PCA. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 394–395. Springer, Heidelberg (2008) CrossRefGoogle Scholar
- 3.Barreno, M., Nelson, B., Sears, R., Joseph, A.D., Tygar, J.D.: Can machine learning be secure? In: ASIACCS ’06: Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, pp. 16–25. ACM, New York (2006)Google Scholar
- 4.Rehák, M., Staab, E., Fusenig, V., Pěchouček, M., Grill, M., Stiborek, J., Bartoš, K., Engel, T.: Runtime monitoring and dynamic reconfiguration for intrusion detection systems. In: Kirda, E., Jha, S., Balzarotti, D. (eds.) RAID 2009. LNCS, vol. 5758, pp. 61–80. Springer, Heidelberg (2009) CrossRefGoogle Scholar
- 5.Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V.V.: Algorithmic Game Theory. Cambridge University Press, New York (2007)CrossRefzbMATHGoogle Scholar
- 6.Blum, A., Mansour, Y.: Learning, regret minimization and equilibria. In: Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V. (eds.) Algorithmic Game Theory, pp. 79–101. Cambridge University Press, New York (2007)CrossRefGoogle Scholar
- 7.Alpcan, T., Başar, T.: A game theoretic approach to decision and analysis in network intrusion detection. In: Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, HI, pp. 2595–2600, December 2003Google Scholar
- 8.Alpcan, T., Başar, T.: An intrusion detection game with limited observations. In: 12th International Symposium on Dynamic Games and Applications, Sophia Antipolis, France, July 2006Google Scholar
- 9.Liu, Y., Comaniciu, C., Man, H.: A bayesian game approach for intrusion detection in wireless ad hoc networks. In: GameNets ’06: Proceeding from the 2006 Workshop on Game Theory for Communications and Networks, p. 4. ACM, New York (2006)Google Scholar
- 10.Chen, L., Leneutre, J.: A game theoretical framework on intrusion detection in heterogeneous networks. IEEE Trans. Inf. Forensics Secur.
**4**(2), 165–178 (2009)CrossRefGoogle Scholar - 11.Zhu, Q., Basar, T.: Dynamic policy-based IDS configuration. In: Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference, pp. 8600–8605 (2009)Google Scholar
- 12.Jain, M., Pita, J., Tambe, M., Ordónez, F., Paruchuri, P., Kraus, S.: Bayesian stackelberg games and their application for security at Los Angeles international airport. SIGecom Exch.
**7**(2), 1–3 (2008)CrossRefGoogle Scholar - 13.Becker, G.S.: Crime and punishment: an economic approach. J. Polit. Econ.
**76**(2), 169–217 (1968)CrossRefGoogle Scholar - 14.Ptacek, T.H., Newsham, T.N.: Insertion, evasion, and denial of service: eluding network intrusion detection. Technical report, Secure Networks Inc., Suite 330, 1201 5th Street S.W., Calgary, Alberta, Canada, T2R–0Y6 (1998)Google Scholar
- 15.Porter, R., Nudelman, E., Shoham, Y.: Simple search methods for finding a nash equilibrium. Games Econ. Behav.
**63**(2), 642–662 (2008)CrossRefMathSciNetzbMATHGoogle Scholar - 16.Wagener, G., State, R., Dulaunoy, A., Engel, T.: Self adaptive high interaction honeypots driven by game theory. In: Guerraoui, R., Petit, F. (eds.) SSS 2009. LNCS, vol. 5873, pp. 741–755. Springer, Heidelberg (2009) CrossRefGoogle Scholar
- 17.Rehak, M., Staab, E., Pechoucek, M., Stiborek, J., Grill, M., Bartos, K.: Dynamic information source selection for intrusion detection systems. In: Decker, K.S., Sichman, J.S., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS ’09), IFAAMAS, pp. 1009–1016, May 2009Google Scholar
- 18.Rehák, M., Pechoucek, M., Grill, M., Stiborek, J., Bartoš, K., Celeda, P.: Adaptive multiagent system for network traffic monitoring. IEEE Intell. Syst.
**24**(3), 16–25 (2009)CrossRefGoogle Scholar