A Large-Scale Markov Game Approach to Dynamic Protection of Interdependent Infrastructure Networks

  • Linan HuangEmail author
  • Juntao Chen
  • Quanyan Zhu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10575)


The integration of modern information and communication technologies (ICTs) into critical infrastructures (CIs) improves its connectivity and functionalities yet also brings cyber threats. It is thus essential to understand the risk of ICTs on CIs holistically as a cyber-physical system and design efficient security hardening mechanisms. To this end, we capture the system behaviors of the CIs under malicious attacks and the protection strategies by a zero-sum game. We further propose a computationally tractable approximation for large-scale networks which builds on the factored graph that exploits the dependency structure of the nodes of CIs and the approximate dynamic programming tools for stochastic Markov games. This work focuses on a localized information structure and the single-controller game solvable by linear programming. Numerical results illustrate the proper tradeoff of the approximation accuracy and computation complexity in the new design paradigm and show the proactive security at the time of unanticipated attacks.


  1. 1.
    Chen, J., Zhu, Q.: Interdependent network formation games with an application to critical infrastructures. In: American Control Conference (ACC), pp. 2870–2875 (2016)Google Scholar
  2. 2.
    Chen, J., Zhu, Q.: Optimal contract design under asymmetric information for cloud-enabled internet of controlled things. In: Zhu, Q., Alpcan, T., Panaousis, E., Tambe, M., Casey, W. (eds.) GameSec 2016. LNCS, vol. 9996, pp. 329–348. Springer, Cham (2016). doi: 10.1007/978-3-319-47413-7_19 Google Scholar
  3. 3.
    Chen, J., Zhu, Q.: Resilient and decentralized control of multi-level cooperative mobile networks to maintain connectivity under adversarial environment. In: IEEE Conference on Decision and Control (CDC), pp. 5183–5188 (2016)Google Scholar
  4. 4.
    Chen, J., Zhu, Q.: Security as a service for cloud-enabled internet of controlled things under advanced persistent threats: a contract design approach. IEEE Trans. Inf. Forensics Secur. 12(11), 2736–2750 (2017)CrossRefGoogle Scholar
  5. 5.
    De Farias, D.P., Van Roy, B.: On constraint sampling in the linear programming approach to approximate dynamic programming. Math. Oper. Res. 29(3), 462–478 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Guestrin, C., Koller, D., Parr, R., Venkataraman, S.: Efficient solution algorithms for factored MDPs. J. Artif. Intell. Res. 19, 399–468 (2003)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Huang, L., Chen, J., Zhu, Q.: A factored MDP approach to optimal mechanism design for resilient large-scale interdependent critical infrastructures. In: Proceedings of 2017 Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES), CPS Week, 18–21 April 2017, Pittsburgh, PA, USA (2017)Google Scholar
  8. 8.
    Korkali, M., Veneman, J.G., Tivnan, B.F., Hines, P.D.: Reducing cascading failure risk by increasing infrastructure network interdependency. arXiv preprint arXiv:1410.6836 (2014)
  9. 9.
    Lee II, E.E., Mitchell, J.E., Wallace, W.A.: Restoration of services in interdependent infrastructure systems: a network flows approach. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 37(6), 1303–1317 (2007)CrossRefGoogle Scholar
  10. 10.
    Li, L., Shamma, J.: LP formulation of asymmetric zero-sum stochastic games. In: 2014 IEEE 53rd Annual Conference on Decision and Control (CDC), pp. 1930–1935. IEEE (2014)Google Scholar
  11. 11.
    Malek, A., Abbasi-Yadkori, Y., Bartlett, P.: Linear programming for large-scale Markov decision problems. In: International Conference on Machine Learning, pp. 496–504 (2014)Google Scholar
  12. 12.
    Manshaei, M.H., Zhu, Q., Alpcan, T., Bacşar, T., Hubaux, J.P.: Game theory meets network security and privacy. ACM Comput. Surv. (CSUR) 45(3), 25 (2013)CrossRefzbMATHGoogle Scholar
  13. 13.
    Monga, A., Zhu, Q.: On solving large-scale low-rank zero-sum security games of incomplete information. In: 2016 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6. IEEE (2016)Google Scholar
  14. 14.
    Ouyang, M.: Review on modeling and simulation of interdependent critical infrastructure systems. Reliab. Eng. Syst. Saf. 121, 43–60 (2014)CrossRefGoogle Scholar
  15. 15.
    Pawlick, J., Farhang, S., Zhu, Q.: Flip the cloud: cyber-physical signaling games in the presence of advanced persistent threats. In: Khouzani, M.H.R., Panaousis, E., Theodorakopoulos, G. (eds.) GameSec 2015. LNCS, vol. 9406, pp. 289–308. Springer, Cham (2015). doi: 10.1007/978-3-319-25594-1_16 CrossRefGoogle Scholar
  16. 16.
    Pederson, P., Dudenhoeffer, D., Hartley, S., Permann, M.: Critical infrastructure interdependency modeling: a survey of US and international research. Ida. Nat. Lab. 25, 27 (2006)Google Scholar
  17. 17.
    Rinaldi, S.M., Peerenboom, J.P., Kelly, T.K.: Identifying, understanding, and analyzing critical infrastructure interdependencies. IEEE Control Syst. 21(6), 11–25 (2001)CrossRefGoogle Scholar
  18. 18.
    Rosato, V., Issacharoff, L., Tiriticco, F., Meloni, S., Porcellinis, S., Setola, R.: Modelling interdependent infrastructures using interacting dynamical models. Int. J. Crit. Infrastruct. 4(1–2), 63–79 (2008)CrossRefGoogle Scholar
  19. 19.
    Zhu, Q., Başar, T.: Game-theoretic approach to feedback-driven multi-stage moving target defense. In: Das, S.K., Nita-Rotaru, C., Kantarcioglu, M. (eds.) GameSec 2013. LNCS, vol. 8252, pp. 246–263. Springer, Cham (2013). doi: 10.1007/978-3-319-02786-9_15 CrossRefGoogle Scholar
  20. 20.
    Zhu, Q., Basar, T.: Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems: games-in-games principle for optimal cross-layer resilient control systems. IEEE Control Syst. 35(1), 46–65 (2015)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Zhu, Q., Fung, C., Boutaba, R., Basar, T.: Guidex: a game-theoretic incentive-based mechanism for intrusion detection networks. IEEE J. Sel. Areas Commun. 30(11), 2220–2230 (2012)CrossRefGoogle Scholar
  22. 22.
    Zhu, Q., Li, H., Han, Z., Basar, T.: A stochastic game model for jamming in multi-channel cognitive radio systems. In: 2010 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2010)Google Scholar
  23. 23.
    Zhu, Q., Tembine, H., Başar, T.: Network security configurations: a nonzero-sum stochastic game approach. In: American Control Conference (ACC), pp. 1059–1064. IEEE (2010)Google Scholar
  24. 24.
    Zimmerman, R., Zhu, Q., De Leon, F., Guo, Z.: Conceptual modeling framework to integrate resilient and interdependent infrastructure in extreme weather. J. Infrastruct. Syst. 23, 04017034 (2017)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringNew York UniversityBrooklynUSA

Personalised recommendations