Skip to main content
Log in

Globalized robust Markov perfect equilibrium for discounted stochastic games and its application on intrusion detection in wireless sensor networks: Part I—theory

  • Original Paper
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript


In this article, we study a discounted stochastic game to model resource optimal intrusion detection in wireless sensor networks. To address the problem of uncertainties in various network parameters, we propose a globalized robust game-theoretic framework for discounted robust stochastic games. A robust solution to the considered problem is an optimal point that is feasible for all realizations of data from a given uncertainty set. To allow a controlled violation of the constraints when the parameters move out of the uncertainty set, the concept of globalized robust framework comes into view. In this article, we formulate a globalized robust counterpart for the discounted stochastic game under consideration. With the help of globalized robust optimization, a concept of globalized robust Markov perfect equilibrium is introduced. The existence of such an equilibrium is shown for a discounted stochastic game when the number of actions of the players is finite. The contraction mapping theorem, Kakutani fixed point theorem and the concept of equicontinuity are used to prove the existence result. To compute a globalized robust Markov perfect equilibrium for the considered discounted stochastic game, a tractable representation of the proposed globalized robust counterpart is also provided. Using the derived tractable representation, we formulate a globalized robust intrusion detection system for wireless sensor networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others


  1. Abdalzaher, M.S., Seddik, K., Elsabrouty, M., Muta, O., Furukawa, H., Abdel-Rahman, A.: Game theory meets wireless sensor networks security requirements and threats mitigation: a survey. Sensors 16(7), 1003 (2016)

    Article  Google Scholar 

  2. Agah, A., Das, S.K., Basu, K., Asadi, M.: Intrusion detection in sensor networks: a noncooperative game approach. Third IEEE Int. Sympos. Netw. Comput. Appl. 2004, 343–346 (2004)

    Google Scholar 

  3. Alpcan, T., Basar, T.: A game theoretic approach to decision and analysis in network intrusion detection. In: 42nd IEEE International Conference on Decision and Control (IEEE Cat. No. 03CH37475), Vol. 3, pp. 2595–2600 (2003)

  4. Ben-Tal, A., El Ghaoui, L., Nemirovski, A.S.: Robust Optimization. Princeton University Press, Princeton (2009)

    Book  Google Scholar 

  5. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)

    Book  Google Scholar 

  6. Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer-Verlag, New York (1996)

    Book  Google Scholar 

  7. Fink, A.M.: Equilibrium in a stochastic \(n\)-person game. J. Sci. Hiroshima Univ. Ser. A-I Math. 28(1), 89–93 (1964)

    Article  MathSciNet  Google Scholar 

  8. Fung, C.J., Zhu, Q., Boutaba, R., Baar, T.: Bayesian decision aggregation in collaborative intrusion detection networks. In: 2010 IEEE Network Operations and Management Symposium 2010, pp. 349–356 (2010)

  9. Han, L., Zhou, M., Jia, W., Dalil, Z., Xu, X.: Intrusion detection model of wireless sensor networks based on game theory and an autoregressive model. Inform. Sci. 476, 491–504 (2019)

    Article  Google Scholar 

  10. Hedengren, J.D., Shishavan, R.A., Powell, K.M., Edgar, T.F.: Nonlinear modeling, estimation and predictive control in APMonitor. Comput. Chem. Eng. 70, 133–148 (2014)

    Article  Google Scholar 

  11. Guan, S., Wang, J., Jiang, C., Tong, J., Ren, Y.: Intrusion detection for wireless sensor networks: a multi-criteria game approach. In: IEEE Wireless Communications and Networking Conference, Barcelona, pp. 1–6 (2018)

  12. Kakutani, S.: A generalization of Brouwer’s fixed point theorem. Duke Math. J. 8(3), 457–459 (1941)

    Article  MathSciNet  Google Scholar 

  13. Karde, E., Ordez, F., Hall, R.: Discounted robust stochastic games and an application to queueing control. Oper. Res. 59(2), 365–382 (2011)

    Article  MathSciNet  Google Scholar 

  14. Moosavi, H., Bui, F.M.: A game-theoretic framework for robust optimal intrusion detection in wireless sensor networks. IEEE Trans. Inform. Forensics Secur. 9(9), 1367–1379 (2014)

    Article  Google Scholar 

  15. Patil, S., Chaudhari, S.: DoS attack prevention technique in wireless sensor networks. Proc. Comput. Sci. 79, 715–721 (2016)

    Article  Google Scholar 

  16. Schirmer, H.: Conditions for the uniqueness of the fixed point in Kakutani’s theorem. Can. Math. Bull. 24(3), 351–357 (1981)

    Article  MathSciNet  Google Scholar 

  17. Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. 39(10), 1095–1100 (1953)

    Article  MathSciNet  Google Scholar 

  18. Subba, B., Biswas, S., Karmakar, S.: A game theory based multi layered intrusion detection framework for wireless sensor networks. Int. J. Wirel. Inform. Netw. 25(4), 399–421 (2018)

    Article  Google Scholar 

  19. Wang, K., Du, M., Yang, D., Zhu, C., Shen, J., Zhang, Y.: Game-theory-based active defense for intrusion detection in cyber-physical embedded systems. ACM Trans. Embedded Comput. Syst. 16(1), 18 (2016)

    Article  Google Scholar 

  20. Yu, W., Ji, Z., Liu, K.J.R.: Securing cooperative ad-hoc networks under noise and imperfect monitoring: strategies and game theoretic analysis. IEEE Trans. Inform. Forensics Secur. 2(2), 240–253 (2007)

    Article  Google Scholar 

  21. Sobel, M.J.: Noncooperative stochastic games. Ann. Math. Stat. 42(6), 1930–1935 (1971)

    Article  MathSciNet  Google Scholar 

  22. Filar, J.A., Schultz, T.A., Thuijsman, F., Vrieze, O.J.: Nonlinear programming and stationary equilibria in stochastic games. Math. Program. 50(1–3), 227–237 (1991)

    Article  MathSciNet  Google Scholar 

  23. Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Mathematical Programing Language. In: Wallace, S.W. (eds) Algorithms and Model Formulations in Mathematical Programming. NATO ASI Series (Series F: Computer and Systems Sciences), vol 51. Springer, Berlin, Heidelberg (1989)

    Chapter  Google Scholar 

  24. Byrd, R.H., Nocedal, J., Waltz, R.A.: KNITRO: An integrated package for nonlinear optimization. In: di Pillo, G., Roma, M. (eds.) Large-Scale Nonlinear Optimization, pp. 35–59. Springer, New York (2006)

    Chapter  Google Scholar 

  25. Czyzyk, J., Mesnier, M.P., Moré, J.J.: The NEOS Server. IEEE J. Comput. Sci. Eng. 5(3), 68–75 (1998) (This paper discusses the design and implementation of the NEOS Server)

    Article  Google Scholar 

Download references


The first author gratefully acknowledges the financial support through the Early Career Research Award (ECR/2015/000467), Science & Engineering Research Board, Government of India. Authors are thankful to the reviewers and the editors for their valuable comments to improve the article.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Debdas Ghosh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


Appendix A. Proof of Theorem 4.1


Let us consider two points \(\varvec{W}\) and \(\varvec{X}\) from \({\mathbb {R}}^{nm}\). Here n is the number of players and m is the number of states. For fixed \({\triangle }^{-i}_s\)\(\forall i,s\),

$$\begin{aligned} mu_s^i({\triangle }_s^{-i}, \varvec{W}^i)&= \underset{{\triangle }_s^i}{\min }~\underset{\varvec{t} \in \textit{Q}}{\max } ~~ g_s^i(\varvec{t}_s; {\triangle }_s; \varvec{W}^i) \nonumber \\&= g_s^i(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{*i},\varvec{W}^i); {\triangle }_s^{-i},{\triangle }_s^{*i}; \varvec{W}^i), \end{aligned}$$

where \({\triangle }_s^{*i}\) minimizes the function and the maximizer \(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{*i},\varvec{W}^i)\) depends on (\({\triangle }_s^{-i},{\triangle }_s^{*i}\)). We also have \({\triangle }_s^{\varvec{'}*i}\) and \(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i) \in Q\) such that

$$\begin{aligned} \mu _s^i({\triangle }_s^{-i}, \varvec{X}^i)&= \underset{{\triangle }_s^{'i}}{\min }~\underset{\varvec{t} \in \textit{Q}}{\max } ~~ g_s^i(\varvec{t}_s; {\triangle '}_s; \varvec{X}^i) \nonumber \\&= g_s^i(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i); {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i}; \varvec{X}^i). \end{aligned}$$


$$\begin{aligned}&\mu _s^i({\triangle }_s^{-i}, \varvec{W}^i) - \mu _s^i({\triangle }_s^{-i}, \varvec{X}^i) \\&\quad = g_s^i(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{*i},\varvec{W}^i); {\triangle }_s^{-i},{\triangle }_s^{*i}; \varvec{W}^i) - g_s^i(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i); {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i}; \varvec{X}^i) \\&\quad \le g_s^i(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i); {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i}; \varvec{W}^i) ~ - ~ g_s^i(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i); {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i}; \varvec{X}^i) \\&\qquad \text {(As }({\triangle }_s^{\varvec{'}*i},\varvec{W}^i) \text { is the minimum point)} \\&\quad = \Omega ^i(s,{\triangle }_s^{\varvec{'}*}) + \beta \sum _{\bar{s} \in \textit{S} } W_{\bar{s}}^i t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i) + \alpha ~ \text {dist}(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i), Z_t) \\&\qquad - \Omega ^i(s,{\triangle }_s^{\varvec{'}*}) - \beta \sum _{\bar{s} \in \textit{S} } X_{\bar{s}}^i t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i)- \alpha ~ \text {dist}(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i), Z_t)\\&\quad \le \beta \sum _{\bar{s} \in \textit{S} } W_{\bar{s}}^i t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i) + \alpha ~ \text {dist}(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i), Z_t) \\&\qquad - \beta \sum _{\bar{s} \in \textit{S} } X_{\bar{s}}^i t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i) - \alpha ~ \text {dist}(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i), Z_t) \\&\qquad \text {(As for a given } ({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i), \varvec{t}_s \text { is a maximizer for }g_s^i(\varvec{t}_s; {\triangle }_s; \varvec{X}^i)) \\&\quad \le \beta \sum _{\bar{s} \in \textit{S} }( W_{\bar{s}}^i - X_{\bar{s}}^i) t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i) \\&\quad \le \beta \sum _{\bar{s} \in \textit{S} } t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i)~ \Vert \varvec{W} - \varvec{X}\Vert _{\infty } \\&\quad = \beta \Vert \varvec{W} - \varvec{X}\Vert _{\infty } ,~~ \text {as } \sum _{\bar{s} \in \textit{S} } t(\bar{s} \vert s, {\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{W}^i) = 1, \end{aligned}$$

where \(\beta \in (0,1) \) is the discount factor and \(\Vert \varvec{W}\Vert _{\infty }\) is the infinity norm on the space \({\mathbb {R}}^{nm}\),

$$\begin{aligned} \Vert \varvec{W}\Vert _{\infty } = \underset{ i \in N, s \in S}{\max } \vert W_s^i \vert . \end{aligned}$$

Similarly, for fixed \({\triangle }_s^{-i}\), we see that \(\forall i \in N, \forall s \in S\),

$$\begin{aligned} \mu _s^i({\triangle }_s^{-i}, \varvec{X}^i) - \mu _s^i({\triangle }_s^{-i}, \varvec{W}^i) \le \beta \Vert \varvec{W} - \varvec{X}\Vert _{\infty }. \end{aligned}$$

Thus, \(\Vert \eta _{{\triangle }}(\varvec{W}) - \eta _{{\triangle }}(\varvec{X})\Vert _{\infty } \le \beta \Vert \varvec{W} - \varvec{X}\Vert _{\infty } \) with \(\beta \in (0,1)\). Therefore, the function \(\eta _{{\triangle }}\) is a contraction mapping. \(\square \)

Appendix B. Proof of Lemma 4.1


As \(\Omega ^i(s,\varvec{a}_s)\) is bounded, we have \(\Omega ^i(s,\varvec{a}_s) \le F\) for some finite F. Also, since \(t(\bar{s} \vert s, \varvec{a}_s) \le 1\)\(\forall i \in N,~s \in S \), the robust values are bounded. Therefore, we have \(\mid W_s^i \mid \le E\)\(\forall i \in N,~s \in S,\) for some finite E. We now note that

$$\begin{aligned}&\mid g_s^i(\varvec{t}_s, {\triangle }_s, \varvec{W}^i) - g_s^i(\varvec{t}_s, {\triangle '}_s, \varvec{X}^i) \mid \nonumber \\&\quad = \biggl \vert \sum _{\varvec{a}_s \in \varvec{A}_s}{\triangle }_s(\varvec{a})_s\Omega ^i(s,\varvec{a}_s) + \beta \sum _{\varvec{a}_s \in \varvec{A}_s}{\triangle }_s(\varvec{a}_s) \left( \sum _{\bar{s} \in S} t_s(\bar{s} \vert s, \varvec{a}_s) W_{\bar{s}}^i \right) + \alpha ~ \text {dist}(\varvec{t}_s, Z_t) \nonumber \\&\qquad - \sum _{\varvec{a}_s \in \varvec{A}_s}{\triangle '}_s(\varvec{a}_s)\Omega ^i(s,\varvec{a}_s) - \beta \sum _{\varvec{a}_s \in \varvec{A}_s}{\triangle '}_s(\varvec{a}_s) \left( \sum _{\bar{s} \in S} t_s(\bar{s} \vert s, \varvec{a}_s) X_{\bar{s}}^i \right) \nonumber \\&\qquad - \alpha ~ \text {dist}(\varvec{t}_s, Z_t) \biggr \vert \nonumber \\&\quad = \biggl \vert \sum _{\varvec{a}_s \in \varvec{A}_s} ({\triangle }_s(\varvec{a}_s) -{\triangle '}_s(\varvec{a}_s) )\Omega ^i(s,\varvec{a}_s) \nonumber \\&\qquad + \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S} t(\bar{s} \vert s, \varvec{a}_s) (W_{\bar{s}}^i \varvec{\triangle _s}(\varvec{a}_s) -X_{\bar{s}}^i\varvec{\triangle '_s}(\varvec{a}_s)) \biggr \vert \nonumber \\&\quad \le \biggl \vert \sum _{\varvec{a}_s \in \varvec{A}_s} (\varvec{\triangle _s}(\varvec{a}_s) -{\triangle '}_s(\varvec{a}_s) )\Omega ^i(s,\varvec{a}_s) \biggr \vert \nonumber \\&\qquad + \biggl \vert \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S} t(\bar{s} \vert s, \varvec{a}_s) (W_{\bar{s}}^i {\triangle }_s(\varvec{a}_s) -X_{\bar{s}}^i{\triangle '}_s(\varvec{a}_s)) \biggr \vert \end{aligned}$$
$$\begin{aligned}&\quad \le \sum _{\varvec{a}_s \in \varvec{A}_s} \left| ({\triangle }_s(\varvec{a}_s) -{\triangle '}_s(\varvec{a}_s) )F \right| + \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S} \left| (W_s^i \varvec{\triangle _s}(\varvec{a}_s) -X_s^i\varvec{\triangle '_s}(\varvec{a}_s)) \right| , \end{aligned}$$

where the inequality (B.1) follows from triangle inequality and the inequality (B.2) follows from \(\Omega ^i(s,\varvec{a}_s) \le F \text { and } t(\bar{s} \vert s, \varvec{a}_s) \le 1 \ \forall i \in N, \forall s,\bar{s} \in S, \forall \varvec{a}_s \in \varvec{A}_s\).


$$\begin{aligned} \delta _1(\epsilon ) = \frac{\min (\epsilon ,1)}{3F (2^n -1)l^n}, ~\delta _2(\epsilon ) = \frac{\min (\epsilon ,1)}{3S \beta l^n} ~\text { and } ~ \delta _3(\epsilon ) = \frac{\min (\epsilon ,1)}{3E S \beta (2^n -1)l^n}. \end{aligned}$$

We define \(\delta (\epsilon )= \min \{\delta _1(\epsilon ),\delta _2(\epsilon ),\delta _3(\epsilon )\}\). As \(d_t(\varvec{c},\varvec{b}) \le \delta (\epsilon )\), we have \({\triangle }_s^i(\varvec{a}_s) = {\triangle }_s^{'i}(\varvec{a}_s) + \rho _{sa}^i\) and \(W^i_s = X^i_s + \varrho _{s}^i\)\(\forall i \in N, \forall s \in S, \forall \varvec{a}_s \in \varvec{A}_s\), where \(\mid \rho _{sa}^i \mid < \delta (\epsilon )\) and \(\mid \varrho _{s}^i \mid < \delta (\epsilon )\).

We see that

$$\begin{aligned} \left| \prod _{i=1}^n ({\triangle }_s^{'i}(\varvec{a}_s) + \rho _{sa}^i) - \prod _{i=1}^n ({\triangle }_s^{'i}(\varvec{a}_s) \right|&~=~ \left| \sum _{\begin{array}{c} \kappa \subset N, \mid \kappa \mid \ge 1 \end{array}} \left( \prod _{i \in \kappa }\rho _{sa}^i \right) \left( \prod _{i \in {\bar{\kappa }}}{\triangle }_s^{'i}(\varvec{a}_s) \right) \right| ~ \nonumber \\&~\le ~ \sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}} \left| \prod _{i \in \kappa }\rho _{sa}^i \right| ~ \left| \prod _{i \in {\bar{\kappa }}}{\triangle }_s^{'i}(\varvec{a}_s) \right| , \end{aligned}$$

where \({\bar{\kappa }} = N - \kappa \). Here, \(\mid \prod _{i \in \kappa }\rho _{sa}^i)\mid < (\delta _1(\epsilon ))^{\mid \kappa \mid } \le \delta _1(\epsilon )\). Using (B.3) in (B.2),

$$\begin{aligned}&F ~ \sum _{\varvec{a}_s \in \varvec{A}_s} \left| \prod _{j=1}^n ({\triangle }_s^{'j}(\varvec{a}_s) + \rho _{sa}^j) - \prod _{j=1}^n {\triangle }_s^{'j}(\varvec{a}_s) \right| \\&\quad \le ~F ~ \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}}\left| \left( \prod _{j \in \kappa }\rho _{sa}^j \right) \left( \prod _{j \in {\bar{\kappa }}}{\triangle }_s^{'j}(\varvec{a}_s) \right) \right| \\&\quad \le F ~ \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}} \left| \left( \prod _{j \in \kappa }\rho _{sa}^j \right) \right| \\&\quad < F ~ \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}}\delta _1(\epsilon ) \le \frac{\epsilon }{3}. \end{aligned}$$

The second part of (B.2) is

$$\begin{aligned}&\beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S} \left| W_{\bar{s}}^i \prod _{j=1}^n {\triangle }_s^{j}(\varvec{a}_s) -X_{\bar{s}}^i \prod _{j=1}^n {\triangle }_s^{'j}(\varvec{a}_s) \right| \nonumber \\&\quad \le \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S} \left|W_{\bar{s}}^i \prod _{j=1}^n ({\triangle }_s^{'j}(\varvec{a}_s) + \rho _{sa}^j)-X_{\bar{s}}^i\prod _{j=1}^n {\triangle }_s^{'j}(\varvec{a}_s) \right|\nonumber \\&\quad \le \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\left| \prod _{j=1}^n {\triangle }_s^{'j}(\varvec{a}_s)( W_{\bar{s}}^i - X_{\bar{s}}^i) + W_{\bar{s}}^i \sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array} }\prod _{j \in \kappa }\rho _{sa}^j \prod _{j \in {\bar{\kappa }}}{\triangle }_s^{'j}(\varvec{a}_s) \right| \nonumber \\&\quad \le \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\left| \prod _{j=1}^n {\triangle }_s^{'j}(\varvec{a}_s) \right| \vert (W_{\bar{s}}^i - X_{\bar{s}}^i)\vert \nonumber \\&\qquad + \beta E \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}}\left| \prod _{j \in \kappa }\rho _{sa}^j \right| \left| \prod _{j \in {\bar{\kappa }}}{\triangle }_s^{'j}(\varvec{a}_s) \right| \text { (using triangle inequality)}\nonumber \\&\quad \le \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\vert \varrho _{s}^i \vert + \beta E \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}}\left| \prod _{j \in \kappa }\rho _{sa}^j \right| \nonumber \\&\qquad ( \text {As }\vert \prod _{j \in {\bar{\kappa }}}{\triangle }_s^{'j}(\varvec{a}_s) \vert \le 1\hbox { and } W_{\bar{s}}^i - X_{\bar{s}}^i = \varrho _{s}^i)\nonumber \\&\quad \le \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\delta _2(\epsilon ) + \beta E \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S}\sum _{\begin{array}{c} \kappa \subset N,\\ \mid \kappa \mid \ge 1 \end{array}}\delta _3(\epsilon )\nonumber \\&\quad = \frac{\epsilon }{3} + \frac{\epsilon }{3} = \frac{2\epsilon }{3}. \end{aligned}$$

The last inequality (B.4) follows from \(\vert \varrho _{s}^i \vert< \delta (\epsilon ) \le \delta _2(\epsilon ) \text { and } \vert \prod _{j \in \kappa }\rho _{sa}^j \vert \le \vert \rho _{sa}^j \vert < \delta (\epsilon ) \le \delta _3(\epsilon )\). Therefore,

$$\begin{aligned}&\mid g_s^i(\varvec{t}_s, {\triangle }_s, \varvec{W}^i) - g_s^i(\varvec{t}_s, {\triangle '}_s, \varvec{X}^i) \mid \\&\quad \le ~F\sum _{\varvec{a}_s \in \varvec{A}_s} \left| \varvec{\triangle _s}(\varvec{a}_s) -\varvec{\triangle '_s}(\varvec{a}_s) \right| + \beta \sum _{\varvec{a}_s \in \varvec{A}_s} \sum _{\bar{s} \in S} \left| (W_s^i {\triangle }_s(\varvec{a}_s) -X_s^i{\triangle '}_s(\varvec{a}_s)) \right| \\&\quad < \frac{\epsilon }{3} + \frac{2\epsilon }{3} = \epsilon . \end{aligned}$$

As \(\mid g_s^i(\varvec{t}_s, {\triangle }_s, \varvec{W}^i) - g_s^i(\varvec{t}_s, {\triangle '}_s, \varvec{X}^i) \mid \le \epsilon \) for \(d_t(\varvec{c},\varvec{b}) < \delta (\epsilon )\), the set of functions \( g(\varvec{t}_s, {\triangle }_s, \varvec{W}^i)\), \(\varvec{t}_s \in Q\) are equicontinuous. \(\square \)

Appendix C. Proof of Theorem 4.2


Lemma 4.1 shows that set of functions \(\{ g_s^i(\varvec{t}_s\),\({\triangle }_s\),\(\varvec{W}^i) | \varvec{t}_s \in Q \}\) are equi-continuous. The function \(h_s^i({\triangle }_s; \varvec{W}^i) =\underset{\varvec{t} \in \textit{Q}}{\max } ~~g_s^i(\varvec{t}_s; {\triangle }_s; \varvec{W}^i)\) is continuous in all of its variables \(\forall i \in N, \forall s \in S\), as proved in Lemma 2 from [13]. As the minimum of the function \(h_s^i({\triangle }_s; \varvec{W}^i)\) exists, we have \({{\,\mathrm{arg\,min}\,}}_{{\triangle }_s^i} h_s^i({\triangle }_s; \varvec{W}^i) \ne \emptyset \).

We now define a mapping \(\zeta : \sum \rightarrow 2^{\sum }\) by

$$\begin{aligned} \zeta ({\triangle })= & {} \biggl \{ {\triangle '} \in \sum ~ \biggl \vert {\triangle '}^{i}_s \in \mathop {{\mathrm{arg}}\,{\mathrm{min}}}\limits _{{\triangle }_s^i} h_s^i({\triangle }_s; \varvec{W}^i), W_s^i \\= & {} \underset{{\triangle }_s^i}{\min } ~~h_s^i({\triangle }_s; \varvec{W}^i), \forall i \in N, \forall s \in S \biggr \}. \end{aligned}$$

It is proved in Theorem 4.1 that \(\eta _{{\triangle }}(\varvec{W})\) is a contraction mapping. Using Banach Contraction Mapping Theorem, \(\varvec{W}\) is a unique fixed point for the function \(\eta \). Therefore, we have

$$\begin{aligned} W_s^i = (\eta _{{\triangle }}(W))_{is} =\underset{{\triangle }_s^i}{\min } ~~h_s^i({\triangle }_s; \varvec{W}^i) ~ \forall i \in N, \forall s \in S. \end{aligned}$$

This means that \(\zeta ({\triangle }) \ne \emptyset \). Now, Theorem 4 of [13] establishes that \(\zeta ({\triangle })\) is convex, upper semi-continuous and closed for any \({\triangle }\). As \(\zeta \) completes the requirements of Kakutani’s fixed point theorem [12], it follows that \(\zeta ({\triangle })\) has a fixed point which is also the equilibrium point. \(\square \)

Appendix D. Proof of Theorem 5.1


\(\textit{Part I.}\) The strategy point \(\triangle \) is a globalized robust Markov perfect equilibrium point of the problem (11) if \(\forall i \in N, s \in S\), for a given (\({\triangle }_s^{-i}, \varvec{W}_i\)), \(\exists \varvec{u}_s^i,\varvec{q}_s^i \in {\mathbb {R}}^{2ml^n}\) and \(\exists \varvec{v}_s^i,\varvec{r}_s^i \in {\mathbb {R}}^{l^n}\) such that (\({\triangle }, W_s^i, \varvec{u}_s^i, \varvec{v}_s^i, \varvec{q}_s^i, \varvec{r}_s^i\)) is an optimal solution of the problem (21). Its dual is given by the problem (20). With the help of Strong Duality Theorem and Banach Contraction Mapping Theorem, evidently, the non-linear system stated in the theorem is followed.

\(\textit{Part II.}\) We now prove that given (\({\triangle }, W_s^i, \varvec{u}_s^i, \varvec{v}_s^i, \varvec{q}_s^i, \varvec{r}_s^i\)) is a solution of the non-linear system \(\forall i \in N,\forall s \in S\), and then \({\triangle }\) is an equilibrium point of the game. Let

$$\begin{aligned} p_s^i ~= ~&~ \underset{l'=1,2,\dotsc ,l}{\min } \beta \varvec{e}_{l'}' \varvec{\hat{Z}}_s^{'i} (\varvec{Z}_s + \varvec{tr}_s^i) + \varvec{e}_{l'}' \varvec{{\hat{\Omega }}}'^i(s,{\triangle }_s ^{-i})\varvec{1} \\ \text { and }~ W_s^i ~= ~&~ \Omega ^i(s,{\triangle }_s) + \begin{bmatrix}\varvec{y}_s ; \varvec{1}\end{bmatrix} \begin{bmatrix}\varvec{q}_s^i \\ \varvec{r}_s^i\end{bmatrix}. \end{aligned}$$

Thus, \(\forall i \in N,\forall s \in S\), (\({\triangle }, W_s^i, \varvec{u}_s^i, \varvec{v}_s^i, \varvec{q}_s^i, \varvec{r}_s^i\)) is a feasible point of the problem (21) and (\(p_s^i\), \(\varvec{Z}_s\), \(\varvec{tr}_s^i\)) satisfies the problem (20) with \(p_s^i \ge W_s^i + \alpha \) or \(p_s^i \ge W_s^i\). By weak duality, we have \(p_s^i \le W_s^i\). This leads to \(p_s^i = W_s^i\). Therefore, (\({\triangle }\),\( W_s^i\),\( \varvec{u}_s^i\),\( \varvec{v}_s^i\),\( \varvec{q}_s^i, \varvec{r}_s^i\)) is the optimal solution for the problem (21), i.e., \({\triangle }\) is the globalized robust Markov perfect equilibrium point of the game. \(\square \)

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghosh, D., Sharma, A., Shukla, K.K. et al. Globalized robust Markov perfect equilibrium for discounted stochastic games and its application on intrusion detection in wireless sensor networks: Part I—theory. Japan J. Indust. Appl. Math. 37, 283–308 (2020).

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI:


Mathematics Subject Classification