Abstract
In this article, we study a discounted stochastic game to model resource optimal intrusion detection in wireless sensor networks. To address the problem of uncertainties in various network parameters, we propose a globalized robust game-theoretic framework for discounted robust stochastic games. A robust solution to the considered problem is an optimal point that is feasible for all realizations of data from a given uncertainty set. To allow a controlled violation of the constraints when the parameters move out of the uncertainty set, the concept of globalized robust framework comes into view. In this article, we formulate a globalized robust counterpart for the discounted stochastic game under consideration. With the help of globalized robust optimization, a concept of globalized robust Markov perfect equilibrium is introduced. The existence of such an equilibrium is shown for a discounted stochastic game when the number of actions of the players is finite. The contraction mapping theorem, Kakutani fixed point theorem and the concept of equicontinuity are used to prove the existence result. To compute a globalized robust Markov perfect equilibrium for the considered discounted stochastic game, a tractable representation of the proposed globalized robust counterpart is also provided. Using the derived tractable representation, we formulate a globalized robust intrusion detection system for wireless sensor networks.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdalzaher, M.S., Seddik, K., Elsabrouty, M., Muta, O., Furukawa, H., Abdel-Rahman, A.: Game theory meets wireless sensor networks security requirements and threats mitigation: a survey. Sensors 16(7), 1003 (2016)
Agah, A., Das, S.K., Basu, K., Asadi, M.: Intrusion detection in sensor networks: a noncooperative game approach. Third IEEE Int. Sympos. Netw. Comput. Appl. 2004, 343–346 (2004)
Alpcan, T., Basar, T.: A game theoretic approach to decision and analysis in network intrusion detection. In: 42nd IEEE International Conference on Decision and Control (IEEE Cat. No. 03CH37475), Vol. 3, pp. 2595–2600 (2003)
Ben-Tal, A., El Ghaoui, L., Nemirovski, A.S.: Robust Optimization. Princeton University Press, Princeton (2009)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)
Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer-Verlag, New York (1996)
Fink, A.M.: Equilibrium in a stochastic \(n\)-person game. J. Sci. Hiroshima Univ. Ser. A-I Math. 28(1), 89–93 (1964)
Fung, C.J., Zhu, Q., Boutaba, R., Baar, T.: Bayesian decision aggregation in collaborative intrusion detection networks. In: 2010 IEEE Network Operations and Management Symposium 2010, pp. 349–356 (2010)
Han, L., Zhou, M., Jia, W., Dalil, Z., Xu, X.: Intrusion detection model of wireless sensor networks based on game theory and an autoregressive model. Inform. Sci. 476, 491–504 (2019)
Hedengren, J.D., Shishavan, R.A., Powell, K.M., Edgar, T.F.: Nonlinear modeling, estimation and predictive control in APMonitor. Comput. Chem. Eng. 70, 133–148 (2014)
Guan, S., Wang, J., Jiang, C., Tong, J., Ren, Y.: Intrusion detection for wireless sensor networks: a multi-criteria game approach. In: IEEE Wireless Communications and Networking Conference, Barcelona, pp. 1–6 (2018)
Kakutani, S.: A generalization of Brouwer’s fixed point theorem. Duke Math. J. 8(3), 457–459 (1941)
Karde, E., Ordez, F., Hall, R.: Discounted robust stochastic games and an application to queueing control. Oper. Res. 59(2), 365–382 (2011)
Moosavi, H., Bui, F.M.: A game-theoretic framework for robust optimal intrusion detection in wireless sensor networks. IEEE Trans. Inform. Forensics Secur. 9(9), 1367–1379 (2014)
Patil, S., Chaudhari, S.: DoS attack prevention technique in wireless sensor networks. Proc. Comput. Sci. 79, 715–721 (2016)
Schirmer, H.: Conditions for the uniqueness of the fixed point in Kakutani’s theorem. Can. Math. Bull. 24(3), 351–357 (1981)
Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. 39(10), 1095–1100 (1953)
Subba, B., Biswas, S., Karmakar, S.: A game theory based multi layered intrusion detection framework for wireless sensor networks. Int. J. Wirel. Inform. Netw. 25(4), 399–421 (2018)
Wang, K., Du, M., Yang, D., Zhu, C., Shen, J., Zhang, Y.: Game-theory-based active defense for intrusion detection in cyber-physical embedded systems. ACM Trans. Embedded Comput. Syst. 16(1), 18 (2016)
Yu, W., Ji, Z., Liu, K.J.R.: Securing cooperative ad-hoc networks under noise and imperfect monitoring: strategies and game theoretic analysis. IEEE Trans. Inform. Forensics Secur. 2(2), 240–253 (2007)
Sobel, M.J.: Noncooperative stochastic games. Ann. Math. Stat. 42(6), 1930–1935 (1971)
Filar, J.A., Schultz, T.A., Thuijsman, F., Vrieze, O.J.: Nonlinear programming and stationary equilibria in stochastic games. Math. Program. 50(1–3), 227–237 (1991)
Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Mathematical Programing Language. In: Wallace, S.W. (eds) Algorithms and Model Formulations in Mathematical Programming. NATO ASI Series (Series F: Computer and Systems Sciences), vol 51. Springer, Berlin, Heidelberg (1989)
Byrd, R.H., Nocedal, J., Waltz, R.A.: KNITRO: An integrated package for nonlinear optimization. In: di Pillo, G., Roma, M. (eds.) Large-Scale Nonlinear Optimization, pp. 35–59. Springer, New York (2006)
Czyzyk, J., Mesnier, M.P., Moré, J.J.: The NEOS Server. IEEE J. Comput. Sci. Eng. 5(3), 68–75 (1998) (This paper discusses the design and implementation of the NEOS Server)
Acknowledgements
The first author gratefully acknowledges the financial support through the Early Career Research Award (ECR/2015/000467), Science & Engineering Research Board, Government of India. Authors are thankful to the reviewers and the editors for their valuable comments to improve the article.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A. Proof of Theorem 4.1
Proof
Let us consider two points \(\varvec{W}\) and \(\varvec{X}\) from \({\mathbb {R}}^{nm}\). Here n is the number of players and m is the number of states. For fixed \({\triangle }^{-i}_s\)\(\forall i,s\),
where \({\triangle }_s^{*i}\) minimizes the function and the maximizer \(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{*i},\varvec{W}^i)\) depends on (\({\triangle }_s^{-i},{\triangle }_s^{*i}\)). We also have \({\triangle }_s^{\varvec{'}*i}\) and \(\varvec{t}_s({\triangle }_s^{-i},{\triangle }_s^{\varvec{'}*i},\varvec{X}^i) \in Q\) such that
Thus,
where \(\beta \in (0,1) \) is the discount factor and \(\Vert \varvec{W}\Vert _{\infty }\) is the infinity norm on the space \({\mathbb {R}}^{nm}\),
Similarly, for fixed \({\triangle }_s^{-i}\), we see that \(\forall i \in N, \forall s \in S\),
Thus, \(\Vert \eta _{{\triangle }}(\varvec{W}) - \eta _{{\triangle }}(\varvec{X})\Vert _{\infty } \le \beta \Vert \varvec{W} - \varvec{X}\Vert _{\infty } \) with \(\beta \in (0,1)\). Therefore, the function \(\eta _{{\triangle }}\) is a contraction mapping. \(\square \)
Appendix B. Proof of Lemma 4.1
Proof
As \(\Omega ^i(s,\varvec{a}_s)\) is bounded, we have \(\Omega ^i(s,\varvec{a}_s) \le F\) for some finite F. Also, since \(t(\bar{s} \vert s, \varvec{a}_s) \le 1\)\(\forall i \in N,~s \in S \), the robust values are bounded. Therefore, we have \(\mid W_s^i \mid \le E\)\(\forall i \in N,~s \in S,\) for some finite E. We now note that
where the inequality (B.1) follows from triangle inequality and the inequality (B.2) follows from \(\Omega ^i(s,\varvec{a}_s) \le F \text { and } t(\bar{s} \vert s, \varvec{a}_s) \le 1 \ \forall i \in N, \forall s,\bar{s} \in S, \forall \varvec{a}_s \in \varvec{A}_s\).
Let
We define \(\delta (\epsilon )= \min \{\delta _1(\epsilon ),\delta _2(\epsilon ),\delta _3(\epsilon )\}\). As \(d_t(\varvec{c},\varvec{b}) \le \delta (\epsilon )\), we have \({\triangle }_s^i(\varvec{a}_s) = {\triangle }_s^{'i}(\varvec{a}_s) + \rho _{sa}^i\) and \(W^i_s = X^i_s + \varrho _{s}^i\)\(\forall i \in N, \forall s \in S, \forall \varvec{a}_s \in \varvec{A}_s\), where \(\mid \rho _{sa}^i \mid < \delta (\epsilon )\) and \(\mid \varrho _{s}^i \mid < \delta (\epsilon )\).
We see that
where \({\bar{\kappa }} = N - \kappa \). Here, \(\mid \prod _{i \in \kappa }\rho _{sa}^i)\mid < (\delta _1(\epsilon ))^{\mid \kappa \mid } \le \delta _1(\epsilon )\). Using (B.3) in (B.2),
The second part of (B.2) is
The last inequality (B.4) follows from \(\vert \varrho _{s}^i \vert< \delta (\epsilon ) \le \delta _2(\epsilon ) \text { and } \vert \prod _{j \in \kappa }\rho _{sa}^j \vert \le \vert \rho _{sa}^j \vert < \delta (\epsilon ) \le \delta _3(\epsilon )\). Therefore,
As \(\mid g_s^i(\varvec{t}_s, {\triangle }_s, \varvec{W}^i) - g_s^i(\varvec{t}_s, {\triangle '}_s, \varvec{X}^i) \mid \le \epsilon \) for \(d_t(\varvec{c},\varvec{b}) < \delta (\epsilon )\), the set of functions \( g(\varvec{t}_s, {\triangle }_s, \varvec{W}^i)\), \(\varvec{t}_s \in Q\) are equicontinuous. \(\square \)
Appendix C. Proof of Theorem 4.2
Proof
Lemma 4.1 shows that set of functions \(\{ g_s^i(\varvec{t}_s\),\({\triangle }_s\),\(\varvec{W}^i) | \varvec{t}_s \in Q \}\) are equi-continuous. The function \(h_s^i({\triangle }_s; \varvec{W}^i) =\underset{\varvec{t} \in \textit{Q}}{\max } ~~g_s^i(\varvec{t}_s; {\triangle }_s; \varvec{W}^i)\) is continuous in all of its variables \(\forall i \in N, \forall s \in S\), as proved in Lemma 2 from [13]. As the minimum of the function \(h_s^i({\triangle }_s; \varvec{W}^i)\) exists, we have \({{\,\mathrm{arg\,min}\,}}_{{\triangle }_s^i} h_s^i({\triangle }_s; \varvec{W}^i) \ne \emptyset \).
We now define a mapping \(\zeta : \sum \rightarrow 2^{\sum }\) by
It is proved in Theorem 4.1 that \(\eta _{{\triangle }}(\varvec{W})\) is a contraction mapping. Using Banach Contraction Mapping Theorem, \(\varvec{W}\) is a unique fixed point for the function \(\eta \). Therefore, we have
This means that \(\zeta ({\triangle }) \ne \emptyset \). Now, Theorem 4 of [13] establishes that \(\zeta ({\triangle })\) is convex, upper semi-continuous and closed for any \({\triangle }\). As \(\zeta \) completes the requirements of Kakutani’s fixed point theorem [12], it follows that \(\zeta ({\triangle })\) has a fixed point which is also the equilibrium point. \(\square \)
Appendix D. Proof of Theorem 5.1
Proof
\(\textit{Part I.}\) The strategy point \(\triangle \) is a globalized robust Markov perfect equilibrium point of the problem (11) if \(\forall i \in N, s \in S\), for a given (\({\triangle }_s^{-i}, \varvec{W}_i\)), \(\exists \varvec{u}_s^i,\varvec{q}_s^i \in {\mathbb {R}}^{2ml^n}\) and \(\exists \varvec{v}_s^i,\varvec{r}_s^i \in {\mathbb {R}}^{l^n}\) such that (\({\triangle }, W_s^i, \varvec{u}_s^i, \varvec{v}_s^i, \varvec{q}_s^i, \varvec{r}_s^i\)) is an optimal solution of the problem (21). Its dual is given by the problem (20). With the help of Strong Duality Theorem and Banach Contraction Mapping Theorem, evidently, the non-linear system stated in the theorem is followed.
\(\textit{Part II.}\) We now prove that given (\({\triangle }, W_s^i, \varvec{u}_s^i, \varvec{v}_s^i, \varvec{q}_s^i, \varvec{r}_s^i\)) is a solution of the non-linear system \(\forall i \in N,\forall s \in S\), and then \({\triangle }\) is an equilibrium point of the game. Let
Thus, \(\forall i \in N,\forall s \in S\), (\({\triangle }, W_s^i, \varvec{u}_s^i, \varvec{v}_s^i, \varvec{q}_s^i, \varvec{r}_s^i\)) is a feasible point of the problem (21) and (\(p_s^i\), \(\varvec{Z}_s\), \(\varvec{tr}_s^i\)) satisfies the problem (20) with \(p_s^i \ge W_s^i + \alpha \) or \(p_s^i \ge W_s^i\). By weak duality, we have \(p_s^i \le W_s^i\). This leads to \(p_s^i = W_s^i\). Therefore, (\({\triangle }\),\( W_s^i\),\( \varvec{u}_s^i\),\( \varvec{v}_s^i\),\( \varvec{q}_s^i, \varvec{r}_s^i\)) is the optimal solution for the problem (21), i.e., \({\triangle }\) is the globalized robust Markov perfect equilibrium point of the game. \(\square \)
About this article
Cite this article
Ghosh, D., Sharma, A., Shukla, K.K. et al. Globalized robust Markov perfect equilibrium for discounted stochastic games and its application on intrusion detection in wireless sensor networks: Part I—theory. Japan J. Indust. Appl. Math. 37, 283–308 (2020). https://doi.org/10.1007/s13160-019-00397-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13160-019-00397-9