Skip to main content
SpringerLink
Log in

Continuous action iterated dilemma under double-layer network with unknown nonlinear dynamics and its convergence analysis

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

In this paper, we propose a convergence analysis of evolutionary dynamics with limited learning ability within a double-layer network, exceeding the constraints of current approaches. To model the diversity in the agents’ ability to perceive their surroundings, we first design the dynamics model for continuous action iterated dilemma with limited learning ability. The agents are initialized with a fixed parameter that represents their maximum probability of strategy switching during the evolution process. Secondly, we extend the dynamics model to double-layer networks, in which the agents interact exclusively with neighbors in the same layer and update their strategies based on a weighted sum of their payoff in the two layers. Then, we evaluate the environmental influences on learning capacity using a dynamics formula and adapt to the unknown environment dynamics with radial basis function neural network (RBF-NN). Lastly, we conduct a convergence analysis of the dynamics models and confirm their effectiveness with experiments. This method may be utilized to analyze evolutionary processes in hierarchically structured networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Abbreviations

\({p_0,p_1,p_2,p_3}\) :

The payoff of player

\({x_i}\) :

The strategy of player i

\({F(x_i)}\) :

The fitness of player i

k :

Iteration number

\({p_{ij}}\) :

The strategy switching probability of player i

G :

A complex network

V :

The vertex set of network G

E :

The edge set of network G

N :

The number of players

\({a_{ij}}\) :

Edge weight between vertex i and j

\({\textrm{deg}(v_{i})}\) :

The degree of player i

x(t):

The state of the system at time t

\({\alpha }\) :

The equilibrium point

\({l_i}\) :

The learning ability of player i

\({f_{i}}\) :

The unknown dynamics of player i

m(t):

The weight of high-fitness layer

\({L_{k}}\) :

The Laplacian matrix of graph \(G_k\)

e :

The error between the state x and equilibrium point \(\alpha \)

X :

The strategy of players

W :

Weight vector

\({\varphi (x)}\) :

The Gaussian function

\({\xi }\) :

The fitting error of f

\({\hat{f}}\) :

The approximation of f

\({\hat{W}}\) :

The estimate of the weight matrix W

\({\tilde{W}^{T}}\) :

The estimation error of the weight matrix W

\({\tilde{f}(x)}\) :

The estimation error of function f

\({\Vert W\Vert _{\textrm{F}}}\) :

The Frobenius norm of W

\(W_{M}\) :

The upper bound of W

\({\Vert \varphi \Vert }\) :

The vector norm of \(\varphi \)

\({\varphi _{M}}\) :

The upper bound of \(\varphi \)

References

  1. Apaloo, J., Brown, J.S., Vincent, T.L.: Evolutionary game theory: Ess, convergence stability, and nis. Evol. Ecol. Res. 11(4), 489–515 (2009)

    Google Scholar 

  2. Assenza, S., Gómez-Gardeñes, J., Latora, V.: Enhancement of cooperation in highly clustered scale-free networks. Phys. Rev. E 78(1), 017101 (2008)

    Google Scholar 

  3. Boucek, F.: Rethinking factionalism: typologies, intra-party dynamics and three faces of factionalism. Party Polit. 15(4), 455–485 (2009)

    Google Scholar 

  4. Brandt, H., Hauert, C., Sigmund, K.: Punishment and reputation in spatial public goods games. Proc. R. Soc. Lond. Ser. B Biol. Sci. 270(1519), 1099–1104 (2003)

    Google Scholar 

  5. Cheng, L., Liu, G., Huang, H., Wang, X., Chen, Y., Zhang, J., Meng, A., Yang, R., Yu, T.: Equilibrium analysis of general n-population multi-strategy games for generation-side long-term bidding: an evolutionary game perspective. J. Clean. Prod. 276, 124123 (2020)

    Google Scholar 

  6. Dal Bó, P., Fréchette, G.R.: On the determinants of cooperation in infinitely repeated games: a survey. J. Econ. Lit. 56(1), 60–114 (2018)

    Google Scholar 

  7. Du, W.B., Cao, X.B., Zhao, L., Hu, M.B.: Evolutionary games on scale-free networks with a preferential selection mechanism. Physica A 388(20), 4509–4514 (2009)

    Google Scholar 

  8. Fu, F., Hauert, C., Nowak, M.A., Wang, L.: Reputation-based partner choice promotes cooperation in social networks. Phys. Rev. E 78(2), 026117 (2008)

    Google Scholar 

  9. Fudenberg, D., Maskin, E.: The folk theorem in repeated games with discounting or with incomplete information. In: A long-run collaboration on long-run games, pp. 209–230. World Scientific (2009)

  10. Hammond, R.A., Axelrod, R.: Evolution of contingent altruism when cooperation is expensive. Theor. Popul. Biol. 69(3), 333–338 (2006)

    MATH  Google Scholar 

  11. Hill, K.: Altruistic cooperation during foraging by the ache, and the evolved human predisposition to cooperate. Hum. Nat. 13(1), 105–128 (2002)

    Google Scholar 

  12. Hintze, A., Adami, C.: Punishment in public goods games leads to meta-stable phase transitions and hysteresis. Phys. Biol. 12(4), 046005 (2015)

    Google Scholar 

  13. Hofbauer, J., Sigmund, K., et al.: Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge (1998)

    MATH  Google Scholar 

  14. Lacker, D.: On the convergence of closed-loop nash equilibria to the mean field game limit. Ann. Appl. Probab. 30(4), 1693–1761 (2020)

    MathSciNet  MATH  Google Scholar 

  15. Lewis, F.L., Zhang, H., Hengster-Movric, K., Das, A.: Cooperative Control of Multi-agent Systems: Optimal and Adaptive Design Approaches. Springer, Berlin (2013)

    MATH  Google Scholar 

  16. Liu, Y., Chen, X., Zhang, L., Wang, L., Perc, M.: Win-stay-lose-learn promotes cooperation in the spatial prisoner’s dilemma game. PLoS ONE 7(2), e30689 (2012)

    Google Scholar 

  17. Milinski, M., Lüthi, J.H., Eggler, R., Parker, G.A.: Cooperation under predation risk: experiments on costs and benefits. Proc. R. Soc. Lond. Ser. B Biol. Sci. 264(1383), 831–837 (1997)

    Google Scholar 

  18. Nakamaru, M., Iwasa, Y.: The coevolution of altruism and punishment: role of the selfish punisher. J. Theor. Biol. 240(3), 475–488 (2006)

    MathSciNet  MATH  Google Scholar 

  19. Nowak, M.A.: Five rules for the evolution of cooperation. Science 314(5805), 1560–1563 (2006)

    Google Scholar 

  20. Nowak, M.A., May, R.M.: Evolutionary games and spatial chaos. Nature 359(6398), 826–829 (1992)

    Google Scholar 

  21. Ploderer, B., Reitberger, W., Oinas-Kukkonen, H., van Gemert-Pijnen, J.: Social interaction and reflection for behaviour change. Pers. Ubiquitous Comput. 18, 1667–1676 (2014)

    Google Scholar 

  22. Poncela, J., Gómez-Gardeñes, J., Traulsen, A., Moreno, Y.: Evolutionary game dynamics in a growing structured population. New J. Phys. 11(8), 083031 (2009)

    Google Scholar 

  23. Prandi, L., Primiero, G.: Effects of misinformation diffusion during a pandemic. Appl. Netw. Sci. 5(1), 1–20 (2020)

    Google Scholar 

  24. Sandholm, W.H.: Evolutionary Game Theory. Complex Social and Behavioral Systems: Game Theory and Agent-Based Models pp. 573–608 (2020)

  25. Santos, F.C., Santos, M.D., Pacheco, J.M.: Social diversity promotes the emergence of cooperation in public goods games. Nature 454(7201), 213–216 (2008)

    Google Scholar 

  26. Sastry, S., Sastry, S.: Lyapunov stability theory. Nonlinear Syst. Anal. Stab. Control pp. 182–234 (1999)

  27. Smith, J.M.: The theory of games and the evolution of animal conflicts. J. Theor. Biol. 47(1), 209–221 (1974)

    MathSciNet  Google Scholar 

  28. Stefanone, M.A., Vollmer, M., Covert, J.M.: In news we trust? Examining credibility and sharing behaviors of fake news. In: Proceedings of the 10th International Conference on Social Media and Society, pp. 136–147 (2019)

  29. Su, Q., McAvoy, A., Plotkin, J.B.: Evolution of cooperation with contextualized behavior. Sci. Adv. 8(6), eabm6066 (2022)

    Google Scholar 

  30. Szabó, G., Hauert, C.: Evolutionary prisoner’s dilemma games with voluntary participation. Phys. Rev. E 66(6), 062903 (2002)

    MathSciNet  Google Scholar 

  31. Toma, C., Butera, F.: Cooperation versus competition effects on information sharing and use in group decision-making. Soc. Pers. Psychol. Compass 9(9), 455–467 (2015)

    Google Scholar 

  32. Traulsen, A., Semmann, D., Sommerfeld, R.D., Krambeck, H.J., Milinski, M.: Human strategy updating in evolutionary games. Proc. Natl. Acad. Sci. 107(7), 2962–2966 (2010)

    Google Scholar 

  33. Vukov, J., Szabó, G., Szolnoki, A.: Cooperation in the noisy case: prisoner’s dilemma game on two types of regular random graphs. Phys. Rev. E 73(6), 067103 (2006)

    Google Scholar 

  34. Wang, C., Hill, D.J.: Deterministic Learning Theory for Identification, Recognition, and Control. CRC Press, Boca Raton (2018)

    Google Scholar 

  35. Wang, W.X., Ren, J., Chen, G., Wang, B.H.: Memory-based snowdrift game on networks. Phys. Rev. E 74(5), 056113 (2006)

    Google Scholar 

  36. Wang, Z., Hou, D., Gao, C., Huang, J., Xuan, Q.: A rapid source localization method in the early stage of large-scale network propagation. In: Proceedings of the ACM Web Conference 2022, pp. 1372–1380 (2022)

  37. Wang, Z., Jusup, M., Guo, H., Shi, L., Geček, S., Anand, M., Perc, M., Bauch, C.T., Kurths, J., Boccaletti, S., et al.: Communicating sentiment and outlook reverses inaction against collective risks. Proc. Natl. Acad. Sci. 117(30), 17650–17655 (2020)

    Google Scholar 

  38. Wang, Z., Mu, C., Hu, S., Chu, C., Li, X.: Modelling the dynamics of regret minimization in large agent populations: a master equation approach. In: Raedt, L.D. (ed.) Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pp. 534–540. International Joint Conferences on Artificial Intelligence Organization (2022). https://doi.org/10.24963/ijcai.2022/76. Main Track

  39. Wen-Bo, D., Xian-Bin, C., Han-Xin, Y., Mao-Bin, H.: Evolutionary prisoner’s dilemma on Newman–Watts social networks with an asymmetric payoff distribution mechanism. Chin. Phys. B 19(1), 010204 (2010)

    Google Scholar 

  40. Wiese, S.C., Heinrich, T.: The frequency of convergent games under best-response dynamics. Dyn. Games and Appl. 12(2), 689–700 (2022)

  41. Wu, Z.X., Xu, X.J., Chen, Y., Wang, Y.H.: Spatial prisoner’s dilemma game with volunteering in Newman–Watts small-world networks. Phys. Rev. E 71(3), 037103 (2005)

    Google Scholar 

  42. Xu, H., Yu, D., Sui, S., Zhao, Y.P., Chen, C.P., Wang, Z.: Nonsingular practical fixed-time adaptive output feedback control of mimo nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. (2022)

  43. Yu, D., Chen, C.P., Ren, C.E., Sui, S.: Swarm control for self-organized system with fixed and switching topology. IEEE Trans. Cybern. 50(10), 4481–4494 (2019)

    Google Scholar 

  44. Yu, D., Chen, C.P., Xu, H.: Intelligent decision making and bionic movement control of self-organized swarm. IEEE Trans. Ind. Electron. 68(7), 6369–6378 (2020)

    Google Scholar 

  45. Yu, D., Long, J., Chen, C.P., Wang, Z.: Bionic tracking-containment control based on smooth transition in communication. Inf. Sci. 587, 393–407 (2022)

    Google Scholar 

  46. Yu, D., Xu, H., Chen, C.P., Bai, W., Wang, Z.: Dynamic coverage control based on k-means. IEEE Trans. Ind. Electron. 69(5), 5333–5341 (2021)

  47. Zhang, J., Chen, X., Stegagno, P., Yuan, C.: Nonlinear dynamics modeling and fault detection for a soft trunk robot: an adaptive nn-based approach. IEEE Robot. Autom. Lett. 7(3), 7534–7541 (2022)

    Google Scholar 

  48. Zhang, Y., Feng, Z., Jin, J., Gao, W., Xu, H., Yu, D.: Time-varying formation control with smooth switching communication. AIP Adv. 11(10), 105103 (2021)

    Google Scholar 

  49. Zhu, P., Guo, H., Zhang, H., Han, Y., Wang, Z., Chu, C.: The role of punishment in the spatial public goods game. Nonlinear Dyn. 102, 2959–2968 (2020)

    Google Scholar 

  50. Zhu, P., Wang, X., Jia, D., Guo, Y., Li, S., Chu, C.: Investigating the co-evolution of node reputation and edge-strategy in prisoner’s dilemma game. Appl. Math. Comput. 386, 125474 (2020)

    MathSciNet  MATH  Google Scholar 

Download references

Funding

This work was supported by the National Science Fund for Distinguished Young Scholars (No. 62025602), the National Key R &D Program of China (Grant No. 2018AAA0100905), the National Natural Science Foundation of China (No. 62073263), Key Research and Development Program of Shaanxi Province (Grant Nos. 2022KW-26, 2022KW-05), Technological Innovation Team of Shaanxi Province (Grant No. 2020TD-013) and the fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, 710072, China

    Peican Zhu & Zhen Wang

  2. School of Computer Science, Northwestern Polytechnical University, Xi’an, 710072, China

    Jialong Sun

  3. The Unmanned System Research Institute, Northwestern Polytechnical University, Xi’an, 710072, China

    Dengxiu Yu

  4. School of Ecology and Environmental Sciences, Northwestern Polytechnical University, Xi’an, 710072, China

    Chen Liu

  5. Air and Missile Defense College, Air Force Engineering University, Xi’an, 710043, China

    Yannian Zhou

Authors
  1. Peican Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dengxiu Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yannian Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhen Wang.

Ethics declarations

Conflict of interest

The authors state that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, P., Sun, J., Yu, D. et al. Continuous action iterated dilemma under double-layer network with unknown nonlinear dynamics and its convergence analysis. Nonlinear Dyn 111, 21611–21625 (2023). https://doi.org/10.1007/s11071-023-08865-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-023-08865-1

Keywords

Search

Navigation