Exploiting User Demand Diversity: QoE Game and MARL Based Network Selection

  • Zhiyong DuEmail author
  • Bin Jiang
  • Qihui Wu
  • Yuhua Xu
  • Kun Xu


This chapter studies distributed network selection for multiple user cases with heterogeneous demand. The key challenge is low system efficiency due to user competition. Motivated by the fact that the ultimate goal of communications is to serve users with personalized demand, we introduce a new concept of user demand diversity gain. This gain derives from the elaborate matching between user demand and radio resource, which cannot be directly attained in the existing throughput-centric optimization due to users’ blindness in maximizing throughput. Aiming at obtaining this gain, we propose the user demand centric optimization, where users seek to maximize QoE. To model this problem, we propose a novel game formulation, QoE game. The properties of QoE game and QoE equilibrium and the definition of user demand diversity gain are presented. Two distributed QoE equilibrium learning algorithms, stochastic learning automata (SLA) based algorithm and trail and error (TE) based algorithm, are designed to achieve QoE equilibrium. Simulation results validate the existence of user demand diversity gain and the effectiveness of the proposed learning algorithms in improving the system efficiency and QoE fairness.


  1. 1.
    Trestian R, Ormond O, Muntean G (2012) Game theory-based network selection: solutions and challenges. IEEE Commun Sruv Tut 14(4):1018–1044Google Scholar
  2. 2.
    Liu D et al (2016) User association in 5G networks: a survey and an outlook. IEEE Commun Sruv Tut 18(2):1018–1044CrossRefGoogle Scholar
  3. 3.
    Keshavarz-Haddad A, Aryafar E, Wang M, Chiang M (2017) HetNets selection by clients: convergence, efficiency, and practicality. IEEE ACM Trans Netw 25(1):406–419CrossRefGoogle Scholar
  4. 4.
    Zhu K, Niyato D and Ping W (2010) Network selection in heterogeneous wireless networks: evolution with incomplete information. In: IEEE wireless communications and networking conference (WCNC)Google Scholar
  5. 5.
    Zhu K, Hossain E, Niyato D (2014) Pricing, spectrum sharing, and service selection in two-tier small cell networks: a hierarchical dynamic game approach. IEEE Trans Mob Comput 13(8):1843–1856CrossRefGoogle Scholar
  6. 6.
    Du Z, Wu Q et al (2015) Exploiting user demand diversity in heterogeneous wireless networks. IEEE Trans Wirel Commun 14(8):4142–4155CrossRefGoogle Scholar
  7. 7.
    Deb S, Nagaraj K, Srinivasan V (2011) MOTA: engineering an operator agnostic mobile service. MobiCom (2011)Google Scholar
  8. 8.
    Perlaza SM, Tembine H, Lasaulce S et al (2011) Quality-of-service provisioning in decentralized networks: a satisfaction equilibrium approach. IEEE J-STSP 6(2):104–116CrossRefGoogle Scholar
  9. 9.
    Rakocevic V, Griffiths J, Cope G (2001) Performance analysis of bandwidth allocation schemes in multiservice IP networks using utility functions. In: Proceedings of the 17th international teletraffic congress (ITC)Google Scholar
  10. 10.
    Reis AB, Chakareski J, Kassler A et al (2010) Distortion optimized multi-service scheduling for next-generation wireless mesh networks. In: IEEE INFOCOMGoogle Scholar
  11. 11.
    Kelly FP (1997) Charging and rate control for elastic traffic. Eur Trans Telecommun 8:33–37CrossRefGoogle Scholar
  12. 12.
    Monderer D, Sharpley LS (1996) Potential games. Games Econ Behav 14:124–143MathSciNetCrossRefGoogle Scholar
  13. 13.
    Milchtaich I (2009) Weighted congestion games with separable preferences. Game Econ Behav 67:750–757MathSciNetCrossRefGoogle Scholar
  14. 14.
    Mavronicolas M, Milchtaich I et al (2007) Congestion games with player-specific constants. In: International symposium on mathematical foundations of computer science (MFCS)Google Scholar
  15. 15.
    Sastry P, Phansalkar V, Thathachar M (1994) Decentralized learning of nash equilibria in multi-person stochastic games with incomplete information. IEEE Trans Syst Man Cybern B 24(5):769–777MathSciNetCrossRefGoogle Scholar
  16. 16.
    Xu Y, Wang J, Wu Q et al (2012) Opportunistic spectrum access in unknown dynamic environment: a game-theoretic stochastic learning solution. IEEE Trans Wireless Commun 11(4):1380–1391CrossRefGoogle Scholar
  17. 17.
    Pradelski BS, Young HP (2010) Efficiency and equilibrium in trial and error learning. University of Oxford, Department of Economics, Economics Series Working PapersGoogle Scholar
  18. 18.
    Coucheney P, Touati C, Gaujal B (2009) Fair and efficient user-network association algorithm for multi-technology wireless networks. In: IEEE INFOCOMGoogle Scholar
  19. 19.
    Xue P, Gong P, Park J et al (2012) Radio resource management with proportional rate constraint in the heterogeneous networks. IEEE Trans Wirel Commun 11(3):1066–1075CrossRefGoogle Scholar
  20. 20.
    3GPP TR 36.814 V9.0.0 (2010-03)Google Scholar
  21. 21.
    How much bandwidth does Skype need?
  22. 22.
    Niyato D, Hossain E (2009) Dynamics of network selection in heterogeneous wireless networks: an evolutionary game approach. IEEE Trans Veh Technol 58(4):2008–2017 (2009)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Zhiyong Du
    • 1
    Email author
  • Bin Jiang
    • 1
  • Qihui Wu
    • 2
  • Yuhua Xu
    • 3
  • Kun Xu
    • 1
  1. 1.National University of Defense TechnologyChangshaChina
  2. 2.Nanjing University of Aeronautics and AstronauticsNanjingChina
  3. 3.Army Engineering University of PLANanjingChina

Personalised recommendations