Learning the Optimal Network with Context Awareness: Transfer RL Based Network Selection

  • Zhiyong DuEmail author
  • Bin Jiang
  • Qihui Wu
  • Yuhua Xu
  • Kun Xu


This chapter focuses on how to realize context-aware network selection. The context information provides a characterization of user-, traffic-, and network-related properties, which is able to enable fine-grained optimization for network selection. Following a similar way with Chap.  4, we can formulate context-aware network selection as an MDP model by generalizing the state to context information. However, the high resolution of context information may lead to large state space, which could result in low learning efficiency. To handle this issue, we employ a transfer learning idea. Specifically, the time–location- dependent periodic changing rule of load statistical distributions is used to realize efficient online network selection via knowledge transfer. Simulation results show that the proposed transfer RL algorithm could achieve better convergence performance by reusing learning experience.


  1. 1.
    Sim GH, Klos S et al (2018) An online context-aware machine learning algorithm for 5G mmWave vehicular communications. IEEE ACM T Network 26(6):2487–2500CrossRefGoogle Scholar
  2. 2.
    Pantisano F, Bennis M, Saad W et al (2013) Matching with externalities for context-aware user-cell association in small cell networks. In: IEEE global telecommunications conference (GLOBECOM)Google Scholar
  3. 3.
    Yang H, Du P et al (2019) Reinforcement learning-based intelligent resource allocation for integrated VLCP systems. IEEE Wirel Commun Lett 8(4):1204–1207CrossRefGoogle Scholar
  4. 4.
    Monteiro A, Souto E et al (2019) Context-aware network selection in heterogeneous wireless networks. Comput Commun 135:1–15CrossRefGoogle Scholar
  5. 5.
    Du Z, Wang C, Sun Y, Wu G (2018) Context-aware indoor VLC/RF heterogeneous network selection: reinforcement learning with knowledge transfer. IEEE Access 6:33275–33284CrossRefGoogle Scholar
  6. 6.
    Du Z, Wu Q, Yang P et al (2015) Exploiting user demand diversity in heterogeneous wireless networks. IEEE Trans Wirel Commun 14(8):4142–4155CrossRefGoogle Scholar
  7. 7.
    Bianchi G (2000) Performance analysis of the IEEE 802.11 distributed coordination function. IEEE J Sel Areas Commun 8(3):535–547CrossRefGoogle Scholar
  8. 8.
    Basnayaka DA Haas H (2015) Hybrid RF and VLC systems: improving user data rate performance of VLC systems. In: IEEE Vehicular Technology Conference (VTC)Google Scholar
  9. 9.
    Kavehrad M (2010) Sustainable energy-efficient wireless applications using light. IEEE Commun Mag 48(12):66–73CrossRefGoogle Scholar
  10. 10.
    GigaIR, Infrared Data Association Standards,
  11. 11.
    González MC, Hidalgo CA, Barabási AL (2008) Understanding individual human mobility patterns. Nature 453:779–782CrossRefGoogle Scholar
  12. 12.
    Lee D, Zhou S, Zhong X et al (2014) Spatial modeling of the traffic density in cellular networks. IEEE Wirel Commun 21(1):80–88CrossRefGoogle Scholar
  13. 13.
    Talvitie E, Singh S (2007) An experts algorithm for transfer learning. In: international joint conference on artificial intelligence (IJCAI)Google Scholar
  14. 14.
    Ibrahim M, Khawam K, Tohme S (2010) Congestion games for distributed radio access selection in broadband networks. In: IEEE global telecommunications conference (GLOBECOM)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Zhiyong Du
    • 1
    Email author
  • Bin Jiang
    • 1
  • Qihui Wu
    • 2
  • Yuhua Xu
    • 3
  • Kun Xu
    • 1
  1. 1.National University of Defense TechnologyChangshaChina
  2. 2.Nanjing University of Aeronautics and AstronauticsNanjingChina
  3. 3.Army Engineering University of PLANanjingChina

Personalised recommendations