Advertisement

Deep Reinforcement Learning for Mobile Social Networks

  • F. Richard Yu
  • Ying He
Chapter
Part of the SpringerBriefs in Electrical and Computer Engineering book series (BRIEFSELECTRIC)

Abstract

Social networks have continuously been expanding and trying to be innovative. The recent advances of computing, caching, and communication (3C) can have significant impacts on mobile social networks (MSNs). MSNs can leverage these new paradigms to provide a new mechanism for users to share resources (e.g., information, computation-based services). In this chapter, we exploit the intrinsic nature of social networks, i.e., the trust formed through social relationships among users, to enable users to share resources under the framework of 3C. Specifically, we consider the mobile edge computing (MEC), in-network caching and device-to-device (D2D) communications. When considering the trust-based MSNs with MEC, caching and D2D, we apply a novel deep reinforcement learning approach to automatically make a decision for optimally allocating the network resources. The decision is made purely through observing the network’s states, rather than any handcrafted or explicit control rules, which makes it adaptive to variable network conditions. Google TensorFlow is used to implement the proposed deep Q-learning approach. Simulation results with different network parameters are presented to show the effectiveness of the proposed scheme.

References

  1. 1.
    N. Vastardis and K. Yang, “Mobile social networks: Architectures, social properties, and key research challenges,” IEEE Comm. Surveys Tutorials, vol. 15, no. 3, pp. 1355–1371, Third Quarter 2013.CrossRefGoogle Scholar
  2. 2.
    Z. Su and Q. Xu, “Content distribution over content centric mobile social networks in 5G,” IEEE Comm. Mag., vol. 53, no. 6, pp. 66–72, Jun. 2015.CrossRefGoogle Scholar
  3. 3.
    ETSI, “Mobile-edge computing – introductory technical white paper,” ETSI White Paper, Sep. 2014.Google Scholar
  4. 4.
    N. Kumar, S. Zeadally, and J. J. P. C. Rodrigues, “Vehicular delay-tolerant networks for smart grid data management using mobile edge computing,” IEEE Comm. Magazine, vol. 54, no. 10, pp. 60–66, Oct. 2016.CrossRefGoogle Scholar
  5. 5.
    C. Wang, C. Liang, F. R. Yu, Q. Chen, and L. Tang, “Computation offloading and resource allocation in wireless cellular networks with mobile edge computing,” IEEE Trans. Wireless Comm., vol. 16, no. 8, pp. 4924–4938, Aug. 2017.CrossRefGoogle Scholar
  6. 6.
    Y. Meng, C. Jiang, H. H. Chen, and Y. Ren, “Cooperative device-to-device communications: Social networking perspectives,” IEEE Network, vol. 31, no. 3, pp. 38–44, May 2017.CrossRefGoogle Scholar
  7. 7.
    K. Wang, F. R. Yu, and H. Li, “Information-centric virtualized cellular networks with device-to-device (D2D) communications,” IEEE Trans. Veh. Tech., vol. 65, no. 11, pp. 9319–9329, Nov. 2016.CrossRefGoogle Scholar
  8. 8.
    Y. Zhang, E. Pan, L. Song, W. Saad, Z. Dawy, and Z. Han, “Social network aware device-to-device communication in wireless networks,” IEEE Trans. Wireless Comm., vol. 14, no. 1, pp. 177–190, Jan. 2015.CrossRefGoogle Scholar
  9. 9.
    P. Hui, J. Crowcroft, and E. Yoneki, “Bubble rap: Social-based forwarding in delay-tolerant networks,” IEEE Trans. Mobile Comput., vol. 10, no. 11, pp. 1576–1589, Nov. 2011.CrossRefGoogle Scholar
  10. 10.
    E. Bulut and B. K. Szymanski, “Friendship based routing in delay tolerant mobile social networks,” in Proc. IEEE GLOBECOM, Miami, FL, USA, Dec. 2010, pp. 1–5.Google Scholar
  11. 11.
    S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, “A survey on mobile edge networks: Convergence of computing, caching and communications,” IEEE Access, vol. 5, pp. 6757–6779, Mar. 2017.CrossRefGoogle Scholar
  12. 12.
    K. Zhang, Y. Mao, S. Leng, Q. Zhao, L. Li, X. Peng, L. Pan, S. Maharjan, and Y. Zhang, “Energy-efficient offloading for mobile edge computing in 5G heterogeneous networks,” IEEE Access, vol. 4, pp. 5896–5907, Aug. 2016.CrossRefGoogle Scholar
  13. 13.
    J. Liu, Y. Mao, J. Zhang, and K. B. Letaief, “Delay-optimal computation task scheduling for mobile-edge computing systems,” in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Barcelona, Spain, Jul. 2016, pp. 1451–1455.Google Scholar
  14. 14.
    S. Sardellitti, G. Scutari, and S. Barbarossa, “Joint optimization of radio and computational resources for multicell mobile-edge computing,” IEEE Trans. Signal Inf. Process. over Netw., vol. 1, no. 2, pp. 89–103, Jan. 2015.MathSciNetCrossRefGoogle Scholar
  15. 15.
    X. Chen, L. Jiao, W. Li, and X. Fu, “Efficient multi-user computation offloading for mobile-edge cloud computing,” IEEE/ACM Trans. Networking, vol. 24, no. 5, pp. 2795–2808, 2016.CrossRefGoogle Scholar
  16. 16.
    X. Chen, J. Wu, Y. Cai, H. Zhang, and T. Chen, “Energy-efficiency oriented traffic offloading in wireless networks: A brief survey and a learning approach for heterogeneous cellular networks,” IEEE J. Sel. Areas Commun., vol. 33, no. 4, pp. 627–640, Jan. 2015.CrossRefGoogle Scholar
  17. 17.
    J. Gu, W. Wang, A. Huang, and H. Shan, “Proactive storage at caching-enable base stations in cellular networks,” in Proc. IEEE Int. Symp. on Personal Indoor and Mobile Radio Comm. (PIMRC), London, UK, Sept. 2013, pp. 1543–1547.Google Scholar
  18. 18.
    B. Bai, L. Wang, Z. Han, W. Chen, and T. Svensson, “Caching based socially-aware D2D communications in wireless content delivery networks: a hypergraph framework,” IEEE Wireless Commun., vol. 23, no. 4, pp. 74–81, Aug. 2016.CrossRefGoogle Scholar
  19. 19.
    A. Sengupta, S. Amuru, R. Tandon, R. M. Buehrer, and T. C. Clancy, “Learning distributed caching strategies in small cell networks,” in Proc. 11th Int. Symp. Wireless Commun. Syst. (ISWCS), Aug. 2014, pp. 917–921.Google Scholar
  20. 20.
    Y. Guan, Y. Xiao, H. Feng, C.-C. Shen, and L. J. Cimini, “Mobicacher: Mobility-aware content caching in small-cell networks,” in Proc. IEEE GLOBECOM, Austin, TX, USA, Feb. 2014, pp. 4537–4542.Google Scholar
  21. 21.
    Y. He, C. Liang, F. R. Yu, N. Zhao, and H. Yin, “Optimization of cache-enabled opportunistic interference alignment wireless networks: A big data deep reinforcement learning approach,” in Proc. IEEE ICC’17, Paris, France, May 2017, pp. 1–6.Google Scholar
  22. 22.
    Y. He, F. R. Yu, N. Zhao, V. Leung, and H. Yin, “Software-defined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach,” IEEE Commun. Magazine, Sept. 2017.Google Scholar
  23. 23.
    Y. Zhou, F. R. Yu, J. Chen, and Y. Kuo, “Resource allocation for information-centric virtualized heterogeneous networks with in-network caching and mobile edge computing,” IEEE Trans. Veh. Tech., vol. 66, no. 12, pp. 11 339–11 351, Dec. 2017.Google Scholar
  24. 24.
    C. Wang, C. Liang, F. R. Yu, Q. Chen, and L. Tang, “Computation offloading and resource allocation in wireless cellular networks with mobile edge computing,” IEEE Trans. Wireless Comm., vol. 16, no. 8, pp. 4924–4938, Aug. 2017.CrossRefGoogle Scholar
  25. 25.
    Y. He, N. Zhao, and H. Yin, “Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach,” IEEE Trans. Veh. Tech., vol. 67, no. 1, pp. 44–55, Jan. 2018.CrossRefGoogle Scholar
  26. 26.
    Y. He, F. R. Yu, N. Zhao, V. C. Leung, and H. Yin, “Software-defined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach,” IEEE Commun. Magazine, vol. 55, no. 12, pp. 31–37, Dec. 2017.CrossRefGoogle Scholar
  27. 27.
    Z. Su and Q. Xu, “Security-aware resource allocation for mobile social big data: A matching-coalitional game solution,” IEEE Trans. on Big Data, vol. PP, no. 99, pp. 1–1, 2017.Google Scholar
  28. 28.
    X. Hou, Y. Li, M. Chen, D. Wu, D. Jin, and S. Chen, “Vehicular fog computing: A viewpoint of vehicles as the infrastructures,” IEEE Trans. Veh. Tech., vol. 65, no. 6, pp. 3860–3873, Jun. 2016.CrossRefGoogle Scholar
  29. 29.
    A. Asadi, Q. Wang, and V. Mancuso, “A survey on device-to-device communication in cellular networks,” IEEE Commun. Surveys Tutorials, vol. 16, no. 4, pp. 1801–1819, Fourth Quarter 2014.CrossRefGoogle Scholar
  30. 30.
    C. Yang, J. Li, P. Semasinghe, E. Hossain, S. M. Perlaza, and Z. Han, “Distributed interference and energy-aware power control for ultra-dense D2D networks: A mean field game,” IEEE Trans. Wireless Comm., vol. 16, no. 2, pp. 1205–1217, Feb. 2017.CrossRefGoogle Scholar
  31. 31.
    K. Hamedani, L. Liu, R. Atat, J. Wu, and Y. Yi, “Reservoir computing meets smart grids: attack detection using delayed feedback networks,” IEEE Trans. Ind. Inf., vol. 14, no. 2, pp. 734–743, Feb. 2018.CrossRefGoogle Scholar
  32. 32.
    R. Atat, L. Liu, H. Chen, J. Wu, H. Li, and Y. Yi, “Enabling cyber-physical communication in 5G cellular networks: challenges, spatial spectrum sensing, and cyber-security,” IET Cyber-Phys. Syst.Theory Appl., vol. 2, no. 1, pp. 49–54, Apr. 2017.CrossRefGoogle Scholar
  33. 33.
    I. R. Chen, F. Bao, and J. Guo, “Trust-based service management for social Internet of things systems,” IEEE Trans. Dependable and Secure Computing, vol. 13, no. 6, pp. 684–696, Nov. 2016.CrossRefGoogle Scholar
  34. 34.
    F. Parzysz, M. Di Renzo, and C. Verikoukis, “Power-availability-aware cell association for energy-harvesting small-cell base stations,” IEEE Trans. Wireless Commun., vol. 16, no. 4, pp. 2409–2422, Apr. 2017.CrossRefGoogle Scholar
  35. 35.
    H. S. Wang and N. Moayeri, “Finite-state Markov channel-a useful model for radio communication channels,” IEEE Trans. Veh. Tech., vol. 44, no. 1, pp. 163–171, Feb. 1995.CrossRefGoogle Scholar
  36. 36.
    H. Gomaa, G. G. Messier, C. Williamson, and R. Davies, “Estimating instantaneous cache hit ratio using markov chain analysis,” IEEE/ACM Trans. Netw., vol. 21, no. 5, pp. 1472–1483, Oct. 2013.CrossRefGoogle Scholar
  37. 37.
    T. M. Chen and V. Venkataramanan, “Dempster-Shafer theory for intrusion detection in ad hoc networks,” IEEE Internet Comput., vol. 9, no. 6, pp. 35–41, Nov. 2005.CrossRefGoogle Scholar
  38. 38.
    R. Changiz, H. Halabian, F. R. Yu, I. Lambadaris, H. Tang, and C. M. Peter, “Trust establishment in cooperative wireless networks,” in Proc. IEEE MilCom’10, San Jose, CA, Nov. 2010, pp. 1074–1079.Google Scholar
  39. 39.
    C. Zouridaki, B. L. Mark, M. Hejmo, and R. K. Thomas, “Hermes: A quantitative trust establishment framework for reliable data packet delivery in MANETs,” Journal of Computer Security, vol. 15, no. 1, pp. 3–38, Jan. 2007.CrossRefGoogle Scholar
  40. 40.
    S. Bu, F. R. Yu, P. Liu, P. Manson, and H. Tang, “Distributed combined authentication and intrusion detection with data fusion in high-security mobile ad hoc networks,” IEEE Trans. Veh. Tech., vol. 60, no. 3, pp. 1025–1036, Mar. 2011.CrossRefGoogle Scholar
  41. 41.
    M. Abadi, A. Agarwal et al., “Tensorflow: Large-scale machine learning on heterogeneous systems,” arXiv:1603.04467, Nov. 2015.Google Scholar
  42. 42.
    Y. Zhao, W. Song, and Z. Han, “Social-aware data dissemination via device-to-device communications: Fusing social and mobile networks with incentive constraints,” IEEE Trans. Services Computing, vol. PP, no. 99, pp. 1–1, 2017.Google Scholar
  43. 43.
    B. Kaufman and B. Aazhang, “Cellular networks with an overlaid device to device network,” in Proc. Asilomar Conf. Signals, Systems & Computers (ACSSC), Pacific Grove, CA, Oct. 2008, pp. 1537–1541.Google Scholar
  44. 44.
    D. H. Lee, K. W. Choi, W. S. Jeon, and D. G. Jeong, “Two-stage semi-distributed resource management for device-to-device communication in cellular networks,” IEEE Trans. Wireless Commun., vol. 13, no. 4, pp. 1908–1920, Apr. 2014.CrossRefGoogle Scholar
  45. 45.
    P. Blasco and D. Gunduz, “Learning-based optimization of cache content in a small cell base station,” in Proc. IEEE Int. Conf. Commun. (ICC), Sydney, Australia, Jun. 2014, pp. 1897–1903.Google Scholar
  46. 46.
    J. Wu, S. Guo, J. Li, and D. Zeng, “Big data meet green challenges: big data toward green applications,” IEEE Systems Journal, vol. 10, no. 3, pp. 888–900, Sept. 2016.CrossRefGoogle Scholar
  47. 47.
    J. Wu, S. Guo, J. Li, and D. Zeng, “Big data meet green challenges: Greening big data,” IEEE Systems Journal, vol. 10, no. 3, pp. 873–887, Sept. 2016.CrossRefGoogle Scholar

Copyright information

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019

Authors and Affiliations

  • F. Richard Yu
    • 1
  • Ying He
    • 1
  1. 1.Carleton UniversityOttawaCanada

Personalised recommendations