Deep Reinforcement Learning for Interference Alignment Wireless Networks

  • F. Richard Yu
  • Ying He
Part of the SpringerBriefs in Electrical and Computer Engineering book series (BRIEFSELECTRIC)


Both caching and interference alignment (IA) are promising techniques for next generation wireless networks. Nevertheless, most existing works on cache-enabled IA wireless networks assume that the channel is invariant, which is unrealistic considering the time-varying nature of practical wireless environments. In this chapter, we consider realistic time-varying channels. Specifically, the channel is formulated as a finite-state Markov channel (FSMC). The complexity of the system is very high when we consider realistic FSMC models. Therefore, in this chapter, we propose a novel deep reinforcement learning approach, which is an advanced reinforcement learning algorithm that uses deep Q network to approximate the Q value-action function. We use Google TensorFlow to implement deep reinforcement learning in this chapter to obtain the optimal IA user selection policy in cache-enabled opportunistic IA wireless networks. Simulation results are presented to show that the performance of cache-enabled opportunistic IA networks in terms of the network’s sum rate and energy efficiency can be significantly improved by using the proposed approach.


  1. 1.
    G. Paschos, E. Bastug, I. Land, G. Caire, and M. Debbah, “Wireless caching: technical misconceptions and business barriers,” IEEE Comm. Mag., vol. 54, no. 8, pp. 16–22, Aug. 2016.CrossRefGoogle Scholar
  2. 2.
    C. Liang, F. R. Yu, and X. Zhang, “Information-centric network function virtualization over 5G mobile wireless networks,” IEEE Network, vol. 29, no. 3, pp. 68–74, May 2015.CrossRefGoogle Scholar
  3. 3.
    X. Wang, M. Chen, T. Taleb, A. Ksentini, and V. C. M. Leung, “Cache in the air: exploiting content caching and delivery techniques for 5G systems,” IEEE Commun. Mag., vol. 52, no. 2, pp. 131–139, Feb. 2014.CrossRefGoogle Scholar
  4. 4.
    C. Fang, F. R. Yu, T. Huang, J. Liu, and Y. Liu, “A survey of green information-centric networking: Research issues and challenges,” IEEE Comm. Surveys Tutorials, vol. 17, no. 3, pp. 1455–1472, Thirdquarter 2015.CrossRefGoogle Scholar
  5. 5.
    D. Liu, B. Chen, C. Yang, and A. F. Molisch, “Caching at the wireless edge: design aspects, challenges, and future directions,” IEEE Commun. Mag., vol. 54, no. 9, pp. 22–28, 2016.CrossRefGoogle Scholar
  6. 6.
    V. R. Cadambe and S. A. Jafar, “Interference alignment and degrees of freedom of the K-user interference channel,” IEEE Trans. Inform. Theory, vol. 54, no. 8, pp. 3425–3441, Aug. 2008.MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    C. Suh and D. Tse, “Interference alignment for cellular networks,” in Proc. 46th Annual Allerton Conf. on Commun., Control, and Computing,. Monticello, IL, Sep. 2008, pp. 1037–1044.Google Scholar
  8. 8.
    S. M. Perlaza, N. Fawaz, S. Lasaulce, and M. Debbah, “From spectrum pooling to space pooling: opportunistic interference alignment in MIMO cognitive networks,” IEEE Trans. Signal Proc., vol. 58, no. 7, pp. 3728–3741, 2010.MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    X. Li, N. Zhao, Y. Sun, and F. R. Yu, “Interference alignment based on antenna selection with imperfect channel state information in cognitive radio networks,” IEEE Trans. Veh. Tech., vol. 65, no. 7, pp. 5497–5511, July 2016.CrossRefGoogle Scholar
  10. 10.
    B. C. Jung and W.-Y. Shin, “Opportunistic interference alignment for interference-limited cellular TDD uplink,” IEEE Commun. Lett., vol. 15, no. 2, pp. 148–150, 2011.CrossRefGoogle Scholar
  11. 11.
    Y. He, H. Yin, and N. Zhao, “Multiuser-diversity-based interference alignment in cognitive radio networks,” AEU-Int. J. Electron. C, vol. 70, no. 5, pp. 617–628, 2016.CrossRefGoogle Scholar
  12. 12.
    M. Deghel, E. Baştuğ, M. Assaad, and M. Debbah, “On the benefits of edge caching for MIMO interference alignment,” in Proc. IEEE SPAWC, 2015, pp. 655–659.Google Scholar
  13. 13.
    M. A. Maddah-Ali and U. Niesen, “Cache-aided interference channels,” in Proc. IEEE ISIT, 2015, pp. 809–813.Google Scholar
  14. 14.
    O. E. Ayach, S. W. Peters, and R. W. Heath, “The practical challenges of interference alignment,” IEEE Wirel. Commun., vol. 20, no. 1, pp. 35–42, Feb. 2013.CrossRefGoogle Scholar
  15. 15.
    J. Yang, A. K. Khandani, and N. Tin, “Statistical decision making in adaptive modulation and coding for 3G wireless systems,” IEEE Trans. Veh. Technol., vol. 54, no. 6, pp. 2066–2073, 2005.CrossRefGoogle Scholar
  16. 16.
    A. Z. Ghanavati, U. Pareek, S. Muhaidat, and D. Lee, “On the performance of imperfect channel estimation for vehicular ad-hoc networks,” in Proc. IEEE VTC’10Fall, Sept. 2010, pp. 1–5.Google Scholar
  17. 17.
    R. Xie, F. R. Yu, and H. Ji, “Dynamic resource allocation for heterogeneous services in cognitive radio networks with imperfect channel sensing,” IEEE Trans. Veh. Tech., vol. 61, pp. 770–780, Feb. 2012.CrossRefGoogle Scholar
  18. 18.
    Y. Cai, F. R. Yu, C. Liang, B. Sun, and Q. Yan, “Software defined device-to-device (D2D) communications in virtual wireless networks with imperfect network state information (NSI),” IEEE Trans. Veh. Tech., no. 9, pp. 7349–7360, Sept. 2016.Google Scholar
  19. 19.
    Y. He, F. R. Yu, N. Zhao, H. Yin, H. Yao, and R. C. Qiu, “Big data analytics in mobile cellular networks,” IEEE Access, vol. 4, pp. 1985–1996, Mar. 2016.CrossRefGoogle Scholar
  20. 20.
    Y. Wei, F. R. Yu, and M. Song, “Distributed optimal relay selection in wireless cooperative networks with finite-state Markov channels,” IEEE Trans. Veh. Technol., vol. 59, no. 5, pp. 2149–2158, 2010.CrossRefGoogle Scholar
  21. 21.
    H. S. Wang and P.-C. Chang, “On verifying the first-order Markovian assumption for a Rayleigh fading channel model,” IEEE Trans. Veh. Technol., vol. 45, no. 2, pp. 353–357, 1996.CrossRefGoogle Scholar
  22. 22.
    C. Luo, F. R. Yu, H. Ji, and V. C. M. Leung, “Cross-layer design for TCP performance improvement in cognitive radio networks,” IEEE Trans. Veh. Tech., vol. 59, no. 5, pp. 2485–2495, June 2010.CrossRefGoogle Scholar
  23. 23.
    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.CrossRefGoogle Scholar
  24. 24.
    Y. He, C. Liang, F. R. Yu, N. Zhao, and H. Yin, “Optimization of cache-enabled opportunistic interference alignment wireless networks: A big data deep reinforcement learning approach,” in Proc. IEEE ICC’17, Paris, France, June 2017.Google Scholar
  25. 25.
    Y. He, F. R. Yu, N. Zhao, V. C. M. Leung, and H. Yin, “Software-defined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach,” IEEE Commun. Mag., vol. 55, no. 12, Dec. 2017.Google Scholar
  26. 26.
    K. Gomadam, V. R. Cadambe, and S. A. Jafar, “A distributed numerical approach to interference alignment and applications to wireless interference networks,” IEEE Trans. Inform. Theory, vol. 57, no. 6, pp. 3309–3322, 2011.MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    A. Tatar, M. D. de Amorim, S. Fdida, and P. Antoniadis, “A survey on predicting the popularity of web content,” Springer J. Internet Services and Applications, vol. 5, no. 1, p. 8, 2014.Google Scholar
  28. 28.
    D. Tse and P. Viswanath, Fundamentals of Wireless Communication. Cambridge, U.K.: Cambridge Univ. Press, 2005.zbMATHCrossRefGoogle Scholar
  29. 29.
    R. H. Clarke, “A statistical theory of mobile radio reception,” Bell Syst. Tech. J., vol. 47, no. 6, pp. 957–1000, Jul.-Aug. 1968.CrossRefGoogle Scholar
  30. 30.
    M. Abadi, A. Agarwal et al., “Tensorflow: Large-scale machine learning on heterogeneous systems,” arXiv:1603.04467, Nov. 2015.Google Scholar
  31. 31.
  32. 32.
    N. Zhao, F. R. Yu, H. Sun, and M. Li, “Adaptive power allocation schemes for spectrum sharing in interference alignment (IA)-based cognitive radio networks,” IEEE Trans. Veh. Tech., vol. 65, no. 5, pp. 3700–3714, May 2016.CrossRefGoogle Scholar
  33. 33.
    F. Cheng, Y. Yu, Z. Zhao, N. Zhao, Y. Chen, and H. Lin, “Power allocation for cache-aided small-cell networks with limited backhaul,” IEEE Access, vol. 5, pp. 1272–1283, 2017.CrossRefGoogle Scholar
  34. 34.
    C. M. Yetis, T. Gou, S. A. Jafar, and A. H. Kayran, “On feasibility of interference alignment in MIMO interference networks,” IEEE Trans. Signal Proc., vol. 58, no. 9, pp. 4771–4782, 2010.MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    H. S. Wang and N. Moayeri, “Finite-state Markov channel-a useful model for radio communication channels,” IEEE Trans. Veh. Tech., vol. 44, no. 1, pp. 163–171, Feb. 1995.CrossRefGoogle Scholar

Copyright information

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019

Authors and Affiliations

  • F. Richard Yu
    • 1
  • Ying He
    • 1
  1. 1.Carleton UniversityOttawaCanada

Personalised recommendations